In the early days a meta tag “NO ROBOTS, NO FOLLOW” would supposedly keep an HTML page from being indexed and showing up in search results. I just found an old page of mine from the 1990s that, of course, showed up in the search results. :^)
Honoring “NO ROBOTS, NO FOLLOW” was purely voluntary, as you found out.
What a trove on FR! Could be used for good with all the rich content and could be used for evil.