Free Republic
Browse · Search
News/Activism
Topics · Post Article

To: bigbob

he doesn’t evcen have to “block” google.

Google fully respects the contents of a site’s robots.txt file - which is akin to putting up a friendly notice at the entrance that soandso (or veery robot) is not welcome there.

One line of text in that file on each of his sites, and Mr Murdoch’s content will completely vansih from Google over the course of the next weeks.

A nice letter to Google would also do the trick as well.

Murdoch is just a bloviating fraud here.


5 posted on 10/10/2009 9:26:26 AM PDT by Tullius
[ Post Reply | Private Reply | To 3 | View Replies ]


To: Tullius

“Google fully respects the contents of a site’s robots.txt file”

Hmm. Debatable. If you put something to not be crawled in robots.txt they will crawl it just to find out what’s so interesting. They may not list it, but they go have a look anyway. Others don’t do that.


11 posted on 10/10/2009 11:27:06 AM PDT by chuck_the_tv_out ( <<< click my name: now featuring Freeper classifieds)
[ Post Reply | Private Reply | To 5 | View Replies ]

Free Republic
Browse · Search
News/Activism
Topics · Post Article


FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson