Free Republic
Browse · Search
News/Activism
Topics · Post Article

To: .30Carbine
Yes, I see they didn't include that in the URL. Go to the site linked above, then enter "worldnetdaily.com" into the box labeled "Compare: freerepublic.com vs." Then click on "Compare Sites." You can also enter multiple sites into the compare box. For example, include "newsmax.com" and "lucianne.com", etc.

15 posted on 11/15/2002 3:23:16 PM PST by Jim Robinson
[ Post Reply | Private Reply | To 14 | View Replies ]


To: Jim Robinson
AT the risk of wasting your valuable time, the following is exactly what I get when I click on either your link or that above:

User-agent: googlebot # allow Google crawler
Disallow: /search

User-agent: gulliver # allow Northern Light crawler
Disallow: /search

User-agent: slurp # allow Inktomi crawler
Disallow: /search

User-agent: fast # allow FAST crawler
Disallow: /search

User-agent: scooter # allow AltaVista crawler
Disallow: /search

User-agent: vscooter # allow AltaVista image crawler
Disallow: /search

User-agent: ia_archiver # allow Internet Archive crawler
Disallow: /search

User-agent: * # disallow all other crawlers access
Disallow: /

I must have done something very, very bad, huh?

16 posted on 11/16/2002 6:38:35 AM PST by .30Carbine
[ Post Reply | Private Reply | To 15 | View Replies ]

Free Republic
Browse · Search
News/Activism
Topics · Post Article


FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson