To: simpson96
taken off of the Internet Archive's Wayback Machine thanks to an exclusion request added by the website's operators
HUH?? WTF?? Why would this be permissible??
6 posted on
04/26/2018 6:21:39 AM PDT by
eyeamok
To: eyeamok
HUH?? WTF?? Why would this be permissible??Because we in the web development community expect the robots crawling our web sites to respect our wishes expressed via the robots.txt.
7 posted on
04/26/2018 6:38:07 AM PDT by
Ol' Dan Tucker
(For 'tis the sport to have the engineer hoist with his own petard., -- Hamlet, Act 3, Scene 4)
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson