WhiteHouse.gov and robots.txt
November 9, 2004
I am doing some research into the robots.txt to work with our SharePoint 2003 implementation at work. I googled ‘robots.txt’ and the first several entries were tutorials on how to implement robots.txt. The 5th entry was for the www.whitehouse.gov/robots.txt.
It is almost 2000 lines long and seems to disallow search engine access to pretty much everything.
I wonder what they are trying to hide. Of course, I can go throught the robots.txt file and find out for myself.
Popularity: 3% [?]
Comments
Got something to say?
-
Sponsored Links
-
Recent Posts
- ACL Fest iTunes Music Sampler Download Codes
- Gustavugees
- Antivirus XP 2008 and avxp2008.com
- Antivirus 2009 and browserprotectioncenter.com
- C# using Directive
- What is the difference between a computer virus and spyware?
- Removing Antivirus 2009 - Round 3
- SIGG Can’t Keep Up With Water Bottle Demand
- XP Antivirus 2008 and Antivirus 2009 - Round 2
- XP Antivirus 2008 and Antivirus 2009 are evil!
-
Sponsored Links
-
Categories
-
Archives
- September 2008
- August 2008
- July 2008
- June 2008
- August 2007
- April 2007
- January 2007
- December 2006
- November 2006
- October 2006
- September 2006
- August 2006
- July 2006
- April 2006
- March 2006
- February 2006
- January 2006
- December 2005
- November 2005
- October 2005
- May 2005
- December 2004
- November 2004
- October 2004
- September 2004
- August 2004
-
Links
- Alex King
- alexking.org: Blog > E-mail Signature?
- Community Server 2.0 Beta 2 — now available!
- CS 2.0 Beta 3 is now available…..
- Gadgetopia
- HALT data for P1130 21″ Trinitron monitor - Monitors - Dell Community Forum
- Joel on Software
- Looking for an adequate BitTorrent client to replace Azureus
- Mike Industries
- MyHomePoint
- Optimus Keyboard | Gadgetopia
- PhotoMatt
- Scobleizer - Microsoft Geek Blogger » I made Phil Ripperger stand in line for an Xbox 360
- SimpleBits
- Smcculloch.Net - Modules, Skins, Skin Objects > Forums > AVCalendar Foster Home?
- Smitty.net
- Solutions Design::TinyForum::List all messages of thread: MyHomePoint.com
- Solutions Design::TinyForum::List all messages of thread: Ruby on Rails
- The Post Money Value: The New Digital Life
- WordPress
- Zawodny: Slashdot Will Die in 2006 | Gadgetopia
- Zeldman
-
Subscribe
-
Meta