![]() |
robot.txt help
I'm throwing together a simple website for my civic association and at the moment don't want it to be indexed by the search engines. Does anyone off their head know how to write a robot.txt file that simply excludes everything in the domain?
Any sorry I know the post is a tad off topic, but hopefully my contributions to the forum outweigh this minor diversion. |
Google is pretty useful. ;)
http://www.javascriptkit.com/howto/robots.shtml 1) Here's a basic "robots.txt": And remember that "hiding" the content with robots.txt doesn't really hide anything. The content is still on the internet and anyone who wants to can find it.User-agent: * Disallow: / With the above declared, all robots (indicated by "*") are instructed to not index any of your pages (indicated by "/"). Most likely not what you want, but you get the idea. |
Thanks for the tip. Anyone who really wants to find it will find an early draft of a boring website with typos, comma splices, and a some non-implemented features.
|
| All times are GMT -6. The time now is 8:20 am. |
This site is owned, operated, and maintained by MH Sub I, LLC dba Internet Brands. Copyright © 2026 MH Sub I, LLC dba Internet Brands. All rights reserved. Designated trademarks are the property of their respective owners.