Last fall I put a robots.txt file in my root directory designed to stop search engines from crawling my blog. I did it because those searches were starting to seriously eat into my bandwidth. Whenever I checked my stats after that I kept seeing hundreds of search engine hits every day, so I thought that the file didn't work.
I got busy and didn't check my stats again until this morning. I was thrilled to see that in the last 24 hours only one of my hits came from a search engine (someone using metacrawler.com looked for "blood sports" and found a post I wrote last year on politics as a blood sport.)
My bandwidth usage is now back to the trivial level you'd expect for a blog as boring as mine.
Your blog is by no means boring!
Posted by: Susan | April 15, 2005 at 04:52 PM
GREAT News! I would appreciate knowing where you stuck that code. As you know, I have had serious problems with bandwidth and poachers using my pictures and site as their own. When I contacted TP about using "that file" they told me I could not use it. I really don't care about hits on my site or having a Huge following just the few that visit is great. If you have a moment I would appreciate learning you secret. You know the address I think.
Saturday Hugs Back @ Ya!
Posted by: Sallie | April 16, 2005 at 10:54 AM
Hey, I would like the secret also!
Posted by: Susan | April 16, 2005 at 05:16 PM
Why would I come to a boring blog? Harrumph.(am I boring?) Seriously, I don't understand this whole bandwidth thing.(?)
Posted by: Margaret | April 17, 2005 at 11:50 PM