Thursday, July 13, 2006

Robots.txt - Another lesson from SES Miami implemented

Vanessa explained to me that a sc****d up robots.txt file was probably not helping me. Then, seeing the truly blank look on my face, she explained to me what a robots.txt file was.... Then, she sighed and looked at her watch.....

In any case, I've been studying hard this week and I think I now understand...My google maps, › robots.txt analysis for says:

Last downloaded July 13, 2006 4:36:05 PM PDT Status 200 (Success)
This means that My site returned a "heres the content that you want" message for the request for a robots file. The report continues:

Parsing results, lue Result
robots.txt file does not appear to be valid

Then in the box, it has the source code for my default "page not found". Got it. :> I'm returning the wrong page without an appropriate error message. Since I don't know how to fix that (it requires involvement of my hosting company which is a non-starter....I know, I should move. I should also exercise more and not stare but.....), then I should at least create a proper robots.txt file and place it there. Which I've now done!

Then, the google test has a nifty test ability of a robots.txt file. What a shame I put it up prior to testing.....But then, since Vanessa actually wrote it for me at the show and I emailed it to myself, it was a reasonably safe bet.....Now I've tested it and it seems to be successful.

Check off another base covered....

No comments: