How to Create a WordPress Friendly robots.txt File 阅读原文» Let's get one thing clear. Robots.txt isn't just a fancy file for webmaster-purists and professional SEOs. In fact, every WordPress developer should know a thing or two about the file and why it's so important for every blog's SEO. So first, here's the big question: What is robots.txt and why is it important? Speaking as the captain obvious: it's simply a file. But there is one interesting thing about it. It isn't displayed to the actual visitors anywhere on the blog itself. Instead, it sits in the root directory of the blog and serves only one purpose. It is the file that search engines look at before they start crawling the contents of a blog. And the reason for looking at it is to find information on what they should and shouldn't be crawling. So in essence, by using this file you can inform search engines what you want them to index and rank, and what you DON'T want them to index...
Know You,Know Me! The World is Ours,You need Know it!