.Google.com's John Mueller answered a concern concerning why Google.com marks pages that are forbidden from creeping by robots.txt as well as why the it is actually safe to neglect the associated Look Console files about those creeps.Crawler Website Traffic To Query Specification URLs.The person talking to the inquiry documented that bots were actually producing links to non-existent inquiry parameter Links (? q= xyz) to webpages with noindex meta tags that are actually also blocked out in robots.txt. What triggered the concern is that Google.com is crawling the web links to those pages, receiving obstructed through robots.txt (without envisioning a noindex robotics meta tag) at that point acquiring reported in Google Explore Console as "Indexed, though blocked through robots.txt.".The person talked to the complying with concern:." But listed below is actually the large question: why would certainly Google.com mark pages when they can't even see the content? What is actually the conveniences during that?".Google.com's John Mueller verified that if they can't creep the web page they can not see the noindex meta tag. He also creates an intriguing reference of the site: search operator, recommending to ignore the end results due to the fact that the "typical" users won't observe those outcomes.He wrote:." Yes, you're proper: if we can't crawl the page, our company can not find the noindex. That said, if our team can't crawl the web pages, after that there's not a great deal for our company to index. Therefore while you may see a number of those web pages along with a targeted site:- inquiry, the typical customer won't find all of them, so I definitely would not bother it. Noindex is also great (without robots.txt disallow), it only suggests the URLs will definitely find yourself being actually crept (as well as wind up in the Search Console document for crawled/not catalogued-- neither of these conditions create problems to the remainder of the site). The integral part is actually that you don't produce all of them crawlable + indexable.".Takeaways:.1. Mueller's solution confirms the limitations in using the Website: search accelerated search operator for analysis causes. Some of those factors is actually given that it is actually not hooked up to the frequent hunt index, it is actually a separate point completely.Google.com's John Mueller commented on the website search driver in 2021:." The brief solution is actually that a web site: concern is actually not indicated to be complete, neither used for diagnostics objectives.A web site inquiry is a details sort of hunt that limits the outcomes to a specific internet site. It is actually essentially only words internet site, a bowel, and after that the internet site's domain.This inquiry confines the end results to a details internet site. It's not indicated to become a comprehensive compilation of all the web pages coming from that web site.".2. Noindex tag without making use of a robots.txt is great for these kinds of scenarios where a bot is linking to non-existent web pages that are actually receiving found by Googlebot.3. Links along with the noindex tag will produce a "crawled/not recorded" entry in Browse Console which those won't possess an adverse effect on the rest of the website.Go through the question and also address on LinkedIn:.Why would Google.com index web pages when they can't also view the information?Included Picture by Shutterstock/Krakenimages. com.