Recently, Google updated their search algorithm to favor mobile-friendly websites in mobile search and penalize those sites that do not have mobile-friendly or responsive versions. In order to help website owners confirm whether or not their site is considered mobile-friendly, Google also released a Mobile-Friendly Test Tool. The trouble is, some people are finding that their websites, which they thought were responsive or mobile-friendly, are failing Google’s test!
This can be extremely frustrating, but fortunately, there’s a fairly simple fix.
Why would a mobile site fail google’s mobile-friendly test?
Before we go on, I should specify that in this article, we’re only troubleshooting websites that are responsive, or have fully-functional mobile versions that look completely fine on smartphones and tablets. In cases where these websites fail Google’s mobile-friendly test, the error message is often the same, usually something like this: “this page uses 5 resources which are blocked by robots.txt.”
What does this mean?
In order to fully understand the problem, there are two concepts that are important to grasp:
First, you should know that the purpose of a robots.txt file is to tell search engine crawlers, such as Google, what pages or files it should or shouldn’t look at on your website. A common use of robots.txt is to prevent a login or thank you page from showing up in search results. On websites that are powered by a content management system, robots.txt is often also used to block certain administrative directories on your website which are used to power your content management system.
How can you fix this robots.txt blocking and pass Google’s mobile-friendly test?
If you’re not sure where to find the problematic robots.txt file, or what it says, you can use the Robots.txt Tester in Google Webmaster Tools.
However, there are a few potential pitfalls to watch out for:
- Be careful not to allow access to any directories or files that you don’t want Google to see. For example, if you keep your stylesheets in a directory called “resources,” but you also keep private PDF files (like whitepapers that people have to provide their email addresses to download) in that same directory, Google will begin to index these PDFs as well.
- On some content management systems, such as WordPress or Drupal, if you’re using certain SEO or redirect plugins, your robots.txt file may not be visible through FTP, or it may be dynamically created upon Google’s request. To fix this, you’ll want to double-check your plugin settings. If need be, you can also create a new robots.txt file and place it where Google is looking – this file will typically override anything a plugin is doing.
Google mobile-friendly testing tool isn’t perfect and can be a real headache for site owners who have spent hours making sure their websites work great on mobile devices. Fortunately, Google’s mobile-friendly test is a great diagnostic tool, and can be a big asset for site owners trying to understand why they no longer rank well in mobile search.
Even if your website works fine on your smartphone, you should definitely run it through mobile-friendly test tool to make sure you’re not unknowingly penalized in mobile search.
For more information on Google’s mobile-friendly algorithm update, check out our post Is Having a Mobile-Friendly Site Good for Business?
Need help understanding your results and your resolution options?
See what we can do for you.
Website Spring Cleaning: The GA4 Migration Edition
Spring is finally here, and the change of season presents a perfect opportunity to…Keep Reading
Google Analytics 4: What it Means for Your Website
If you’ve read any marketing newsletter in the past few months, you’ve seen the…Keep Reading
The Essential Guide To SEO for 2023 and beyond
What You Need To Know Now About SEO We’ve put together this Essential Guide…Keep Reading