Day: August 10, 2020

Googlebot is crawling my AMP pages more than they are crawling my desktop pages. I have the appropriate canonical from AMP to desktop and amphtml from desktop to AMP. The desktop version also has a self-referencing canonical. Only canonical pages are in the sitemap.

This is a concern because less than 10% of our traffic is from mobile devices (unique, I know), yet it's more than 50% of our crawl budget.

The one thing that we do, which I'm not sure if this is appropriate or not, is whenever a desktop page 1 links to an internal page 2, the AMP version of page 1 links to the AMP version of page 2. Therefore, there are internal links pointing to AMP pages, but only from other AMP pages.

The other thing I was wondering is whether anyone has heard of Google serving AMP pages to desktop users behind low bandwidth connections, where they could benefit from AMP. Supposedly AMP doesn't have to be for only mobile anymore, but Google hasn't really demonstrated this. We don't get a lot of mobile traffic, but we do get a lot of third-world / low-bandwidth traffic.

I am trying to remove an entire folder of thin content from Google to help me recover from a Panda/EAT-related penalty. I want to keep the content on the site for the benefit of users, but not waste crawl budget or have Google think that we have so many pages of thin content.

I added the folder to robots.txt quite a few months ago. While some pages are showing up as "Blocked by robots.txt", the majority of pages now show up in my coverage report as "Indexed, though blocked by robots.txt". About 2 months ago, I submitted a removal request for all URLs that begin with the prefix, but there's been no change. Google Search Console's report updates every few days, but the number of URLs that say, "Indexed, though blocked by robots.txt" is increasing, even months after the removal request for those same pages.