Troubleshooting guide: Technical SEO
Here are some methods of troubleshooting technical SEO issues:
Use the “info:https://www.domain.com/page” search operator on Google to look for issues on a specific page. Often, Google will treat two or more pages as a duplicate and fold them into one resultant page. The problem arises when the page you want is not what Google considers the canonical version of the page. If you use the above operator and get a different page from the one you were expecting, then you have some problems you need to address immediately.
Use the “site:domain.com” to list all the pages from your domain in the Google index. You need to look for any pages in the index that appear in ways you did not expect such as unknown operators in the URL.
Check your robots.txt file regularly to ensure that there are no blocks on the page you want part of the index. If you block a page that another page marks as canonical, then you risk removing that page form the Google index entirely. Often, webmasters do not realize how an old entry in a robots.txt file can cause problems to new changes to the site.
Large websites run by more than one team can result in changes that can affect other teams. For example, when you make country-specific changes, you can prevent an entire another geography from seeing the content. Change your user-agent in Chrome to appear as Googlebot and make sure that changes don’t block out search engine spiders.