There are a lot of myths and misinformation pushed out about Local SEO, and it can be difficult for business owners who are already preoccupied with running their businesses to know what they should worry about and what is a contrived sales pitch. Here’s a free SEO tip that you can investigate yourself. This will be particularly helpful if you are using content marketing and want to get the most out of it.
Google Is Getting Stingy with Its Crawlers, and Your Content May Not Be Indexed
The internet grows at a rate that boggles the mind. Millions of new pages every day and with the availability of tools like ChatGPT and other Artificial Intelligence writing tools that allow users to generate written content almost instantly, that rate of expansion is only going to increase.
Google has to take in all of that content, and the volume has become an energy and processing dilemma. To deal with it, Google will assign websites a crawl budget. This is the amount of time and data it will invest in your site at a given interval, and the reality is Google is becoming much more stingy with its crawlers.
Unfortunately, there is no straightforward way to determine your crawl budget as it is not publicly disclosed by Google. However, monitoring your crawl stats in Search Console can give you an idea of how often Google is crawling your site and if any changes impact your crawl frequency. It is possible to use your crawl history to reverse engineer, but the reality is unless you are prepared to do the heavy lifting of optimizing your site for an improved crawl budget, the actual budget is not what you should focus on.
If you have Search Console set up for your site, it’s easy to see the status of all your pages.
- Log in to Search Console, and click on Pages. Scroll down to the section labeled: Why pages aren’t indexed.
You will see a number of reasons listed. Most of these have to do with technical SEO and the way your site is set up. If you have done the basic SEO on-site optimization, you will have your canonical set up as well as a full complement of pages that you intentionally do not want to be crawled. There is a full write-up of all the listed Reasons from Google here.
For our purposes the only two categories we want to check are:
- Crawled – currently not indexed
- Discovered – currently not indexed
These categories are actionable in Search Console, and I have found that they do get content crawled. Here’s how you do it.
Click on the Crawled – currently not indexed. You will see a list of URLs. Click on one of them to get the side menu to appear and then click on Inspect URL. 
Google will go out and inspect the URL to see if it’s online and its current state. Once it’s done retrieving data, the system will test the URL to see if it can be indexed. Once this is complete and the page is found to be in good order, you will have the option to Request Indexing. Click on this option, and it will run until you see the following window.
This does not immediately put you into the index however I have found that it moves things along in a few days to a week.
Just a little SEO tip that can help improve the performance of your Content Marketing. I hope it helps your business marketing.
