Google Indexation Bug Starting Around Mid-April 2020

Joined
Sep 3, 2015
Messages
423
Likes
196
Points
1
They should start by fixing their miserable indexing in GSC.

Am I the only one who is experiencing issues with getting new pages indexed or updated/edited pages reindexed?
Never had any issues like that myself. I was of the opinion that if they aren't reindexing as quick as you'd like then its a quality/authority issue.
 

becool

BuSo Pro
Joined
May 10, 2018
Messages
79
Likes
53
Points
0
I haven't either until three days ago. I'm of a similar opinion. However, given the site, I've been inclined to believe it's a bug or fluke or update, rather than a lack of quality or authority.
 

Ryuzaki

お前はもう死んでいる
Staff member
BuSo Pro
Digital Strategist
Joined
Sep 3, 2014
Messages
4,158
Likes
8,009
Points
8
I know someone struggling with new posts being indexed, started last week. I've confirmed as far as I can tell there's nothing on the page, HTTP headers, meta tags, or htaccess, or robots.txt to cause the problem. This seems to be an on-going issue with Google over the last year or so: intermittent trouble with indexing.
 
Joined
Jan 27, 2020
Messages
50
Likes
64
Points
0
They should start by fixing their miserable indexing in GSC.

Am I the only one who is experiencing issues with getting new pages indexed or updated/edited pages reindexed?

330 articles uploaded for a month. 100 indexed.

800 referring domains in total, only 35 shown in Google.

Redirects not taking effect for a month, or at least not being shown in GSC.

All in all, total shitshow.
 
Joined
Dec 12, 2019
Messages
28
Likes
22
Points
0
330 articles uploaded for a month. 100 indexed.

800 referring domains in total, only 35 shown in Google.

Redirects not taking effect for a month, or at least not being shown in GSC.

All in all, total shitshow.
I'm also experiencing this indexing issue on a project. There are lots of strange things happening in serps, for example, some weird position shuffle. Initially, I've assumed that it was because of the tracker but after checking on 3 different trackers and manual search, it seems that there is some strange volatility. For trackers checking only once per day, the shuffle graph does not appear to be as aggressive.
 
Joined
Jan 27, 2020
Messages
50
Likes
64
Points
0
I'm also experiencing this indexing issue on a project. There are lots of strange things happening in serps, for example, some weird position shuffle. Initially, I've assumed that it was because of the tracker but after checking on 3 different trackers and manual search, it seems that there is some strange volatility. For trackers checking only once per day, the shuffle graph does not appear to be as aggressive.
And another funny thing. 30 articles uploaded yesterday are all in the index. The ones from month ago still aren't. IMO it's caused by the amount of content everyone is churning during quarantine, combined with the core updated that's been rolling out today. Waiting game I suppose.
 
Joined
Dec 12, 2019
Messages
28
Likes
22
Points
0
And another funny thing. 30 articles uploaded yesterday are all in the index. The ones from month ago still aren't. IMO it's caused by the amount of content everyone is churning during quarantine, combined with the core updated that's been rolling out today. Waiting game I suppose.
I just checked two domains that a friend owns and were totally penalized and now rank in top5 for the main keyword. Nothing was done to them.
 

stackcash

I Sell Words
BuSo Pro
Digital Strategist
Joined
Nov 19, 2014
Messages
722
Likes
1,158
Points
3
They should start by fixing their miserable indexing in GSC.

Am I the only one who is experiencing issues with getting new pages indexed or updated/edited pages reindexed?
I'm having annoying index issues, too. We updated the site design on 4/20/20. Since then, any new blog posts added will not index. Static web pages index just fine.

The most curious bit is that SC is telling me that the pages are available to Google:



But, when I check with the site: operator, I get nothing:



Already tweeted @ searchliason and Danny for this...but no response yet.
 

CCarter

King of America
Staff member
BuSo Pro
Boot Camp
Digital Strategist
Joined
Sep 15, 2014
Messages
2,834
Likes
6,266
Points
7
@stackcash - The "site:" operator was turned semi-off on purpose. That's not how you are suppose to check if a page is in the index.

Do the following: go into google with the whole URL and type in cache:https://wordagents.com/how-long-does-it-take-to-write-1000-words/

--

Here is a result - your page was cached on April 23, 2020 and is already in the index:



(You guys need to allow CSS files to be indexed or something)

--

Here is a cache of BuSo cache:https://www.buildersociety.com/ - cached on May 4th, 2020:



--

Another solution is to input the keywords you are targeting into a rank tracker, like the exact quote of the article "How Long Does It Take To Write 1000 Words?" - if it shows up in the top 100 you should see it.

--

If you people are having indexing problems were using the "site:" operator to find out whether Google has indexed it, that was broken on purpose. I believe it happened last quarter or so - and that John Mueller guy even stated it in a tweeted that it is no longer going to work correctly or something.

Use the cache: command going forward - IF you allow your website to be cached, I don't.
 
Joined
Jan 27, 2020
Messages
50
Likes
64
Points
0
If you people are having indexing problems were using the "site:" operator to find out whether Google has indexed it, that was broken on purpose. I believe it happened last quarter or so - and that John Mueller guy even stated it in a tweeted that it is no longer going to work correctly or something.
They did, yes, however I can clearly see, that only the pages that are shown with "site:domain.tld" are actually ranking for something. Regardless whether the GSC says they are indexed.

Not to mention that Mueller has said a lot of crap over the past few years.
 

CCarter

King of America
Staff member
BuSo Pro
Boot Camp
Digital Strategist
Joined
Sep 15, 2014
Messages
2,834
Likes
6,266
Points
7
@Nargil - It's broken in terms it will not show not all pages, especially newer pages - the indexing problem. It still exists but should not be used to figure out whether a page is indexed or not.
 
Joined
Dec 12, 2019
Messages
28
Likes
22
Points
0
Ok, it seems that using cache the URLs seem to be indexed in my case. The strange thing is that in the results for x keyword, they still serve the old page even if ~40 days passed.
 

Ryuzaki

お前はもう死んでいる
Staff member
BuSo Pro
Digital Strategist
Joined
Sep 3, 2014
Messages
4,158
Likes
8,009
Points
8
I just checked out my own site. Of my last two posts, published last week one day apart, one is indexed and cached, the other isn't. At the end of the day, it's inconsequential, but something to get easily annoyed about and try to fix.

This seems like a reoccurring bug we'll run into every 6 months or so. Like the Search Console and Mobile Friendly Test bug of not loading CSS and JS and image files that's been going on for years now. ~JustGoogleThings☆★☆★
 
Joined
Sep 3, 2015
Messages
423
Likes
196
Points
1
OK I just checked some of my smaller sites and can now see that posts are not being indexed. GSC says they are, cache: reports them as nowhere to be found!
 

becool

BuSo Pro
Joined
May 10, 2018
Messages
79
Likes
53
Points
0
They ought to rollback a dozen or so updates to get back to when things actually almost kind of sort of made sense.

I've had delayed indexing/reindexing for a week plus on new and old pages. I'll report back with any noteworthy developments or observations.

Best line from Google: "We're not aware that there is anything on our end causing this."
I like this. It isn't Google that's broken. It's LinkedIn and all the other sites.
 
Joined
Dec 12, 2019
Messages
28
Likes
22
Points
0
I've encountered some strange behavior.

So, I've added 4 posts and waited 2 days to see what happens. According to the server logs, Google visited the index sitemap(sitemap_index.xml) BUT did not visit the other included sitemaps(rankmath makes one for posts, another for pages, and so on). Even if they have the date on which they were modified.

After that, I've used the "request as Google" and "Submit to the index" to hurry the indexing process for 2 out of 4 posts. Google visited the pages but did not index them even after that. I've waited 12 hours.


It indexed the 2 pages instantly(checked with proxy and incognito) only after I've resubmitted the sitemap that contains the posts. The other 2 posts were indexed in one hour.

This behavior makes me think about the way hacked sites are used to spam content. While the content is on the server, the pages are not in the sitemap(most of the time).
 

becool

BuSo Pro
Joined
May 10, 2018
Messages
79
Likes
53
Points
0
Quick update. It has been two weeks since my original post regarding this issue and I'm still encountering the problem. Specifically, I cannot seem to get pages reindexed based on the last crawl date in GSC and the title tags and meta description shown in the SERPs (which is displaying outdated information). I also cannot seem to get a new page to show up in the SERPs, which I'd ordinarily view as an entirely separate problem. I first began experiencing the original issue before the recent Google update.
 
Joined
Nov 5, 2014
Messages
391
Likes
275
Points
1
Quick update. It has been two weeks since my original post regarding this issue and I'm still encountering the problem. Specifically, I cannot seem to get pages reindexed based on the last crawl date in GSC and the title tags and meta description shown in the SERPs (which is displaying outdated information). I also cannot seem to get a new page to show up in the SERPs, which I'd ordinarily view as an entirely separate problem. I first began experiencing the original issue before the recent Google update.
What's your site's CitationFlow (or some other calculation of PageRank)? If you google your site's homepage and check it's cache, what's the date that the catch show? That's the last time Google indexed your homepage, the page with the highest PR on your whole site. The other pages would be indexed at a slower rate.

GoogleBot crawls the web according to PageRank. The higher a domain's PR, the more often it's crawled. 2 weeks might not be enough time, especially if your domain is low PR. Also, a poor site structure would make GoogleBot have a hard time finding new pages too.
 
Joined
Nov 15, 2019
Messages
11
Likes
10
Points
0
Not that you have not been doing this too, but it has worked for me for awhile. I am getting new website updated and indexed fast. Do the normal Request Indexing, then when done, then do Test Live URL and refresh the search console page and you will see Last Crawl was moments ago. My updated content, or Title, Description will show within 24hrs in index. This works with my established sites as well as new hobby sites I slap up together and make regular changes. Hope this helps
 

becool

BuSo Pro
Joined
May 10, 2018
Messages
79
Likes
53
Points
0
What's your site's CitationFlow (or some other calculation of PageRank)? If you google your site's homepage and check it's cache, what's the date that the catch show? That's the last time Google indexed your homepage, the page with the highest PR on your whole site. The other pages would be indexed at a slower rate.

GoogleBot crawls the web according to PageRank. The higher a domain's PR, the more often it's crawled. 2 weeks might not be enough time, especially if your domain is low PR. Also, a poor site structure would make GoogleBot have a hard time finding new pages too.
Citation Flow is 35. The date of the cache is a week ago. Historically, I have not had any issues with indexing. When I request indexing in GSC, the last crawl date does not change.

Not that you have not been doing this too, but it has worked for me for awhile. I am getting new website updated and indexed fast. Do the normal Request Indexing, then when done, then do Test Live URL and refresh the search console page and you will see Last Crawl was moments ago. My updated content, or Title, Description will show within 24hrs in index. This works with my established sites as well as new hobby sites I slap up together and make regular changes. Hope this helps
I appreciate it. This didn't work for me, at least based on the date shown under "Last Crawl"
 
Joined
Dec 12, 2019
Messages
28
Likes
22
Points
0
So, after doing some more tests, it seems that Google checks the parent sitemap if notified but it does not the internal sitemaps.




I think having only one sitemap will accelerate the process.
 

Ryuzaki

お前はもう死んでいる
Staff member
BuSo Pro
Digital Strategist
Joined
Sep 3, 2014
Messages
4,158
Likes
8,009
Points
8


It looks like this is going to be an on-going ordeal. They tried to brush it off as "we don't index low quality crap" but we know they index everything they can get their hands on, even blank pages.

EDIT - looks like they're saying they resolved the issues today. The problem started well before June the 2nd, though. Took them a while to acknowledge it, possibly to make it look like they fixed it quickly. Who knows. Index our pages!
 

becool

BuSo Pro
Joined
May 10, 2018
Messages
79
Likes
53
Points
0

It looks like this is going to be an on-going ordeal. They tried to brush it off as "we don't index low quality crap" but we know they index everything they can get their hands on, even blank pages.

EDIT - looks like they're saying they resolved the issues today. The problem started well before June the 2nd, though. Took them a while to acknowledge it, possibly to make it look like they fixed it quickly. Who knows. Index our pages!
It appears that they have resolved the issue, at least on my end and with respect to one of the new pages I created on an existing site (that I talked about earlier/above). By way of background, I had issues getting existing pages reindexed and at least one new page actually indexed on a site that did not previously suffer from indexation issues.