Recovery efforts after getting hit by last 2 updates

[makoloko]

Hi - this is my first post here.

I've been working on my site for 3 years and have 400 posts. I use Wordpress. A few months ago my traffic hit 170k/mo. Then it got hit by the two latest Google updates. 50% drop. Ouch.

A month ago I started working on fixing SEO problems. I've been going through the "kitchen sink method" and checking everything.

Fixes
First, here's specifically what I've done so far:
1. 75% of indexed pages were tag pages. 1200 tags total. I deleted them all and then put them in the temporary removal tool in GSC (to prevent 404s in SERPs while they drop out of the index)

2. I noindexed category pages. These are 0 effort. I use them as nav menu items. I don't want them competing with the actual content.

3. My homepage was terrible. I was linking to manually updated directory pages. Awful. I changed the homepage to simply display the latest posts (paginated) and deleted the directory pages.

So lots of thin content removed from the index. It's been a month since I did those changes, and I just got hit by the spam update, which motivated me to keep going.

4. I got Screaming Frog. I've fixed the following:

• 45 pages had images with no alt tag
• 100 links with redirects. I fixed all of the links. One post had several of these. This was one of my first posts and it was very low quality, so I completely rewrote it (kinda fun after fixing links all day)
• A few missing images
• Changed posts to show "Last updated on..." instead of published date. Right now I'm naturally updating lots of posts, so they should all get a boost from this.
• Removed thin categories from the nav menu
• Rewrote 15 low quality posts so far. I plan on doing a quality check on all content systematically going forward.

So that's what I've done so far. It seems like index bloat was the biggest problem, but I'm going to keep gping. There's more problems to fix, like titles / descriptions.

Questions
I have a few questions.

1. My homepage shows the latest posts with pagination. I've noticed Google is now indexing these pages. That seems bad and is just contributing to index bloat. Should these pages be noindexed? I'm pretty confused about this, because I thought I read that Google knows how to properly deal with these (sure doesn't look like it).

2. I'm not sure what to do with categories. I have them noindexed. I have 5 main categories and about 20 subcategories. I just have these as nav menu items, but they get very little traffic. Regardless, i want to at least add descriptions so they don't look like trash. I don't think they have value in the index though.

3. How many posts should I show on paginated results (homepage, categories, search)? It's currently 10. This shows the title and a snippet. As a user, 10 isnt enough. I prefer to skim through lots and lots of posts rather than having to click through paginated results. I'm thinking of boosting it up to 25 or 30. Apparently this would be good for SEO because it reduces crawl depth?

 

[Darth]

It is unlikely that any of the issues/fixes you've considered (on-site) are the cause of your problems in the latest Google updates.

1. What niche are you?
2. What is your content like compared to competitors?
3. How do you build links?

I have posted about index bloat and noindexing categories and tags in here very often, but recently put them all indexed again (somewhere in my post history I link to an article about why). I have only seen positive movements with the recent updates. I don't think index bloat due to tags and categories is a big deal any more.

 

[Ryuzaki]

@makoloko, GREAT work fixing up all you have. It's always worth doing.

What I can tell you right now is this kind of thing has been going on since the May Update. Tons of people's sites were essentially disappearing from the SERPs and taking ~50% hits. While a bunch of those have recovered in the July and September Updates, more sites are taking these kind of hits. My own took a 50% dive in September and I'm positive I'm doing nothing wrong.

The advice I was giving back in May and July was for everyone to hold tight and the sites would pop back, and that Google was probably making a mistake. It's likely now that they aren't making a mistake and while I don't have a great theory that answers all the questions as to why this is happening, one was provided here. It doesn't answer all questions, but it does fit.

In my opinion, many of us are probably doing nothing wrong and it's just a matter of time for us to pop back. But these moments are always a great time to audit our sites like you're doing. I'm collecting ideas of things to add instead of remove to my site to improve things and will implement them when I'm satisfied with my collection.

Anyways, I agree and disagree with @Darth. Indexation bloat does matter from tags and categories, but not if you're using them appropriately. I'm not going to name the site or share it, but I know of a venture backed news site that has failed to thrive and actually got penalized and the answer is because for every one of their posts, they add like 3-5 brand spanking new tags that they only use once. That's ~99% of their indexation at this point and it totally tanked them. That kind of bloat will hurt you for sure, and I'm glad you cleaned yours up.

But having 100's of paginated pages actually filled with posts won't hurt you. That's where I agree with Darth.

However, If I was you, I would increase beyond showing 10 posts per paginated (Wordpress calls them "paged" pages) page. I'd show 25 or so. It'll make each page have a higher quality score.

What I also would recommend would be to definitely index your category pages. If you're linking to them sitewide, then you're endorsing those pages and Google will possibly index them anyways. But I'd index them, and if you don't want anything else to be shown, you can set the "paged" pages to be noindex. You can search for a code snippet @Darth and I have both posted many times now that will help you achieve this.

So what happens is your first page of an archive (category, tag, date archive, author archive) are indexed, but /page/2/ through /page/999999/ are noindex. This will work on your homepage issue, too. This isn't necessary as Darth is pointing out, but I do set them to noindex because I like a squeaky clean indexation. What you can do from here is make your theme use the category descriptions Wordpress offers and set them to only show on the first page of the archive, and then add all the text, images, headers, and links you want in there. The boxes accept HTML of all kinds. That way you can get some unique content on these first pages of the archives to increase their quality scores further.

I disagree that the technical and on-page issues would not have an impact in the current updates. One was a Broad update, and that's where they usually hide the fact that they're refreshing Panda data. As a matter of fact, I don't care what they name an update any more because it's all obfuscation. They can call it the "X Update" but that doesn't mean it doesn't also include W, Y, and Z in it, too.

We want to do everything the best we can at all times to avoid issues in any update. I'm not saying you did take a hit due to the issues you've outlined. The indexation would be the main problem. But "everyone" is getting their turn being hit for ~50% since May. DR 5 to DR75 sites are taking beatings. High quality and low quality sites are getting their turn. It may have little to do with our sites this time around, but every time around is a good chance to take a step out of our normal workflow and run an audit with fresh eyes and improve anything we can.

 

[Darth]

Here's some code if you want to noindex tag pages with only 1 post allocated to them. Tag archives with 2 or more posts will be index.

add_filter( 'wp_robots', 'wpse_cleantags_add_noindex' );
function wpse_cleantags_add_noindex( $robots ) {
    global $wp_query;


    if ( is_tag() && $wp_query->found_posts < 2 ) {
        $robots['noindex'] = true;
        $robots['follow']  = true;
    }


    return $robots;
}

 

[makoloko]

Thanks for the great feedback @Darth and @Ryuzaki. This gives me very specific things to work on.

1. Change "paged" results to 25.
2. Noindex paged results > 1. I agree @Ryuzaki - I want a squeaky clean index. Ignoring what's in the index got me in a lot of trouble.
3. Improve category pages and re-index ones with > X posts (@Darth's code gave me that idea). I want these to be useful to users when they click them in the menu, and some of them, especially categories would probably match exact queries.

a great theory that answers all the questions as to why this is happening, one was provided here. It doesn't answer all questions, but it does fit.

That's really interesting. So it's a gigantic, rolling experiment by Google with randomized sites. Based on the sheer randomness of who's been hit, that does make a lot of sense.

I know of a venture backed news site that has failed to thrive and actually got penalized and the answer is because for every one of their posts, they add like 3-5 brand spanking new tags that they only use once. That's ~99% of their indexation at this point and it totally tanked them

That's exactly what I did. Honestly, I was mindlessly adding tags. I ended up with 1200 tags total, 800 of them were used once. Regardless of it being bad for indexation, it's just terrible UX when these are used wrong. And to make matters worse, they were competing with my actual content - and winning in some cases!

1. What niche are you?
2. What is your content like compared to competitors?
3. How do you build links?

1. Programming
2. I'll try to answer this objectively. I know what my target audience wants, and all metrics indicate they like my content. Compared to my competitors, I am brief. No fluff, no filler. If a word is there, it had to be. Sometimes that requires 100 words, other times 1000 words.

3. I participate in my niche community (twitter and newsletters). Besides that, it's all organic. According to Moz, my DA is 27. Of course, I get lots of spam links. I haven't disavowed them or anything, since I'm afraid I'll cause more harm than good.

 

[Hanuman]

Here's some code if you want to noindex tag pages with only 1 post allocated to them. Tag archives with 2 or more posts will be index.

add_filter( 'wp_robots', 'wpse_cleantags_add_noindex' );
function wpse_cleantags_add_noindex( $robots ) {
    global $wp_query;


    if ( is_tag() && $wp_query->found_posts < 2 ) {
        $robots['noindex'] = true;
        $robots['follow']  = true;
    }


    return $robots;
}

Where exactly should one be adding this?

 

[Darth]

Where exactly should one be adding this?

Into functions.php of your child theme.

 

[makoloko]

@Hanuman Personally, I have my private plugin for putting customizations like this. Most people should probably stick with child themes though - as @Darth mentioned

 

[makoloko]

I just wanted to give a final update about this, just in case it's useful for someone reading this in the future.

I've completed looking at all SEO issues on my site. I fixed most of the issues. My index is now super clean! For completeness, I've listed all of the issues I fixed below, and explained which issues I'm not fixing.

My next step is to do a content quality check, and this will be a long-term, continuous process. I'll be checking all content, starting with ones with poor quality indicators. Meanwhile, I'll continue writing new content.

Index bloat

• Removed 1200 tag pages. This was 75% of the index, and mostly single use tags.
• Removed manually updated directory pages (good riddance!)
• Noindexed archive pages past page 1.
• Noindexed author pages that I didn't realize existed. I had author links going to the site's about page. Google somehow discovered these default author pages that have 0 inlinks (probably from way before)

Before, there were about 1700 indexed pages. Now there's 700 in GSC (it's slowly going down), and the site: search is now showing exactly the right results (I verified every single result).

Crawl depth and internal linking

• Increased pagination from 10 results to 25.
• Reindexed category pages, improved titles/descriptions, and consolidated thin categories.

Before, there were 40 pages with crawl depth >= 4. Now all pages have crawl depth < 4. Much tighter crawl graph.

Before, according to Screaming Frog the avg link score was 6, now it's 8.6.


Fixed 4xx client errors

• Fixed about 450 internal links with redirects (mostly missing the trailing slash)
• Fixed 50 external redirects where possible
• Eliminated about 90 broken external links

Images

• 45 pages had 1+ image with no alt tags.
• Eliminated a few missing images
• Replaced several unnecessary images with text-based content types (code blocks, quote blocks, lists).

Other

• Show updated dates instead of published dates
• Changed author link titles to correctly describe that they are pointing at the site's about page (what you see when you hover, and probably for screen readers). Probably a non-issue, but I fixed it anyway.
• I removed category and tag link "clouds" from the page footer. I originally added these for my misguided attempt at extreme internal linking. I realized these were useless for users. And now that I've been reading BuSo, I know this was probably bad for SEO (supplementary content).

Not fixing - Page title/description lengths

I have many, many posts with page titles and meta descriptions that are considered "too long" by Screaming Frog.

This would be a massive effort to try to fix (i.e. shorten the titles) immediately. I wanted to know, is there a correlation between title / description length and CTR?

My title lengths and CTR have a very weak correlation (.24). Description length and CTR have no correlation. I'm guessing this is because Google probably tends to rewrite these in SERPs. I'm going to consider this a non-issue. Instead, I'll check title/description quality as I go through and quality check every single page (this will take awhile).

 

[makoloko]

Ok, I wanted to mention one more issue I fixed. This seems important.

I had an above-the-fold ad pushing content below the fold (besides 1-2 sentences) on many pages. I removed it. This is terrible for UX and it's bad for SEO (Google Fred?).

Note: I think this poor ad placement was a big reason why 35% of users were leaving immediately. Interestingly, removing this ad will probably improve RPM. I'll monitor this.

 

[Darth]

Ok, I wanted to mention one more issue I fixed. This seems important.

I had an above-the-fold ad pushing content below the fold (besides 1-2 sentences) on many pages. I removed it. This is terrible for UX and it's bad for SEO (Google Fred?).

Note: I think this poor ad placement was a big reason why 35% of users were leaving immediately. Interestingly, removing this ad will probably improve RPM. I'll monitor this.

Let me guess - Ezoic?

 

[makoloko]

Let me guess - Ezoic?

Yes

 

[Darth]

Yes

Ezoic is less of an ad-tech company and more akin to email spammers. I would check out Adthrive or Mediavine.

 

[JanLG]

Wow. Impressive how much effort you have put into recovering your site!
It is good for me to see that I am apparently not the only freak who goes into full-on optimization mode after being hit by a Google update. @makoloko

This thread has been super helpful and I have been implementing a lot of it.
I also put @Darth 's code snippet into my functions.php, but I am still looking for the code snippet that no-indexes "paged" pages.

I did the site:domain search and the first two sites of the SERP are just pages from my website. No posts, whatsoever. So I no-indexed all pages now via Yoast, but I am neither sure that this is gonna work, nor if this is the right thing to do.

Maybe I will find the legendary code snippet via the search as @Ryuzaki has mentioned. So far I have not been lucky enough to do so.

 

[Ryuzaki]

Maybe I will find the legendary code snippet via the search as @Ryuzaki has mentioned. So far I have not been lucky enough to do so.

It's in this thread: Sub-Pages on Archives / Paged Pages - To Noindex or Not?

Here's the code snippet:

// Set Sub-Pages of Archives to Noindex
function ryu_noindex_paged()  {
    if ( is_paged() ) {
        echo '<meta name="robots" content="noindex,follow" />';
    }
}
add_action('wp_head', 'ryu_noindex_paged');

 

[Roy Cohen]

I've lost 2 sites recently and everyone I showed them to said that they didn't think they should've gotten hit. It's sad, but this is the state of the internet these days.

 

[Bob_the_Builder]

@makoloko How did this work out so far? Do you see any (early) signs of recovery?

 

[makoloko]

@makoloko How did this work out so far? Do you see any (early) signs of recovery?

I haven't seen any signs of traffic recovering yet. I always take a huge hit during the holidays, so that kinda obscures things.

The only thing I've noticed recently was my DR on Ahrefs increasing from 12 to 34. During the same time, my DA on Moz stayed the same (27), so who knows if that means anything.

 

[Zyzz]

I haven't seen any signs of traffic recovering yet. I always take a huge hit during the holidays, so that kinda obscures things.

The only thing I've noticed recently was my DR on Ahrefs increasing from 12 to 34. During the same time, my DA on Moz stayed the same (27), so who knows if that means anything.

Did your competitors also lose traffic?

 

[makoloko]

Did your competitors also lose traffic?

You mean during the holidays? Yes, this niche is highly impacted by holidays.

 

[makoloko]

This is a continuous recovery effort, so here's a periodic update.

Status: No sign of recovery yet. Dropped an additional 15% after December updates.

Continued Efforts: I'm writing new content, improve existing content, learning SEO, and applying what I learned.

Latest SEO Fixes/changes

• Added LinkedIn profile and started sharing (I was only on Twitter before).
• Added social links (dofollow) to the footer.
• Fixed 280 articles that were somehow missing social images (due to a bug probably). This was a grueling brute force effort.
• Removed comment URL field. Two reasons 1) Reduced spam attempts by 95% 2) Eliminated about 20 outbound links from commenters (legit comments, so I originally let them link).
• Removed social share buttons. Note: People share without these, so these were just all downside
• Increase domain registration to 3 years. I read this was a trust signal
• 410 (Gone)'d all of the deleted pages (about 1200) because Google has continued to crawl all these 404'd pages after 4 months. This is a waste of everyone's resources and clogs the GSC Index stats. Furthermore, me telling them "quit wasting your resources on these" is a sign of quality/trust. I don't care what they say about 404/410, getting 404's for 75% of pages has to stink of low quality.

Learning SEO and improving my processes
I'm continuing to learn A LOT about SEO by reading posts here. After reading the Digital Strategy Crash Course, I have a much better understanding of keyword research now. I've refined my processes (monthly, weekly, and daily) and am now using the appropriate tools to make me way more efficient and effective.

I believe this is the final ingredient that I needed to get to the next level. My peak was 170k. One of my competitors is at 700k. This isn't just about recovery anymore. I'm aiming for 1m.

 

[Prentzz]

This is a continuous recovery effort, so here's a periodic update.

Status: No sign of recovery yet. Dropped an additional 15% after December updates.

Continued Efforts: I'm writing new content, improve existing content, learning SEO, and applying what I learned.

Latest SEO Fixes/changes

• Added LinkedIn profile and started sharing (I was only on Twitter before).
• Added social links (dofollow) to the footer.
• Fixed 280 articles that were somehow missing social images (due to a bug probably). This was a grueling brute force effort.
• Removed comment URL field. Two reasons 1) Reduced spam attempts by 95% 2) Eliminated about 20 outbound links from commenters (legit comments, so I originally let them link).
• Removed social share buttons. Note: People share without these, so these were just all downside
• Increase domain registration to 3 years. I read this was a trust signal
• 410 (Gone)'d all of the deleted pages (about 1200) because Google has continued to crawl all these 404'd pages after 4 months. This is a waste of everyone's resources and clogs the GSC Index stats. Furthermore, me telling them "quit wasting your resources on these" is a sign of quality/trust. I don't care what they say about 404/410, getting 404's for 75% of pages has to stink of low quality.

Learning SEO and improving my processes
I'm continuing to learn A LOT about SEO by reading posts here. After reading the Digital Strategy Crash Course, I have a much better understanding of keyword research now. I've refined my processes (monthly, weekly, and daily) and am now using the appropriate tools to make me way more efficient and effective.

I believe this is the final ingredient that I needed to get to the next level. My peak was 170k. One of my competitors is at 700k. This isn't just about recovery anymore. I'm aiming for 1m.

IMO all you've changed the entire time is stuff that almost certainly had no impact.

You had all of these technical SEO problems before, and you ranked well. Large updates pass and along with a ton of other sites, you got hit. That's a pretty good indicator that it wasn't technical SEO issues in the first place, because the updates were not going to directly target something like indexation of tag pages. That's something that's easy for Google to handle day to day, if it cares enough.

The bulk of the rankings changes in updates happen because of a shuffling of weightings put on factors. Yes, there's more to it, especially behind the scenes, but that's essentially the impact that you see in the data.

The content on your pages and the links to the page dictate your ranking more than everything else combined, so I'd just focus on those.

I will say that the "authority" factor has been dialed all the way to 11 for most of 2022. It's insane the amount of parasite spam out there, more than I've ever seen. I'm fairly confident what we talk about as "authority" is almost entirely driven by factors Google pulls from the link graph. So, yeah, look there for issues.

EDIT: Just read you're in the "programming" niche. If this means answering questions like "how to add a column in pandas", this niche is driven by links. The content is complete shit and it makes no difference, since it's basically just scraped from StackOverflow and spun or re-written using AI anyways. If these are the types of queries you're trying to answer, your approach of writing only the right amount of content with no fluff or filler is not going to work. You need to aggressively optimize for keywords which typically means adding more content. Also, you're competing with groups who are using massive PBN networks to rank.

 

[makoloko]

IMO all you've changed the entire time is stuff that almost certainly had no impact.

The content on your pages and the links to the page dictate your ranking more than everything else combined, so I'd just focus on those.

According to the "there's not one smoking gun" theory, everything matters - on-page, off-page, technical SEO. All the problems add up until you hit a tipping point and are then smacked by Panda (estimated 6 month recovery). The solution is to fix everything, including optimizing content (new + updating).

I agree though. Content is the most important thing. If I wasn't fixing content too, this would all be pointless. Since I started this recovery effort, I've written 45 new articles + updated 25. By update, I mean optimizing based on keyword research and fixing the writing quality, readability, and code examples. This may not seem like a lot of updates, but I'm getting the hang of it and I'm way more productive now (i wasn't doing updates or optimizations at all before, so this is all new).

Before: 100% focused on writing new content with no proper keyword research. Basically wildly throwing darts at a wall.

Now: Carefully throwing darts at a target.

The content is complete shit and it makes no difference, since it's basically just scraped from StackOverflow and spun or re-written using AI anyways. I

You need to aggressively optimize for keywords which typically means adding more content.

I love my shitty competition. I'm not worried about the trash. I'm focusing on the next-level competition. They're definitely optimized and have a good content plan, but their content is fluffy. I know my content is better (in terms of helping actual coders). So you're right, it's all about optimizing.

 

[ronaldo]

Awesome job with being proactive.

"Carefully throwing darts at a target" is definitely the way to move forward from here.

I created a brand new site last year that was "carefully targeted" and grew insanely quickly but got hit in September. A lot of pages were getting de-indexed daily so I paused content production.

I knew my technical and on-page was on point, so I did nothing and just let it sit.

It was hard for me to let it sit since I'm the type of person who must be actively doing something but that strategy seems to be paying off since metrics are starting to shoot back up and pages are getting re-indexed.

 

[Prentzz]

According to the "there's not one smoking gun" theory, everything matters - on-page, off-page, technical SEO. All the problems add up until you hit a tipping point and are then smacked by Panda (estimated 6 month recovery). The solution is to fix everything, including optimizing content (new + updating).

Note: This is just my opinion, the majority of SEOs probably disagree.

There is a smoking gun though in many, many cases.

There are two ways you can get impacted negatively by an update. The first is that the weighting of signals changes, which is the most common. In a simplified example, maybe referring domains had a 20% impact and keyword density 10%, now it's 17% and 13% say. So, if your referring domains was strong but your keyword density was poorer, you total score is now worse. The smoking gun is your keyword density. A change to parameters can be tiny, but impact the SERPs enormously, especially in less competitive spaces where sites tend to rank based on a single factor because pages are generally not perfectly optimized to all factors.

The second is what some people are calling MC4, which is real. It's essentially Penguin but for all signals. They look for patterns of manipulation across all signals. So, what's happening is that people are ranking based primarily on a single factor. Let's say that for a given algorithm you're ranking primarily because of referring domains. You just have 10x the referring domains of the competition, and so you score higher than the competition, but solely because of referring domains. That's a clear pattern of manipulation.

What MC4 (or whatever you want to call it) is doing is saying, you're manipulating this single factor and so we're just going to cut your score on that factor by 50% or completely ignore it entirely.

Then, people say "oh no, this is a links problem, I should disavow all those links" but the solution is not to remove the signals that you're doing really well on, it's to improve the other signals so that MC4 doesn't see that signal as manipulation. If you disavow the links, the "penalty" goes away, because you drop your score for that factor. So you could actually go up by disavowing, because it's better to have 20% of your referring domain score than 0%... but it would be way better to have 100% of that score, and also get a higher score on all other signals / factors.

That's the "no smoking gun" theory and how why the "kitchen sink" approach discussed on BuilderSociety by many actually works. But, things like technical SEO are not big factors (for most people, let's just ignore extreme cases here and be realistic), and so if you think you have an MC4 issue you should again focus on content and links and figure out where you are massively over optimized and are triggering the algorithmic MC4 penalty. Then, bring up all the under-optimized factors so that you're so optimized for every factor that the algorithm can no longer say that your page is trying to manipulate a single factor, it really is just that amazing across the board.

With so many people on this forum, it's pretty obvious that they've triggered MC4 with backlinks. I've noticed this especially with anchor text. People saying less than 15% exact match is not extreme... that's very extreme, just look at the natural anchor text profiles for non-optimized but highly ranking pages (ignoring extreme niches / keywords here).

You can test this very easily and a few smart people here have noticed it. Scrape a ton of results and associated Ahrefs data. In a lot of cases, if you're using an anchor text multiple times, the page will not rank for that exact phrase, but will still rank for others. Those pages triggered MC4 for that query. The pages which are using exact match and still rank for that keyword did not trigger MC4.

You'll see that one site can have 5% exact match and trigger MC4. Another page will have 7% and not trigger MC4. Clearly, it's not just the percentage that matters, it's the optimization of anchor text relative to something else. The "something else" is the optimization of other factors. This obfuscation is what leads people to say that you can use a ton of exact match anchors and still win. But they aren't looking at the whole picture.

There's also certain factors which are easier to over optimize than others. Keyword density is nearly impossible to over optimize with real content. Anchor text is very easy to over optimize and trigger algorithmic penalties.

If your traffic loss happens in-line with a widespread update, particularly core updates which I believe are solely adjustments to the strength of factors in the algorithm, the issue is the first case. If your loss occurs outside of an update, it's more likely you triggered an algorithmic penalty similar to MC4.