Crawled but not included

Joined
Mar 27, 2015
Messages
824
Likes
1,469
Degree
3
So I was looking through search console and I noticed a ton of urls that are classed as crawled but not included.

When I investigated they were like this -

mysite.com/realpostname/feed

I am glad they are not indexed as the real post is already indexed.

However I then thought - am I wasting crawl budget on these urls that are meaningless? If yes how would I stop Googlebot crawling them to spend more time crawling my real content?
 
That looks like some kind of RSS feed for comments on each post.

In the case that you're using a pre-built theme, you'll either want to create a child theme and edit the comments.php to add rel="nofollow" to links to RSS feeds or reply links/buttons.

Or preferably you'd find a function that will add that for you through a filter.
 
Will doing either of the above prevent articles from being syndicated via RSS? I have the same type of thing with the feed urls showing crawled but not indexed. I'd like to not have those crawled, but I do want my articles syndicated via RSS.
 
I don't think so. It's just the links that are removed. RSS takes the data from example.com/feed
 
Back