Testing the weight of fresh content in the travel niche

Nov 5, 2014
For travel content, how much weight is freshness in the search engine ranking algorithm? My boss tasked me with this the other week and I've been pondering how to conduct the test. This post is me working through this task. To do this, we'd need an apples to apples comparison; therefore, simply comparing the rankings of new posts to old posts doesn't work. It doesn't take into account the optimization of competitors for those keywords. Also, we'd need a control group, one that didn't receive fresh content, to account for changes in the SE ranking algorithm. An increase or decrease in rankings for one group might just be a new algorithm update that we're unaware of.

I think I need an ANOVA or a t-test? I remember discussing this in a statistics class for my psychology degree. What we'll have is two groups, a control group and an experiment group. The control group is just measured for its rankings across the time of the experiment. The experiment group will receive new content at time T. The rankings will be observed from before T and after T. The data for before T already exists thanks to Google Search Console. It takes 2 weeks for Google to update our site's rankings since GoogleBot visits caches our blog posts, at the minimum, once every 3 days.

So, questions are, how many blog posts are needed in the control and experiment group to accurately represent the site? We have 22,000 indexed URLs in Google at the moment, 12,000 of those are in English. Google treats each language differently, so I'll just focus on English for this study. So the sample needs to be representative of 12,000 indexed posts.

Finally, how long should the study go for? I'm thinking 3 months.

Lastly.... what statistical function is needed for this study?! I gotta go read up on statistics again or... if you could help me... that'd be great!