Don't get me wrong I love the Internet Archive and really depend on it for the work I do. But I recently found out that maybe my love for it needs to be slightly limited. I own 6 sites that I bought in 1998 all the same serp. Most of these sites at one time or another were ranked on page one of my serp for long periods of time since 1998. But in the past two years or so all 6 have mostly been listed on the very last page of my serp. I have made many many changes at these 6 sites over the years to compensate for google and yahoo algo changes to keep my sites ranked high. The changes I made mostly worked up until about two years ago. Then I got this crazy idea that maybe all the different versions and changes of my sites listed at the internet archive might be effecting my sites ranking in google and causing them to all do very badly. So I added a robots.txt file at all 6 sites blocking the internet archive last week. Since doing that all 6 sites listed on the last page of my s
erp have made an enormous jump upwards in ranking and are now between pages 17-25 of my serp in just one week. So I think I have hit upon something or it's the greates coinicidense going? Anyway thought I'd share this one with you if anyone also wants to try it. Here's what you need for the robots.txt file to block the internet archive from archiving your sites:
User-agent: ia_archiver
Disallow: /
ps-if you have made no changes to your sites over the years or hardly any changes then maybe blocking the IA won't make any difference. But if you've made many changes as I have then it's worth the try!
|
|