This post doesn’t relate to the Khmer language or books, but recently, when we launched our new site we accidentally used our beta site robots.txt file on our live site! Our beta site had a robots.txt with the following code:
User-agent: * Disallow: /
This was so that no search engine would add our beta site to its index. But when used on our live site, it had devastating results! Our daily average of search impressions from Google was around 2,000 before, but then it went all the way down to only 177 impressions! It basically killed our SEO and rank on Google searches (but there is good news as you see in the image below!).
When we noticed the decrease in traffic we first thought there was a design flaw in our new site code, but after farther testing we found the mistakenly used robots.txt
We changed robots.txt to this (because we use WordPress):
User-agent: * Disallow: /feed/ Disallow: /trackback/ Disallow: /wp-admin/ Disallow: /wp-content/ Disallow: /wp-includes/ Disallow: /xmlrpc.php Disallow: /wp- Allow: /wp-content/uploads/ Sitemap: https://sbbic.org/sitemap_index.xml
Be sure that the “Search Engine Visibility” option is also UNCHECKED in your WordPress -> Settings -> Reading page.
Then sign in to your Google Webmaster Tools Account (if you don’t have one, sign up, it’s free) and re-submit a sitemap (we use the WordPress SEO by Yoast plugin to create our sitemap automatically), and then for faster results, re-crawl your most popular pages by using the “Fetch as Google” tool:
After you fetch a page, click on the “Submit to Index” button. Google will then crawl the page almost instantly, and you should start seeing your pages re-appear in the Google index and your site traffic should begin to increase (our pages started re-appearing in searches in a few hours, but the time might vary).
Within a few days your site should almost recover to its original rank in Google searches – but it still might take up to two weeks for all the damage to be reversed (and in some cases it can take longer than that).
Well, at least this worked for us! We hope you find this useful if you ever make the same mistake as we did.
2 Comments. Leave new
Leave a Reply Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
- Sophat on SBBIC Khmer Unicode Keyboard for Mac OS X
- Nathan Wells on Free English to Khmer and Chuon Nath Dictionary Download
- Sopanha on Download Every Known Khmer Font All At Once
- Vanneth on Khmer Grammar
- Hok on Download All Khmer Unicode Fonts
I am very happy to see this post. My site http://www.newbesttop10.com also meet the same problem and now i edit robots.txt file and resubmit sitemap but my rank and search is decrease everyday. Could you advice me some and please check my site to see what is errors.
Thank in advance!
Glad the post was helpful. Your robots.txt looks ok. There are a few tools you might find useful: http://www.feedthebot.com/tools/spider/
Also make sure you are using Webmaster Tools by Google (which I assume you are since you submitted a sitemap). It can take some time to get back in the Google index (sometimes 2 weeks). In the mean time try optimizing your site with the tips from feedthebot.com and see if that helps.