How to add custom robot.txt file in blogger Blog

The Current Interface by Blogger is more SEO friendly. Yes, it has many Search Engine friendly features and among them robot.txt is one of the new SEO friendly feature where your file can be customized with this according to your choice. Really this robot.txt plays a vital role in the search engine optimization process of your blog. In this post let us discuss about how to add custom robot.txt file in Blogspot Blog.

Add robot.txt file in Blogger BlogAdd robot.txt file to blogspot blog

Add custom robot.txt file in Blogger BlogAdd robot.txt file to blogspot blog

Add robot.txt file to blogspot blog

Add robot.txt file to blogspot blogWhat is meant by Robot.txt?

Actually Robot.txt is simple text file with simple codes which is used by the owner of the website to write specific commands for search engine’s spiders or crawlers. These files instruct the crawlers about their limitation to index a particular page. Let me throw some extra light on this , remember whenever the web crawlers crawl your website before indexing your web pages they first search for robot.txt file to check for the exclusions protocol. Hence Search Engine Crawlers get an idea of which webpage to index and which page not to Index. This is a great tool to increase the ranking of your blog and also it helps you to get more traffic provided if it is used properly. Let us dig deep into this and learn about how to add custom robot.txt file in Blogspot blog.

 

Add custom robot.txt file in blogger Blog

 

1)      Go to Blogger.com

2)      Login To your Account

3)      Select the blog if you have multiple blog otherwise ignore this step

4)      Now look for settings>> Search Preferences

How To ADD Custom Robot.txt File in Blogger Blog

 

5)     If you click search preferences mentioned above you will get the below display

How To Add Robot.txt File in Blogger Blog

If you click edit option marked above you will get the below display. You can see a text area which you can use for content exclusion by web crawlers  or  spiders.

How to Add Robot.txt file in Blogger BlogNow enter the following code in the above box

User-agent: Mediapartners-Google

Disallow:

User-agent: *
Disallow: /search
Allow: /
Sitemap: http://yourblogname.com/feeds/posts/default?orderby=UPDATED

NOTE: Replace “yourblogname.com” with the name of your blog.

You are now done with adding custom robot.txt file to your Blogspot Blog

Explanation of each code

Now you will wonder to see there are many terms such as User-agent, Disallow, Disallow:/search,Allow, Sitemap so if you were not aware of these terms then you check my explanation to each code.

User-agent: Mediapartners-Google

This code is for Google Adsense.The Adsense bots will crawl this so that they can serve better ads for your blog.So this code is only for the use of goole Adsense.If you are not using google Adsense then you can simply ignore this step

User-agent:*

The User agent marked with asterisk allows bots or crawlers of all search engine it may be either yahool or bing etc., to crawl your blog.

Disallow:
By adding this you are instructing crawlers not to index your web pages

Disallow:/search

This means you are not not allowing the search results of your blog by default.That is you are not allowing the crawlers to crawl the search directory which comes very next to your domain name.It will look like

http://yoursite.blogspot.com/search/label/yourlabel

This page will not be crawled abd never be indexed

Allow

As it name states allow this will allow crawlers to crawl the home page of your blog

Sitemap

This code refers the sitemap of our blog.This helps the crawlers to index all webpages which can be accessed.Web crawlers will always find a path to sitemap because it makes their job easier as sitemap have all the links of our published posts.

You can add the xml sitemap upto first 500 posts only

Sitemap: http://example.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500

If you have more than 500 posts say 1000 then you can add two sitemap as below

Sitemap: http://example.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500

Sitemap: http://example.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=1000

Hope you liked the article . Kindly support the page by linking it and sharing it in facebook and google plus.

Get Free UPdates Delivered TO Your Email

Enter your email address:

Comments

  1. test@gmail.com'
    priya

    Reply

    • Reply

  2. Reply

    • Reply

  3. Reply

  4. Reply

Leave a Reply

Your email address will not be published. Required fields are marked *