How to block dynamic urls by robots txt in blogger



URL structure in blogger is restricted feature. It can not be either modified or deleted, specially when it relates to post & page url. Post url has fixed date inclusion in every new article you write. Similarly Pages has pre-fixed /P/ inclusion in every new page you create in blog spot as shown in below examples.

Example:
Post permalink
http://www.freshknowledgecenter.com/2016/02/how-to-remember-lengthy-texts-in-short.html

Page permalink
http://www.freshknowledgecenter.com/p/contact-us.html

According to google's policy these permalink cant be changed by users, Unlike word-press it is default and prefixed.

That is why it becomes biggest issue in terms of SEO. However Unique article with date inclusion can generates good traffic but as far as i know after some time those websites articles without date will win. No matter how good you write or represent.

Read more: Optimize Blogger SEO by Choosing right meta description
Read more: Basic Blogger SEO Tricks And Tips Forever | Best SEO Help For Your Blog/Website

Dynamic and static URL

In simple words auto generated link structure which includes special characters like "?", "#" and "=" called as dynamic url and rest of static.

Example:
Dynamic link
https://www.blogger.com/blogger.g?blogID=1763716379433603602#editor/target=

Static link
http://www.freshknowledgecenter.com/2015/09/how-to-give-h3-tage-for-home-page-title.html

According to google blog post now they recommend that do not rewrite dynamic urls that looks ugly on your blog.

However it is not mandatory but refers to those websites which has selling, trading and other facility services like Online games, tools, tips based on use queries.

Google has now smarter solution for both static ans dynamic links.

Dynamic URL in blogger

As mentioned above there are certain limitations in blogger specially with permalink. You can only modify the post link once while you are writing it first time.


Editable post Permalink in Blogger before published
Editable post Permalink in Blogger before published 


Fixed Link after post published
Fixed Link after post published

Whenever user comments on your blog, or spamming with unusual queries generates Dynamic-links. It will not harm your website but will affects your reputation. Because it will throw 404 error when user click on it.

Once Dynamic url generated it can not be deleted or modified later. So that to improve website reputation and completely block that unusual links we need to configure ROBOTS TXT file.

ROBOTS configuration gives us total control over web-crawlers.

Block Every ugly Urls with ROBOTS TXT

Disallow function blocks every links to be fetched by web crawlers, So by implementing certain rules we can also use it on blogger.

For example if you need some page to be hidden from search engines, following rule will resist it to be fetch by web-crawlers
Disallow: /search
Disallow: /*?*
Disallow: /*=*

Here Disallow /Search will block all links which includes Search, Similarly second and third line will obstruct all link which contains "?" and ''=" characters.


Thus no dynamic urls will ever fetch by search engines and will not occur in any search results. Although wrong configuration will affect your entire website, may cause traffic lost or may ignored by Search engines, so be careful while working with ROBOTS configurations.

You can check and validate it in Google webmaster tools.
Go to Crawl > Robots txt tester

Test your Sitemap and it will gives you over all idea if everything is fine or not.

Result
After successful implementation, you can see the effect on your google webmaster tool's crawl errors section. All the 404 Errors will dramatically decrees. Thus it will also impact on your overall Search engine optimization score and website reputation.


13 comments:
  1. Good information, however this code will block every url which contain = and ? So also mobile sites. We need exclusion for mobile site so that it can not be block by mobile bots!

    ReplyDelete
  2. Thanks for the information shared, quite helpful for us Grocery at Cheap Price Online

    ReplyDelete
  3. I just see the post i am so happy the post of information's.So I have really enjoyed and reading your blogs for these posts.Any way I’ll be subscribing to your feed and I hope you post again soon.

    ccna training in chennai guindy

    ReplyDelete
    Replies
    1. I have read your blog its very attractive and impressive. I like it your blog.

      Email Marketing India Email Marketing Services India

      Delete
  4. Superb i really enjoyed very much with this article here. Really its a amazing article i had ever read. I hope it will help a lot for all. Thank you so much for this amazing posts and please keep update like this excellent article Voice Actor

    ReplyDelete
  5. iOS Training in Chennai
    Android Training in Chennai
    php Training in Chennai

    I wondered upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.

    ReplyDelete
  6. Thank you for sharing such a nice and interesting blog with us. Hope it might be much useful for us. keep on updating...!!
    seo company in india
    digital marketing company in india

    ReplyDelete

  7. I wondered upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.

    Android App Development Company

    ReplyDelete
  8. great and nice blog thanks sharing..I just want to say that all the information you have given here is awesome...Thank you very much for this one.
    web design Company
    web development Company
    web design Company in chennai
    web development Company in chennai
    web design Company in India
    web development Company in India

    ReplyDelete
  9. I wondered upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.
    iOS App Development Company
    iOS App Development Company

    ReplyDelete
  10. I wondered upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.

    Fitness SMS
    Salon SMS
    Investor Relation SMS

    ReplyDelete
  11. it is really amazing...thanks for sharing....provide more useful information...
    Mobile app development company

    ReplyDelete
  12. great and nice blog thanks sharing..I just want to say that all the information you have given here is awesome...Thank you very much for this one.
    web design Company
    web development Company
    web design Company in chennai
    web development Company in chennai
    web design Company in India
    web development Company in India

    ReplyDelete