URL, URL Redirection Optimization and Robot.Txt

URL is a combination of protocol, company name and extension.

A web address is also called Uniform Resource Locator and defined by Tim Berners Lee in the year 1994.

Example of Uniform Resource Locator:

What is a URL?

URL is a unique web address to identify each page on website.

HTTP:// – Hyper Text Transfer Protocol

In simple terms HTTP is an In-secured version, where communication between website and browser are not encrypted.

HTTPS:// – Hyper Text Transfer Protocol Secure

In this communication between website and browser are encrypted as it is a secured version.

HTTPS improves ranking in search engine result page as it assures safety to use website.

WWW-World Wide Web

WWW is a system to create interconnected documents with interlinking the documents and other web sources by hypertext links.

Usage of WWW is an optional.

Domain Name

Domain Name is combination of Company Name or Business Name and the services you offer.

Further, Extension can be classified in two types

1. Business Type–

.Com, .gov, .net,  .edu, .tv, .mobi

2. Country–

.in, .jp, .ae, .co.uk, .co.in

After choosing the domain, the next important step is Uniform Resource Locator optimization.

URL Optimization:

It is important because the keywords in the URL will help to increase the rank in search engine result page. So you have to create the perfect Uniform Resource Locator for each page by including the target keyword.

You might have seen two types of Uniform Resource Locator

• http://everestvision.com/sample-post/
• http://everestvision.com/?p=123

1. First URL is with simple words and it is a SEO friendly Uniform Resource Locator.
2. Second URL is complex URL as it with some parameters. This Uniform Resource Locator is confusing and unfriendly.

It is necessary to describe Uniform Resource Locator as SEO friendly, because it will be easy to understand for search engines and users.

URL would provide a clear idea of what will be the content of the page is about and chances are more for users to click the URL.

URL as SEO friendly also helps for better crawling of your page by search engine.

As we have already discussed URL act as a one of the ranking factor if we include the keyword. We should include the keyword to determine a particular page, don’t create unusual URL simply by including irrelevant keyword.

Rules and Regulation we need to follow for Optimizing the URL.

• Use the target keyword in the URL
• Create the URL base on hierarchy of Site
• Search Engine identifies hyphen as space, so separate the URL with hyphen “-“.
• Keep URL as simple, accurate and most importantly keyword should be relevant of that particular page.
• Don’t use Space, Underscore or any other Character to separate words.
• Don’t use capital letters in URL.
• The best practice is to use only words as it is understandable by search engine, more importantly users should understand.
• Avoid using parameters in the URL
• In WordPress we need to enable permalinks to create simple and SEO friendly URL.

URL Redirection:

Uniform Resource Locator direction is also known as Uniform Resource Locator forwarding, the purpose of this URL redirection is to send a user or search engine to a correct or preferred page.

It means when users open a URL that has been redirected, it will be forwarded to a different URL page.

In a same way domain redirection works when you redirect domain URL, all pages in domain URL will be forwarded to a different domain from the one they originally requested.

Example of redirecting Domain URL:

Suppose if, Everestvision.net and everestvision.org domain URL redirected to everestvision.com.

So in the above case, if users try to open a domain or page with everestvision.net or everestvision.org. Users will be redirected to everestvision.com domain or everestvision.com particular page.

Types of Redirection URL:

301- Moved Permanently and recommended for SEO.

302-Moved Temporarily

URL direction is to prevent broken links and also to allow multiple domain owner to redirect to a single website. So in a simple way URL will fetch content information easily and URL redirection is best practices to maintain SEO value.

URL Redirection in WordPress:

In WordPress install redirection plugin to redirect Uniform Resource Locator page to preferred page Uniform Resource Locator.

Conclusion:

These are the steps we need to follow for URL and URL redirection optimization.

Robots.txt:

Robots.txt is a text file use to give instruction for search engine crawler to crawl the web pages.

By default search engine will crawl all the pages in the website.

So to avoid some of the pages to stop crawling from search engine we need to use robot.txt file.

Robot.txt file indicates the crawler which are the page accessible to crawl and which are the pages not accessible to avoid crawling.

Basic format of Robot.txt:

1)

User-Agent: *

Disallow:

So in the above format, we instruct all the robots can crawl all the pages on the website. We can accomplish same results, if we are not using robot.txt file.

2)

User-Agent: *

Disallow: /

In the above robot.txt file we are indicating all crawlers to stay out from Crawling the website.

3)

User-Agent: *

Disallow: /Privacy/

Disallow: /Junk/

So in the above robot.txt file format, we are indicating all the Crawlers to crawl the web pages except Privacy and Junk Pages.

4)

User-Agent: Googlebot

Disallow: /Privacy/

So in this above robot.txt file format we are indicating only Google Crawler to Crawl all the pages except Privacy pages.

5)

User-Agent: Bingbot

Disallow: /Privacy/

In this robot.txt file format, we are giving instructions only for Bing Crawler to Crawl all the pages except above file i.e. Privacy file.