Preface: Meta Tags Robots Definition
The Robots meta tag ( Meta Tags Robots) is a code that contains instructions on how search engine site pages are monitored. While the Robots.txt file( types of robots meta tags) specifies search engine robots’ access restrictions. More detailed instructions on how to monitor and index pages are provided by the Robots meta tag.
There are two types of Robots meta tags: one is part of the page’s HTML code, and the other is the webserver sending it under HTTP headers (such as the x-robots-tag). The same parameters are used to control this type of meta tags (such as “noindex” and “nofollow”) and their differences in how they relate to search engine monitors.
Meta tags are instructions that help search engine robots index the pages and give them information about them. If these meta tags are viewed by the monitors, it will affect their behavior on that page. The following is an introduction to parameters that search robots understand and follow. These parameters are not case-sensitive, but be aware that some search engines may only understand some of these parameters and their behavior may be slightly different concerning some of these parameters.
Parameters Index Index Controls
- Noindex: tells search engines not to index that page.
- Index tells search engines to index that page. The important point here is that you do not need to include this parameter in all pages, because search engines index the pages by default.
- Follow: With this command, search engines even check pages that are not indexed and follow the links inside. This will also give the page credibility to the links within it.
- Nofollow: This parameter tells search engines not to follow any links on the page and not to give them credit.
- Noimageindex: This parameter tells the viewers not to index the screenshots.
- Note: This parameter is equivalent to the simultaneous use of Noindex and Nofollow on the page.
- Noarchive : With this parameter, the search engines do not display the page link in the SERP (search results page).
- Nocache: This parameter is similar to Nocache, except that it is used by IE and Firefox.
- Nosnippet: This parameter tells search engines not to display page snippets (such as meta description ) on the search results page.
- Noodyp / Noydir: This parameter has been deprecated and was used in the past for not displaying DMOZ site descriptions on the search results page.
- Unavailable_after: This parameter tells search robots that they should not index that page from a specified date.
Types of Robots meta tags
There are two types of Robot meta tags: meta robots tag and x-robots-tag. All of the above parameters apply to both types of tags. Here’s a look at both types of tags.
Meta robots tag
This type of tag is used in the HTML code of a page and is included in the <head> section.
Sample code
[jv_label background_color = "# 09c" font_size = "13" border_radius = "0" text_color = "# ffffff"] <meta name = "robots" content = "[PARAMETER]"> [/ jv_label]
The <meta name = “robots” content = “[PARAMETER]”> command applies to all browser robots, and you can specify your robot by changing the robot’s name. For example, if your target is a Google bot, you can use the following example command:
[jv_label background_color = "# 09c" font_size = "13" border_radius = "0" text_color = "# ffffff"] <meta name = "googlebot" content = "noindex"> [/ jv_label]
If you want to add more than one parameter to this code, you must separate the parameters with a comma. See the following example:
[jv_label background_color = "# 09c" font_size = "13" border_radius = "0" text_color = "# ffffff"] <meta name = "robots" content = "noimageindex," "nofollow," "nosnippet"> [/ jv_label]
This tag tells search engines not to index page images, to follow all inbound links and not display any snippets on the search results page.
If you want to use different commands for each of the search robots, you should write a separate line for each of these instructions and specify the name of each robot.
X-robots tag
The Meta robots tag allows index management at the page level, while the X-robots tag can be part of the site’s HTTP header and can even control parts of the index using it.
The X-robots tag has all the capabilities of a Meta robots tag, but it also offers a much wider range of features. The X-robots tag can handle more complex instructions and can be used at the overall site level and even non-HTML files such as Flash.
To use the X-robots tag, you need to access one of the header.php, .htaccess, or server access file. In these files, you can use the X-robots tag with any parameter. Here are some examples of X-robots tag features:
- Indexing non-HTML content such as flash or video
- Prevent indexing a specific part of a page (such as a specific image or video) instead of not indexing the entire page
- Control the index of pages when you do not have direct access to HTML pages (especially the <head> section) or when the site uses a general header for all pages
- Add rules to index pages (for example, index a user’s profile page after they have 20 comments enabled)
Conclusion: SEO tips on using Robots meta tags
- All metadata can only be viewed by search engines when URLs can be monitored. That is to say, if access to a URL is blocked in the robots.txt file, the meta of that page (whether in HTML code or HTTP headers) will never be viewed and affected by robots.
- In many cases, using the Meta robots tag (Meta Tags Robots )with the “noindex, follow” parameters is the best way to restrict access to pages, and from an SEO standpoint, it’s better to limit that page to the robots.txt file.
- It should be noted that these meta tags are ignored by malicious robots and their use is not a good mechanism to safeguard the pages. If you have content on a page that you don’t want to be seen by the public, instead of using meta tags to access them, use more sophisticated methods such as passwords.
- There is no need to use the Meta robots and X-robots tags on one page at the same time. There is nothing special to use and just overlap between these two tags.
ABOUT US
Working with Digital marketing, SEO services, and website design with a highly experienced team for years, َAvenger IT Next Generation has been able to meet the needs of people in various businesses and help businesses grow. Continuously updating their level of knowledge and exploring different markets has surpassed the pioneers in this field and incorporate successful experiences into their careers.
Avenger IT Next Generation is a website design and development agency and an SEO agency to promote your business, call with us.
