book a call
+1 (347) 418-03-62
Starting a new project?
get advice
17
1

*For updates on 308 redirects and the time to save redirects, see p. 1.1 and 4, respectively.

It is important to configure the correct server responses during the internal optimization of your website. Search engines regularly interact with the website: crawl pages, make changes to the index, rank. The goal of your online business is to provide correct information about the work of the entire resource in order to get more points when ranking.

You need to check what response codes the server is sending and whether page redirects are configured correctly to do this. Since the search robots initially turn to the server, and only after that - to the pages of the website. When the server responses are configured correctly, search engines do not need to re-check the entire website when crawling - it only indexes the changed or new ones. This is how robots save resources.

Page redirect in simple terms - redirecting the search robot and the user to a page that they did not request initially. This often happens when the url of some page was changed, then they set up a redirect using redirects in order for the user to get to the desired one.

Example:

Redirects

Fig. 1 - Sample server response (response contains HTTP status code and Expires header)

Let's figure out what response codes are exist and how to redirect the page.


1. Status codes

If you are transferring a website to the https protocol or to a version of the website without www, or moving to a new domain, then redirects are the first thing you need to set up.

Consider the most important and demanded classes of page state and outline the details of work.


1.1 Redirects 3**

  • 300 Multiple Choice.

The query has several possible answers and the user must choose one of it.
Since there are no standard response codes, this redirect is rarely used.

  • 301 Moved Permanently.

Indicates that the site has permanently changed the address and further website requesting should be redirected to the new URL. The weight of pages from the old one is transferred to the new one.

Redirects 301

Fig. 2 — Example for redirects 301

  • 302 Moved Temporarily, Found.

Code for temporary redirects. Search engines don’t update their links to the website, but the browser redirects users to the new page.

Disadvantages of 302 redirects:

  1. 302 redirect does not transfer page weight;
  2. if the 302 code is used for more than 7 days in a row, the site may fall under the search engine filter (the 302 answer is sometimes regarded by bots as a component of “black SEO”).

As a result, when linking, the website loses the weight not only of the internal pages, but also of the external link mass.

  • 303 See Other.

Similar to code 302, but 303 does not indicate a movement of the requested URL.

A 303 redirect shows that there is no suitable URL for the requested page, but there are several URLs that conditionally satisfy the request.

  • 304 Not Modified.

Indicates that there is no need to resend the requested address if the page has not been modified since the last transfer. The browser will redirect the user to the saved copy of the page.
And if there were changes, then the answer is “200 OK”.

  • 305 Use Proxy.

Forwarding through a proxy server.
The 305 code is used for anonymity, or to speed up page loading (then the content is cached).
But not all browsers handle this response code correctly (for Explorer and Mozilla). Therefore, it is not used often.

  • 306 Switch Proxy.

Used to indicate the use of a specific proxy.
The code is not used now, but remains for reserve.

  • 307 Temporary Redirect.

Used to refine 302 redirects.
Response 307 indicates that the site is available at a different URL, but will return to the previous URL shortly.
Like 302, 307 redirects are used for temporary redirects. The difference is that on a 307 redirect, the page response code ensures that the method and body remain unchanged.

  • 308 Permanent Redirect.

Analog of 301 code, transfers the weight to the page. But with a limitation – it does not allow changing the request method from Post to Get.
308 redirect uses Google Drive – shows the Client that the download of data was interrupted.

Code 301 of the page’s response is believed to convey most of the link weight. Therefore, 301 redirects (permanent redirection to a new address) are used more often.

Upd. 12.07.2021 Google has officially confirmed that a 308 redirect is treated as 301. Literally: “status codes 301 and 308 mean that the page has moved to a new location forever” (source).

How to check if the transfer of a 301 redirect is correct?

  • manually (services Header checker tool, bertal.ru);
  • automatically (Screaming frog seo spider, Netpeak Spider, Serpstat).

Checking redirects can take up to 3 weeks, be prepared for this – search engines must bypass redirected pages and assign weight to new ones.

What you need to consider when affixing:

  • if the site has 302 redirects, and the main URL cannot be set, then replace the 302 response with 301.

The results of site crawling will show most of the errors associated with redirects.


1.2 Internal links and redirects

  • we eliminate all internal redirects (if possible) and replace them with target URLs with a 200 server response code;
  • select the main mirror (with or without www), and from the secondary website to the main one we make a page-by-page 301 redirect;
  • we monitor the absence of cyclic redirects (when the redirection occurs in a closed loop);
  • avoiding redirection chains;
  • all URLs must end with either a slash or no slash.

It should not be that part of the URL with a slash, and part without it. Ensuring consistency.

There must also be a 301st redirect from a URL with a slash to a URL without a slash (or vice versa – depending on the option chosen).

  • set up a 301 redirect if there are multiple slashes inside the URL.

Example: https://site.com/razdel////tovar should redirect to a normal url https://site.com/razdel/tovar

  • URLs with invalid character set are redirected to the normal URL;

Example: https://site.com/razdel12 to https://site.com/razdel

  • URLs with capital letters are redirected to the normal URL.

Example: https://site.com/razdel/ToVar на https://site.com/razdel/tovar

Example URL redirection with uppercase letters

Fig. 3 — Example URL redirection

If it is not possible to correctly determine where to redirect the user, we recommend setting up the processing of such a rule:

Main/block1/blockn/

If there is an error in block n, and you did not find its match with any of the previous points, redirect using the 301st code to the n-1 nesting page up to the Main.


1.3 Details of redirection

Things to consider:

  • when transferring a website to the https version, we monitor the correct transfer of comments: so that comments from the disqus application are either transferred correctly or are absent in the code;
  • we monitor the absence of a redirect to a redirect;
  • redirects are configured only to pages with a status of 200 OK;
  • check the .htaccess file for correctness and consult with the admin;
  • all elements of the website are accurately transferred to the https protocol (experts often forget to translate image URLs).

Recommendation:

  • when redirecting from http to https, it is better to do all internal linking with relative links.

1.4 Codes 404 and 410 (remote page addresses)

What services uses to search for “broken” links?

  1. manually (if the website is small, you can scroll through it all and click all the links on the website, or ask the search engine a command with the operator “site: yoursite.com”);
  2. automatically:
  • Google Search Console (“Crawling – Crawl Errors” tab);
  • Google Tag Manager (through the creation of a separate variable);
  • Yandex.Webmaster (section “Internal links – Broken internal links on the website”);
  • Online broken link checker (free check up to 3000 pages – only HTML documents);
  • Xenu’s link sleuth;
  • Screaming frog (free check up to 500 pages);
  • Broken link checker (WordPress plugin);
  • Check my links (extension for Google Chrome).

Broken links found, what’s next?

404 server response

Fig. 4 — Example 404 server response

1. create a custom 404 permanent page like https://site.com/404;

2. check if she gives the 404th response code accurately (the 200th response is often set up by mistake);

3. we send all 404 pages to it, for example — https://site.com/folfdsfods

4. page https://site.com/404 contents:

  • warning that such a page does not exist;
  • an offer to go to other sections of the website, or use the search.

That is, there should be links to popular sections or the main menu.

5. with the help of crawlers, we identify and eliminate the causes of 404x so-called “broken” links;

6. so that data about missing or error pages is reflected in Google Analytics reports:

  • add the Google Analytics tracking code to the page with the 404 code;;
  • + line before GTM code:

Now the collected data can be viewed in the Content (Top pages, Top landing pages, Top exit pages) section of the Google Analytics report.

Information on “bad” 404 pages will show the volume of “broken” links. So we can safely remove or change them. Moreover, they will have to be removed from the index as well. We do this through the Google and Yandex panels. We formalize it with a 404 error to delete the broken page. The next time it crawls, the robot will execute deletion requests and they will disappear from Search.

  • Soft 404 error

404 Soft appears when the server should respond with a 404 (page does not exist), but responds with 200 OK.
This may be a product card with products that are no longer in production, i.e. the product will definitely not be in stock.

If in this case we register a redirect to a product category in order to transfer weight, then Google, due to the dissimilarity of page parameters, will perceive this as a Soft 404 error (Google defines them as “soft error pages”).

The same problem can occur when batch redirecting a large number of pages to one.
All 404 Soft errors need to be found and fixed to 404.

Where can you find all server response processing errors? In file logs.
This is an additional tool for correcting problems with internal website optimization.

  • 410 Gone

Analogue of 404 error. The response code notifies that the page has been removed and is no longer available. During the subsequent check, the bot will not bypass it and make changes to the index.

Convenience of 410 redirects in notifying search robots that the page has been deleted correctly and can definitely be excluded from the index.


1.5 Access Log and report in Search Console

The text file access.log collects site statistics – all calls to the server of search robots.

What this list of URLs gives:

  • search for low-quality pages that the parser does not see, but which the robot knows about.

In addition, it can be “black SEO” from competitors (generated duplicates to deteriorate promotion).

  • identifying sections of the website that do not receive visits from the robot, which can lead to poor indexing if there are many pages in such a section;
  • understanding the reasons for the absence in the index of a group of pages with low crawler traffic.

2. Server response headers

Headers — exchange commands between the server and the search robot.

Headers contain information about the protocol, encoding, language and other components of the website.


2.1 Last-Modified

Stores information about the date the page was last edited.

Mechanism of action:

  1. if the Last-Modified header was received during the crawl, then the next time this URL is accessed (and if this page is in the browser cache), the robot will ask If-Modified-Since (were there any changes on the page after the date in Last-Modified);
  2. further, the server compares the received time with the time of the last page change;
  3. if the date has not changed, it replies “304 Not Modified”.

Last-Modified header is good for speeding up indexing of multi-page websites.

Simply put, if the website has changed 5 pages out of 100, then the scanner does not need to bypass all 100. The spider will first index pages with the title Last-Modified, and after that – all the others, if it fits within the limited time allotted for crawling 1 website.


2.2 If-Modified-Since

Indicates whether the page has been modified after the date that was received in Last-Modified.

Mechanism:

  • Client receives an If-Modified-Since request;
  • checks the date with the time of the last page change;
  • if they match, the server responds with 304 Not Modified and the page is loaded from the cache;
  • if the time of the last change is different, the answer will be 200 OK.

2.3 Expires

Stores a timestamp after which the server response is considered obsolete.

Expire date is needed to cache the contents of URLs so that these URLs are not loaded again on next calls to the server.

  • page creation date = real time of the last page change.

Or, if the content is static, it is equal to the moment the page was requested and the date the file was created.

  • Expire date of all pages is equal to the current time + 3600 seconds in order to cache static elements for 1 hour.

In total, the server status codes make it clear to search bots how to work with the page, The robot either skips the content by responses and headers (since the page has not changed since its last crawl), or enters the index, or returns to it later.


3. How to set up redirects in different CMS

1. If you are running an Apache web server, the easiest way to set up a redirect is through the .htaccess file.

WordPress

.htaccess file is located in the root folder of the site. To do this, go to the hosting control panel: Files – File manager. Or, if you are using cPanel, then File Manager – Files. Or you can connect via FTP.

For OpenCart, you need to write the code:

RewriteCond %{QUERY_STRING} ^_route_=address_old_page.html$

RewriteRule ^(.*)$ http://your_domain.ua/new_page/? [R=301,L]

Bitrix

RewriteEngine On

RewriteCond %{HTTP_HOST} ^www.itforce.ua$ [NC]

RewriteRule ^(.*)$ http://itforce.ua/$1 [R=301,L]

Joomla

Configuration takes place through the admin panel in the Components – Redirection section.

Here you can also track pages with broken links and immediately set up redirection to the correct pages.

2. The second way – plugins for CMS.

For WordPress:

We go to the admin panel, then Tools – Redirection.

Redirects Adding new

Fig. 5 — Adding new

Editing old WP

Fig. 6 — Editing old

OpenCart

There is a ready-made paid module “301 redirect manager”.

Bitrix

There are several paid modules: Redirect master, Redirect links, Redirector, etc.

Joomla

This CMS has a built-in component “Components – Redirection”.


4. Что же по SEO?

Pages that are promoted should give a 200 response code.
Relocated — 301.
And 404 — all pages with errors.
Each code must match the assigned task. This is how search robots work more efficiently with website pages.

Upd. 26.07.2021. About saving redirects – Twitter post from Google employee Harry Ilsh:

According to SEO, it is advisable to leave redirects for at least a year. This will help ensure that all signals are constantly being transmitted to the new URL. However, even if you decide to delete them after a year, it is advisable for users to keep redirects forever (this is not a year from the moment of creation, but from the moment of their first Google crawl).

Hopefully now you won’t have any problems configuring the answers.

Ask questions in the comments if there is any misunderstanding. Our specialists will be happy to answer it.


Targeted advertising for the aroma product website

Clutch Names ITForce as Top Search Engine Marketing Agency

Targeted advertising for the aroma product website

SERM of search results

Subscribe to our updates
More useful articles and manuals are yet to come Stay up to date!

Вы уже подписаны на нашу рассылку!

Подтвердите свой Email для завершения подписки.

Order
website promotion
More useful articles and manuals are yet to come Stay up to date!

Thank you! Our manager will contact you soon.