Back

Error - The server does not return an HTTP 200 code

The server does not return an HTTP 200 code

An HTTP code is a code returned by a WEB server during a client request (browser or robots). These codes are categorized in family :

  • code 2xx (200): code informing that the exchange went well and that the answer was provided.
  • code 3xx (from 300 to 399) : these are specific codes informing the customer of a redirection, i.e. a change of URL to be followed.
  • code 4xx (from 400 to 499): these are the so-called "client" errors where access to the page is not possible.
    • code 403 : access to the page has been refused by the server : because of authentication needs, anti-bots filters etc...
    • code 404 resource not found, the page the client is trying to access does not exist, it does not recognize the URL
  • code 5xx (from 500 to 599) : these are the so-called "server" errors where the server/application has encountered an internal error and is unable to provide the page.

When the server does not return an HTTP 200 code for the URL entered, our robots are not able to analyze the page since the response has not been validated. Indeed, it is quite normal that a 404 error returns an HTML page, it informs the user that it was not found, but it is not an error page and it is not intended to be parsed.

When the server does not return an HTTP 200 code, it is necessary to :

  • check the URL entered, an unrecognized URL will return an HTTP 404 code.
  • Check that it is not a configuration or code error
  • add an exception on your firewall/proxy for our bots, in most cases on HTTP 403 error. Our robots are identified by their user-agents (they do not have fixed IPs) :
    •  "Mozilla/5.0 (Linux ; Android 6.0.1 ; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible ; Cocolyzebot/1.0; +https://cocolyze.com/bot) for mobile scanning
    • "Mozilla/5.0 (compatible; Cocolyzebot/1.0; https://cocolyze.com/bot)" for computer analysis
  • contact your providers, a 5xx error can come from a database failure, incompatibility with a new module, etc
Back