In what cases will HTTP_REFERER be empty

Question

I know it's possible to get an empty HTTP_REFERER. Under what circumstances does this happen? If I get an empty one, does it always mean that the user changed it? Is getting an empty one the same as getting a null one? and under what circumstances do I get that too?

Answer 1

I have found the browser referer implementation to be really inconsistent.

For example, an anchor element with the "download" attribute works as expected in Safari and sends the referer, but in Chrome the referer will be empty or "-" in the web server logs.

<a href="http://foo.com/foo" download="bar">click to download</a>

Is broken in Chrome - no referer sent.

Answer 2

BalusC's list is solid. One additional way this field frequently appears empty is when the user is behind a proxy server. This is similar to being behind a firewall but is slightly different so I wanted to mention it for the sake of completeness.

Answer 3

It will/may be empty when the enduser

• entered the site URL in browser address bar itself.
• visited the site by a browser-maintained bookmark.
• visited the site as first page in the window/tab.
• clicked a link in an external application.
• switched from a https URL to a http URL.
• switched from a https URL to a different https URL.
• has security software installed (antivirus/firewall/etc) which strips the referrer from all requests.
• is behind a proxy which strips the referrer from all requests.
• visited the site programmatically (like, curl) without setting the referrer header (searchbots!).

Answer 4

HTTP_REFERER - sent by the browser, stating the last page the browser viewed!

If you trusting [HTTP_REFERER] for any reason that is important, you should not, since it can be faked easily:

1. Some browsers limit access to not allow HTTP_REFERER to be passed
2. Type a address in the address bar will not pass the HTTP_REFERER
3. open a new browser window will not pass the HTTP_REFERER, because HTTP_REFERER = NULL
4. has some browser addon that blocks it for privacy reasons. Some firewalls and AVs do to.

Try this firefox extension, you'll be able to set any headers you want:

@Master of Celebration:

Firefox:

extensions: refspoof, refontrol, modify headers, no-referer

Completely disable: the option is available in about:config under "network.http.sendRefererHeader" and you want to set this to 0 to disable referer passing.

Google chrome / Chromium:

extensions: noref, spoofy, external noreferrer

Completely disable: Chnage ~/.config/google-chrome/Default/Preferences or ~/.config/chromium/Default/Preferences and set this:

{
   ...
   "enable_referrers": false,
   ...
}

Or simply add --no-referrers to shortcut or in cli:

google-chrome --no-referrers

Opera:

Completely disable: Settings > Preferences > Advanced > Network, and uncheck "Send referrer information"

Spoofing web service:

http://referer.us/

Standalone filtering proxy (spoof any header):

Privoxy

Spoofing http_referer when using wget

‘--referer=url’

Spoofing http_referer when using curl

-e, --referer

Spoofing http_referer wth telnet

telnet www.yoursite.com 80 (press return)
GET /index.html HTTP/1.0 (press return)
Referer: http://www.hah-hah.com (press return)
(press return again)

Answer 5

It will also be empty if the new Referrer Policy standard draft is used to prevent that the referer header is sent to the request origin. Example:

<meta name="referrer" content="none">

Although Chrome and Firefox have already implemented a draft version of the Referrer Policy, you should be careful with it because for example Chrome expects no-referrer instead of none (and I have seen also never somewhere).