Which browsers/plugins block HttpReferer from bein

问题:

I am trying to interpret HttpReferer strings in our server logs. It seems like there is quite a high number of empty values.

I am wondering how many of these empty values are due to direct hits from people entering our URL directly into a browser and how many might be due to some kind of blocking utility that prevents the Referer from being sent.

I really have no idea how many people are using tools or browsers or 'anonymizers' that might block the refer. Any input?

回答1:

I personally disable it using "Web Developer" extension of Firefox, only because of some "helpful" sites that highlight the search terms that I used to get to that page.

Thanks, I am fully capable of installing a highlighter plugin, or search for the words inside your page.

回答2:

I think a large proportion may actually be caused by ISPs' restrictions. I know my ISP (BT, in the UK) filters it out (probably at the router) which is bloody annoying at times.

As it turns out, the block is actually put in place by Zone Alarm, a software firewall, which is often supplied by ISPs.

回答3:

Opera has a quick toggle in the F12 menu that you can switch on "Send Referrer Information" or not to the site(s) you're surfing around.

回答4:

I used to log all this stuff in my blogging app - pretty much all bots never send referrer info.

You should be able to make an educated guess as to whether it's down to it being filtered out or just people entering the URL.

If the first hit has no referrer but the loading of images/CSS etc has referrer info then they just entered the URL directly.
If they only ever pull down HTML with no images or CSS they are most likely a bot (or using Lynx perhaps).
If they pull down HTML, images and CSS with no referrer then it's being filtered out.

回答5:

Some antivirus software is retarded and also started doing this for "security" reasons.

We had an email form that used referrer tracking to eliminate the gist of the random bot-spam an some people moaned that it didn't work.

Not entirely wonderful, but there are far more good uses of the referrer header than for just 'lets be evil and watch where people came from' to legitimise it.

( Some antivirus packages have been known to stop email working altogether for instance, and the clients will ring you and tell you its your fault until you tell them to get rid of their rubbish i've never heard of that company before' antivirus for the 40th time and they listen and their problem magically resolves )

Addendum on security

Referrer tracking is very useful for keeping state within a site. (Without needing cookies)

Referrer tracking is very useful to acknowledge that a users origin was from the site itself ( without needing cookies )

Though I see a legitimate privacy concern with leaking 3rd party sites leaking data via referrer, and the recipient seeing that.

So:

3rd-party => site  # referrer preferred blank
local     => local # referrer preferred kept

At least here you can easily distinguish between a "hotlink" from an external source and an internal link.

Also, because of this, cross-domain referrals from SSL websites are blocked by default by some browsers.