How a PHP notice revealed a quirk of Norton Internet Security
April 4th, 2006 by Ivo
I had a strange thing just now.
I was working on a website which was working just fine and notice free.
Out of the blue, my website had the following notice:
Notice: Undefined index: HTTP_ACCEPT_ENCODING in /mnt/clusterdata/home/ivo/beta.epointment.nl/atk/ui/class.atkoutput.inc on line 153
I thought 'Que?!', as I have not modified the code in class.atkoutput.inc in weeks, and certainly not tonight. The code in question:
if (atkconfig("output_gzip") &&
phpversion() >= '4.0.4pl1' &&
(strstr($_SERVER["HTTP_USER_AGENT"],'compatible') || strstr($_SERVER["HTTP_USER_AGENT"],'Gecko')) &&
strstr($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip')
)
{
header('Content-Encoding: gzip');
echo $this->gzip($res);
}
else
{
echo $res;
}
This code worked in all browsers, for years, without notices, because the HTTP_ACCEPT_ENCODING header is usually set for most major browsers.
Then I realised that simultaneously I was installing Norton Internet Security on my new laptop (a very cool Toshiba Tecra M4 tablet PC, thank you Ibuildings!). It turned out not to be a coincidence. Apparently, Norton Internet Security puts itself between my browser in the network and intercepts headers. It seems to remove the HTTP_ACCEPT_ENCODING header from any browser request.
That explained the sudden appearance of the notice.
The (undocumented?) side effect is that with Norton Internet Security active, no page will be send gzipped. This is a performance penalty I think. They probably do it to be able to scan the text before it arrives in the browser (unzipping, scanning and rezipping would probably take too much time).
I'll be turning off such features in Norton Internet Security anyway, but it was funny how a PHP notice revealed this.
April 04, 2006 at 11:30 pm, Chris D said:
Are you trying to say that NIS blocks pages that are compressed at the server?
April 04, 2006 at 11:51 pm, Ivo Jansch said:
No, it doesn’t block them. It removes the HTTP_ACCEPT_ENCODING from the outgoing request header. Therefor, a server checking this header to see if it can send zipped content, will notice the absence and send an unzipped page.
I’m not sure what happens if the server forces a zipped page though.
April 05, 2006 at 10:22 am, Gaetano Giunta said:
To have some real fun haed to PORT80 blog (the guys who develop some nifty tools to bring to IIS the capabilities that Apaceh has for free, such as content compression): they dissect all the evildoing in the Norton Internet Security crap – not only it adds, removes and modifies http headers, but gets it completely wrong with respect to RFCs. It goes as far as sending out mangled http headers that crash flash servers, iirc…
April 07, 2006 at 11:21 pm, Jacques Marneweck said:
You would hate to see what their Symantec Firewall thing does to web requests. Eventually I ripped it out and replaced it with a FreeBSD-based firewall which sorted out weird issues with web-browsing, mail, etc.
April 19, 2006 at 4:30 pm, Harrie Verveer said:
There are more of these headers removed by several firewalls and virusscanners.
For example; I host a website on a shared hosting server with very few rights and barely any statistics about my website. I wanted to receive an e-mail when people visited my website with the referrer in it, since I’m interested in where my visitors come from. The idea is simple: check if there is a referrer (and it is not your own domain), and if so send an e-mail. I tried to test it but for some reason $_SERVER['HTTP_REFERER'] was never set! Turned out that norton personal firewall was screwing up my headers as well!
These issues can be very anoying. Good plan to turn of all firewalls and virusscans when you encouter a strange problem with missing headers or something! This can often solve the problem!
April 20, 2006 at 12:11 pm, Gaetano Giunta said:
Note that directly typing the URL in the address bar (and also clicking on a previously saved bookmark) will also cause the browser to not send any referrer header, but it is accepted as the correct behaviour in that case.