Internet Explorer meets the Vary: header

So you've just installed mod_gzip and you're ready to watch you bandwidth usage drop by 50-75%. Problem is, instead of seeing your bandwidth usage go down it actually goes... well.. up. What's going on here? The chances are your problem is caused by yet another obscure bug in Microsoft Internet Explorer. This one prevents the browser caching responses with anything but a mindlessly simple Vary: header.

Let's first have a look at the Vary: header, and what function it is supposed to perform. This header is described in RFC 2616 [1] which describes HTTP 1.1:

A server SHOULD use the Vary header field to inform a cache of what
request-header fields were used to select among multiple
representations of a cacheable response subject to server-driven
negotiation.

For example, Vary: User-Agent might cause a HTTP proxy to cache several different versions of a web page based to be served to different browsers. Vary: Accept-Language can be used to make sure caches return the correct page if your application localises it's response based on the user's language preference. mod_gzip uses Vary: Accept-Encoding so that proxies can cache a compressed and uncompressed version of a page.

So how does IE handle the Vary: header? Well, there is a thread on mod_gzip mailing list [2] from 2002 which goes into the problem quite thoroughly, but in summary:

• IE will cache compressed responses because they must be written to the disk to be decompressed.
• IE will cache uncompressed responses with Vary: User-Agent
• Any other, or additional Vary: values will be treated as Vary: * which prevents the response being cached.
• The defect is 7 years old and exists in IE 4, 5, and 6 (and probably 7).

What a mess.

Unfortunately there isn't a foolproof solution. The best I've come up with is to simply disable the Vary: header. In theory, a user behind a proxy using a browser which doesn't support gzip encoding might be served a compressed file which they cannot handle. We haven't had any complaints so far, and they'd have to be using something like Netscape 3 for this to happen. Even lynx supports gzip encoding!

mod_gzip allows you to disable the Vary: header with:

# don't send vary header, or IE will not cache uncompressed responses
mod_gzip_send_vary            Off

References

[1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.44
[2] http://lists.over.net/pipermail/mod_gzip/2002-December/006826.html

About Roger Keays

Roger Keays is an artist, an engineer, and a student of life. He has no fixed address and has left footprints on 40-something different countries around the world. Roger is addicted to surfing. His other interests are music, psychology, languages, the proper use of semicolons, and finding good food.