Question
mod_deflate: Excluding Binary MIME Types to Prevent Double Compression
Ira Cedar
0 reputation · 04 Oct 2024, 21:25 UTC
148.9K views1
Loading…
0 reputation · 04 Oct 2024, 21:25 UTC
Ensure that the Apache mod_deflate module compresses only text‑based responses, preventing corrupted binary assets from being double‑compressed.
While AddOutputFilterByType DEFLATE targets common text MIME types, the list of binary types to exclude is not exhaustive. An incomplete exclusion list may lead to broken images, PDFs, or archives.
It is unclear which MIME types should be explicitly exempted in a given deployment, and how changes to the list affect CPU usage and perceived performance.
mod_deflate exclusion list for a typical web application?Content‑Encoding: gzip?mod_deflate is active?34775 reputation · 05 Oct 2024, 04:56 UTC
Yes, you can prevent double compression by excluding binary MIME types from mod_deflate. The most reliable method is to use SetEnvIfNoCase to set the no-gzip environment variable for known binary types, or to whitelist only text types with AddOutputFilterByType. The latter is simpler and avoids maintaining an exclusion list, but it requires you to list every text MIME type you serve.
mod_deflate compresses any response whose MIME type matches its filter rules, regardless of whether the content is already compressed (e.g., JPEG, PNG, ZIP). If a client sends Accept-Encoding: gzip, Apache will gzip the binary again, wasting CPU and potentially increasing file size or causing corruption in rare cases.
Whitelist text types explicitly, which implicitly excludes all binaries:
apacheAddOutputFilterByType DEFLATE text/html text/plain text/xml text/css application/javascript application/json application/xmlIf you prefer an exclusion list, use SetEnvIfNoCase to mark binary types:
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png|webp|pdf|zip|gz|mp4|mp3|avi)$ no-gzip dont-varyThis matches file extensions, but be aware that extension mapping depends on your AddType directives. For content-type–based matching, use:
SetEnvIfNoCase Content-Type "^(image/|video/|audio/|application/(pdf|zip|gzip))" no-gzip dont-varysudo apachectl graceful or sudo systemctl reload apache2.curl -I -H "Accept-Encoding: gzip" https://example.com/image.png — the response should not include Content-Encoding: gzip.curl -I -H "Accept-Encoding: gzip" https://example.com/style.css — this should include Content-Encoding: gzip.Excluding binaries reduces CPU usage because compression is skipped for large files. Memory overhead is negligible; mod_deflate uses a streaming buffer, so the savings come almost entirely from CPU cycles. For typical web applications, the performance gain is modest but measurable under high traffic.
To give you a precise exclusion list, I need to know which binary MIME types your application actually serves. If you provide a sample of the Content-Type headers from your static assets, I can tailor the configuration accordingly.
Use comments to ask for clarification. Post a solution as an answer.
34,775 reputation · 04 Oct 2024, 22:20 UTC
When implementing exclusions via SetEnvIf, it is important to consider the Vary: Accept-Encoding header. By default, mod_deflate adds this header to tell downstream caches (like CDNs or browser caches) that the response varies based on the client's compression support.
If you use dont-vary in your environment variable settings, you explicitly tell Apache not to send the Vary header for those specific binary assets. This is generally recommended for binary files because their content is identical regardless of the Accept-Encoding request, reducing the number of cache variants stored by intermediate proxies.
To verify this behavior in Apache 2.4.x, check the headers of a binary asset using curl:
curl -I -H "Accept-Encoding: gzip" https://example.com/image.pngEnsure that Content-Encoding: gzip is absent and check if Vary: Accept-Encoding is also omitted when dont-vary is applied.