FIT 14.6: Web Accelerator with next-generation Image Compression

This year’s last release is a big one. FIT 14.6 introduces our next-generation Image Compression engine.

To find the optimal representation for an image, the Image Compression engine performs a number of optimizations suitable for the input image. For example, lossy compression techniques (WebP/JPEG) may be applied to a photograph for substantial size improvements. Transparent PNGs may benefit from ZorroSVG mapping, or graphics from WebP lossless coding or quantization. The experiments are traffic-aware, so that frequently requested images are taken care of with higher priority.

<image-compression> can be used if Image Scaling is disabled. As a bonus, the handling of Image Compression URLs in HTML (and CSS) documents is much faster, which reduces the footprint of HTML processing in the critical path.

As a BETA feature, we have added an automatic image quality assessment tool. Lossy compression usually achieves the highest byte savings. However, it is impossible to find a compression level (ironically often called “quality”) that balances compression and quality for all images. Different images show different patterns of degradation if the compression level is increased. While some images still look good with JPEG “quality” 30, other images may start showing visible artifacts already at high levels like 90. The common solution is to raise the quality for all images. You can now define the compression level by means of quality: “Compress an image as much as possible, as long it looks like the original”. This can be a real byte-saver! It allows us to apply lossy compression to input images in lossless formats without the risk of quality degradation. As this is beta, we would love to hear your feedback about this!

Another interesting beta feature is the HTML5 compliant parser. It parses HTML documents exactly like your browser, and you can still use all FIT development features like XSLT and XPath.

As this is a major release, the various bug fixes, 3rd party updates and smaller improvements are accompanied by a couple of removals of previously deprecated features. So please pay close attention to the changelog.

The SDK for 14.6 is available from Github.

Refer to the full changelog for a more detailed list of changes.

FIT 14.5.3 & Web Accelerator: Delayed iframes and more

In addition to fixing bugs FIT 14.5.3 introduces new and improved Web Accelerator features.

In many modern projects, iframes are used for social media widgets and user tracking. They usually have no impact on the perceived performance of a web page. But there are also sites that have iframes with heavy-weight media content such as youtube videos. Loading megabytes of videos slows down page loading tremendously, even in iframes.

To ensure that users will only load content they are about to see, we introduce delayed iframes. The delayed loading for iframes works exclusively with visibility based loading: The source of the iframe is going to be loaded when the iframes comes close to or into the browser’s viewport, for example by scrolling the page. Due to the unknown nature of the included content of iframes (small tracking and social media vs large videos), the feature cannot be activated globally. You have to enable it in config.xml and mark the individual iframe element with the ai-delay="true" attribute.

Probably even more exciting is an improvement we made for delayed images. If the dimensions of images are already known (i.e. the image is in the FIT cache) FIT will generate an SVG image with the same intrinsic size as the final image (instead of a transparent 1x1px GIF). Hence, the layout of the page will remain unchanged when the final image is loaded and replaces the placeholder. This saves a lot of browser reflows and avoids “jumping pages”. The SVG placeholder also clears the way for more promising improvements to the delayed image placeholders, so stay tuned for further news on the topic in upcoming releases.

The SDK for 14.5.3 is available from Github.

Refer to the full changelog for a more detailed list of changes.

Sevenval FIT 14.5.2

FIT 14.5.2 is another release bringing mostly bug fixes and improvements for existing features.

Delayed images prioritized to be loaded when they are going to become visible considerably reduce traffic. But if visitors want to print sites, they are basically missing every image that neither is nor has been in the viewport. The new parameter of the loadHQImages function makes the client load all images, whether they are in the viewport or not. And the new event delayedimagesloaded informs you when the client finished doing so. So this combination offers means to prepare the site for printing and lets you know when preparing has finished.

FIT 14.5.1 already addressed noise in the logs. We identified another source for noise, namely overly greedy bots (such as Google spiders) interpreting JavaScript strings starting with a / as relative URLs. So when you write something like <script>var foo = '/bar';</script> into your document, it will not take long until you notice said spiders requesting the resource /bar. This will probably lead to unnecessary 404 responses, inflating the FIT server logs and backend server logs, thus probably hiding real problems from you. FIT 14.5.2 will no longer generate JavaScript strings starting with a /.

Additionaly, fixing issues with the image scaling cache has reduced the CPU usage noticeably.

The SDK for 14.5.2 is available from Github.

Refer to the full changelog for a more detailed list of changes.


FIT 14.5.1 – Web Accelerator vs Bigimage

This month’s release is mostly a “smoothing” release comprising a lot of bug fixes and smaller improvements, as well as 3rd party updates and security patches.

One of the most common errors that we have encountered in our log files was a complaint about source images being too large: … image area exceeds limit of 8 Mpixels. Of course, 8 million are a lot of pixels. But with the proliferation of high resolution cameras (in smartphones) and high density displays (also in smartphones), prepare to encounter large images in Web sites more frequently. However, the worst part of the error message was: Continue with original source. That means that of all things the largest images on your Web site are “too large to optimize”. The client has to load all these big files as they are. Even if those might be the outliers of your image files, a single large image with megabytes of data will outweigh every little byte saved by thoroughly optimizing everything else.

Acknowledging this problem, the Web Accelerator now supports sources images with up to 100 mega pixels. Image Scaling will drastically reduce the file size of large images by allowing optimizations such as format conversion or lossy compression. On our systems, we have seen savings of multiple mega bytes per image. To mitigate the operational costs of processing huge pixel arrays, images with more than 8 mega pixels are processed with an algorithm that is less precise but faster than our usual scaling method. Given so many pixels, there is no noticeable difference.

Don’t let the legendary Bigimage eat your bandwidth! Update today!

Refer to the full changelog for a more detailed list of changes.


FIT 14.5 – Web Accelerator Feature Release

FIT 14.5 is a feature release for the Web Accelerator.

Brotli is a new data compression library that shrinks Web payload better than proven methods such as gzip or deflate. The Web Accelerator now includes brotli as an Apache module. We have seen an improvement of over 10% of the average compression ratio compared to gzip. This means that compressible payloads (almost all formats except images) are transferred to clients in less than a third of its original weight.

Brotli also accounts for much of the benefits of the WOFF2 font format. Those font files load significantly faster than the predecessor format WOFF or the old TTF format. The Web Accelerator beta feature webfont-compression converts WOFF fonts on-the-fly into the WOFF2 format, resulting in 20% or more reduction in payload. As fonts are usually loaded in the critical render path, this is a welcome diet. To keep things simple, the Web Accelerator handles the necessary changes to your CSS font definitions as well!

To further accelerate loading of CSS files, we have improved our style-concat feature to work with any external CSS files. All adjacent <link rel=stylesheet> definitions within the same media query selector are combined into a single request. This not only frees up request slots in HTTP/1 setups, it also allows for more efficient data compression than single requests would have. Furthermore, style-minifying now removes even more redundant code and has improved runtime performance.

In 1999 Microsoft gave birth to the infamous favicon.ico. Of course everyone loves icons, but what made the favicon so annoying was that it was a request the browser conducted itself without the developer referencing it in the page. That makes it so easy to forget. Later, Apple tuned in with its apple-touch-icon.png that some (interestingly non-iOS) clients attempt to load from every single site visited. If your site does not handle those well-known icon URLs, it may result in unnecessary requests and wasted bandwidth. For example, the 404 error document may be much larger than a regular favicon. Many Web sites have developed the habit of sending redirects (to potentially better suited URLs) instead of 404s. In this case, the browser could be redirected to the home page in search for the favicon, resulting in two (or more) requests which may return lots of unnecessary HTML. Our new error handling for missing-icons does not paint an icon for you, but it replaces all non-icon answers for well-known icon URLs with a tiny error document.

To save even more bytes, more clients including Microsoft Internet Explorer and Mozilla Firefox now qualify to use Zorro images as a replacement for transparent PNG images with photo characteristics.

There are many more improvements and bug fixes in this release. And as usual, you will find all third-party libraries freshened up.

Refer to the full changelog for a more detailed list of changes.

FIT 14.4.2 released with new Web Accelerator options

https:// should be considered the norm for Web traffic. It allows for secure communication, the green padlock looks cool, and most important: it enables the efficient and fast data transfers of HTTP/2. However, switching your site to HTTPS may not be trivial. The force-https feature removes the hassle of changing URLs in your backend. But what can you do about URLs pointing to other domains, like ad scripts, social media or user images? Your browser will show mixed content errors and block the loading of these assets. The new force-https option external-media handles this by rewriting foreign http:// URLs to their secure counterpart. Switch to https:// now!

Speaking of security, we have improved our transport mapping feature, which is crucial when using the same domain name for Web Accelerator and source, to validate TLS certificates on mapped traffic. E.g. your certificate is accepted when the requests are routed to the IP resolved for

Besides a lot of bug fixes, this version ships ImageMagick 7 to be up-to-date with the frequent upstream updates, many of which are security related.

Refer to the full changelog for a more detailed list of changes.

FIT 14.4.1

This release is a maintenance release comprising mostly bug fixes, security patches and 3rd party updates. However, there are some interesting improvements, too.

But security comes first! The Web Accelerator uses the ImageMagick library for image scaling. Recently, a number of security vulnerabilities were discovered in ImageMagick. With this release, we address these with a policy.xml as recommended by the vendor, as well as some patches that remove potentially exploitable code that we do not use. This is why we recommend all customers to update to 14.4.1 as soon as possible.

Until now, the cache for scaled or compressed images was valid for one day – regardless of the resource’s actual caching headers. Now these headers are respected, allowing shorter TTLs. However, you may still override this with the interval cache.

With the new scaling option add-size-attributes, the width and height attributes of images are set in the HTML document. This allows browsers to render the page quicker, because no reflows are necessary after the images have been downloaded. For pages with delayed images, this avoids “jumping pages” if image dimensions are missing.

Still in Beta, but nevertheless exciting, we provide an optional new URL scheme. With trailing marks enabled, the URL marks needed for URL optimization (such as caching) are moved to the last path segment. In 1:1 setups where a single origin is mounted onto /, all paths are identical to the origin. In combination with the new force-transparent switch, this allows for transparent cookies with arbitrary path directives.

Refer to the full changelog for a more detailed list of changes.

FIT 14.4: Web Accelerator Update

This release brings a performance update to the “rear side”. When communicating with an origin server, a TCP connection needs to be established. This comprises multiple round trips. For encrypted connections (HTTPS), an additional TLS handshake is necessary to setup parameters for secure communication. Depending on the network latency this adds up to a significant amount of waiting time in the critical path. Furthermore, the throughput of a freshly established TCP connection is limited. Only after a number of network packages have been exchanged, the throughput increases.

With 14.4, we start recycling TCP connections to avoid these performance penalties. This reduces connection overhead and improves request performance. The savings are especially high for TLS backends and (very) remote origins. On our production systems we have seen average speed-ups of up to 20%. But even in low latency environments with plain HTTP there are some milliseconds to scrape. Check your fit_request.log to see what’s in it for you!

Besides that, we have fixed a number of bugs. But in case something goes wrong nevertheless, you can now fail with grace with a custom error page.

Please note that this major release includes deprecations and removals.

In the Beta section we have introduced a page cache for PPL: When navigating back and forth in the browser history, no HTTP request or HTML parsing will be required. Instead, the previous pages’ DOM objects may be reused, resulting in much faster and less interruptive history navigation.

Refer to the full changelog for a more detailed list of changes.

FIT 14.1.5

14.1.5 is available! It is the 11th and last planned release in 2015. The main focus was on improving the image handling of the Web Accelerator.

However, the most interesting feature is certainly the script manager that now supports loading remote scripts, too. According to HTTP Archive, after image media, JavaScript is the second largest chunk of content used in Web pages. At the same time, JS code is often delivered in many small files. FIT combines its caching, minifying and script loading facilities to turn the situation into a strength: All JS files, both from backend servers or local files, will be delivered in a single request. This results in better compression, a reduced HTTP overhead and faster browsing – especially on networks that suffer from higher latency like mobile connections.

For Image Scaling (and compression), we have improved the handling of PNG images. This image type delivers sharp graphics, well suited for logos and glyphs, but they can eat up your bandwidth – especially when the number of colors exceeds the maximum palette size. Our new algorithm considers the number of colors, transparency and client capabilities to decide whether the image is suitable for lossy compression. For graphic-like images, WebP lossless compression is used, if supported. These measures drastically reduce the resulting file size.

These advanced compression techniques may now be used for even more images: The parse action detects image content and then applies image scaling to it. These images are scaled according to the default image settings. As the URLs are the same for every client, the response is not publicly cacheable. Nevertheless, this allows you to reduce image payload, even if your image URLs are created or modified on-the-fly with JavaScript, as many responsive image libraries do.

Speaking of which, the new responsive-image-filtering reduces the HTML size by filtering out all image variants (picture/source or srcset) that a capable client would not use anyway. At the same time, image scaling and compression is applied to these images, too.

In addition to these features, we have improved our HTTP compatibility to properly pass HTML form uploads (multipart/form-data) through FIT, regardless of the field names used. In the other direction, HTML fragment responses can be detected automatically, which is often useful for sites making heavy use of AJAX.

The SDK available from Github is updated to 14.1.5, too.

Refer to the full changelog at our developer site for a more detailed list of changes.

FIT 14.1.4

We are happy to announce the release of FIT 14.1.4. It is a maintenance release comprising bugfixes and a number of enhancements. Many of which aim to provide seamless integration of FIT into existing Web site deployments to make use of the Web Accelerator features.

The SDK available from Github is updated to 14.1.4, too.

Refer to the full changelog at our developer site for a more detailed list of changes.