FIT 14.6: Web Accelerator with next-generation Image Compression

This year’s last release is a big one. FIT 14.6 introduces our next-generation Image Compression engine.

To find the optimal representation for an image, the Image Compression engine performs a number of optimizations suitable for the input image. For example, lossy compression techniques (WebP/JPEG) may be applied to a photograph for substantial size improvements. Transparent PNGs may benefit from ZorroSVG mapping, or graphics from WebP lossless coding or quantization. The experiments are traffic-aware, so that frequently requested images are taken care of with higher priority.

<image-compression> can be used if Image Scaling is disabled. As a bonus, the handling of Image Compression URLs in HTML (and CSS) documents is much faster, which reduces the footprint of HTML processing in the critical path.

As a BETA feature, we have added an automatic image quality assessment tool. Lossy compression usually achieves the highest byte savings. However, it is impossible to find a compression level (ironically often called “quality”) that balances compression and quality for all images. Different images show different patterns of degradation if the compression level is increased. While some images still look good with JPEG “quality” 30, other images may start showing visible artifacts already at high levels like 90. The common solution is to raise the quality for all images. You can now define the compression level by means of quality: “Compress an image as much as possible, as long it looks like the original”. This can be a real byte-saver! It allows us to apply lossy compression to input images in lossless formats without the risk of quality degradation. As this is beta, we would love to hear your feedback about this!

Another interesting beta feature is the HTML5 compliant parser. It parses HTML documents exactly like your browser, and you can still use all FIT development features like XSLT and XPath.

As this is a major release, the various bug fixes, 3rd party updates and smaller improvements are accompanied by a couple of removals of previously deprecated features. So please pay close attention to the changelog.

The SDK for 14.6 is available from Github.

Refer to the full changelog for a more detailed list of changes.

FIT 14.5.3 & Web Accelerator: Delayed iframes and more

In addition to fixing bugs FIT 14.5.3 introduces new and improved Web Accelerator features.

In many modern projects, iframes are used for social media widgets and user tracking. They usually have no impact on the perceived performance of a web page. But there are also sites that have iframes with heavy-weight media content such as youtube videos. Loading megabytes of videos slows down page loading tremendously, even in iframes.

To ensure that users will only load content they are about to see, we introduce delayed iframes. The delayed loading for iframes works exclusively with visibility based loading: The source of the iframe is going to be loaded when the iframes comes close to or into the browser’s viewport, for example by scrolling the page. Due to the unknown nature of the included content of iframes (small tracking and social media vs large videos), the feature cannot be activated globally. You have to enable it in config.xml and mark the individual iframe element with the ai-delay="true" attribute.

Probably even more exciting is an improvement we made for delayed images. If the dimensions of images are already known (i.e. the image is in the FIT cache) FIT will generate an SVG image with the same intrinsic size as the final image (instead of a transparent 1x1px GIF). Hence, the layout of the page will remain unchanged when the final image is loaded and replaces the placeholder. This saves a lot of browser reflows and avoids “jumping pages”. The SVG placeholder also clears the way for more promising improvements to the delayed image placeholders, so stay tuned for further news on the topic in upcoming releases.

The SDK for 14.5.3 is available from Github.

Refer to the full changelog for a more detailed list of changes.

Sevenval FIT 14.5.2

FIT 14.5.2 is another release bringing mostly bug fixes and improvements for existing features.

Delayed images prioritized to be loaded when they are going to become visible considerably reduce traffic. But if visitors want to print sites, they are basically missing every image that neither is nor has been in the viewport. The new parameter of the loadHQImages function makes the client load all images, whether they are in the viewport or not. And the new event delayedimagesloaded informs you when the client finished doing so. So this combination offers means to prepare the site for printing and lets you know when preparing has finished.

FIT 14.5.1 already addressed noise in the logs. We identified another source for noise, namely overly greedy bots (such as Google spiders) interpreting JavaScript strings starting with a / as relative URLs. So when you write something like <script>var foo = '/bar';</script> into your document, it will not take long until you notice said spiders requesting the resource /bar. This will probably lead to unnecessary 404 responses, inflating the FIT server logs and backend server logs, thus probably hiding real problems from you. FIT 14.5.2 will no longer generate JavaScript strings starting with a /.

Additionaly, fixing issues with the image scaling cache has reduced the CPU usage noticeably.

The SDK for 14.5.2 is available from Github.

Refer to the full changelog for a more detailed list of changes.


WirUm4-Vortrag: Web Architektur bei Sevenval

WirUm4 ist unsere Vortragsreihe von Kollegen für Kollegen, die stets nachmittags stattfindet. Zuletzt hat Robert Großer unsere Web Architektur vorgestellt. 

Um unseren Kunden moderne Webtechniken anbieten zu können, betreiben wir bei Sevenval in unseren Rechenzentren eine komplexe Infrastruktur. In seinem WirUm4-Vortrag erläuterte Robert die Komponenten der Web-Architektur und legte besonders Augenmerk auf die verschiedenen Stationen beim Request-Durchlauf, die für Caching eingesetzt werden können. So befindet sich, neben dem auf den FIT Servern eingesetzten FIT HTTP Cache, sowohl vor den FIT Servern ein Cachingserver (Varnnish Cache), als auch hinter den FIT Servern zu den Kundenbackenservern für http-Requests ein Proxy Cache (Squid).

In einer Simulation eines Requests beginnend vom anfragenden Handy bis hin zu dem Kundenbackend erläuterte Robert den Teilnehmern den vollständigen Ablauf inklusive DNS-Auflösung und SSL-Entschlüsselung. Das erworbene Wissen soll vor allem dabei helfen, ungenutzte Caching-Potentiale in Projekte zu erkennen, diese zu nutzen und die Identifizierung aufgetretener Fehler zu beschleunigen.


Ladezeiten sparen mit dns-prefetch, preconnect, preload und HTTP/2 push

Unser Kollege Johannes stellte in einem internen Vortrag Resource Hints und diverse andere Methoden vor, mit denen Ladezeiten von Websites und externen Inhalten bzw. Ressourcen verkürzt und die Performance verbessert werden können. Thematisiert wurden dns-prefetch, preconnect, preload und HTTP/2 push.

Durch explizites Aufrufen von DNS-Prefetching kann die Auflösung von IP-Adressen bereits vor dem Aufruf von Inhalten externer Domains durchgeführt werden. Da die Einbindung externer Domains ihren Aufruf mit sich bringt, wird dadurch möglicherweise das Laden einer Website verlangsamt. Die Verwendung von DNS-Prefetch-Links kann also dabei helfen, die Ladezeit um einige (Mili-)Sekunden zu verbessern.

WPT ohne alles

Ersten Laden ohne Optimierung (Usecase: Font Loading)

wpt dns prefetch

Ladezeiten mit DNS-prefetch (Usecase: Font Loading)

Preconnect hingegen ermöglicht es dem Browser Verbindungen einzurichten, bevor eine HTTP-Anfrage an den Server gesendet wird. Teure Roundtrips, die durch DNS-Lookups, TCP-Handshake und ggfs. TLS-Negotiaton entstehen, können hierdurch aus dem Request-Ablauf eliminiert und Ladezeiten verkürzt werden.

wpt mit preconnect

Ladezeiten mit preconnect (Usecase: Font Loading)

Bei Preload handelt es sich um einen neuen Web-Standard, der mehr Kontrolle bei der Anforderung bestimmter Ressourcen ermöglicht. Einen nützlichen Anwendungsfall stellt unter anderem die frühe Beschaffung kritischer Ressourcen dar, deren Ausführung erst später erwartet wird.

HTTP/2 push erlaubt das Senden von Subressourcen, bevor diese von Client angefordert wurden. Die Zeit, in der das HTML „zusammengebaut“ wird, kann somit überbrückt und die Performance hierdurch verbessert werden. Cache-bare Ressourcen sollten jedoch nicht gepusht werden, da nicht alle Browser den Cache prüfen und den Push ggfs. unterbrechen. Die Verwendung von HTTP/2 push sollte daher nur in Verbindung mit einer sinnvollen Cache-Lösung stattfinden.

FIT 14.5.1 – Web Accelerator vs Bigimage

This month’s release is mostly a “smoothing” release comprising a lot of bug fixes and smaller improvements, as well as 3rd party updates and security patches.

One of the most common errors that we have encountered in our log files was a complaint about source images being too large: … image area exceeds limit of 8 Mpixels. Of course, 8 million are a lot of pixels. But with the proliferation of high resolution cameras (in smartphones) and high density displays (also in smartphones), prepare to encounter large images in Web sites more frequently. However, the worst part of the error message was: Continue with original source. That means that of all things the largest images on your Web site are “too large to optimize”. The client has to load all these big files as they are. Even if those might be the outliers of your image files, a single large image with megabytes of data will outweigh every little byte saved by thoroughly optimizing everything else.

Acknowledging this problem, the Web Accelerator now supports sources images with up to 100 mega pixels. Image Scaling will drastically reduce the file size of large images by allowing optimizations such as format conversion or lossy compression. On our systems, we have seen savings of multiple mega bytes per image. To mitigate the operational costs of processing huge pixel arrays, images with more than 8 mega pixels are processed with an algorithm that is less precise but faster than our usual scaling method. Given so many pixels, there is no noticeable difference.

Don’t let the legendary Bigimage eat your bandwidth! Update today!

Refer to the full changelog for a more detailed list of changes.


New Web Accelerator Tutorials Published

We have just published some new tutorials to get you up and running with the latest features of the Sevenval Web Accelerator (WA). The tutorials look at how to use WA for image optimization, content optimization, and request reduction. If you are new to WA, or if you simply want a general, hands-on overview of its most common features, then these tutorials are for you!

Continue reading

Progressive Web App Dev Summit 2016 in Amsterdam

I had the privilege to attend the Progressive Web App Dev Summit 2016 in Amsterdam. Google invited developers to learn and share experiences about Progressive Web Apps: web sites that comprise a combination of HTTPS, Service Workers, application manifest for features like add to home screen or push notifications and – maybe more as a result from making great web apps rather than being a PWA feature itself – reliable performance.

Google did a great job in choosing top-of-the-line speakers. Continue reading

FIT 14.5 – Web Accelerator Feature Release

FIT 14.5 is a feature release for the Web Accelerator.

Brotli is a new data compression library that shrinks Web payload better than proven methods such as gzip or deflate. The Web Accelerator now includes brotli as an Apache module. We have seen an improvement of over 10% of the average compression ratio compared to gzip. This means that compressible payloads (almost all formats except images) are transferred to clients in less than a third of its original weight.

Brotli also accounts for much of the benefits of the WOFF2 font format. Those font files load significantly faster than the predecessor format WOFF or the old TTF format. The Web Accelerator beta feature webfont-compression converts WOFF fonts on-the-fly into the WOFF2 format, resulting in 20% or more reduction in payload. As fonts are usually loaded in the critical render path, this is a welcome diet. To keep things simple, the Web Accelerator handles the necessary changes to your CSS font definitions as well!

To further accelerate loading of CSS files, we have improved our style-concat feature to work with any external CSS files. All adjacent <link rel=stylesheet> definitions within the same media query selector are combined into a single request. This not only frees up request slots in HTTP/1 setups, it also allows for more efficient data compression than single requests would have. Furthermore, style-minifying now removes even more redundant code and has improved runtime performance.

In 1999 Microsoft gave birth to the infamous favicon.ico. Of course everyone loves icons, but what made the favicon so annoying was that it was a request the browser conducted itself without the developer referencing it in the page. That makes it so easy to forget. Later, Apple tuned in with its apple-touch-icon.png that some (interestingly non-iOS) clients attempt to load from every single site visited. If your site does not handle those well-known icon URLs, it may result in unnecessary requests and wasted bandwidth. For example, the 404 error document may be much larger than a regular favicon. Many Web sites have developed the habit of sending redirects (to potentially better suited URLs) instead of 404s. In this case, the browser could be redirected to the home page in search for the favicon, resulting in two (or more) requests which may return lots of unnecessary HTML. Our new error handling for missing-icons does not paint an icon for you, but it replaces all non-icon answers for well-known icon URLs with a tiny error document.

To save even more bytes, more clients including Microsoft Internet Explorer and Mozilla Firefox now qualify to use Zorro images as a replacement for transparent PNG images with photo characteristics.

There are many more improvements and bug fixes in this release. And as usual, you will find all third-party libraries freshened up.

Refer to the full changelog for a more detailed list of changes.

FIT 14.4.2 released with new Web Accelerator options

https:// should be considered the norm for Web traffic. It allows for secure communication, the green padlock looks cool, and most important: it enables the efficient and fast data transfers of HTTP/2. However, switching your site to HTTPS may not be trivial. The force-https feature removes the hassle of changing URLs in your backend. But what can you do about URLs pointing to other domains, like ad scripts, social media or user images? Your browser will show mixed content errors and block the loading of these assets. The new force-https option external-media handles this by rewriting foreign http:// URLs to their secure counterpart. Switch to https:// now!

Speaking of security, we have improved our transport mapping feature, which is crucial when using the same domain name for Web Accelerator and source, to validate TLS certificates on mapped traffic. E.g. your certificate is accepted when the requests are routed to the IP resolved for

Besides a lot of bug fixes, this version ships ImageMagick 7 to be up-to-date with the frequent upstream updates, many of which are security related.

Refer to the full changelog for a more detailed list of changes.