FIT 14.5.1 – Web Accelerator vs Bigimage

This month’s release is mostly a “smoothing” release comprising a lot of bug fixes and smaller improvements, as well as 3rd party updates and security patches.

One of the most common errors that we have encountered in our log files was a complaint about source images being too large: … image area exceeds limit of 8 Mpixels. Of course, 8 million are a lot of pixels. But with the proliferation of high resolution cameras (in smartphones) and high density displays (also in smartphones), prepare to encounter large images in Web sites more frequently. However, the worst part of the error message was: Continue with original source. That means that of all things the largest images on your Web site are “too large to optimize”. The client has to load all these big files as they are. Even if those might be the outliers of your image files, a single large image with megabytes of data will outweigh every little byte saved by thoroughly optimizing everything else.

Acknowledging this problem, the Web Accelerator now supports sources images with up to 100 mega pixels. Image Scaling will drastically reduce the file size of large images by allowing optimizations such as format conversion or lossy compression. On our systems, we have seen savings of multiple mega bytes per image. To mitigate the operational costs of processing huge pixel arrays, images with more than 8 mega pixels are processed with an algorithm that is less precise but faster than our usual scaling method. Given so many pixels, there is no noticeable difference.

Don’t let the legendary Bigimage eat your bandwidth! Update today!

Refer to the full changelog for a more detailed list of changes.

 

New Web Accelerator Tutorials Published

We have just published some new tutorials to get you up and running with the latest features of the Sevenval Web Accelerator (WA). The tutorials look at how to use WA for image optimization, content optimization, and request reduction. If you are new to WA, or if you simply want a general, hands-on overview of its most common features, then these tutorials are for you!

Continue reading

Progressive Web App Dev Summit 2016 in Amsterdam

I had the privilege to attend the Progressive Web App Dev Summit 2016 in Amsterdam. Google invited developers to learn and share experiences about Progressive Web Apps: web sites that comprise a combination of HTTPS, Service Workers, application manifest for features like add to home screen or push notifications and – maybe more as a result from making great web apps rather than being a PWA feature itself – reliable performance.

Google did a great job in choosing top-of-the-line speakers. Continue reading

FIT 14.5 – Web Accelerator Feature Release

FIT 14.5 is a feature release for the Web Accelerator.

Brotli is a new data compression library that shrinks Web payload better than proven methods such as gzip or deflate. The Web Accelerator now includes brotli as an Apache module. We have seen an improvement of over 10% of the average compression ratio compared to gzip. This means that compressible payloads (almost all formats except images) are transferred to clients in less than a third of its original weight.

Brotli also accounts for much of the benefits of the WOFF2 font format. Those font files load significantly faster than the predecessor format WOFF or the old TTF format. The Web Accelerator beta feature webfont-compression converts WOFF fonts on-the-fly into the WOFF2 format, resulting in 20% or more reduction in payload. As fonts are usually loaded in the critical render path, this is a welcome diet. To keep things simple, the Web Accelerator handles the necessary changes to your CSS font definitions as well!

To further accelerate loading of CSS files, we have improved our style-concat feature to work with any external CSS files. All adjacent <link rel=stylesheet> definitions within the same media query selector are combined into a single request. This not only frees up request slots in HTTP/1 setups, it also allows for more efficient data compression than single requests would have. Furthermore, style-minifying now removes even more redundant code and has improved runtime performance.

In 1999 Microsoft gave birth to the infamous favicon.ico. Of course everyone loves icons, but what made the favicon so annoying was that it was a request the browser conducted itself without the developer referencing it in the page. That makes it so easy to forget. Later, Apple tuned in with its apple-touch-icon.png that some (interestingly non-iOS) clients attempt to load from every single site visited. If your site does not handle those well-known icon URLs, it may result in unnecessary requests and wasted bandwidth. For example, the 404 error document may be much larger than a regular favicon. Many Web sites have developed the habit of sending redirects (to potentially better suited URLs) instead of 404s. In this case, the browser could be redirected to the home page in search for the favicon, resulting in two (or more) requests which may return lots of unnecessary HTML. Our new error handling for missing-icons does not paint an icon for you, but it replaces all non-icon answers for well-known icon URLs with a tiny error document.

To save even more bytes, more clients including Microsoft Internet Explorer and Mozilla Firefox now qualify to use Zorro images as a replacement for transparent PNG images with photo characteristics.

There are many more improvements and bug fixes in this release. And as usual, you will find all third-party libraries freshened up.

Refer to the full changelog for a more detailed list of changes.

FIT 14.4.2 released with new Web Accelerator options

https:// should be considered the norm for Web traffic. It allows for secure communication, the green padlock looks cool, and most important: it enables the efficient and fast data transfers of HTTP/2. However, switching your site to HTTPS may not be trivial. The force-https feature removes the hassle of changing URLs in your backend. But what can you do about URLs pointing to other domains, like ad scripts, social media or user images? Your browser will show mixed content errors and block the loading of these assets. The new force-https option external-media handles this by rewriting foreign http:// URLs to their secure counterpart. Switch to https:// now!

Speaking of security, we have improved our transport mapping feature, which is crucial when using the same domain name for Web Accelerator and source, to validate TLS certificates on mapped traffic. E.g. your example.com certificate is accepted when the requests are routed to the IP resolved for origin.example.com.

Besides a lot of bug fixes, this version ships ImageMagick 7 to be up-to-date with the frequent upstream updates, many of which are security related.

Refer to the full changelog for a more detailed list of changes.

FIT 14.4.1

This release is a maintenance release comprising mostly bug fixes, security patches and 3rd party updates. However, there are some interesting improvements, too.

But security comes first! The Web Accelerator uses the ImageMagick library for image scaling. Recently, a number of security vulnerabilities were discovered in ImageMagick. With this release, we address these with a policy.xml as recommended by the vendor, as well as some patches that remove potentially exploitable code that we do not use. This is why we recommend all customers to update to 14.4.1 as soon as possible.

Until now, the cache for scaled or compressed images was valid for one day – regardless of the resource’s actual caching headers. Now these headers are respected, allowing shorter TTLs. However, you may still override this with the interval cache.

With the new scaling option add-size-attributes, the width and height attributes of images are set in the HTML document. This allows browsers to render the page quicker, because no reflows are necessary after the images have been downloaded. For pages with delayed images, this avoids “jumping pages” if image dimensions are missing.

Still in Beta, but nevertheless exciting, we provide an optional new URL scheme. With trailing marks enabled, the URL marks needed for URL optimization (such as caching) are moved to the last path segment. In 1:1 setups where a single origin is mounted onto /, all paths are identical to the origin. In combination with the new force-transparent switch, this allows for transparent cookies with arbitrary path directives.

Refer to the full changelog for a more detailed list of changes.

FIT 14.4: Web Accelerator Update

This release brings a performance update to the “rear side”. When communicating with an origin server, a TCP connection needs to be established. This comprises multiple round trips. For encrypted connections (HTTPS), an additional TLS handshake is necessary to setup parameters for secure communication. Depending on the network latency this adds up to a significant amount of waiting time in the critical path. Furthermore, the throughput of a freshly established TCP connection is limited. Only after a number of network packages have been exchanged, the throughput increases.

With 14.4, we start recycling TCP connections to avoid these performance penalties. This reduces connection overhead and improves request performance. The savings are especially high for TLS backends and (very) remote origins. On our production systems we have seen average speed-ups of up to 20%. But even in low latency environments with plain HTTP there are some milliseconds to scrape. Check your fit_request.log to see what’s in it for you!

Besides that, we have fixed a number of bugs. But in case something goes wrong nevertheless, you can now fail with grace with a custom error page.

Please note that this major release includes deprecations and removals.

In the Beta section we have introduced a page cache for PPL: When navigating back and forth in the browser history, no HTTP request or HTML parsing will be required. Instead, the previous pages’ DOM objects may be reused, resulting in much faster and less interruptive history navigation.

Refer to the full changelog for a more detailed list of changes.

FIT 14.3.1: Web Accelerator and RDL Update

Today we have released version 14.3.1 of our FIT software, including updates for the Web Accelerator and Rapid Development Layer

We have learned a lot from analyzing our image traffic. Therefore, this minor release focusses on bug fixes and enhancements of image processing.

Many images are small enough, in terms of dimensions, to fit into current clients’ viewports. That is because many mobile devices have high density displays (e.g. Apple’s retina display) resulting in lots of pixels. Those images don’t need to be scaled. Therefore, Image Scaling and Compression are now separately configurable. This can be useful for CSS documents with image sprites that should not be scaled but still be compressed.

If an image is known to be small enough for the requesting client, FIT will automatically switch from Image Scaling to Image Compression. The latter uses image URLs without sizing parameters, resulting in a better cache hit rate (in FIT and in downstream caches) and reduced disk and CPU usage.

Another finding in our image statistics is that in certain cases image weight (i.e. file size) may be increased by our image processing. Now, the smaller original image will be sent to the client instead of the processed image. We call that pass-on-enlargement. Since animated GIF images are hard to optimize and are practically never scaled, they are no longer processed at all.

We have further improved SVG as a replacement for transparent PNGs (also known as “zorro”). The alpha map is smaller. And a bug showing artifacts at image borders was fixed.

Developers will find some interesting enhancements, too. The content DC properties for example let you run Flow actions depending on the type of content (e.g. html or js). The detection page can now be disabled on a per-URL basis.

The SDK for 14.3.1 is available from Github.

Refer to the full changelog for a more detailed list of changes.

Mo’ Speed – 25% faster Web Accelerator in FIT 14.3

This update brings a significant performance boost to the server-side! The overall CPU usage is reduced by 25% on average. Depending on the features in use, sites may see even larger savings in CPU time. This is not only a benefit in terms of hardware savings. Since the responses are generated more quickly, resulting in shorter times-to-first-byte, it can be a major improvement to your site’s critical render path. These performance gains result from the update of the underlying PHP engine to the much acclaimed version PHP7.

The Web Accelerator was improved, too. As images are a very important part of the user experience, support for WebP and SVG images is now determined automatically by the client feature detection. Before 14.3, these capabilities were taken from our Client Description Repository. With the new on-the-fly detection we have more accurate information when dealing with new, unknown clients. FIT now uses URL encoding instead of base64 when inlining SVG images, which results in more efficient transfer compression and a reduced payload for the client.

The documentation now has a section on beta features. It offers an overview of experimental features that we make available for pre-production use. Any developer feedback on these features is highly appreciated!

With this major release, we introduce support for OpenSUSE Leap 42.1. We have also removed some formerly deprecated features. Please note that Session Management and the Load Brake have been removed. Support for

The SDK for 14.3 is available from Github.

Refer to the full changelog at our developer site for a more detailed list of changes.

 

FIT 14.2 – Reduce Image Weight, Raise Quality

The first release of 2016, FIT 14.2 primarily brings a set of new features regarding image optimization.

First and foremost we have implemented a new algorithm that allows us to reduce the transport size of truecolor PNGs with transparency by up to 80%. This is especially interesting for relevant browsers that do not support WebP images (looking at you, Apple Safari). While we were inspired by Mario Klingemann’s Zorro SVG we were able to take the idea several steps ahead and not only resolve issues that it raised, but also improve methods further and in consequence produce even smaller result images without the need of JavaScript.

FIT now also uses 16-bit algorithms for internal image calculation and a better gamma correction. Better handling for truecolor PNGs that are transcoded to WebP format was implemented and developers now may use ai-scaling-width/height values that will scale images bigger than the target viewport. Additionally bringing new scroll speed dependent loading of visibility delayed images and options to control the viewport offset for loading these, FIT 14.2 arguably offers the best automated image optimization we ever had.

For assuring high quality in development and maintenance of projects we introduced an XPath function and a fit.ini setting. The watch() XPath function allows you to watch XPath expressions and in case of errors will create entries in the fit_alert.log. In combination with the new fit.ini option FIT_DEPRECATION_EXCEPTION, which will cause exceptions to be thrown when deprecated features are used, you can minimize the chance that your projects will run into future issues.

We also did a rework of our user agent detection. FIT now uses the common UA Parser project for client matching. It provides a robust set of regular expressions that are less sensitive to the frequent version updates (of both operating systems and browsers) seen in user agent strings. However, we still maintain our own Client Description Repository that stores client capabilities, features and properties. After identifying the client, our CDR data is accessed to build up the Delivery Context.

Updates of third party software and bugfixes complete this release.

The SDK available from Github is updated to 14.2, too.

Refer to the full changelog at our developer site for a more detailed list of changes.