Brotli Compression in .NET Standard

In 2015, Google introduced a new compression algorithm for the internet named Brotli, which was an upgrade to their own Zopfli compression algorithm released in 2013. The specification of the Brotli Compressed Data Format is defined in RFC 7932. Brotli is already supported by most browsers such as Google Chrome, Mozilla Firefox, Opera, and Microsoft Edge. In fact, according to caniuse.com, 66.33% of visitors to this blog already support Brotli.

Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression.

In layman's terms, the smaller compressed size allows for better space utilization and faster page loads.

Unlike most general purpose compression algorithms, Brotli uses a predefined 120 kilobyte dictionary, in addition to the dynamically populated ("sliding window") dictionary. The pre-defined dictionary contains over 13000 common words, phrases and other substrings derived from a large corpus of text and HTML documents. Using a predefined dictionary has been shown to increase compression where a file mostly contains commonly-used words.

If you're interested in seeing in-depth results of Brotli performance, CloudFlare has a splendid blog post on their results of experimenting with Brotli for dynamic web content.

Implementing Brotli Compression in a .NET project

I've already written about gzip compression in ASP.NET Core, but on the 27th July 2017, Microsoft released an alpha-quality preview of System.IO.Compression.Brotli. Before this, ASP.NET had two compression methods available: Deflate or gzip; the trade-off being compression time and size reduction.

Brotli compression is available for .NET Standard 1.4, so either a .NET Framework or a .NET Core application will work.

Since, at time of writing, this is still an alpha release, the NuGet Package is found in Microsoft's MyGet Feed, so we should register that first. As a best practice, you shouldn't register this feed globally as it might bring down other preview packages in your production projects!

Instead, register this feed only for the projects you want to use preview bits in. You can do that by creating a file named NuGet.config and putting it next to your solution file:

<configuration>
    <packageSources>
        <add key="dotnet.myget.org dotnet-corefxlab" value="https://dotnet.myget.org/F/dotnet-corefxlab/" />
    </packageSources>
</configuration>

Once we have this included, we can go ahead and install the pre-release System.IO.Compression.Brotli package and the Microsoft.AspNetCore.ResponseCompression package from the default NuGet Package source.

Once we have those 2 packages downloaded, we can now write our own BrotliCompressionProvider:

public class BrotliCompressionProvider : ICompressionProvider
{
    public string EncodingName => "br";
    public bool SupportsFlush => true;
    public Stream CreateStream(Stream outputStream)
    {
        return new BrotliStream(outputStream, CompressionMode.Compress);
    }
}

Now we need to open the Startup.cs and modify the ConfigureServices to add the Brotli compression provider (we are enabling this on SVGs as well, since these are essentially text):

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    // Add framework services.
    services.AddMvc();

+   services.AddResponseCompression(options =>
+   {
+       options.Providers.Add<BrotliCompressionProvider>();
+       options.MimeTypes = ResponseCompressionDefaults.MimeTypes.Concat(new[] { "image/svg+xml" });
+   });
}

To enable automatic response compression, we also need to modify Configure by adding this line:

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
    loggerFactory.AddConsole(Configuration.GetSection("Logging"));
    loggerFactory.AddDebug();

    if (env.IsDevelopment())
    {
        app.UseDeveloperExceptionPage();
        app.UseBrowserLink();
    }
    else
    {
        app.UseExceptionHandler("/Home/Error");
    }

+   app.UseResponseCompression();
    app.UseStaticFiles();

    app.UseMvc(routes =>
    {
        routes.MapRoute(
            name: "default",
            template: "{controller=Home}/{action=Index}/{id?}");
    });
}

Your application should now be able to compress and encode dynamic content (and SVGs to Brotli)! To prove this, we can easily open up Chrome Developer Tools and have a look at the Content-Encoding Response Header. This should contain: br.

If we inspect Network Traffic, with Brotli turned on, we can see that for the demo site we transferred 128 KB worth of data.

With Brotli turned off (no compression), we transferred 550 KB worth of data. This means that Brotli compressed the original payloads to 23%!

If we go ahead and compare this with gzip, we can see that gzip compressed this to 174 KB. So, in this particular use case, Brotli was able to compress the site to down 74% the size of gzip.

Of course, these are really quick (and inaccurate metrics). Your mileage may vary and if you want more accurate metrics I suggest you read the CloudFlare post referenced above. But it is clear that Brotli is an improvement over gzip and now that Microsoft is officially endorsing Google's Brotli it is worth more than ever taking in into serious consideration.

As usual, source code used in this post is located on GitHub cdemi/Brotli-ASP.NET-Core-Compression.

If you want to see the changes needed to add Brotli Response Compression you can look at commit 5605681e76f11214cc0f6eedecbf4e5ce3e97de7, which will give you a diff from the default Microsoft template.