Re:Why Snaps are slow. From technical view.
From
Sativa GNutella@21:1/5 to
All on Wed Aug 31 23:13:12 2022
Why LZO was chosen as the new compression method
by Ian Johnson on 23 December 2020
Everyone wants fast applications. Recently, we provided a
mechanism to make snap applications launch faster by using the
LZO format. We introduced this change because users reported
desktop snaps starting more slowly than the same applications
distributed via traditional, native Linux packaging formats like
Deb or RPM.
After a thorough investigation, we pinpointed the compression
method as the primary slowdown. Once we introduced the change,
some users started wondering why we chose LZO as the new
compression method for snaps, given that there are ?better?
algorithms available. Here, we want to take you through the
journey of understanding why we picked LZO, and what is next for
the snap compression story.
The old way
Previously, the only supported compression format for snaps was
XZ. This decision was borne out of two main determining factors:
compatibility and size. One of the primary delivery targets for
snaps (in addition to desktop users) is IoT devices, and so for
those, we wanted to have the smallest possible size.
Additionally, cross-distro support is very important with snaps,
and so we also wanted to make sure that the compression format
chosen would be compatible with the widest range of kernels. In
both cases, the XZ method fit the bill nicely. Additionally, at
the time the decision was made, some of the new algorithms such
as ZSTD were not even in existence.
It?s important to mention a few overarching design goals of snaps
before we go much further. The first thing is the choice of
squashfs as the packaging mechanism ? instead of distributing
individual files as tarballs, etc. We wanted both determinism and
ease of deployment whereby all users of a snap get the same
files, and those files are not modifiable (by the user). To
satisfy both goals, we selected squashfs, which is a compressed
filesystem format that is mounted read-only.
Orthogonal to that goal was package integrity. It was crucial to
have a design where the files that were delivered to users were
cryptographically verified, as well. We wanted a system where the
bits that make up the .snap file are uploaded to the store by a
snap developer, and those same exact bits are delivered to the
user without modification. A delta upload/download functionality
is used to save on network bandwidth, but that merely
reconstructs the original exact bits that the snap developer
uploaded to the store. Therefore, we aren?t able to recompress
the snap either on the user?s computer or on our servers as that
would change the snap files. As such the snap that is uploaded
can currently only be uploaded and distributed with a single
compression setting.
Why we needed to switch
When we started looking into why desktop applications packaged as
snaps were slower, we explored multiple hypotheses, and we
focused on the difference in startup times between the various
packaging formats. For example, here is a graph created at the
start of these explorations, demonstrating how many milliseconds
it takes to launch a snap application vs the same application
packaged natively:
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)
From
Sativa GNutella@21:1/5 to
All on Wed Aug 31 23:14:39 2022
One hypothesis was centered around the decompression of the
squashfs snap taking some time, so we set up tests to run and
compare the performance and timing of various supported
compression algorithms for squashfs, including: no compression,
GZIP, LZO, ZSTD, and of course XZ.
Another idea was that dynamic library search paths may be slowing
things down. While optimizing these does provide some improvement
in snap startup times, the contribution was less significant than
the change of the compression algorithm. We explored this topic
in greater detail in a snapcraft.io forum post.
Yet another theory was that perhaps snaps were slow due to desktop
?helper scripts?, which are auxiliary shell scripts used to
configure applications to ensure compatibility with the graphical
system such as X11 or Wayland. Like the dynamic library search
paths, the optimization of these scripts did not contribute a
significant change or reduction in the overall snap startup
times.
Eventually, we narrowed down our integration on the compression,
and found that indeed, using no compression for the squashfs
files resulted in the quickest startup time for many different
snaps, reducing it by an order of magnitude in some scenarios, on
some systems.
Here is a graph of the various types of compression for the
chromium snap specifically.
The graph above shows the startup time in milliseconds of the
chromium snap, with various different compression algorithms
used. All times were averaged across multiple runs from a ?cold
start? with no caching activated.
This also demonstrated that in fact the format we initially chose
had the worst performance when it comes to this kind of
decompression at application startup. At the time when we chose
XZ as the format, we were mostly concerned with size and it was
expected, if not fully appreciated, that XZ would be slower than
other alternatives.
This investigation made clear that we should evaluate a use of a
new compression format, to improve the startup times for snaps
geared towards desktop experience rather than size-constrained
IoT devices.
As a middle ground between size and decompression performance, we
decided to allow snaps to ?opt-in? to a better performing
algorithm. Developers can choose an alternative compression
format at build time (before uploading to the store). We decided
it would be best not to enable the use of every available
compression format, because users on older kernels and distros
may not have necessary support for the new algorithm. Thus, if a
user tried to install a snap compressed this way, they would not
be able to install the application, which creates a poor user
story.
What we switched to
Given all our constraints, the bit-for-bit guarantee, the wide
cross-distro support and the security-driven consideration
(currently) not to re-compress on the fly, we decided to select
LZO as the second allowed compression format for snaps. We chose
it because it is widely supported by old kernels and systems, it
has relatively good performance (much better than XZ and not much
worse than ZSTD, for example), and it still provides a good
compression ratio, minimizing disk space usage (and network
bandwidth).
Above, you can see a graph of the compression ratio for various
representative snaps, where a higher compression ratio is better.
We can see that LZO (in purple) has a worse compression ratio
than XZ, and in fact is the worst of the 4 formats (except for
?none? which by definition has no compression). However, it still
has a ratio of around 2.5-3 for reasonably sized snaps such as
chromium and mari0. For very large snaps like supertuxkart, the
ratio is very close to 1 for all compression formats and so
picking LZO vs XZ is not a big win either way. If you are a snap
author and you want to enable your snap to use LZO, have a look
at this forum post to see how you can easily make the switch.
What?s next
Eventually, we would like to be able to support some sort of
dynamic re-compression either on the store side or on the user?s
machine, which could potentially allow users with newer kernels
to use better compression formats, but still allow users with
older kernels to use legacy formats that are supported there.
This could also allow users who want to save space but are not as
concerned about performance to choose a compression method with a
higher compression ratio, or users who want the best performance
and for whom space isn?t an issue to choose a more performant
algorithm or even no compression.
Summary
User feedback and cross-distro support are both important with
snaps. We originally chose xz for various compatibility reasons,
but over time we discovered this to be suboptimal for desktop
users looking for the best performance comparable with
traditional packaging mechanisms. We investigated and selected
LZO as an alternative which offers a good balance for many
applications between size and speed.
We hope you found this article interesting and helpful. If you
have any ideas or suggestions, please join the discussion at
forum.snapcraft.io and let us know your thoughts. We look forward
to improving snaps together with your feedback.
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)