GitHub Action for binary crates installation

5-minute read

Rust crates ecosystem has this amazing and thriving part, which is focused on linting, auditing, fuzzing and packaging other Rust projects: cargo-audit, cargo-geiger, cargo-release and so on. Obviously, it is very common to use them in the CI in order, for example, to fail builds automatically if changes introduced security vulnerabilities or new dependencies has the incompatible licenses.

There is a one small problem, though: most of these crates are your usual executable binaries, meaning that you need to compile them before use. While it is fine for a developer environment, adding cargo install line into your CI workflow impacts execution time drastically as it takes minutes to build the requested program and a couple seconds then to execute it.

Few ways to solve this irritating quirk exist, and in this post I’m going to talk about them and introduce another potential solution I’ve been working on during these boring self-isolating days. Bear in mind that for obvious reasons it will be all about the GitHub Actions and not other available CI solutions.

Cache it!

First idea is to compile our binary executable and put it into the Actions cache storage, so it can be reused later. Unfortunately, there is no programmatic access to the GitHub Actions cache storage yet, but you can always try to use @actions/cache Action to cache ~/.cargo/bin directory:

- name: Cache cargo plugins
  uses: actions/cache@v1
  with:
    path: ~/.cargo/bin/
    key: ${{ runner.os }}-cargo-plugins

Starting with Rust 1.41 cargo install command is now able to upgrade binary crates only if there is a new version exist, meaning that it will not recompile crate each time, even if no updates were published. The only potential problem is the cache usage limits and eviction policy; since Rust creates a lot of weighty files during the compilation, it is possible for big projects to evict cache entries with our binaries and force them to be compiled once again. Also, please note that this CI step was not tested thoughtfully, so consider checking and testing it before copying it into your projects.

Releases?

Another option is to use prebuilt binaries published by maintainers via GitHub Releases, GitLab Releases or other similar things. While it is very easy to bring such a binary into your CI environment (@XAMPPRocky/get-github-release Action is one example of it), there also few caveats exist:

  • Not all maintainers are publishing compiled binaries for their crates
  • Repository could be moved or removed, effectively breaking CI in a most unexpected way

@actions-rs/install

I worked on another approach to this problem for a while and it is now in a good enough state to be announced publicly. This @actions-rs/install GitHub Action aims to get the best from both of these approaches in order to provide better and faster solution for the binary crates installation.

In a most simple form it acts the same as cargo install command, compiling the requested crate each time (cargo-audit tool will be used for this and other examples):

- uses: actions-rs/install@v0.1
  with:
    crate: cargo-audit

Prebuilt binaries cache

General idea to speed up the process is very simple — we can compile these binaries only once, put them somewhere and then download each time when we need them. Separate tool-cache repository does exactly that: it maintains a list of crates to be compiled, builds them and then uploads into the cache storage.

Now, with enabled use-tool-cache input, this Action checks first if requested crate is available in the cache storage, and if it does — it will be downloaded and put into ~/.cargo/bin:

- uses: actions-rs/install@v0.1
  with:
    crate: cargo-audit
    use-tool-cache: true

What are the advantages of this approach?

  1. It is obviously faster, roughly one second to download already prepared file is a huge win comparing to the time spent on a crate compilation
  2. We do not rely on a crate maintainers publishing prebuilt binaries anymore
  3. All crates are installed from the same trusted source: crates.io

Of course, there are disadvantages too:

  1. Not all crates are available in the cache; some of them are just not added yet (diesel_cli, for example) and some are just not intended to be in there, like battop, because why do you even need that in your CI?
  2. Downloading and executing random executables is a bad idea in general, even if it happens in the isolated CI environment.

It must be understood that all executed Actions and downloaded files has an access to the source code, environment variables and secrets, so how can we be sure that this one executable was not tampered by some malicious third party? In general, we can’t guarantee that without reproducible builds and some kind of verified updates (TUF maybe?), so in order to provide at least some guarantees, build scripts are publicly available in the tool-cache repo (check the build.py) and all produced files are signed with RSA key.

When executed, @actions-rs/install Action verifies downloaded file with provided signature and public key, which is bundled directly into the Action repository; you can read more about the whole process in the Action documentation.

Of course, the only thing it proves is that files were not changed in the S3 bucket by a third party without private key; there is still no guarantee provided about other parts of the process, and that’s why this whole thing is disabled by default and in order to use it you should acknowledge risks and explicitly opt-in into using this feature by enabling use-tool-cache Action input.

What’s next?

This solution does the job really fast, but also adds too much complexity; so the next thing in a roadmap is to make the same thing as in “Cache it!” section, but compiled crate will be stored in the GitHub cache totally opaque for users. Such an approach will effectively eliminate the need for a third party cache and reduce the potential attack vectors, and will be implemented as soon as programmatic access to the cache will appear.

Note that other @actions-rs Actions (such as cargo with use-cross enabled, grcov or audit-check) are not using this functionality yet, but as temporary hack you can call this Action first and they will pick up the installed binary:

- uses: actions-rs/install@v0.1
  with:
    crate: cargo-audit
    use-tool-cache: true
- uses: actions-rs/audit-check@v1
  with:
    token: ${{ secrets.GITHUB_TOKEN }}

In a meantime, current implementation provides an opinionated, but greatly working way for speeding up slow CI workflows; let me know how that works for you.