Caching All Native Ruby Gem Platforms

TL;DR

Some dependencies, like nokogiri, ship with multiple libraries for different architectures. If you cache your gems, you may need to cache multiple platforms, because your development team is spread across various platforms or you deploy to a different platform. To do this, you can use:

bundle cache # cache gems in vendor/cache
bundle lock --add-platform x86_64-linux # add additional platforms
bundle package --all-platforms # cache multiple platforms

On bundler version 1.x, add:

bundle config specific_platform true

Native Nokogiri

Nokogiri 1.11.0 has been released, and one of the exciting updates is the inclusion of pre-compiled native gems for various platforms! If you're using a supported platform, your days of installing nokogiri with native extensions may be over. These changes result, "in much faster installation and more reliable installation". Many thanks to the maintainers and contributors for this great update.

Updating to these pre-compiled gems should be a seamless experience. Bundler will grab the appropriate pre-compiled .gem file, if you're on a supported version, and use that. However, if you cache your gems, and you'd like to cache multiple platforms, you have a few more steps to complete.

Cache Hit

Gem dependencies can be cached along with your app and then you can use that cache to retrieve your application's dependencies, rather than RubyGems. We take advantage of this on a number of projects for various reasons, but the most important one that requires all gems to be vendored is that some applications are deployed to, and the deployments are created in, environments where they cannot access RubyGems directly.

We need to tell bundler to cache our gems.

bundle cache

Running that on an existing application will add .gem files into the vendor/cache directory.

Platform Dependence

You need to tell bundler that you require multiple platforms. In the case of this example, I'm developing on a computer running macOS, so installing nokogir will give me the pre-compiled gem for that architecture. That's great, but I also need the linux native gem for my deployment environments.

First, I need to tell bundler to add the platform.

bundle lock --add-platform x86_64-linux

After doing that, the Gemfile.lock file is updated to list that platform.

Platform Independence

However, even if you add the platform before installing the dependency, adding the platform will still not retrieve and cache both platform's .gem files. We also need to tell bundler to cache those other platforms.

bundle package --all-platforms

Now our other platform is cached, along with the existing platforms.

If you are using Bundler version 1.x, you may also need to set the specific_platform configuration setting.

bundle config specific_platform true

Now you should have all your gem dependencies cached across all platforms specified in your Gemfile.lock. You no longer need to compile nokogiri!

Learn more about how The Gnar builds Ruby on Rails applications.