Haskell Version Lockdown

Update

Cabal now supports version freezing. Just install cabal version 1.20 and use the cabal freeze command. So I deleted information in this post about our custom solution. We have switched to using cabal freeze now.

I find that almost nobody understands version freezing since they aren't doing it yet, so you can still benefit from reading the rest of this.

Motivation

Cabal is a great tool for library authors. As a library author we could give you a few minor nitpicks but we have few substantial complaints.

What we often forget is that the needs of library authors are different from application builders. It has been said before that Cabal is not a package manager. But cabal-install's constraints are greater than not managing user dependencies properly: it still does not provide basic essential tools for application builders.

Application builders need to produce reliable, re-playable builds. Haskellers will often attempt to do this with a Cabal file. But .cabal file versioning is meant for library authors to specify maximum version ranges that a library author hopes will work with their package. Pegging packages to specific versions in a .cabal file will eventually fail because there are dependencies of dependencies that are not pegged.

Related Work

or, just because Ruby does it does not mean that it is criminally unsafe.

Our solution is essentially the same as Ruby gems Gemfile.lock but not as heavy-weight since Haskell does not decide dependencies at application startup like Ruby (no bundle exec is required).

Talking with some other industrial users also validated what we are doing: most adopt techniques for limiting what can be installed, and attempt to achieve the same end result.

Library authors vs. Application developers

Library authors have no need for lock files since they need to build across as many versions as possible. However, they may find it useful to publish lock files of successfully built versions.

Application developers no longer need to*peg packages to specific versions in their cabal file. Instead they specify version ranges that they want to install from when they change or upgrade their dependencies.

Usage with cabal-meta

This versioning solution is similar to how cabal-meta functions. cabal-meta keeps a separate list of special packages to install and feeds that to cabal-install. cabal-meta was mainly designed to deal with building multiple local packages at once, which to a certain extent you can use the add-source command for from cabal-dev or the new cabal sandbox feature. cabal-meta also helps automatically build remote dependencies. One problem with cabal-meta is that it is a separate executable and it does nothing to stop you from accidentally not using it and just using cabal/cabal-dev. We will look into cabal-meta integration with lock files in the future.

Usage with cabal 1.18 sandboxes

If you aren't using cabal sandboxes, please immediately stop what you are doing, read Mikhail's awesome introduction to Cabal Sandboxes, upgrade cabal, and type cabal sandbox init in your project. We have been using the new sandboxes for weeks now, and it is great. We don't have to remember to use cabal-dev instead of cabal, w just have to type cabal sandbox init once.

Another very important feature of Cabal 1.18 is that it adds the ability to build individual targets. So if you have a library foo and a test-suite foo-test, you can type cabal build foo to build the library and cabal build foo-test to build the test suite. This makes using a lock file easier because you can always use cabal configure --enable-tests to start with to write out the lock file and then you can choose your build target later.

Using Cabal 1.18 combined with our lock files means we now spend none of our time on installation issues that tools should easily solve for us. Right before we rolled out our lock file one of our team members had an installation failure issue on the continuous integration server. This is the kind of build issue that is still common place with Haskell. After rolling out the lock file we had him re-merge his branch and install from the lock file, and the build passed, proving the value of what we had done.

It was just posted that Hackage2 is going to be officially release soon. In the span of a few months Haskell is changing from ridiculously harder to manage packages than mainstream programming languages to being on par.