Linux: 适合所有Linux发行版的包管理器
Nix fixes dependency hell on all Linux distributions
Share
Print
CommentsA next-generation package manager called Nix provides a simple distribution-independent method for deploying a binary or source package on different flavours of Linux, including Ubuntu, Debian, SUSE, Fedora, and Red Hat. Even better, Nix does not interfere with existing package managers. Unlike existing package managers, Nix allows different versions of software to live side by side, and permits sane rollbacks of software upgrades. Nix is a useful system administration tool for heterogeneous environments and developers who write software supported on different libraries, compilers, or interpreters.
Why provide yet another package manager? Because current package managers fall short in the upgrade cycle. Everyone gets burnt by software dependencies, at some point. In particular, with a major release of any given distribution, many people choose not to upgrade until it is time to do a fresh install. With Nix, upgrades are always safe: they don't overwrite previously installed packages. This means previous versions will continue to work, and you can easily roll back.
Nix started as an academic project at Utrecht University in the Netherlands. The name is tongue in cheek; in Dutch it means "nothing."
The problems: destructive upgrades, software versioning, heterogenous environments
All popular package managers, including APT, RPM and the FreeBSD Ports Collection, suffer from the problem of destructive upgrades. When you perform an upgrade -- whether for a single application or your entire operating system -- the package manager will overwrite the files that are currently on your system with newer versions. As long as packages are always perfectly backward-compatible, this is not a problem, but in the real world, packages are anything but perfectly backward-compatible.
Suppose you upgrade Firefox, and your package manager decides that you need a newer version of GTK as well. If the new GTK is not quite backward-compatible, then other applications on your system might suddenly break. In the Windows world a similar problem is known as the DLL hell, but dependency hell is just as much a problem in the Unix world, if not a bigger one, because Unix programs tend to have many external dependencies.
Also, destructive upgrades make it hard to undo, or roll back, an upgrade. Unless you or your package manager makes a backup of all the files that got replaced, you cannot easily undo an upgrade.
Finally, while the package manager is busy overwriting all the files that belong to a package, your system is temporarily in an inconsistent state in which a package may or may not work properly. Hit the power switch on your computer halfway through your next OS upgrade and see if the system will still boot properly!
Usually, an upgrade of a package will make the older version disappear. Sometimes a package manager allows a few versions next to each other -- say gcc-3.4 and gcc-4.3. However, this only works if the packager has arranged for this by making sure that the two versions install to different paths. What if you want to test gcc-4.0.3 without disrupting your system? Or if you want to test software using different compiler, library, or interpreter versions and combinations? What if you want to try the latest beta version of an application without risking your existing installation?
Then there is the problem of multi-distribution support. Developers and systems administrators have no way of knowing what combinations of kernel, libraries, and packages a user is running. When a user tries to install some software and complains about a missing library, you can suggest he try to find the appropriate packages, but you have not tested them in the same combination of dependencies. You can ask him to build from source, but he may not be using appropriate versions of libraries and compiler. Even if you are lucky enough to work for an institution that standardizes on a single distribution, you may find users' libraries are outdated and do not support the software you want to deploy.
Existing package managers do well enough in furnishing stable systems because they depend on a long period of testing by a lot of people. However, when users need more recent software, because of a bug or some missing functionality, they turn to so-called "testing" or even "unstable" packages, which often come with a range of dependencies that also get updated on the users' systems, potentially introducing instabilities in other software that depends on those components.
Nix has a more graceful approach that lets different versions of software coexist. Rather than installing packages in global locations such as /usr, it instead stores each package in its own directory under /nix/store. The top-level directory for each package contains a cryptographic hash based on all the inputs used to build the package. That creates a unique identifier such that multiple versions of a package don't interfere with each other, and different packages on your system can use different versions of some dependency without causing a conflict. This means that you can atomically upgrade and roll back packages.
Getting started with Nix
Ironically, the best way to install Nix is to fetch and compile it from source using the normal ./configure ; make ; make install commands, as root, as this will work for any system, including Mac OS X and the BSDs. You'll need a standard GNU g++ build environment and curl installed to bootstrap Nix. Once Nix is installed it no longer depends on those tools. Nix stores packages and some metadata under the directory /nix, so make sure there is enough space there. If / doesn't have enough space, either use a special partition for /nix , or bind-mount /nix to a filesystem that has 5 to 50GB of space available. You should also add the file /usr/local/etc/profile.d/nix.sh to your bashrc file. It sets up some environment variables (like $PATH) that Nix needs in order to work properly.
Once Nix itself is installed, you can start using it to install software. You can get Nix Packages (Nixpkgs), a large collection of packages for Nix, by downloading it from the Nix Web site or checking it out from its Subversion repository. However, the easiest way to use it is to subscribe to the Nixpkgs channel, which is just a way to distribute the latest version of Nixpkgs automatically to users:
nix-channel --add http://nixos.org/releases/nixpkgs/channels/nixpkgs-unstable nix-channel --update
Nixpkgs consists of a large set of Nix expressions, which are Nix's language for describing how packages are built, similar to Gentoo ebuilds. Nix is essentially a source-based package manager, like Portage, but Nix will automatically download precompiled binaries of packages if it can locate them by their hash values. The Nixpkgs channel contains a lot of precompiled binaries, which speeds up package installation significantly.
Once nix-channel has downloaded Nixpkgs, you can see what packages are available. The nix-env tool acts much like yum and apt-get. You can see a list of packages by running nix-env -qa '*', or to list all the packages named Firefox, nix-env -qa firefox. That should return a list like:
firefox-2.0.0.17 firefox-2.0.0.17-with-plugins firefox-3.0.4 firefox-3.0.4-with-plugins
You can add the -s flag to see which packages are pre-built in the Nixpkgs channel. An S in the listing indicates that the package can be downloaded in binary form.
--S firefox-2.0.0.17 --- firefox-2.0.0.17-with-plugins --S firefox-3.0.4 --- firefox-3.0.4-with-plugins
Installing a package with dependencies
To see the power of Nix, let's say you want to install Firefox 2, with all its dependencies and a bunch of plugins, such as Flash, without overwriting anything on your existing system. Just type nix-env -i firefox-2.0.0.17-with-plugins and Nix downloads and compiles these packages. Even though firefox-2.0.0.17-with-plugins is not available precompiled in the channel, most of its dependencies are. Nix will download binaries where possible, and compile from source otherwise. Firefox has lots of dependencies, from the GNU C library to GTK. Your distro will already have most of these, but Nix installs its own copies to make sure that you get exactly the right versions and that they don't interfere with rest of your system. After all, nowadays, why not sacrifice a bit of disk space to get predictability?
So now you should be able to run the Firefox you just downloaded by simply typing firefox on the command line. (You may need to set the FONTCONFIG_FILE environment variable to /etc/fonts/fonts.conf to make sure it can find your system's fonts.)
Now let's try upgrading to Firefox 3, either with the command nix-env -i firefox-3.0.4-with-plugins, or nix-env -u firefox, which would have made Nix pick the latest version. After this, you should have Firefox 3 installed, but the old Firefox is not gone -- it has not been overwritten. The two versions automatically end up in different paths:
/nix/store/vskr06rlblihz22...-firefox-2.0.0.17-with-plugins /nix/store/w1i05b7s30zqz...-firefox-3.0.4-with-plugins
You can see from this why packages do not interfere with each other. You can run either version of Firefox cleanly from the above paths.
Outside of Nix, you can still use the installed packages on your native system. The Nix packages are extra and independent.
Profiles
One can invoke a program directly from its path, or by adding paths to the system's search paths, but the long path names obviously make this ugly. That is why nix-env automatically generates trees of symlinks to the installed packages automatically. To make this work, you would have in your $PATH something like ~/.nix-profile/bin, which points at one of these trees of symlinks. This is incidentally how Nix can do atomic upgrades and rollbacks: it just generates a new tree of symlinks to the new packages, then flips ~/.nix-profile to point at the new one.
You can have more than one of these trees of symlinks, or profiles, each containing different activated packages. This lets you set up a profile for a specific version of any application within a profile. For instance, you could set up a profile to try out the latest version of Firefox by entering the command:
nix-env --profile my-firefox-test -i firefox-3.0.4-with-plugins
then running ./my-firefox-test/bin/firefox (which is a symlink to the actual location of Firefox 3 in /nix/store) -- it won't affect your 'normal' Firefox in ~/.nix-profile/bin.
Rollback
Since installing Firefox 3 did not overwrite the old Firefox 2, it is straightforward to roll back changes. If you want to go back to Firefox 2, all you need to do is say nix-env --rollback to return to the previous situation.
Of course, you'll probably want to get rid of old versions eventually. If you are sure you do not need to roll back any more, you can say nix-collect-garbage -d to delete all unneeded packages from the system -- in this case Firefox 2 and any dependencies that are not used by some other (still enabled) package.
Creating your own packages
Finally, Nix comes with a domain-specific language for writing package descriptions. This functional yet straightforward language provides an elegant way of providing and maintaining packages with their assorted dependencies. A package for PHP with dependencies, looks like:
# Nix expression for building PHP. This is a function that given some arguments # (like flex and libxml2) builds an instance of PHP in the Nix store. # These are the function arguments... {stdenv, fetchurl, flex, bison, libxml2, apacheHttpd, postgresql ? null, mysql ? null}: # and this is the result of the function: a build action. stdenv.mkDerivation { name = "php-5.2.4"; src = fetchurl { url = http://nl3.php.net/distributions/php-5.2.4.tar.bz2; sha256 = "1h513j7crz08n7rlh8v7cvxfzisj87mvvyfrkiaa76v1wicm4bsh"; }; inherit flex bison libxml2 apacheHttpd; builder = ./builder.sh; buildInputs = [flex bison libxml2 apacheHttpd]; patches = [./fix.patch]; }
This Nix expression requires libxml2, among other packages, and allows a choice of SQL back ends on installation. Nix expressions come with a large number of commands for package definition, which usually makes package descriptions compact and easy to understand. At build time the environment is clean and isolated from all other libraries, so dependencies always have to be included explicitly. A missing dependency will always be detected, which is a great feature for software developers.
The ability to pass options to the package configuration allows for flexible package management. For instance, you could build a package with GUI support on desktops and without GUI support on servers. Stripped-down versions of software and even documentation are feasible with Nix. More complex examples can be found in the package source tree.
Conclusion
With Nix, you can use a rock-solid and stable distribution like Debian stable or Red Hat Enterprise Linux as a foundation. When you need to deploy recent software, not provided by the distribution, Nix provides support for multiple versions, complete dependencies, and rollback. Nix allows you to escape dependency hell by creating a predictable system for software managment.About Nix
Nix is a purely functional package manager. This means that it treats packages like values in purely functional programming languages such as Haskell — they are built by functions that don’t have side-effects, and they never change after they have been built. Nix stores packages in the Nix store, usually the directory /nix/store, where each package has its own unique subdirectory such as
/nix/store/r8vvq9kq18pz08v249h8my6r9vs7s0n3-firefox-2.0.0.1/
where r8vvq9kq… is a unique identifier for the package that captures all its dependencies (it’s a cryptographic hash of the package’s build dependency graph). This enables many powerful features.
Multiple versions
You can have multiple versions or variants of a package installed at the same time. This is especially important when different applications have dependencies on different versions of the same package — it prevents the “DLL hell”. Because of the hashing scheme, different versions of a package end up in different paths in the Nix store, so they don’t interfere with each other.
An important consequence is that operations like upgrading or uninstalling an application cannot break other applications, since these operations never “destructively” update or delete files that are used by other packages.
Complete dependencies
Nix helps you make sure that package dependency specifications are complete. In general, when you’re making a package for a package management system like RPM, you have to specify for each package what its dependencies are, but there are no guarantees that this specification is complete. If you forget a dependency, then the component will build and work correctly on your machine if you have the dependency installed, but not on the end user's machine if it's not there.
Since Nix on the other hand doesn’t install packages in “global” locations like /usr/bin but in package-specific directories, the risk of incomplete dependencies is greatly reduced. This is because tools such as compilers don’t search in per-packages directories such as /nix/store/5lbfaxb722zp…-openssl-0.9.8d/include, so if a package builds correctly on your system, this is because you specified the dependency explicitly.
Runtime dependencies are found by scanning binaries for the hash parts of Nix store paths (such as r8vvq9kq…). This sounds risky, but it works extremely well.
Multi-user support
Starting at version 0.11, Nix has multi-user support. This means that non-privileged users can securely install software. Each user can have a different profile, a set of packages in the Nix store that appear in the user’s PATH. If a user installs a package that another user has already installed previously, the package won’t be built or downloaded a second time. At the same time, it is not possible for one user to inject a Trojan horse into a package that might be used by another user.
More details can be found in Section 3 of our ASE 2005 paper.
Atomic upgrades and rollbacks
Since package management operations never overwrite packages in the Nix store but just add new versions in different paths, they are atomic. So during a package upgrade, there is no time window in which the package has some files from the old version and some files from the new version — which would be bad because a program might well crash if it’s started during that period.
And since package aren’t overwritten, the old versions are still there after an upgrade. This means that you can roll back to the old version:
$ nix-env --upgrade some-packages $ nix-env --rollback
Garbage collection
When you install a package like this…
$ nix-env --uninstall firefox
the package isn’t deleted from the system right away (after all, you might want to do a rollback, or it might be in the profiles of other users). Instead, unused packages can be deleted safely by running the garbage collector:
$ nix-collect-garbage
This deletes all packages that aren’t in use by any user profile or by a currently running program.
Functional package language
Packages are built from Nix expressions, which is a simple functional language. A Nix expression describes everything that goes into a package build action (a “derivation”): other packages, sources, the build script, environment variables for the build script, etc. Nix tries very hard to ensure that Nix expressions are deterministic: building a Nix expression twice should yield the same result.
Because it’s a functional language, it’s easy to support building variants of a package: turn the Nix expression into a function and call it any number of times with the appropriate arguments. Due to the hashing scheme, variants don’t conflict with each other in the Nix store.
Transparent source/binary deployment
Nix expressions generally describe how to build a package from source, so an installation action like
$ nix-env --install firefox
could cause quite a bit of build activity, as not only Firefox but also all its dependencies (all the way up to the C library and the compiler) would have to built, at least if they are not already in the Nix store. This is a source deployment model. For most users, building from source is not very pleasant as it takes far too long. However, Nix can automatically skip building from source and download a pre-built binary instead if it knows about it. Nix channels provide Nix expressions along with pre-built binaries.
Binary patching
In addition to downloading binaries automatically if they’re available, Nix can download binary deltas that patch an existing package in the Nix store into a new version. This speeds up upgrades.
Nix Packages collection
We provide a large set of Nix expressions containing hundreds of existing Unix packages, the Nix Packages collection (Nixpkgs).
Service deployment
Nix can be used not only for rolling out packages, but also complete configurations of services (such as this web server). This is done by treating all the static bits of a service (such as software packages, configuration files, control scripts, static web pages, etc.) as “packages” that can be built by Nix expressions. As a result, all the features above apply to services as well: for instance, you can roll back a web server configuration if a configuration change turns out to be undesirable, you can easily have multiple instances of a service (e.g., a test and production server), and because the whole service is built in a purely functional way from a Nix expression, it is repeatable so you can easily reproduce the service on another machine.
You can read more about this in our SCM-12 paper.
Portability
Nix should run on most Unix systems, including Linux, FreeBSD and Mac OS X. It is also supported on Windows using Cygwin.
NixOS
NixOS is a Linux distribution based on Nix. It uses Nix not just for package management but also to manage the system configuration (e.g., to build configuration files in /etc). This means, among other things, that it’s possible to easily roll back the entire configuration of the system to an earlier state. Also, users can install software without root privileges. Read more…
More information
If you want to use Nix, you’ll want to read the manual (see the Documentation links on the left). Nix is described in detail in Eelco Dolstra’s PhD thesis, and in a number of papers.
About us
Nix was developed at the Department of Information and Computing Sciences, Utrecht University by the TraCE project. The project is funded by the Software Engineering Research Program Jacquard to improve the support for variability in software systems
博文
没有评论:
发表评论