Packaging a single crate
The Rust team uses a somewhat unusual workflow to package single crates.
Instead of one packaging repository per crate or upstream repository, it goes with a shallow monorepo approach. The repo is called debcargo-conf, sometimes abbreviated as dc-c or dcc.
It's called "shallow" because actual upstream sources and a full debian/
directory are not stored, only an "overlay". The overlay consists of
debian/debcargo.toml
, a declarative configuration for the debcargo tool;
debian/copyright
, the mandatory, most of time the only file needing manual
editing; and other debian/
files as needed.
From there, a Debian packaging is generated out-of-tree with debcargo, fixed and patched, built, and uploaded.
Some packaging techniques found during the years are recorded in Hacks.
Setting up
It's recommended to work in Debian unstable, unless you have specific backport needs.
First, install tools and clone the repo:
$ sudo apt update && sudo apt install cargo rustc dh-cargo debcargo git devscripts quilt
$ git clone git@salsa.debian.org:rust-team/debcargo-conf.git
$ cd debcargo-conf/
Second, set up a build environment. The "official" build tool in the Rust team is sbuild, as is used by Debian buildds. Adapt accordingly if using a different tool.
Using sbuild with unshare:
$ sudo apt install mmdebstrap sbuild uidmap autopkgtest
$ mkdir -p ~/.cache/sbuild # default cache location of sbuild
$ mmdebstrap --variant=buildd --include dh-cargo,cargo,lintian unstable ~/.cache/sbuild/debcargo-unstable-amd64.tar.xz
You may want to set up apt-cacher-ng to reduce bandwidth consumption and speed up package downloads.
Note that you need to set CHROOT_MODE=unshare
to use ./build.sh with unshare.
Using sbuild with schroot:
$ sudo apt install reprepro debootstrap sbuild schroot autopkgtest
$ sudo sbuild-createchroot --include=dh-cargo,cargo,lintian \
--chroot-prefix debcargo-unstable unstable \
/srv/chroot/debcargo-unstable-amd64-sbuild http://deb.debian.org/debian
# replace deb.debian.org with a mirror of your preference if applicable
These are basic setups of sbuild. For more detailed usage and customization, consult the sbuild page on wiki.
Packaging scripts
debcargo only prepares a Debian packaging. Some other scripts in the debcargo-conf repo automates the updating, building, and releasing (uploading) of it, conveniently named update.sh, build.sh, release.sh, respectively. Sections below introduce their basic usage, but more usage instructions are in their headers - just read them! And improve them if you have an idea ;)
Steps to work on a crate
- Prepare a new version/revision of a crate
- Fix patches if failing to apply, then go to 1
- Build
- Fix build and test failures if present, then go to 2
- Upload if you have upload rights; if not, send in the work and Request for Sponsorship
Prepare a new version/revision of crate
$ ./update.sh <crate-name>
and follow its instructions.
You should usually also write collapse_features = true
in
src/<crate>/debian/debcargo.toml
. See the reason.
Co-installable older versions ("versioned" or "semver-suffixed" packages)
Even if we strive to maintain only one version of a crate, there are times we have to split out an older version of it.
Only do it when absolutely necessary, e.g. when an important crate used by many dependents introduces a breaking, incompatible major version, yet dependents differ in the pace of migration.
If possible, please consider these before going split:
- Downgrading or upgrading the dependency
- Ask upstream to upgrade if there is not already similar asks
- If it's an optional dependency, consider temporarily disabling it (and if an optional feature depends on it, also such feature)
- Wait until upstream upgraded
To split out, or update the split package, run:
$ ./update.sh <crate-name> <old-version>
and follow its instructions.
To save time, when first splitting out, you can first copy anything relevant
from src/<crate-name>
to src/<crate-name>-<old-version>
, then run
the command above and adapt as needed.
Hold packages at versions older than the newest
If you need to keep the unversioned package in Debian at an older version than
is released on crates.io, e.g. to upload an important bugfix without being
blocked on dependencies of the newest version, you can prefix the scripts with
REALVER=<wanted-version>
:
$ REALVER=<wanted-version> ./update.sh <crate-name>
$ REALVER=<wanted-version> ./build.sh <crate-name>
$ REALVER=<wanted-version> ./release.sh <crate-name>
Repackage/recreate an existing revision
To build the version of a package as packaged in debcargo-conf:
$ ./repackage.sh <crate-name>
If this package is already in the archive and you want to recreate that
exactly, you will need to use the exact same version of debcargo that was used
previously. Its version is mentioned in debian/changelog
, like this:
* Package foo a.b.c from crates.io using debcargo x.y.z
Fix patches
Due to the way this workflow organizes files, we can't use other options like gbp-pq. See "Patching crates" in Hacks.
Build
Run:
$ ./build.sh <crate-name>
$ ./build.sh <crate-name> <version> # for a versioned package
$ REALVER=<wanted-version> ./build.sh <crate-name> # for older version of unversioned package
If you need to pass additional options to sbuild, like --arch=i386
, set them
in the SBUILD_OPTS
environment variable:
$ SBUILD_OPTS="--arch=i386" ./build.sh <crate-name>
Normally, ./build.sh
will fail early if not all the build dependencies are
available in your local apt cache. If you are packaging a large dependency tree
however, to avoid many round-trips through NEW it is possible to bypass this
check and build all the packages together. Suppose package B depends on package
A, then you can run something like:
$ ( cd build && ./build.sh A )
$ ( cd build && ./build.sh B librust-A*.deb )
The librust-A*.deb
arguments after ./build.sh B
are extra deb files to pass
to sbuild to use as dependencies. Alternatively, use the environment variable
EXTRA_DEBS
, like so:
$ EXTRA_DEBS=librust-A*.deb ./build.sh B
$ EXTRA_DEBS=librust-A.deb,librust-B.deb ./build.sh C
There is also a dev/chain_build.py
script that lets you build multiple
packages in a "chain". Run it without arguments to see its help message.
Upload
Note: this is usually only needed for uploading the package to the Debian archive. If you don't have upload rights, and weren't asked by someone who has to send a built package to them, this is likely not for you. See the "Request for Sponsorship" section below.
To build for upload, run
$ ./release.sh <crate-name> # or
$ ./release.sh <crate-name> <old-version> # as appropriate
DISTRO=experimental ./release.sh <crate-name> # to target another distro
EXTRA_DEBS=foo.deb ./release.sh <crate-name> # use local dependencies
A source-only build is generated (e.g. source.changes, .dsc only). The script
shows some information about its upload, but does not do it; you need to do the
actual upload yourself, e.g. dput build/rust-foo_1.2.3-1_source.changes
.
It also creates a Git branch named pending-<crate>
to signal the upload. See
conventions.
Send in the work
The Rust team usually uses the merge request (MR) workflow. There isn't too much about it - commit your changes and send them in with a MR to the debcargo-conf repo.
One thing though - we have some conventions on branch names and commit messages.
Request for Sponsorship (RFS)
If you are new to the Rust team, or to Debian packaging in general, after packaging, you should ask for its sponsorship with a Request for Sponsorship (RFS).
The linked page describes the RFS process for packages on mentors.debian.net. It's not one process rules all, though.
For single crate process, create src/<crate>/debian/RFS
, in the debcargo-conf
repo. It should include relevant information like what is needed to build it,
what it's for, etc. After the packaging is merged, ask someone in the team with
upload rights to sponsor it.
For workspace process, it's more like the "usual" way in Debian: you can either upload to mentors.d.n and file a RFS bug, or ask a DD you've worked with to directly upload your package.
Expert mode & packaging multiple packages
You should get used to the single-packaging workflow a bit first, including doing a few test builds of your package. Otherwise the instructions below may seem a bit opaque.
- Depending on the situation, optionally clean the
build/
directory. ./update.sh <CRATE>
all your relevant packages.- Do any manual updates.
(cd build && ./build.sh <CRATE> *.deb)
for all your relevant packages, in dependency order.- Deal with any issues that come up.
- Push your updates to debcargo-conf.
- Run
dev/list-rdeps.sh <CRATE> [<CRATE> ...]
on all the crates you updated. If a breaking version is introduced, any reverse dependencies affected also need to be updated, and you should repeat steps 1-7 (including this step) for them as well until this step lists no new packages that are affected. ./release.sh <CRATE>
all the packages you updated, running the build again if necessary. It may be possible to do this out of dependency order, assuming you didn't have to make significant changes in step (5). If you did, then this step also has to be done in dependency order.- Upload them.
- Push your
pending-*
branches to debcargo-conf.
There are also various scripts in dev/*
that might help you. They should have
a few lines at the top of the source code describing their functionality and
some brief usage instructions.
Whew, thanks for all your work!
Collapsing features
TL;DR: Set collapse_features = true
in debcargo.toml, unless something
breaks. This eliminates empty feature packages and speeds up the NEW trip.
Rust and Debian have two different levels of abstraction when handling dependencies and the relationship between them. In Rust the lowest level is a feature, while in Debian it's the binary package.
This means that the following dependency chain is not a problem in rust:
- crate A with feature AX depends on crate B with feature BY
- crate B with feature BX depends on crate A with feature AY
This is a perfectly valid situation in the Rust ecosystem. Notice that there is
no dependency cycle on the per-feature level, as this is enforced by cargo; but
if collapse_features = true
then package A+AX+AY would cyclicly depend on
package B+BX+BY in Debian.
This is reflected in the Debian packages by producing Provides
lines for all
combinations of features, and this can become a quite large section.
Setting collapse_features = true
in debcargo.toml removes this behaviour and
is generally recommended, unless when it leads to dependency cycles of Debian
packages. If that happens, those must be broken up by having some or all of the
packages set this feature to false.
Changed original tarballs
Sometimes the orig.tar generated by debcargo might change e.g. if you are using
a newer version of debcargo and one of the dependencies relating to generating
the tarball was updated and its behaviour changed - compression settings,
tarball archive ordering, etc. This will cause your upload to get REJECTED by
the Debian FTP archive for having a different orig.tar. In this case, set
REUSE_EXISTING_ORIG_TARBALL=1
when running ./release.sh
.