Packaging "hacks"
Scope of this document
This document aims to document some nice "hacks" and tricks to employ when packaging crates.
Getting an overview of missing crates
A nice tool to generate a graphical overview of a rust projects' dependency
tree is cargo debstatus
. Install it like that: apt install cargo-debstatus
(trixie and sid). Then download either a release or clone the git project and
cd
into there. Run cargo debstatus
to get a nice graph about dependencies
and reverse dependencies.
Please note that cargo debstatus
may be unable to find
some of the crates which are already packaged in Debian, so before packaging
new crates double-check they are not in the archive using
apt search $CRATE_NAME
or apt list '*$CRATE_NAME*'
(replace underscores
with dashes in $CRATE_NAME
).
Patching crates
If a crate needs a) a newer dependency or b) an older dependency than the one in the archive you need to patch the crate. This is relatively common. You can also use this to patch out features in Cargo.toml or make changes to the source code.
Since the source is pulled from crates.io and not from github/lab, patching requires downloading the source from there. There are two ways to achieve this:
- Use
cargo-download
:cargo install cargo-download
- Download the source:
cargo download foocrate version > foocrate.tar.gz
- Extract the crate:
tar -xf foocrate.tar.gz
cd foocrate-version && git init && git add . && git commit -m "d"
- Edit e.g. Cargo.toml to relax the dependencies: Instead of version 0.4 of a crate bump it to 0.5 ( if that is the corresponding debian version) or down to 0.3 (if that is the debian version)
- Generate a patch with your changes:
git diff -p >> relax-deps.diff
cp relax-deps.diff debcargo-conf/src/foocrate/debian
cd debcargo-conf/src/foocrate/debian && mkdir patches
mv relax-deps.diff patches/ && cd patches && echo relax-deps.diff >> series
cd ../../../../
- run
./update.sh foocrate
again. The patch should get applied and allow you to build against an older/newer dependency / with features disabled. - Document your patches in d/changelog
wget http://crates.io/api/v1/crates/foocrate/version/download -O foocrate-version.tar.gz
then follow 1.1
Alternatively you can do it directly in the build directory with quilt:
- In the root directory of
debcargo-conf
,mkdir -p src/foo/debian/patches && ./update.sh foo && ln -srf src/foo/debian/patches build/foo/debian/
(linking it there so it's "sync"ed) cd build/foo
quilt series
to check existing patches,quilt push -a
to test apply all of them, andquilt pop -a
to unapply them,quilt push patch-name
to apply the series until exactly that patchquilt new patch-name.patch
to create a new patch,quilt edit path/to/file
to edit a file with changes saved in current patch,quilt header -e --dep3
to add a DEP-3 patch headerquilt refresh
to update current patch and prevent fuzz (not allowed when building)
You may want to also rm -r build/foo/.pc
if quilt complains about something
you don't recognize.
Note that update.sh
deletes and re-creates build/foo
, so an open terminal
in it needs to go up and down once, into the new directory.
capitol did a nice writeup which can be read here: https://blog.hackeriet.no/packaging-rust-part-II/
built-using-dh-cargo
If you get an error like this:
You must patch build.rs of CRATE to output 'println!(\"dh-cargo:deb-built-using=$lib=\$s={}\", env::var(\"CARGO_MANIFEST_DIR\").unwrap());'
where: $s is 1 if the license(s) of the included static libs require source distribution alongside binaries, otherwise 0"
when building a FFI rust library you need to patch build.rs like stated above. $s is 0 for BSD-like licenses such as MIT and 1 for copyleft licenses like GPL.
Special d/rules overrides
In case you need special overrides for d/rules not provided by debcargo:
touch src/foocrate/debian/rules && ./update.sh foocrate
. This will generate arules.debcargo.hint
alongside it, which you can use as template, similar to the case of d/copyrightcd src/foocrate/debian && cp rules.debcargo.hint rules
- use your favorite editor to edit the rules file to your needs.
# runs all tests on a single thread
override_dh_auto_test:
dh_auto_test -- test -- --test-threads 1
Packaging binary crates
File an ITP for them as this is team policy. Add the following content (as minimum) in debcargo.toml:
summary = "short program summary"
description = """
long, exhaustive description of the program. this equals the extended-description line in debian/control
for "regular" packages.
"""
# the section for the application
[source]
section = rust
Examples to take inspiration from include lsd, bat, alacritty, ...
If a package ships a binary but you only want to use it as library add this stanza in debcargo.toml:
[packages.lib]
bin = false
Packaging a lot of crates
Use the dev/chain_build.py
script. It has a self-explanatory help message,
but basically you run
dev/chain_build.py lowest_dep_1 lowest_dep2 lower_dep_1 lower_dep_2 target_crate
And solve build failures, press enter to continue, solve, continue, repeat, until the target crate is successfully built.
debcargo.toml tweaks
Excluding files
In case you need to exclude certain files from debcargo.toml
, there is an
easy way to do that: just add
#![allow(unused)] fn main() { excludes = ["foo/bar.rs", "bar/non-dfsg-file.c"] }
in debcargo.toml. This has the following usecases:
- Exclude non-dfsg/unnecessary files form the orig tarball (Example: svg-metadata) This is also required for some sys-crates to exclude vendored copies of the C library already in debian
- Exclude broken tests that do not run (when in doubt, ask on #debian-rust) (Example: cxx )
Passing external packages to the buildsystem
If the crate needs external packages (such as -dev libraries) you can also pass
those conveniently via debcargo.toml
: depends = ["libfoo-dev"]
. This is
essentially needed for all -sys crates which provide rust bindings to C
developement libraries such as GTK, for instance.
Passing runtime test dependencies to autopkgtest
In rare cases the autopkgtest can fail on the official runners compared to the local one because it has a slightly different setup. You can do it like this:
test_depends = ["foo"]
That happens very rarely, see cxx
for an example.
Whitelisting files
Sometimes debcargo marks files as suspicious, most of the time those are tests written in C for -sys crates. Whitelist them like this:
whitelist = ["tests/foo.c"]
Example: libadwaita-sys
Pre-release dependencies
Some crates depend on a crate with an alpha/beta version strings. debcargo will
emit an error if that is the case. To allow those deps, pass the following:
allow_prerelease_deps = true
. Do this only if you are sure this will work !
Collapsing features
If a crate has features, collapse_features = true
should be set in
debcargo.toml
. This is strongly recommended. See issue #17 in the debcargo
repo for the reasoning.
Marking feature tests as broken
Let's assume you are building a package and get the following output at the end:
autopkgtest [22:45:42]: test librust-foo-dev:: -----------------------]
autopkgtest [22:45:42]: test librust-foo-dev:: - - - - - - - - - - results - - - - - - - - - -
librust-foo-dev: PASS
autopkgtest [22:45:42]: @@@@@@@@@@@@@@@@@@@@ summary
rust-foo:@ FAIL non-zero exit status 101
librust-foo-dev:default PASS
librust-foo-dev: PASS
librust-foo-dev:serde: FAIL non-zero exit status 101
Tests of serde
and @
features failed. Since the other features (and the
package itself) passes, those need to be marked as flaky so the package can
migrate to testing.
Do it like this:
#serde and @ feature fail the autopkgtest
[packages.lib]
test_is_broken = false
[packages."lib+@"]
test_is_broken = true
[packages."lib+serde"]
test_is_broken = true
Examples: cxx, hashbrown, uom, ...
Do this only if some features fail. If all feature tests fail, read the test logs and look at the upstream test system. Make sure that some tests pass or that the tests aren't meant to be run. Some upstream projects run all test in a specific container or use specific setup. Also some crates are only tested with their default features enabled by upstream. If that's the case mark all tests as broken:
# tests need a postgres container to run
[packages.lib]
test_is_broken = true
Examples: tokio-postgres, ...
Sometime you need to combine some tricks to run at least some tests: Exclude a faulty test, patch dependencies away and mark tests as broken.
More resources
For a full documentation of all keywords available in debcargo.toml
refer to
debcargo.toml.example
in the the debcargo repo.