One of the main jobs of a package manager like apt is to download packages (ideally in a secure way) from a repository so that they can be processed further – usually installed. FSVO "normal user" this is all there ever is to it in terms of getting packages.

Package maintainers and other users rolling their own binary packages on the other hand tend to have the packages they want to install and/or play-test with already on their disk. For them, it seems like an additional hassle to push their packages to a (temporary) repository, so apt can download data from there again… for the love of supercow, there must be a better way… right?

For the sake of a common start lets say I want to modify (and later upload) hello, so I acquire the source via apt source hello. Friendly as apt is it ran dpkg-source for me already, so I have (at the time of writing) the files hello_2.10.orig.tar.gz, hello_2.10-1.debian.tar.xz and hello_2.10-1.dsc in my working directory as well as the extracted tarballs in the subdirectory hello-2.10.

Anything slightly more complex than hello probably has a bunch of build-dependencies, so what I should do next is install build-dependencies: Everyone knows apt build-dep hello and that works in this case, but given that you have a dsc file we could just as well use that and free us from our reliance on the online repository: apt build-dep ./hello_2.10-1.dsc. We still depend on having a source package built previously this way… but wait! We have the source tree and this includes the debian/control file so… apt build-dep ./hello-2.10 – the later is especially handy if you happen to add additional build-dependencies while hacking on your hello.

So now that we can build the package have fun hacking on it! You probably have your preferred way of building packages, but for simplicity lets just continue using apt for now: apt source hello -b. If all worked out well we should have now (if you are on a amd64 machine) also a hello_2.10-1_amd64.changes file as well as two binary packages named hello_2.10-1_amd64.deb and hello-dbgsym_2.10-1_amd64.deb (you will also get a hello_2.10-1_amd64.buildinfo which you can hang onto, but apt has currently no way of making use of it, so I ignore it for the moment).

Everyone should know by now that you can install a deb via apt install ./hello_2.10-1_amd64.deb but that quickly gets boring with increasing numbers, especially if the packages you want to install have tight relations. So feel free to install all debs included in a changes file with apt install ./hello_2.10-1_amd64.changes.

So far so good, but all might be a bit much. What about install only some debs of a changes file? Here it gets interesting as if you play your cards right you can test upgrades this way as well. So lets add a temporary source of metadata (and packages) – but before you get your preferred repository builder setup and your text editor ready: You just have to add an option to your apt call. Coming back to our last example of installing packages via a changes file, lets say we just want to install hello and not hello-dbgsym: apt install --with-source ./hello_2.10-1_amd64.changes hello.

That will install hello just fine, but if you happen to have hello installed already… apt is going to tell you it has already the latest version installed. You can look at this situation e.g. with apt policy --with-source ./hello_2.10-1_amd64.changes hello. See, the Debian repository ships a binary-only rebuild as 2.10-1+b1 at the moment, which is a higher version than the one we have locally build. Your usual apt-knowledge will tell you that you can force apt to install your hello with apt install --with-source ./hello_2.10-1_amd64.changes hello=2.10-1 but that isn't why I went down this path: As you have seen now metadata inserted via --with-source participates as usual in the candidate selection process, so you can actually perform upgrade tests this way: apt upgrade --with-source ./hello_2.10-1_amd64.changes (or full-upgrade).

The hello example reaches its limits here, but if you consider time travel a possibility we will jump back into a time in which hello-debhelper existed. To be exact: Right to the moment its maintainer wanted to rename hello-debhelper to hello. Most people consider package renames hard. You need to get file overrides and maintainerscripts just right, but at least with figuring out the right dependency relations apt can help you a bit. How you can feed in changes files we have already seen, so lets imagine you deal with with multiple packages from different sources – or just want to iterate quickly! In that case you want to create a Packages file which you would normally find in a repository. You can write those by hand of course, but its probably easier to just call dpkg-scanpackages . > Packages (if you have dpkg-dev installed) or apt-ftparchive packages . > Packages (available via apt-utils) – they behave slightly different, but for our proposes its all the same. Either way, ending up with a Packages file nets you another file you can feed to --with-source (sorry, you can't install a Packages file). This also allows you to edit the dependency relations of multiple packages in a single file without constant "fiddle and build" loops of the included packages – just make sure to run as non-root & in simulation mode (-s) only or you will make dpkg (and in turn apt) very sad.

Of course upgrade testing is only complete if you can influence what is installed on your system before you try to upgrade easily. You can with apt install --with-source ./Packages hello=2.10-1 -s -o Dir::state::status=/dev/null (it will look like nothing is installed) or feed a self-crafted file (or some compressed /var/backups/dpkg.status file from days past), but to be fair that gets a bit fiddly, so at some point its probably easier to write an integration test for apt which are just little shellscript in which (nearly) everything is possible, but that might be the topic of another post some day.

Q: How long do I have to wait to use this?

A: I think I have implemented the later parts of this in the 1.3 series. Earlier parts are in starting with 1.0. Debian stable (stretch) has the 1.4 series, so… you can use it now. Otherwise use your preferred package manager to upgrade your system to latest stable release. I hope it is clear which package manager that should be… 😉︎

Q: Does this only work with apt?

A: This works just the same with apt-cache (where the --with-source option is documented in the manpage btw) and apt-get. Everything else using libapt (so aptitude included) does not at the moment, but potentially can and probably will in the future. If you feel like typing a little bit more you can at least replicate the --with-source examples by using the underlying generic option: aptitude install -s hello-dbgsym -o APT::Sources::With::=./hello_2.10-1_amd64.changes (That is all you really need anyhow, the rest is syntactic sugar). Before you start running off to report bugs: Check before reporting duplicates (and don't forget to attach patches)!

Q: Why are you always typing ./packages.deb?

A: With the --with-source option the ./ is not needed actually, but for consistency I wrote it everywhere. In the first examples we need it as apt needs to know somehow if the string it sees here is a package name, a glob, a regex, a task, … or a filename. The string "package.deb" could be a regex after all. And any string could be a directory name… Combine this with picking up files and directories in the current directory and you would have a potential security risk looming here if you start apt in /tmp (No worries, we hadn't realized this from the start either).

Q: But, but, but … security anyone?!?

The files are on your disk and apt expects that you have verified that they aren't some system-devouring malware. How should apt verify that after all as there is no trustpath. So don't think that downloading a random deb suddently became a safe thing to do because you used apt instead of dpkg -i. If the dsc or changes files you use are signed and you verfied them through, you can rest assured that apt is verifying that the hashes mentioned in those files apply to the files they index. Doesn't help you at all if the files are unsigned or other users are able to modify the files after you verified them, but apt will check hashes in those cases anyhow.

Q: I ❤︎ u, 🍑︎ tl;dr

Just 🏃︎ those, you might 😍︎ some of them:

apt source hello
apt build-dep ./hello-*/ -s
apt source -b hello
apt install ./hello_*.deb -s
apt install ./hello_*.changes -s
apt install --with-source ./hello_*.changes hello -s
apt-ftparchive packages . > ./Packages
apt upgrade --with-source ./Packages -s

P.S.: If you have expected this post to be published sometime inbetween the last two months… welcome to the club! I thought I would do it, too. Lets see how long I will need for the next one… I have it partly written already, but that was the case for this one as well… we will see.

This blog has no centralized comment section. Feel free to comment on it in your own blog, some other page or channel you are on to discuss stuff and/or contact me by mail. I am happy to add pointers and updates as needed.