Week 13: Progress reporting

On the public side of things I did a bunch of things this week which weren't exactly related to the GSoC project which were triggered by incoming mails (and IRC highlights) with bugreports and interesting whichlists, which isn't completed yet as there are two new responses with debug logs waiting for me next week.

I say that as I brushed up a smallish commit for merge this week which is supposed to deal better with triggers in progress reporting. That sounds boring – writing progress reporting stuff – but many people care deeply about it. While this one is more of a cosmetic change its conceptionally a big one: With the actions the planner proposed, apt builds for each package a list of states it will pass through while it is installed/upgraded/removed/purged. Triggers rain in this parade as we don't know beforehand that a package will be triggered. Instead, a status message from dpkg will tell us that a package was triggered, so if we don't want to end up in a state in which apt tells via progress report that it is done, but still has a gazillion triggers to run we have to notice this and add some progress states to the list of the triggered package – easy right? The "problem" starts with packages which are triggered but are destined to be upgraded to. A triggered package will loose its trigger state if it is unpacked, so our progress report has to skip the trigger states if the package is unpacked – we can't just exclude packages if they will be unpacked as it can easily be that a package is triggered, the trigger is acted upon and is upgraded "ages" later in this apt run.

My wip branch contains many more progress related commits as there is a big gotcha in the description above: I said "with the actions the planner proposed", so what about the actions the planner isn't proposing but will happen as part of dpkg --configure/--remove/--purge --pending calls? And what about hook scripts like apt-listbugs/changes which get told the actions to perform their own magic?

The solution is simple: We don't tell dpkg about this, but for our own usage we do trivial expansions of the --pending commands and use these for progress report planning as well as for telling the hookscripts about them. That sounds like a very simple and optional thing, but it was actually what blocked the activation of various config options I had implemented years ago which delay trigger execution, avoid explicit configuration of all packages at the end and all that which I could now all enable – a bit more on that after this hits the master branch. 🙂

I also looked into supporting crossgrades better. In apts conception a crossgrade is the remove of a package of arch A and the installation of a new package (with the same name) of arch B. dpkg on the other hand deals with it like a 'normal' upgrade, just that the architecture of the package changes. The issue with that isn't gigantic usually, but it becomes big with essential packages like if you try to crossgrade dpkg itself with apt: APT refuses to do that by default, but with enough force it will tell dpkg to remove dpkg:A and then it tells dpkg to unpack dpkg:B – just that there is no dpkg anymore which could unpack itself. At least in that case we can skip the remove of dpkg:A, but we can't do it unconditionally as that might very well be some part of an order requirement, so progress reporting should be prepared for either to (not) happen… That isn't finished yet and will surely leak into next week.

Next week will also see my freshly built planner 'dpkg' get a proper tour: With all the --pending calls it seems like a good idea to try to be extra dumb and have a planner just unpack everything in one go, let the rest be covered by --pending calls and see what breaks: Obviously the harder stuff, but I have two directions I would like to explore based on this minimal planner to make it viable. First I have to finish the crossgrading through, my usual self-review of commits and the bugreports I participated in this week want to trigger further actions, too… see you next week!