Week 11+12: Where did all the time go

Julian said that in his talk about the last year in APT in regards to (further) fixes to the performance of pdiff and apt in general. I want to say that about the last two week in general through. I was two days at the university (last two weeks of lectures even), a day helping moving, a day away with the young group, a combined day busy with mental and organisational buildup and teardown of that away-day, 5 days building, tearing down and in between manning our wine booth as well as other booths at a festival… Somehow that left only 4 days for day-to-day business like gsoc work… not ideal, especially if the plan isn't working out as it should either. I had to deal with a few issues uncovered by our testsuite caused (on non-amd64 architectures) more or less by the EIPP merge. Mind you, not actual hardcore bugs, but mostly: For some esoteric reason like additional lines in output the testcases are unhappy, but its not reproducible always and absolutely not on your local system.

Turns out that one of them was a deeper issue with how compression methods can be dynamically configured and how the testcases do this to force testing of certain aspects which interfered with me hardcoding xz-compression for EIPP – time-dependent of course as EIPP tends to run faster than /bin/false (no joke, as the errors were pipe write errors which only happen if the program we called closes the pipes aka exits). Sounds simple, but the fix is split over 6 logical commits (which all just change a few lines, but finding them is the trick…). Another dpkgs inability to change the architecture it believes to be native, while apt can do that easily and uses it in the testcases frequently.

Another issue was an a lot bigger bug in which apt ended up changing permissions, ownership & access/modification times on files it should actually only read through symlinks created by the file:// method. Finding and fixing this was an interesting walk in the kernel/libc API park as even if it isn't strictly needed to change the ownership of a symlink, it feels better too avoid to much of special casing if it can be helped and what we really don't want is singling out the Linux kernel against other kernels – I did that once accidentally (#738567) and try to avoid it ever since (that report is related by the way as that call would have been handy if it could have been used here…).

With the weeks so grossly derailed I purred some time into an issue with our relatively new SRV records support which was working only if the first connect would be successful, which is kinda against the point of SRV. Testing this was fun as I was actually convinced I would need a fullblown DNS server setup (much like we need a fullblown webserver setup so much that we implemented our own), but it turned out that its dead simple to add SRV records via dnsmasq, so that was easy to test (after I figured that out that is) at least interactively.

Also I implemented support for "install ./foo.changes" partly in response to the possibility of securing the download of individual debs question in the APT talk – now you can distribute a bunch of debs (and dscs) via a changes file, which can be signed (which the USER has to verify!) and includes the hashes of said files (which apt checks against). Mostly I was doing it for me through as I do local rebuilds at times (like dpkg with a fix for #830267) and picking the individual debs can be a bit annoying and with changes its slightly less bad.

Apart from that I looked into the progress reporting thing as intended and tried a few options, but that isn't all too easy given that apt is expecting to know EVERY action performed… I started with this, but I will probably tell you more about that next week as I fully intend to quickly make up for last two weeks, so I better get started right about now…