With Fedora 19 comes a change that is rather bitter-sweet. Yum-presto, the plugin that I originally wrote to allow the use of deltarpms in Fedora, has been retired, its functionality merged directly into yum. While this is a necessary step to achieve things like parallel downloads of deltarpms and regular rpms, it’s still hard to see the death of a project that I’ve been involved with since its beginning, six years ago.

I started using Linux in 2000, switching from Red Hat Linux to Mandrake in 2001, and then to Fedora Core in September 2004, just before Fedora Core 3 was released. I had just returned to the Lebanon Evangelical School as a teacher and sysadmin after finishing university, and I remember downloading the FC3 Test 2 ISO image on our impressively fast 128kbps link. And I still remember my disgust when Test 2 was still quite buggy and I had to download the FC2 ISO.

Somewhere around the end of 2005, I came up with the brilliant idea of running a local mirror of Fedora Core for our school. The problem was that updates took forever to download, so I had a flash of inspiration. What if… we could download only the parts of the rpm that had changed? Updates would be much smaller and running a local mirror would become feasible. With that kind of motivation, I quickly hacked together a brilliantly elegant mess of code that was able to delta two small binary files in sometime under a day and apply the resultant patch in sometime under an hour. Definitely not my proudest coding moment, but I announced my work in the Fedora Forum (which I’m sure made sense to me at the time).

I got a single response to my post. Rahul Sundaram pointed out, much to my disgust, that Michael Schroeder of Suse had already created an efficient and fast (at least compared to my mess of code) program that could create and apply deltas to rpms. This program was cleverly named deltarpm. After looking at deltarpm and verifying that, yes, it did do what I wanted, I then put the whole project on the back burner for a while.

2006 passed. I got married, in the process missing a war in Lebanon, and spent most of rest of the year trying to adapt to the new rules of married life. Apparently leaving my dirty clothes all over the house was no longer acceptable, and there was some difference of opinion on what the definition of a well-balanced breakfast was. I thought that the peanuts in the Snickers bar did a great job of balancing out my Coke, while my new Irish wife thought that my logic needed to be balanced with a sharp smack on the head. She won. She usually does.

Early in 2007, Ahmed Kamal started posting about some work he was doing on a yum plugin that would download deltarpms. This plugin was named yum-presto. In early March, I got involved and started reworking some of the code. By the end of March, we started testing the plugin with Fedora Core 6, and, in early April, Kevin Fenzi sponsored me and reviewed yum-presto for Fedora Extras. We hit bugs. Then we fixed them. We were on a roll.

Then things ground to a halt. We had the client side working reliably, but there was no way that the Fedora infrastructure was going to use my hacky scripts for generating deltarpms. Someone needed to do some work to properly integrate deltarpm generation into Fedora’s infrastructure, but I had no idea how to go about it.

So I ended up in a rut, daily generating deltarpms for our FC6 i386 mirror, but it was anything *but* official. If anyone wanted to use them, they had to manually change the url in fedora-updates.repo. FC6 turned into Fedora 7, then Fedora 8, then Fedora 9, and finally Fedora 10. Jeremy Katz, Casey Dahlin and James Antill all contributed major changes to the client code, with James essentially rewriting the whole thing, but the infrastructure side of things stayed stagnant.

I opened a ticket against bodhi to try to figure out where the deltarpm generation should be happening, and, after a few rounds of pass the parcel, Seth Vidal finally decided to put it straight into createrepo. With that work done in 2009, Fedora 11 was the first release with official deltarpms across all the platforms that Fedora supported.

Four years have passed since Fedora 11 came out. Now I’m the one trying to convince my kids that Snickers and Coke weren’t meant for breakfast. And yum-presto has been mostly in maintenance mode. Last May, Lars Gullik Bjønnes added code to rebuild the rpms in parallel, and, in August, Zdenek Pavlas added code to download drpms in parallel. Both features made it into Fedora 18. I didn’t realize at the time that those would be the last features added to yum-presto.

On February 21, Zdenek posted this on yum-devel:

The native drpm support is complete enough to be used and tested. I’m quite satisfied with the performance, but the applydeltarpm backend needs more work. I’m going to make a rawhide release today.

Yum-presto had just died. And yum was now able to do things with deltarpms that we never could do as a plugin.

I do want to say thank you to everyone who helped test and develop yum-presto. A special thank you to Michael Schroeder, whose work on deltarpm made yum-presto possible. Thank you to the Lebanon Evangelical School for hosting the i386 deltarpms and Angel Marin for hosting the x86_64 deltarpms for the two years it took to get the Fedora infrastructure up and running. And thank you to all those whose hard work makes Fedora the exciting place that it is.

At our school we have around 100 desktops, a vast majority of which run Fedora, and somewhere around 900 users. We switched from Windows to Fedora shortly after Fedora 8 was released and we’ve hit 8, 10, 13, 16, and 17 (deploying a local koji instance has made it easier to upgrade).

As I finished putting together our new Fedora 18 image, there were a few things I wanted to mention.

The Good

1. Offline updates: Traditionally, our systems automatically updated on shutdown. In the 16-17 releases, that became very fragile as any systemctl scriptlets in the updates would block because systemd was in the process of shutting down. Now, with systemd’s support for offline updates, we can download the updates on shutdown, reboot the computer, and install the updates in a minimal system environment. I’ve packaged my offline updater here.
2. btrfs snapshots: This isn’t new in Fedora 18, but, with the availability of offline updates, we’ve finally been able to take proper advantage of it. One problem we have is that we have impatient students who think the reset button is the best way to get access to a computer that’s in the middle of a large update. Now, if some genius reboots the computer while it’s updating, it reverts to its pre-update state, and then attempts the update again. If, on the other hand, the update fails due to a software fault, the computer reverts to its pre-update state and boots normally. Either way, the system won’t be the half-updated zombie that so many of my Fedora 17 desktops are.
3. dconf mandatory settings: Over the years we’ve moved from gconf to dconf, and I love the easy way that dconf allows us to set mandatory settings for Gnome. This continued working with only a small modification from Fedora 17 to Fedora 18, available here and here.
4. Javascript config for polkit: I love how flexible this is. We push out the same Fedora image to our school laptops, but the primary difference compared to the desktop is that we allow our laptop users to suspend, hibernate and shutdown their laptops, while our desktop users can’t do any of the above. What I would really like to do is have the JS config check for the existence of a file (say /etc/sysconfig/laptop), and do different things based on that, but I haven’t managed to work out how to do that yet. My first attempt is here.
5. systemd: This isn’t a new feature in 18, but systemd deserves a shout-out anyway. It does a great job of making my workstations boot quickly and has greatly simplified my initscripts. It’s so nice to be able to easily prevent the display manager from starting before we have mounted our network directories.
6. Gnome Shell: We actually started experimenting with Gnome Shell when it was first included in Fedora, and I switched to it as the default desktop in Fedora 13. As we’ve moved from 13 to 16, then 17, and now 18, it’s been a nice clean evolution for our users. When I first enabled Gnome Shell in our Fedora 13 test environment, the feedback from our students was very positive. “It doesn’t look like Windows 98 any more!” was the most common comment. As we’ve upgraded, our users have only become more happy with it.

The Bad

The bad in Fedora 18 mainly comes down to the one area where Linux in general, and Fedora specifically, is weak – being backwards-compatible. This was noticeable in two very specific places:

1. Javascript config for polkit: While I was impressed with the new javascript config’s flexibility, I was most definitely not impressed that my old pkla files were completely ignored. As a system administrator, I find it frustrating when I have to completely rewrite my configuration files because “now we have a better way”. I’ve read the blog post explaining the reasoning behind the switch to the JS config, but how hard would it have been to either keep the old pkla interpreter, or, if it was really desired, rewrite the pkla interpreter in javascript? The ironic part of this is that the “old” pkla configuration was itself a non-backwards-compatible change from the even older PolicyKit configuration a little less than four years ago.
2. dconf mandatory settings: With the version of dconf in Fedora 18, we now have the ability to have multiple user dconf databases. This is a great feature, but it requires a change in the format of the database profile files, which meant my database profile files from Fedora 17 no longer worked correctly. In fact, they caused gnome-settings-daemon to crash, which crashed Gnome and left users unable to log in. Oops. To be fair, this was a far less annoying change because I only had to change a couple of lines, but I’m still not impressed that dconf couldn’t just read my old db profile files.

As a developer, I totally understand the “I have a better way” mindset, but I think backwards compatibility is still vital. That’s why I love rsync and systemd, but have very little time for unison (three different versions in the Fedora repositories because newer versions don’t speak the same language as older versions).

I know some people will say, “If you want stability, just use RHEL.” That’s fine, but I’m not necessarily looking for stability. I like the rate of change in Fedora. What I dislike is when things break because someone wanted to do something different.

All in all, I’ve been really happy with Fedora as our school’s primary OS, and each new release’s features only make me happier. Now I need to go fix a regression in yum-presto that popped up because of some changes we made because we wanted to do something different.