Last week, I had the opportunity to be at Flock, Fedora’s contributer conference, in Dresden. As I was preparing to leave for Flock, I seriously wondered whether it was going to be worth it, given that I’ve just moved countries and am still in the process of getting settled in and of job-searching. But as I got on the plane after it was over, I knew that it was definitely worth it! This year was, by far, the best Flock I’ve been too. There were a number of great talks, but the thing I appreciated the most was the sense of community evident at the conference.

Over the last few months, I’ve been working on getting Fedora’s metadata zchunked, so this was a great chance to meet with Igor Gnatenko and Neal Gompa, who have both been helping me. I was also able to talk with some of the DNF and Fedora Infrastructure guys, and got a lot of good feedback. A huge thank you to everyone who listened (sometimes unwillingly, I’m sure) to me talk about zchunk! (And kudos to Randy Barlow and Jeremy Cline who are thinking about doing a zchunk implementation in Rust!)

Talks

I made it to quite a number of talks, but there were three that really stood out to me.

The power of one: For the good of the community - Rebecca Fernandez

This was a great talk, mainly because it focuses on the soft skills that we geeks can sometimes be very weak on. She talked about how important it is to speak up, especially when someone is being offensive in the community and gave great ideas on how to speak up in a way that moves the conversation forward rather than increasing drama. She didn’t give us any magic wands that will supernaturally change people’s behavior, but, as a teacher who’s been stuck mediating many a student dispute, I think her methods are far more effective than most realize.

I’d love to be able to point to a video of her talk, but it doesn’t look like it’s online yet.

Ansible

I made it to a couple of the ansible talks, and they were quite fascinating. We’ve been using ansible in the school for workstation deployment for a few years now, but the talks quickly made it clear how little I actually know about ansible.

The first revelation was that you can use templates for configuration files, something that I had never caught onto. You can create a template for foo.conf, set a {{ variable }} in the template, and then watch as the the variable is replaced in the deployed file. This is the kind of thing that makes deployment so much easier.

We’ve been using roles for our ansible playbooks since the beginning, but my second revelation was that you could change the default task that runs in a role. When running a role, normally the default task is main.yaml, but, by setting the tasks_from: attribute in include_role, you can change that to any task you want. This allows us to specify different sets of tasks within a role, so we can just run the correct set when including the role.

The final revelation is that ansible allows you to create plugins that run on the controller for various purposes. These plugins can be used for purposes that range from writing a custom connection method for connecting to your hosts, to creating a custom strategy for running ansible on the hosts, to loading external data into your playbooks. This allows you to extend ansible far beyond its normal limitations.

The beauty of ansible is that you don’t have to be an expert to create functional playbooks, but it’s nice to see ways that I could make our playbooks more efficient.

Modularity

My last post deals with my experience building a module for Fedora, so I’m not going to spend much time discussing it, but I do have a lot of hope for where modularity could take us, especially as it relates to server packaging.

For those who want to try my LizardFS development module, on F28 server or Rawhide, run the following:

# dnf --enablerepo=updates-testing-modular module enable lizardfs:devel
# dnf --enablerepo=updates-testing-modular list "lizardfs*"

You’ll see and be able to install the 3.13.0 release candidate packages rather than the default 3.12 packages. To go back to 3.12, just run:

# dnf --enablerepo=updates-testing-modular module disable lizardfs:devel
# dnf --enablerepo=updates-testing-modular distro-sync "lizardfs*"

Once the LizardFS module has been in updates-testing-modular for a week, I’ll push it to stable and you’ll be able to remove “–enablerepo=updates-testing-modular” from your commands.

Lightning talks

I also gave a lightning talk explaining how zchunk works, and got to hear a lot of quick interesting things that people are working on. Luka described a new project called ignition that’s meant to be an interpreter-free replacement for cloud-init, Praveen shared about building a Fedora minishift ISO, Florian made an interesting proposal to remove the changelog from the spec file and automatically generate it from git, and Adam S. spoke about using Fedora containers in MacOS. Adam’s proposal is to try to make Fedora the default choice when a developer wants to use containers on MacOS, which could greatly expand the userbase.

Evening events

There were a few evening events, but the two that stuck out were the Dresden treasure hunt and the Roller Coaster Restaurant. For the treasure hunt, we were placed in teams of seven or eight and sent around the city center, filling in a “treasure” map. I was lucky enough to be on a team that included Matej, who was in charge of AV at Flock and is quite competitive, along with a couple of guys from Dresden. With Matej pushing us forward and the local guys acting as guides, we were the first team finished (though we came in second overall due to missing bonus tasks).

The second memorable event was the dinner organized at the Roller Coaster Restaurant. I was mildly concerned that we would be expected to eat on a roller coaster, but it turns out that the food is sent on a roller coaster to you. My first thought was, “What could possibly go wrong?” The answer, as it turns out, is that a beer bottle might just fall from five meters up, you might have flaming sparklers dropping sparks on your head, and your food might get stuck. Other than that, though… In all seriousness, it was a fun place to hang out at, and I got the opportunity to chat with Igor a bit and then with Zbigniew for quite a while. Definitely made up for the slight risk of having a bottle dropped on your head or being lit on fire.

Summary

I feel like this year’s Flock was the best that I’ve been to yet. I got the opportunity to get to know a number of the people that I normally only see on IRC or the mailing lists. Putting faces to names and getting a feel for personalities is important, so I’m really glad that I was able to make it again this year! I would strongly recommend that anyone who wants to be part of the Fedora community makes a point of being there if at all possible. A huge thank you to everyone involved in putting Flock together!

Today is the last day of Flock in Dresden, and it has been a really good Flock! On Thursday, I got the opportunity to build my first module for Fedora Modularity, and I just wanted to document the experience. As things go, it was quite easy, but there were a couple of things that tripped me up.

Please note that the tooling is still being worked on, so if you’re reading this much after it was published, some things will have probably changed.

Background

First, for those who haven’t been following, Modularity is a system that allows the user to select different streams of software in the same release. Traditionally in Fedora, there’s only one available version of any given package. There are situations where this can be limiting (say, if you need a particular version of a certain Python framework, or if you want to test drive the latest release of a package without upgrading your whole system to something unstable), and Modularity tries to fill this gap.

I thought this might be useful for LizardFS in Fedora. The current stable version of LizardFS is 3.12.0 (available in all active Fedora releases and in EPEL), but a release candidate for 3.13.0 has been published, and I thought it would be useful for those who actively want to test LizardFS to be able to install it.

On Thursday, I went to the Expert Helpdesk: Module Creation session and noticed that the room was fairly full. I was a bit concerned that I might not get the expert help that I needed, but that worry was put to rest when I found out that I was the only person in the room that hadn’t built a module before.

A certain Modularity developer (who shall remain unnamed to protect the guilty) volunteered me to go to the front and plug my laptop into the projector so the room full of experts could watch and advise me as I built my first module. No pressure at all.

I was advised that for my suggested use-case, I should just keep packaging the stable releases in Fedora as usual, but create a devel module stream for the unstable releases.

The process

The first step in building a module is to read the documentation. If you’re like me, you’ll read the first step, see that it talks about getting your package into Fedora, and quickly move to the next step. It turns out, though, that you do need to go back and read that section because you’ll need to create a new branch in dist-git for each module stream you are going to create.

Branching

One thing that isn’t (at the time of writing) documented is that, when requesting a branch, you must specify a service level, even though it seems that these aren’t going to be used. The date must end in 12-01 or 06-01, so I just made something up.

To request my branches, I ran:

fedpkg --module-name=lizardfs request-branch devel --sl rawhide:2020-12-01

Mohan approved my requests in record time (there are advantages to having everybody watching you), and I was ready to work with both dist-git, and the modulemd git repo. I pushed the release candidate to my new devel branch in dist-git, and then it was time to create the module definition.

Creating the module definition

Step two, creating the module definition, should have been straightforward, but there was talk about using fedmod to automate the process, so I installed it, ran into problems getting it working, and we decided it would be easier to just generate the modulemd file manually. I used the minimum template here as a starting point, saving it as lizardfs.yaml in the modulemd git repo. It was pretty simple to setup and the only slightly tricky thing to remember is that the license is the license of the module definition, not the package itself.

One thing I didn’t do, but really need to, is to setup some profiles for LizardFS. A profile is a group of packages that fit a use-case, and make it easier for the end-user to install the packages they need.

Building the module

Once I had pushed the modulemd, it was time to build! I pushed the build and waited… and waited… The way my module was defined, it was going to be built for every Fedora release that has modules (currently F28 (Server) and F29), so it took a while. Our time was running out, and we had a walking tour and scavenger hunt in Dresden (a brilliant idea, huge thanks to the organizers!), so that was the end of the workshop.

While on the scavenger hunt (our team placed second, thanks to some very competitive members of the team), I got a notification that the module build had failed. I found that I was missing a dependency, so I fixed that and rebuilt, but ran into a rather strange problem: it tried to redo the first build rather than doing a new one.

It turns out that, as documented, you need to push an empty commit to the modulemd git repo in order for it to build from the latest dist-git repo.

I had another failed build because lizardfs-3.13.0 doesn’t build against 32-bit architectures, so I wrote a small patch to fix that, pushed another empty commit to the modulemd git repo, and finally managed to build my very first module! Cue slow clap.

Thoughts

The module building process is actually quite simple and being able to build a module stream for all active Fedora releases simplifies the workload quite a bit. I’m seriously thinking about retiring LizardFS in Rawhide and only providing it via modules (with the stable stream being the default stream that you would get if you ran dnf install lizardfs-client).

The workflow is still a bit clunky, especially with having to manage both dist-git and the modulemd git repo, and I’d love to see that simplified, if possible, but it’s not nearly as difficult as I thought it might be.

A huge thanks to everyone (even Stephen) who put the time and effort into making modules work! I think they have a lot of potential in making the distribution far more relevant to developers and users alike. And a huge thanks to all the experts at the session. I know how hard it is to sit and watch someone make mistakes as they try to use something you’ve created, and you all were very patient with me.