!chair nirik jnsamyak patrikp

Meeting started at 2026-08-24 15:03:56 UTC

The Meeting Name is now releng

The Meeting name is 'RELENG (2026-08-24)'

morning

Hi.

here with my phone my laptop crashed

patrikp: can you take over just rebooting my system

Yes, sure.

We have a retrospective planned for today so let's get through the first topics quick.

I don't have anything for init. Does anybody else?

not here

Pending reviews maybe?

would like to have: https://forge.fedoraproject.org/releng/tooling/pulls/13051 an eye from you folks as well

BTW, epel is doing branching right now, in case they need help.

Also need a fresh pair of eyes at https://forge.fedoraproject.org/releng/tickets/issues/13441 - there is an edge case I dont seem to understand

that's all from me

I can take it.

yeah quick heads up might be on pto that day

Next up is Beta Freeze tomorrow, we already have a ticket [1] and Samyak self-assigned it, so I think we're good there. Anything to add to that, or anything else for upcoming actions?

[1] https://forge.fedoraproject.org/releng/tickets/issues/13474

nothing atm.

Cool. Would you like to do the retrospective now?

Yes RETRO time :3

Let's start with good things and appreciations first, and then we dive deeper onto things we could make better.

does meetbot have a subtopic?

!subtopic What Went Well?

I guess not so let's do it like this

The branching used automation bits this time, manually cutting down a lot of sshing here and there, and make sure things remains intact through playbooks - jnsamyak

There was no delays, so people didn't have to hold and work late - kevin

Also documentations were good and updated to follow for folks doing it first and second time, so kudos there - jnsamyak

*asking for fas because I could make a better agenda in the end*

Documentation more or less corresponded with reality so we knew what was done and what still had to be done and divided it up between the 3 of us. We worked as a team. - patrikp

🎉 good job everyone on branching, anything else to record here

🎉 good job everyone on branching, anything else to record here?

Then Adam joined it an started saving the day, self-evident, needless to say. 😆

If not, I'll move to the next two to three important sections

2️⃣ What Did Not Go Well

3️⃣ Things That Surprised Us

4️⃣ Manual Steps and Pain Points - things that we can still automate.

Let's go over next one

The rawhide release in bodhi was using 'f46' as branch instead of 'rawhide'. Not sure how that happened, but broke gating - kevin

The releases/44 dirs somehow got permissions moved to 700 which caused a lot of mirror churn, but not sure how that happened - kevin

That was very likely my doing. I am sorry.

thats ok, it's confusing... but it should match the branch in git. :)

I think one point is to make bodhi release section more clearer, this is not happened for the first time, I have seen this before also, either add a warning or big note to make sure to check out the branch name before adding a release or do a releaseinfo and confirm, thios was the reason the critpath tests weren't working and we had to waive and merge faulty updates - jnsamyak

this is a blameless retro, everyone does mistake we just need to make sure how things can be better for future!

There was a unrelated to branching change just before branching that broke composes and caused a lot of work to get composes working again - kevin

this is a blameless retro, everyone does mistake we just need to make sure how things can be better for future.

Yes I want to discuss this in next bit, but same unrelated, and unknown changes should not go just before branching to either critical packages or things touching the releng branching and compose process - jnsamyak

+1

Any more rant for branching to do this can be your dear section ;)

for my point I'll talk about it in next topic more on how we can make it better.

Last branching there was also a massive update that caused us to delay the branching altogether. Not the same thing but related.

But what to do about these things sounds like a next topic thing. I don't have anything else for this one.

But what to do about these things sounds like a next section thing. I don't have anything else for this one.

Okay so we had a suprise and pain point, if that change wouldn't have landed we could have saved adam's side quest for fixing it.

The process itself went more or less smoothly. This was not expected. -patrikp

yeah, we could look at reverting any last minute things like this, but I hope this is a one off type of thing - kevin

I want to have it explicitly mentioned to avoid pushing changes to either critical packages or things touching the releng branching and compose process; This can be handled in two manners, in my opinion

- If the change has to land in anyways, please use the tracker ticket to inform releng about the change has been introduced, there are lots of moving part at the branching day we can look at everything, so person who wants to get those in should let us know Why it is important to get it in before branching? And, it's impact. We can evaluate this request if things have to go in we will wait for branching to start, we have done this.

- But if not explicity mention, releng will unpush all the unfamiliar changes.

If this needs to be run by fesco - I'm happy to take it there.

- jnsamyak :)

`- jnsamyak :)

well, this was a approved change, but the scope of it wasn't clear too much... but also it should have landed much sooner - kevin

But this should be added to email template, can we add this as an Action item please? - Keep / Change / Remove?

Not the problem with the change but the timing it landed on, the branching mail was sent out mentioning it has to happen that day. - jnsamyak

Not the problem with the change but the timing it landed on, the branching mail was sent out mentioning it has to happen that day, so the change could have holded off until we were done there. - jnsamyak

Need opinions on how we can handle such suprises?

Need opinions on how we can handle such suprises? - What should we do differently?

I mean, the template already says it, at least implicitly. It says that we have a ticket open, links to the ticket, tells people that if they have any questions or requests (i.e. a big update) to post it in there, and then informs people that all pending updates will be unpushed. But if it's not clear enough it can be reworded.

We have a tracker ticket [2] open on the Release Engineering issue tracker. In case of any questions and/or requests feel free to leave a comment on the ticket.

Rawhide builds in Koji will be disabled for the duration of Mass Branching. All builds running at that time will be canceled and can be resubmitted by maintainers after the Mass Branching. All Rawhide updates that are pending will be unpushed.

It doesn't explicitly mention Bodhi updates I suppose. It's lumped in with Koji.

Yeah let's reword it, just hard emphasis on unpush :D

well, it wasn't 'during' the mass branching, it was just before. ;(

True...

I think perhaps just a note to check for recent builds/changes in doc would help/

I think perhaps just a note to check for recent builds/changes in doc would help?

And I suppose the author didn't have any questions or requests either so there was nothing to comment in the ticket.

good point, can atleast we can check on things that is used for branching and compose generation

mainly two rpms we have repos and release mainly

Daaaaaaamn. Good luck...

patrikp: can you take care of these action items in your doc update?

Yeah.

okay last bit of retro

Last thing to bring up and then you are all free from retro disco 0/

I am already working on introducing playbooks for koji and dist-git branching but would like to ask your opinions as well - jnsamyak

I think one more bit can be having a playbook for koji release tags automated - jnsamyak

For bodhi releases can we automate those bits? Setting eols, creating a release, updating flatpaks one? - jnsamyak

I don't have answers to these at this time.

sure no rush brainstorm either ping me with those before next meeting, I'll make sure to send out a summary in the next call.

But that's it from me, thank you for your time, and discussing it with me.

Thanks for running it.

Any final thoughts? Otherwise we can close this/

Any final thoughts? Otherwise we can close this.

Nothing from me and Kevin seems to be fighting fires, as firefighters tend to do.

Okay then!