18:00:22 <bcotton> #startmeeting F32 Beta Go/No-Go meeting
18:00:22 <zodbot> Meeting started Fri Mar 13 18:00:22 2020 UTC.
18:00:22 <zodbot> This meeting is logged and archived in a public location.
18:00:22 <zodbot> The chair is bcotton. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:22 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:22 <zodbot> The meeting name has been set to 'f32_beta_go/no-go_meeting'
18:00:23 <bcotton> #meetingname F32-Beta-Go_No_Go-meeting
18:00:23 <zodbot> The meeting name has been set to 'f32-beta-go_no_go-meeting'
18:00:28 <bcotton> #topic Roll Call
18:00:42 * coremodule is here
18:00:42 <nirik> morning
18:00:52 <sgallagh> .hello2
18:00:53 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
18:01:39 <mboddu> .hello mohanboddu
18:01:41 <zodbot> mboddu: mohanboddu 'Mohan Boddu' <mboddu@bhujji.com>
18:01:44 <cverna> Hello :)
18:02:03 <bcotton> okay, we're well represented, so let's get started!
18:02:15 <bcotton> i'll skip the boilerplate, y'all remember it from yesterday :-)
18:02:23 <bcotton> #topic Current status - blockers
18:02:24 <bcotton> #link https://qa.fedoraproject.org/blockerbugs/milestone/32/beta/buglist
18:03:27 <bcotton> looks like we verified the one accepted blocker and one of the proposed blockers
18:03:38 <adamw> .hello adamwill
18:03:38 <sgallagh> 1813237 would be a proposed previous release blocker, FTR
18:03:40 <zodbot> adamw: adamwill 'Adam Williamson' <awilliam@redhat.com>
18:03:56 <bcotton> #info BZ 1798792 is an accepted blocker, but is VERIFIED
18:04:19 <bcotton> #info BZ 1812026 is a proposed blocker, but is VERIFIED
18:04:33 <bcotton> #topic (1813237) list of installed modules is not remembered across reboots
18:04:35 <bcotton> #link https://bugzilla.redhat.com/show_bug.cgi?id=1813237
18:04:36 <bcotton> #info Proposed Blocker, PackageKit, MODIFIED
18:04:57 <bcotton> so this was discovered overnight, but i believe it goes to previous release, yes?
18:05:15 <adamw> yeah. we could probably just close it and handle it as part of the existing acceptedpreviousrelease blocker
18:05:35 <bcotton> anyone opposed to this?
18:06:28 <coremodule> +1
18:06:35 <bcotton> #agreed We will handle BZ 1813237 as part of the exisiting AcceptedPreviousRelease blocker
18:06:37 <bcotton> so!
18:06:57 <bcotton> #topic (1804564) Cannot upgrade to Fedora 32: Modules blocking the upgrade path
18:06:58 <bcotton> #link https://bugzilla.redhat.com/show_bug.cgi?id=1804564
18:06:59 <bcotton> #info Accepted Previous Release Blocker, PackageKit, ON_QA
18:07:43 <sgallagh> We have a fix for this that we've tested via COPR builds
18:07:45 <kparal> PackageKit-1.1.12-14.fc31 works fine, I just checked
18:07:49 <sgallagh> Official builds are in-progress
18:08:01 <sgallagh> kparal is fast, that just completed 10 minutes ago :)
18:08:21 <bcotton> #info PackageKit-1.1.12-14.fc31 works as expected
18:08:35 <bcotton> is there an fc30 build, too?
18:09:07 <sgallagh> bcotton: Currently building
18:09:19 <sgallagh> There was a glitch with buildroot overrides
18:09:28 <bcotton> okay, have we tested it unofficially at least?
18:09:52 <sgallagh> adamw, kparal ?
18:10:13 <sgallagh> bcotton: Can we go with "we absolutely will know for sure by Monday"?
18:10:29 <adamw> yeah. if kparal didn't test yet, we will shortly
18:10:35 <bcotton> sgallagh: maybe, but what happens if the sure answer is "nope, still broken"?
18:10:55 <nirik> then we are all slip city. ;)
18:10:56 <sgallagh> Suggest that people do a stepped upgrade instead of jumping direct and fix that for GA?
18:11:01 <kparal> I have tested only the previous broken packagekit on FC30
18:11:14 <bcotton> i don't love it, but that's a not-terrible thing for beta, i suppose
18:11:15 <kparal> which worked, just had some flaws we had to fix
18:11:41 <sgallagh> The fixes for the flaws are identical to the fixes for F31, so I have a high confidence level
18:11:49 <bcotton> #info Fedora 30 package is in the works
18:12:08 <sgallagh> It just finished building, kparal
18:12:15 <sgallagh> If you want to test it during this meeting :)
18:12:20 <kparal> work, work
18:12:31 <sgallagh> zug zug
18:12:35 <bcotton> proposed #agreed Fixes for BZ 1804564 are sufficient and will be in the stable repos in time for the release
18:12:51 <sgallagh> ack
18:12:58 <kparal> sgallagh: I wondered if anybody gets that in text only form :)
18:13:11 <nirik> sgallagh: thanks much for sheparding that fix!
18:13:32 <sgallagh> You're welcome.
18:13:35 <adamw> ack
18:13:40 <sgallagh> (I am never doing that again!)
18:13:48 <adamw> he lied
18:13:49 <bcotton> sgallagh++ to never doing that again :-D
18:13:56 <mboddu> sgallagh: Until f32 final :P
18:14:10 <bcotton> #agreed Fixes for BZ 1804564 are sufficient and will be in the stable repos in time for the release
18:14:50 <bcotton> #topic (1767351) Cannot upgrade to Fedora 32: Modules blocking the upgrade path
18:14:52 <bcotton> #link https://bugzilla.redhat.com/show_bug.cgi?id=1767351
18:14:53 <bcotton> #info Accepted Previous Release Blocker, dnf-plugins-extras, MODIFIED
18:15:13 <bcotton> i think this one is solved, but the package push changed the state deceptively
18:16:30 <adamw> well, the f30 update was never pushed stable yet
18:16:37 <adamw> even the earlier one for the dnf upgrade side only
18:16:40 <adamw> it just didn't get enough karma
18:16:50 <nirik> no one uses f30 anymore. ;)
18:16:50 <adamw> still, we're pretty sure the fixes for that side of things are good
18:16:57 <adamw> we'll re-test and karma and stuff, but i'd say this is addressed
18:17:13 <bcotton> proposed #agreed Fixes for BZ 1767351 are sufficient and will be in the stable repos in time for the release
18:18:26 <nirik> ack
18:18:54 <sgallagh> ack
18:18:55 <sumantro> ack
18:19:34 <bcotton> #agreed Fixes for BZ 1767351 are sufficient and will be in the stable repos in time for the release
18:19:58 <bcotton> #topic Current status - blockers
18:20:08 <bcotton> #info All outstanding blockers are in an acceptable state
18:20:15 <bcotton> #topic Current status - test matricies
18:20:17 <bcotton> #link https://fedoraproject.org/wiki/Category:Fedora_32_Test_Results
18:20:28 <bcotton> so, we've had another day with Beta 1.2. How do we feel about it?
18:21:28 <sgallagh> Best Fedora Beta Ever. Five Stars. Would run it again.
18:21:51 <nirik> better than cats!
18:21:57 <mboddu> Haha :)
18:22:06 <adamw> well now let's not be *hasty*
18:22:07 <bcotton> nirik: the animal or the musical?
18:22:07 <sgallagh> Have you seen the reviews of that movie? That's a damned low bar...
18:22:09 <adamw> better than dogs, sure
18:22:22 <sgallagh> adamw++
18:22:25 <nirik> the musical...
18:23:01 <nirik> anyhow...
18:23:18 <sgallagh> adamw, kparal: How's the coverage?
18:23:24 <adamw> i'm just looking at it now
18:23:30 <adamw> please talk among yourselves
18:23:47 <kparal> I spent the whole day breaking packagekit
18:24:10 <sgallagh> I'm glad he pluralized "yourselves". A lot of people forget that and the implications are mind-boggling and disturbing.
18:25:25 <adamw> not having any ec2 xen coverage makes me v. slightly sad
18:25:34 <adamw> i don't recall if we committed to having any in that whole discussion
18:25:58 <adamw> sgallagh: today's pet grammar peeve: people who start "from X to Y" clauses but don't finish them
18:26:01 <bcotton> iirc, we decided to try both xen and kvm based ec2, but that was a long time ago
18:26:05 <adamw> GIVE ME THE 'TO', DAMNIT
18:26:22 <adamw> they're both in the matrices, i just don't recall how strongly we committed to testing both and when
18:26:40 <adamw> coremodule: any chance you could quickly test on a xen instance?
18:26:54 <coremodule> ahhhhhh
18:26:57 <coremodule> ummmm
18:26:59 <coremodule> lemme see
18:27:45 <coremodule> isn't xen always a pain in our backside?
18:27:57 <sgallagh> No, sometimes it's a pain all-over.
18:28:07 <coremodule> despite guarantees that the xen folks will make sure to test it each release
18:29:24 <adamw> coremodule: more or less, but this was the resolution to that
18:29:32 <adamw> we decided we only care about xen in the context of ec2 instances that are backed by xen
18:29:45 <adamw> we stopped caring about literally installing a xen dom0 on a fedora box and then booting a domU on it
18:30:00 <adamw> ec2 xen instances *should* be less painful. i think.
18:30:07 <adamw> (as amazon gets to deal with most of the xen fun.)
18:30:54 <adamw> aside from that, we're looking pretty good
18:31:14 <adamw> there's just a *few* holes in ARM coverage, but nothing i'd lose sleep over, only a few things and there's no reason they'd differ from x86_64 really
18:31:27 <adamw> a couple of upgrade configs, and printing with a real (not virtual) printer
18:32:47 <bcotton> #info Test coverage is "looking pretty good" apart from a few gaps in ARM coverage and EC2 Xen coverage (which may be done by the time the meeting ends)
18:33:11 <bcotton> any questions, comments, or concerns on test coverage?
18:34:00 <kparal> adamw: sgallagh: I tested F30->F32 upgrade, works fine, with my "test suite"
18:34:05 <bcotton> kparal++
18:34:17 <nirik> cool
18:34:22 <bcotton> #topic Current status - RC
18:34:26 * kparal going afk now
18:34:27 <sgallagh> kparal++
18:34:28 <zodbot> sgallagh: Karma for kparal changed to 6 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
18:34:39 <bcotton> So Beta 1.2 still exists, right? It hasn't been stolen in the dead of night? :-D
18:34:49 <sgallagh> <.<
18:34:52 <sgallagh> >.>
18:35:00 * adamw swallows, looks guilty
18:35:06 <adamw> i, uh, may have eaten it.
18:35:12 <bcotton> was it tasty?
18:35:17 <adamw> in my defence it was right there! and i was hungry!
18:35:22 <adamw> kinda of spiky
18:35:40 <bcotton> hm, that may have been a coronavirus, not an RC
18:35:52 <adamw> *slight cough*
18:36:01 <bcotton> #info Beta 1.2: still a thing
18:36:04 <sgallagh> *panicked vacating of the room*
18:36:09 <sumantro> kparal++
18:36:09 <zodbot> sumantro: Karma for kparal changed to 7 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
18:36:24 <bcotton> okay, so i guess we've covered all the bases and now it's time to make a decision
18:36:32 <mboddu> bcotton: I thought of stealing 1.2, but then I decided not to :P
18:36:40 <bcotton> unless we want to stall while coremodule does the ec2 xen test?
18:36:42 <pwhalen> in my defence for arm coverage, workstation is new for release blocking this release, so dont think we needed those and didnt have time with all the other testing needed
18:36:44 <bcotton> mboddu: we appreciate it
18:37:02 <pwhalen> those being upgrade tests
18:37:30 <adamw> pwhalen: sorry, yeah, should've mentioned that's a bit unclear
18:37:44 <adamw> whether we need to/should worry about workstation upgrades
18:37:47 <pwhalen> f33, fair game.
18:38:21 <bcotton> ideally, we'd cover that for final, but i would 100% buy the "it's not required because it's never been blocking before" interpretation
18:39:32 <bcotton> #topic Go/No-Go decision
18:39:34 <bcotton> I will poll each team. Please reply “go” or “no-go”
18:39:41 <bcotton> FESCo?
18:39:43 <sgallagh> GO!
18:39:43 <nirik> go
18:39:52 <bcotton> #info FESCo is go
18:39:55 <bcotton> Releng?
18:40:06 <mboddu> I still feel like we should punt the release, but from releng perspective we are GO
18:40:23 <bcotton> #info Releng is go
18:40:52 <bcotton> you said 'go' so it counts, but why do you feel like we should punt?
18:41:44 <mboddu> bcotton: Just last minute sheparding makes me feel we might have missed something, thats all
18:41:58 * bcotton nods
18:42:07 <bcotton> mboddu: you officially have the right to say "i told you so" :-)
18:42:23 <mboddu> But I do appreciate all the people who worked hard in the last couple of days
18:42:24 <frantisekz> :)
18:42:50 <bcotton> so that leaves us with QA?
18:42:50 <mboddu> bcotton: I am reserving that right from the last couple of releases (at least 1 IIRC)
18:43:07 <bcotton> mboddu: hold on to them as long as you can and then use them all at once :-)
18:43:33 * adamw sets aside a day to be told so by mboddu
18:44:01 <mboddu> bcotton: If you remember about the missing spins and labs for which I had to rerun the unofficial images...
18:44:19 <bcotton> mboddu: ah yes, that's several ITYSes, i believe :-)
18:44:48 <frantisekz> adamw, I think you deserve to say the magical word on QA behalf
18:45:11 <nirik> we haven't automated that yet?
18:45:12 <bcotton> frantisekz: take his power
18:45:13 * nirik runs
18:45:35 <adamw> =)
18:45:40 <frantisekz> (hmm, after missing most of the meeting... I'd say go :D )
18:45:53 <bcotton> #info QA is go
18:45:55 <bcotton> :-D
18:45:57 <frantisekz> :O
18:46:05 <adamw> i have similar reservations to mboddu, but on policy, we're go
18:46:16 <adamw> and at least i haven't seen any weirdness in manual or openqa testing with 1.2
18:46:23 <frantisekz> I'd like to avoid this late minute testing and fixing for final...
18:46:27 <adamw> nothing specific that makes me nervous. we don't have any weird feedback on updates or lists
18:46:34 <adamw> yeah, i would not want to do final this way
18:46:45 <bcotton> so here's the thing: the policy is more like...guidelines
18:46:47 <adamw> but for beta i think we'll go
18:46:55 <adamw> bcotton: no, the QA team policy is...rules
18:46:56 <adamw> =)
18:46:58 <mboddu> Yeah, I am definitely saying "hard no" for final if we do like this
18:47:14 <bcotton> for sure, we will not be doing this for final. and it will take a lot of cash payments to get me to do this for the next few betas :p
18:47:26 <bcotton> okay, well since no one is willing to pull the ejection handle...
18:47:36 <bcotton> #agreed Fedora 32 Beta is GO
18:47:47 * mboddu goes to work
18:47:48 <bcotton> #info Fedora 32 Beta will release on 2020-03-17
18:47:55 <bcotton> flip those bits, mboddu!
18:48:06 <bcotton> #action bcotton to announce decision
18:48:12 <bcotton> #topic Open floor
18:48:13 <bcotton> Anything else we need to discuss before closing?
18:48:16 <mboddu> bcotton: That has to wait until Monday :)
18:48:59 <nirik> thanks for all the hard work folks
18:49:34 <bcotton> yes, thank you everyone!
18:49:52 <bcotton> sgallagh: it's go home time. tell pfrields i'll fight him if he objects
18:50:06 <adamw> there's 'what do we do about IoT and silverblue' i guess
18:50:25 <mboddu> adamw: Oh right, good question
18:52:04 <pwhalen> iot has been tested as well https://fedoraproject.org/wiki/User:Pwhalen/QA/IoT/Fedora-IoT-32-20200312.0
18:52:22 <coremodule> pwhalen++
18:52:22 <zodbot> coremodule: Karma for pwhalen changed to 4 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
18:52:29 <adamw> i note the empty column for igniton
18:52:36 <adamw> is that working yet?
18:52:47 <bcotton> does ignition need a remix?
18:52:53 * bcotton couldn't help himself
18:53:10 <adamw> .fire bcotton
18:53:10 <zodbot> adamw fires bcotton
18:53:28 <pwhalen> it should work, but I've not yet tested it. Been frantically trying to get through it all.
18:53:55 <pwhalen> during our test day we lost chronyd, which I think is why it failed.
18:54:48 <pwhalen> I need to also write a test case for it, which I'll do today
18:55:39 <adamw> rgr
18:56:23 <bcotton> pwhalen++
18:56:23 <zodbot> bcotton: Karma for pwhalen changed to 5 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
18:56:35 <bcotton> wow, how have i not enkarma'ed pwhalen already?
18:57:46 <bcotton> okay, so are we done here?
18:57:57 <adamw> silverblue?
18:58:03 <cmurf> USS Ship It
18:58:13 <adamw> (assuming for iot the plan is 'ship that nightly if it passes the rest of the tests', i guess)
18:58:15 <bcotton> sliverblue missed the boat, i guess
18:58:16 * cmurf says that most every release at some point
18:58:19 <pbrobinson> chrony is long back, there's been a bunch of fixes for the install/provisioning in the last few days
18:58:21 <bcotton> cmurf++
18:58:49 <adamw> btw, that iot build is probably missing the bits we pulled into beta?
18:58:51 <adamw> like the updated kernel?
18:58:52 <coremodule> cmurf, what about HMS Ship It for those members of the commonwealth?
18:58:53 <nirik> well, there are nightlys for silverblue? or did those fail as well?
18:59:10 <pbrobinson> today's build has the new kernel
18:59:11 <adamw> nirik: sure, if the plan is 'ship a nightly' that's fine, was just thinking it might be good to write down what the plan is...
18:59:23 <nirik> indeed.
18:59:32 <adamw> pbrobinson: right, so, there's a thing: do we re-test and ship that?
18:59:34 <pwhalen> right adamw, today's compose has it, and I will repeat testing on that now
18:59:36 <pbrobinson> adamw: it's on my list, I've been absolutely smashed the last 2 weeks
18:59:53 <adamw> or wait till we push everything that was sideloaded to the beta stable, then test and ship the next nightly after *that*, for IoT and SB?
19:00:21 <cmurf> coremodule: haha UMM HMMM, not sure I want to extend the metaphor too far :D
19:00:24 <adamw> (i will try and get a stable push request for that done today, it depends if everything has sufficient karma yet, if not i'll be bugging people for karma)
19:00:30 <nirik> and if we do ship a nightly do we actually put it somewhere, or do we just say 'go find a nightly and use that' ?
19:00:34 <adamw> right
19:00:35 <pbrobinson> most of the sideloading was mostly gnome? Or was there others?
19:00:40 <adamw> pbrobinson: i don't remember...
19:00:54 <adamw> of course, GNOME is sort of significant for SB :P
19:01:04 <adamw> https://pagure.io/releng/issue/9319 has the list
19:01:06 <pbrobinson> right, and irrelevant for IoT
19:01:50 <cmurf> Silverblue can catch up asynchronously when a successful compose happens? Or not?
19:02:08 <pbrobinson> I'm happy with today's or doing another run/test when the other bits land
19:02:46 <nirik> one important point... if we want a compose with just beta stuff in it, we should NOT open the flood gates on updates stable f32 until after composes tonight.
19:02:54 <mboddu> cmurf: There wont be an image, but they can update based on nightly branched commits
19:02:56 <adamw> nirik: right.
19:03:02 <pbrobinson> it's not hard to do a compose using the sideload tag, I meant to do that yesterday and then got dragged backwards into another slew of meetings
19:03:25 <cmurf> mboddu: ok thanks
19:04:49 <mboddu> I guess once we have the final stable push request, we can run a branched compose, IOT compose and people can submit them as unofficial beta images
19:05:03 <adamw> i'd say: me and mboddu will get a stable push done today, then we won't push anything else and wait for the nightly compose, that should get us a silverblue, IoT can do a compose off that or whatever
19:05:13 <adamw> yeah, that
19:05:16 <pbrobinson> WFM
19:05:17 <nirik> adamw: +1
19:05:30 <nirik> mboddu: or that. :)
19:05:59 <mboddu> Haha :)
19:06:17 <bcotton> +1
19:06:39 <cmurf> you guys remember to turn off kparal? he's gonna find another blocker today if you haven't shut him down
19:07:17 <bcotton> cmurf: too late, once we're go he can't derail us
19:07:30 <pbrobinson> lol
19:07:52 <cmurf> only 52 minutes left in Czech beer hours, maybe he's busy anyway, lightweight tho he is
19:08:20 <adamw> okay, that's all the awkward i had
19:08:25 <cmurf> haha
19:08:29 <adamw> back to debugging mysterious aarch64 machines
19:10:02 <bcotton> alright, time to bang the gavel then
19:10:04 <bcotton> #endmeeting