17:04:39 <amoloney> #startmeeting F39 Final Go/No-Go meeting 17:04:39 <zodbot> Meeting started Thu Nov 2 17:04:39 2023 UTC. 17:04:39 <zodbot> This meeting is logged and archived in a public location. 17:04:39 <zodbot> The chair is amoloney. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions. 17:04:39 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 17:04:39 <zodbot> The meeting name has been set to 'f39_final_go/no-go_meeting' 17:04:39 <saluki> .hello saluki 17:04:41 <zodbot> saluki: saluki 'Seth Maurice-Brant' <fedoraproject.kwudj@passinbox.com> 17:04:50 <Son_Goku> .hello ngompa 17:04:51 <zodbot> Son_Goku: ngompa 'Neal Gompa' <ngompa13@gmail.com> 17:04:51 <thebeanogamer> .hello thebeanogamer 17:04:55 <zodbot> thebeanogamer: thebeanogamer 'Daniel Milnes' <daniel@daniel-milnes.uk> 17:05:03 <amoloney> #meetingname F39-Final-Go_No_Go-meeting 17:05:03 <zodbot> The meeting name has been set to 'f39-final-go_no_go-meeting' 17:05:07 <geraldo_simiao> .hello geraldosimiao 17:05:08 <zodbot> geraldo_simiao: geraldosimiao 'Geraldo S. Simiรฃo Kutz' <geraldo.simiao.kutz@gmail.com> 17:05:25 <amoloney> #topic Roll Call 17:05:30 <coremodule> .hello2 17:05:31 <zodbot> coremodule: coremodule 'Geoffrey Marr' <gmarr@redhat.com> 17:05:58 <nirik> morning again 17:06:28 <amoloney> evening :) 17:06:30 <decathorpe> .hi 17:06:31 <zodbot> decathorpe: decathorpe 'Fabio Valentini' <decathorpe@gmail.com> 17:06:38 <adamw> .hello adamwill 17:06:39 <zodbot> adamw: adamwill 'Adam Williamson' <awilliam@redhat.com> 17:06:43 <neil> o/ 17:06:45 <neil> hi 17:06:46 <neil> .hi 17:06:47 <zodbot> neil: neil 'Neil Hanlon' <neil@shrug.pw> 17:07:08 <Son_Goku> I guess I should say it again for the "roll call" 17:07:12 <Son_Goku> .hello ngompa 17:07:13 <zodbot> Son_Goku: ngompa 'Neal Gompa' <ngompa13@gmail.com> 17:08:07 <adamw> davdunc: pre-emptive ping for later 17:08:19 <amoloney> ok 3 mins is enough for roll call I think 17:08:29 <amoloney> we have important business to be about! 17:08:32 <neil> :D 17:08:41 <amoloney> #topic Purpose of this meeting 17:08:48 <adamw> amoloney: i'd chair a couple of people at this point, just in case you lose your connection or whatever 17:08:55 <adamw> always good to have backup chairs 17:09:10 * nirik can override and add chairs if needed too. ;) 17:09:44 <amoloney> crap now I have to admit I can never remember the way to do that...I think its #chair<someone>? 17:10:01 <adamw> yeah 17:10:04 <adamw> well, with a space 17:10:05 <amoloney> excellent 17:10:08 <amoloney> tag youre it 17:10:14 <amoloney> #chair adamw 17:10:14 <zodbot> Current chairs: adamw amoloney 17:10:17 * mattdm_not_sus picks up a paving stone to place on the accelerator pedal 17:10:20 <amoloney> #chair nirik 17:10:20 <zodbot> Current chairs: adamw amoloney nirik 17:10:37 * Son_Goku stares at mattdm_not_sus 17:10:41 <amoloney> anyone else want to be added as chair? 17:11:42 <amoloney> Ok ping if you change your mind. We shall continue then... 17:11:45 <adamw> Son_Goku: seems like very mattdm-like behaviour 17:11:55 <amoloney> #info Purpose of this meeting is to check whether or not F39 Final is ready for shipment, according to the release criteria. 17:12:21 <amoloney> #info This is determined in a few ways: 17:12:58 <amoloney> #info 1. Release candidate compose is available 17:13:08 <amoloney> #info 2. No remaining blocker bugs 17:13:19 <amoloney> #info 3. Test matrices are fully completed 17:13:34 <amoloney> #info 4. Fedora CoreOS and IoT are ready 17:13:59 <amoloney> and with that... 17:14:05 <amoloney> #topic Current status - RC 17:14:33 <adamw> well, we've got one! 17:14:45 <adamw> https://dl.fedoraproject.org/pub/alt/stage/39_RC-1.5/ is the current RC 17:15:03 <adamw> it addresses all current *accepted* blockers except the shim one (2113005) 17:15:26 <amoloney> which has been a known blocker for several releases, correct? 17:15:34 <pjones> correct 17:16:05 <nirik> sadly so 17:16:24 <adamw> right 17:16:48 <amoloney> silver lining would be this is not a blocker for this one then either though! Glass half full and all that 17:17:29 <adamw> yes, basically. we can vote to waive it later 17:17:30 <nirik> yeah, we usually accepted it as a blocker then waived it as 'too hard/long to fix' 17:17:41 <amoloney> ack 17:18:02 <nirik> note that we are missing some artifacts in RC-1.5... but we can ignore that unless we are go and discuss later. 17:19:23 <amoloney> If theres nothing we need to talk about on the RC Im going to info it and move to the next item 17:19:25 * dustymabe ๐ 17:20:02 <amoloney> #info RC-1.5 is the current release candidate 17:20:21 <nirik> +1 move on 17:20:37 <amoloney> #topic Current status - blockers 17:21:11 <amoloney> #Link https://qa.fedoraproject.org/blockerbugs/milestone/39/final/buglist 17:21:18 <amoloney> #chair adamw 17:21:18 <zodbot> Current chairs: adamw amoloney nirik 17:22:24 <adamw> ahoy 17:22:29 <adamw> time for mini blocker review! 17:22:39 <dustymabe> ๐ 17:22:42 <adamw> #topic (2246871) F39 Server Edition for aarch64 SBC disk image is not bootable 17:22:42 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=2246871 17:22:42 <adamw> #link https://pagure.io/fedora-qa/blocker-review/issue/1429 17:22:42 <adamw> #info Proposed Blocker, arm-image-installer, NEW 17:23:28 <pboy> Well, most of the issue could be cleared. 17:23:31 <nirik> So, it seems this may have been something particualr to pboy's system? but we don't know what? 17:23:40 <nirik> hey pboy! 17:23:52 <pboy> nirik no it is not articular to my system, but to server. 17:24:11 <pwhalen> Seems so, I will try to work it out and fix it, but its fine for me on multiple installations. 17:24:11 <pboy> But we know now the reason 17:24:11 <nirik> so any server install? thats... odd. There's not many differences... 17:24:22 <adamw> so...this one is a bit messy. i have attempted to summarize the current status, to the best of my knowledge, in https://bugzilla.redhat.com/show_bug.cgi?id=2246871#c35 . 17:24:48 <adamw> nirik: the significant thing, I think, is that Server still uses LVM by default. no other edition does. 17:25:09 <pboy> nirik: I you use Server to cfeate the disk image, it is always non-bootlbe for non-RPi devides, ยด. g you use workstation yuou always get bootable image. 17:25:16 <pwhalen> adamw: summary is great. I have one system that uses lvm and fedora as the vg to test this. 17:25:17 <adamw> having a VG (or LV, I forget which) called 'fedora' makes things tricky for the image installer script because the image also contains a VG called 'fedora'. 17:25:28 <nirik> yeah, lv/xfs... is one of the few differences 17:25:46 <adamw> so the script has to go down a different codepath which tries to handle that, which explains why you can get a different result in this case. 17:26:04 <adamw> pwhalen, have you tried writing an f39 server image on that system yet? can you reproduce pboy's issue that way? 17:26:13 <nirik> so, in theory we could fix this in a 0 day update of arm-image-installer right? 17:26:14 <pwhalen> yes 17:26:14 <adamw> i was going to try it this morning but sort of ran out of time trying to juggle work on all three blockers 17:26:32 <pwhalen> adamw: yes, on both sysytems. One with lvm and vg fedora, the other btrfs. 17:26:48 <pwhalen> nirik: yes. 17:26:50 <adamw> pwhalen: and did you see the problem trying to write the image on the lvm system? 17:27:05 <pwhalen> adamw: no. 17:27:14 <adamw> yayyyy 17:27:24 <pboy> Well, for me that is no longer a blocker. We know, just use workstation live media on your server and it works. 17:27:27 <nirik> Love those heisenbugs 17:28:00 <nirik> I think I am ok saying -1 blocker but we should still try and get to the bottom of it and fix it if we can... 17:28:16 <pboy> nirik +1 17:28:56 <nirik> perhaps it's worth making a debug version of the installer and having pboy run that and see more clearly what goes wrong... just a thought. 17:29:01 <pwhalen> this bug also conflates multiple issues. There is another issue with lvm that has nothing to do with the script . 17:29:23 <adamw> yeah, my overall sense is that nothing here needs to block the release. we can commonbugs that you should remove the /etc/lvm/devices file if necessary, and try to fix whatever bugs may remain in the script as we go along. 17:29:44 <adamw> nirik: the 'debug version' is just 'run it with sh -x'. :P 17:29:48 <nirik> yeah, that sounds like a installer issue... it should wipe that 17:29:50 <geraldo_simiao> Common bugs +1 17:30:02 <nirik> sure 17:30:06 <pboy> The think is now, how serious we take the ARM SBC. The current image is inferior according to Server installation media standards. 17:30:33 <pboy> But it is OK as a Nerd toy,## 17:31:07 <Son_Goku> Common bugs +1 17:31:11 <Son_Goku> and just let it go at this point 17:31:32 <nirik> ideally we would fix things like this, but in order to do that we probibly need to add more test cases and test eariler next time... 17:32:41 <adamw> pboy: i agree with peter and disagree with you that a relatively minor bug like a leftover file that affects operations you don't need to do on first boot makes the image a "toy". 17:32:44 <pboy> nirik yes, probably it is not worth it in the current time frame. 17:32:54 <lruzicka_> nirik, if only people who use it would report their findings 17:33:11 <adamw> nirik: i think it probably would make more sense to wipe it at image build time, but we could potentially see if the script can remove it as a workaround 17:33:26 * nirik nods 17:33:27 <pboy> adamw We would never accept this in our x86 DVD installation media!! 17:33:33 <adamw> i think we probably would 17:33:38 <pwhalen> right 17:33:39 <Son_Goku> yeah, we have before 17:33:52 <Son_Goku> technically we still are with the shim thing 17:34:12 <adamw> well, that's a waived blocker. but anyhow 17:34:18 <nirik> anyhow, more discussion? voting? 17:34:21 <adamw> it seems like there's mostly a consensus here 17:34:24 * Son_Goku mumbles deprecations about the shim thing 17:34:25 <adamw> -1 blocker 17:34:32 <Son_Goku> -1 FB 17:34:37 <thebeanogamer> -1 blocker 17:34:40 <nirik> -1 blocker 17:34:47 <lruzicka> -1 blocker 17:34:56 <mattdm_not_sus> -1 blocker 17:37:28 <adamw> proposed #agreed 2246871 - RejectedBlocker (Final) - this bug kinda rolls up two issues: possible problems writing the Server image from a system with a VG called 'fedora' or writing the image multiple times to the same media, and problems caused by a file on the image that shouldn't be there. as best we understand this right now, nothing here is bad enough to block the release on 17:37:33 <adamw> sorry, that's a bit hard to write concisely :)O 17:38:43 <lruzicka> ack 17:38:50 <geraldo_simiao> ack 17:39:07 <nirik> ack 17:39:31 <coremodule> ack 17:39:41 <thebeanogamer> ack 17:39:54 <adamw> #agreed 2246871 - RejectedBlocker (Final) - this bug kinda rolls up two issues: possible problems writing the Server image from a system with a VG called 'fedora' or writing the image multiple times to the same media, and problems caused by a file on the image that shouldn't be there. as best we understand this right now, nothing here is bad enough to block the release on 17:41:43 <adamw> #topic (2247611) [aarch64] F39 RC1.5 unxzed images too large 17:41:44 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=2247611 17:41:44 <adamw> #link https://pagure.io/fedora-qa/blocker-review/issue/1432 17:41:44 <adamw> #info Proposed Blocker, distribution, NEW 17:41:44 <adamw> #info Ticket vote: FinalBlocker (+0,0,-4) (-adamwill, -kevin, -lruzicka, -frantisekz) 17:42:05 <adamw> so, well, this one's kind of a bit murky (I did a bit more research since my ticket vote) 17:42:11 <adamw> see https://pagure.io/fedora-qa/relval/issue/27 17:42:42 <adamw> basically, it seems like we were sort of relying on the `disk_size` limit in fedora pungi config to enforce these size restrictions...but at some point we institutionally forgot about that, and have just been bumping those sizes when the images fail to compose 17:43:41 <nirik> oof. Someday we will get good at this releasing things process. 17:43:45 <adamw> the disk_size for workstation got bumped from 14 to 15 and then from 15 to 16 in the last few years. also, those sizes are specified in power-of-two gigabytes. (I think perhaps we picked 14 because 14 power-of-two GiB is under 16 power-of-ten GB) 17:44:40 <lruzicka> adamw, Did not we want them to be copyable to certain flash sizes? 17:44:44 <adamw> so...i think i might actually want to accept this as a blocker, but waive it on the basis we've kinda messed this up lately and it's way too late to try and slice the workstation image back down to 14 for f39 17:44:57 <mattdm_not_sus> yes to that 17:44:58 <adamw> lruzicka: yes, that's why I said "(I think perhaps we picked 14 because 14 power-of-two GiB is under 16 power-of-ten GB)" 17:45:07 <nirik> yeah, I was just thinking that... I think we need to sort this out better, but it shoudl not be right now. 17:46:02 <Son_Goku> +1 FB 1+ waive as too-late-and-too-hard 17:46:04 <nirik> also note that if some release blocking image goes over size in rawhide, it's breaking all composes until addressed and sometimes it's hard to do process to increase the size. 17:47:04 <mattdm_not_sus> 32GB flash drives are down to $2.99 in single quantities at microcenter 17:47:12 <Son_Goku> wow... that's cheap 17:47:20 <mattdm_not_sus> $3.99 for 64GB 17:47:24 <adamw> heh 17:47:41 <Son_Goku> I bought a pile of 16GB sticks (~4 of them) for $15 a couple years ago 17:47:49 <Son_Goku> I think I got ripped off 17:47:53 <lruzicka> mattdm_not_sus, yeah, we just need to decide that we do not want to support smaller ones and we can kick the limits 17:47:56 <mattdm_not_sus> $1.99 for 16GB but they only have 2 in stock and only one brand 17:49:05 <nirik> ok, so where are we ? revoting? 17:49:05 <mattdm_not_sus> anyway. not the point right now, but I feel like it's safe to increase the limit 17:49:28 <nirik> yeah which should be workstation working group for workstation image... 17:51:15 <adamw> we're not revoting, just voting. :P 17:51:22 <adamw> well, i guess if you voted on the ticket you can revote here. 17:51:35 <adamw> i'm +1 on the current criteria (but will also vote to waive) 17:52:03 <nirik> Ditto. Change vote to +1, but will vote to waive 17:52:12 <geraldo_simiao> +1 FB and +1 waive 17:52:54 <adamw> we'll vote on waivers a bit later, just...process 17:54:05 <adamw> proposed #agreed 2247611 - AcceptedBlocker (Final) - this is accepted as a violation of the image size criterion, with the test case wording supporting the interpretation that size limits for disk images should be read as applying to the uncompressed size 17:54:11 <Son_Goku> ack 17:54:13 <adamw> (we can clarify this in the criteria later) 17:54:17 <geraldo_simiao> ack 17:54:19 <lruzicka> ack 17:54:55 <nirik> ack 17:55:04 <thebeanogamer> ack 17:55:57 <adamw> #agreed 2247611 - AcceptedBlocker (Final) - this is accepted as a violation of the image size criterion, with the test case wording supporting the interpretation that size limits for disk images should be read as applying to the uncompressed size 17:56:08 <adamw> #topic (2247659) User switching causes gnome-shell to crash occassionally. 17:56:08 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=2247659 17:56:08 <adamw> #link https://pagure.io/fedora-qa/blocker-review/issue/1433 17:56:08 <adamw> #info Proposed Blocker, gnome-shell, NEW 17:56:15 <adamw> so, openQA test history here: 17:56:28 <adamw> https://openqa.fedoraproject.org/tests/2243821#next_previous 17:56:32 <adamw> (set the dropdown to 100) 17:56:42 <adamw> the test's failed two times in the last month 17:56:49 <Son_Goku> our old friend fast user switching 17:56:59 <nirik> -1 FinalBlocker... I think this is too rare to be blocking 17:57:05 <Son_Goku> -1 FinalBlocker 17:57:10 <adamw> rawhide is pretty similar - https://openqa.fedoraproject.org/tests/2243208#next_previous 17:57:21 <adamw> the bug is pretty bad if you hit it, because you lose whatever you had in the session 17:57:32 <adamw> but on the whole...it's probably okay to fix it as a post-release update 17:57:37 <thebeanogamer> -1 FinalBlocker, 2/100 is an acceptable level of intermittency 17:57:46 <nirik> you should always be ready to loose your session. ;) Be prepared! 17:57:49 <lruzicka> Son_Goku, no, none of a fast switching. Regularly switched to another user. Happened on the first attempt. 17:58:15 <Son_Goku> lruzicka: two users logged in at once is "fast user switching" 17:58:31 <Son_Goku> (yes I know it makes no sense as a name, but that's what it's called) 17:58:33 <lruzicka> Son_Goku, Linux is a multi-user system. Please, do not forget about this. 17:58:51 <Son_Goku> I am aware, but GUI user switching has been off and on broken for almost six Fedora releases 17:58:59 <adamw> note it's not 2/100 (I can't give an exact number out of 100 as some of the failures are not this crash) 17:59:15 <adamw> Son_Goku: well, it's "fast" because you don't have to log out first. 17:59:24 <lruzicka> For me, this happened like 4/5 times 17:59:31 <lruzicka> I tried 5 times :D 17:59:33 <Son_Goku> adamw: yes I figured that 17:59:37 <adamw> unfortunately we don't run the openqa test on aarch64 17:59:44 <Son_Goku> ... yet 17:59:57 <adamw> nah, the GUI tests just aren't terribly reliable on aarch64 18:00:00 <adamw> and this is a *long* test 18:00:03 <Son_Goku> ugh 18:00:04 <adamw> so it'd probably fail a whole lot 18:00:23 <adamw> i keep meaning to try and figure out why it's still pretty unreliable but never get the roundtuits. the hardware we have in prod now ought to be fairly powerful :( 18:01:22 <nirik> never enough time... 18:01:33 <adamw> #10 0x0000ffffa2d475d8 in g_assertion_message_expr (domain=domain@entry=0xffffa2770018 "libmutter", file=file@entry=0xffffa279fb18 "../src/backends/native/meta-onscreen-native.c", line=line@entry=196, func=func@entry=0xffffa27a1868 <__func__.14> "meta_onscreen_native_notify_frame_complete", expr=expr@entry=0xffffa279fae8 "!cogl_onscreen_peek_head_frame_info (onscreen)") at ../glib/gtestutils.c:3523 18:01:33 <adamw> s = 0xaaaaefc5fd50 "assertion failed: (!cogl_onscreen_peek_head_frame_info (onscreen))" 18:01:35 <neil> esp. not with daylight savings ... 18:01:37 <adamw> seems to be the key problem, btw. 18:02:38 <nirik> well, sure, it seems obvious now... 18:02:40 <nirik> (I kid) 18:02:53 <adamw> hehe 18:03:03 <adamw> jadahl can probably figure it out. but we're not gonna fix it now 18:03:17 <adamw> it doesn't seem to be new code, so the problem must be in something leading up to it, i guess. 18:04:01 <adamw> so, we have -3 atm 18:04:03 <adamw> anyone want to argue +1? 18:04:18 <lruzicka> nah, I am fine 18:04:37 <geraldo_simiao> -1 FB 18:04:44 <lruzicka> Or accept and waive 18:06:06 <adamw> proposed #agreed 2247659 - RejectedBlocker (Final) - this is a bad bug if you hit it, but openQA test results indicate it doesn't happen terribly often (at least on x86_64), so we think it's acceptable to try and fix this post-release 18:06:12 <Son_Goku> ack 18:06:21 <nirik> ack 18:06:36 <coremodule> ack 18:06:50 <adamw> #agreed 2247659 - RejectedBlocker (Final) - this is a bad bug if you hit it, but openQA test results indicate it doesn't happen terribly often (at least on x86_64), so we think it's acceptable to try and fix this post-release 18:07:05 <adamw> okay 18:07:31 <adamw> let's move on to accepted blocker review 18:07:38 <adamw> #topic (2247311) Update Firefox in Fedora 39 Final to fix CVE-2023-5721 18:07:38 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=2247311 18:07:38 <adamw> #link https://pagure.io/fedora-qa/blocker-review/issue/1431 18:07:38 <adamw> #info Accepted Blocker, firefox, ON_QA 18:07:44 <adamw> #info this is addressed in RC-1.5, so all good 18:07:50 <adamw> #topic (2113005) Live image made with BOOTX64.EFI from latest shim-x64-15.6-2 fails to boot on some boards 18:07:50 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=2113005 18:07:50 <adamw> #link https://pagure.io/fedora-qa/blocker-review/issue/1154 18:07:50 <adamw> #info Accepted Blocker, shim, NEW 18:07:57 <adamw> i think now is an appropriate time to do waiver votes! 18:08:14 <adamw> proposal: as for previous releases, let's waive this as being impossible to fix in a reasonable timeframe. we are still waiting on upstream merges 18:08:39 <nirik> +1 18:08:51 <thebeanogamer> +1 waive, +1 cursing Microsoft's name 18:09:00 <geraldo_simiao> +1 waive 18:09:13 <Son_Goku> +1 waive +1 curse everyone involved in it 18:10:21 <mattdm_not_sus> +1 waive, -1 cursing, +1 back to making it a prioritized bug (which was rejected 'cause it was already a blocker) 18:11:19 * Son_Goku stares at mattdm_not_sus again 18:11:21 <lruzicka> +1 waive 18:11:28 <adamw> #agreed 2113005 is waived to Fedora 40 Beta under the "Difficult to fix blocker bugs" provision: https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process#Exceptional_cases 18:11:32 <lruzicka> ack 18:11:35 <Son_Goku> ack 18:11:45 <adamw> i don't think making this prioritized will accomplish anything, everyone involved very much wants to fix it, as soon as it's practical to do so it's gonna happen 18:11:56 <adamw> pjones: is that accurate? 18:12:14 <nirik> ack 18:12:24 <pjones> yeah, that's a fair summary 18:12:26 <adamw> oh, sorry, i didn't make that a proposal. ah well, retroactive acks? 18:12:49 <lruzicka> as if it happened 18:13:03 <Son_Goku> adamw: timetraveled ack 18:13:28 <pjones> I also don't think it's worth *cursing* people about it. Have some respect for work other people are putting in. 18:13:36 <adamw> yeah, i think that was poorly put. 18:14:01 <thebeanogamer> My apologies, that's not how I meant it 18:14:35 <adamw> alrighty, so moving on 18:14:55 <adamw> #topic (2247611) [aarch64] F39 RC1.5 unxzed images too large 18:14:55 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=2247611 18:14:55 <adamw> #link https://pagure.io/fedora-qa/blocker-review/issue/1432 18:14:55 <adamw> #info Accepted Blocker, distribution, NEW 18:15:01 <adamw> so, back to this one, as an accepted blocker 18:15:14 <mattdm_not_sus> (prioritized doesn't mean people don't want to fix it; it's just another way of tracking critical things. but okay to not bother in this case) 18:15:17 <nirik> +1 waive as being too late/hard to fix now 18:15:21 <adamw> proposal: waive this as both "reported too late" and "difficult to fix" (we can't just magically hack a gig off workstation at this point) 18:15:29 <Son_Goku> ack 18:15:30 <nirik> +1 18:15:33 <mattdm_not_sus> +1 18:15:34 <Son_Goku> +1 18:15:35 <lruzicka> +1 18:15:50 <adamw> mattdm_not_sus: i just meant, the additional tracking isn't really going to achieve anything except bugging people unnecessarily. everyone is already aware this needs fixing as soon as we can. 18:16:51 <adamw> proposed #agreed 2247611 is waived to Fedora 40 Beta under both "Last minute blocker bugs" and "Difficult to fix blocker bugs" provisions (https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process#Exceptional_cases ). ahead of F40 we will look at tightening up the process here and probably bumping the size requirement 18:17:20 <nirik> ack 18:17:35 <Son_Goku> ack 18:19:05 <lruzicka> ack 18:19:25 <adamw> #agreed 2247611 is waived to Fedora 40 Beta under both "Last minute blocker bugs" and "Difficult to fix blocker bugs" provisions (https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process#Exceptional_cases ). ahead of F40 we will look at tightening up the process here and probably bumping the size requirement 18:19:59 <adamw> okay. I believe that's everything for blockers. so as things stand we have no accepted, unaddressed, unwaived blockers 18:20:29 <adamw> #info current blocker state: we have three accepted blockers, one is addressed, two are waived. we have no 'active' blockers 18:20:40 <nirik> \o/ 18:20:57 <mattdm_not_sus> Good enough! 18:21:00 <amoloney> hurray! 18:21:08 <geraldo_simiao47> Great 18:21:20 <lruzicka> Wait, my test machine now crashed terribly. 18:21:31 <dustymabe> lruzicka: mine too.. wait a second let me paste some logs 18:21:41 <lruzicka> :D 18:22:09 <neil> I was just about to ๐ฅณ but I guess I will wait 18:22:17 <dustymabe> :D - /me jokes 18:22:26 <amoloney> is this some dark humour that Im not getting, or is this real? 18:22:34 <dustymabe> .fire dubiousness 18:22:34 <zodbot> adamw fires dubiousness 18:22:36 <dustymabe> oops 18:22:39 <dustymabe> .fire dustymabe 18:22:39 <zodbot> adamw fires dustymabe 18:22:45 <Son_Goku> .fire lruzicka 18:22:45 <zodbot> adamw fires lruzicka 18:22:57 <Son_Goku> amoloney: I seriously hope it's dark humor 18:23:07 <adamw> ./kickban lruzicka 18:23:11 <adamw> did anyone hear anything? i didn't 18:23:12 <Son_Goku> otherwise we're in for another cycle where a release party happens before the release _again_ 18:23:19 <amoloney> small favour for the newbie? please wait a few releases that Im involved in before making 'jokes' :-D 18:23:35 <adamw> haha, sorry amoloney 18:23:37 <amoloney> my heart dropped ha 18:23:47 <Son_Goku> I was a little scared too :) 18:23:47 <adamw> i think the first time someone did `.fire lruzicka` he was terrified 18:23:56 <Son_Goku> lol wut 18:24:07 <Son_Goku> it's not like anyone here is his boss 18:24:07 <adamw> it's a bit disconcerting if you don't know it's a meme :D 18:24:12 <Son_Goku> lol 18:24:19 * decathorpe has always wondered why it's adamw's responsibility to fire people 18:24:31 <lruzicka> adamw, I am still terrified :D 18:24:41 <amoloney> its because we just accept his will :-D 18:24:45 <geraldo_simiao47> For me the first time I was fired, was an honour... 18:24:52 <michel-slm> amoloney++ 18:24:52 <zodbot> michel-slm: Karma for amoloney changed to 1 (for the release cycle f39): https://badges.fedoraproject.org/tags/cookie/any 18:24:53 <tflink> decathorpe: it started as a joke and then someone made it into a zodbot command and the joke has lived on 18:24:55 <geraldo_simiao47> :) 18:25:04 <amoloney> right lets 'fire' up the test matrices! 18:25:22 <amoloney> #topic Current status - test matrices 18:25:51 <adamw> er, let me...take a look 18:26:07 <thebeanogamer> amoloney++ 18:26:07 <zodbot> thebeanogamer: Karma for amoloney changed to 2 (for the release cycle f39): https://badges.fedoraproject.org/tags/cookie/any 18:26:33 * nirik gets more coffee while adamw looks 18:27:02 <amoloney> #link https://fedoraproject.org/wiki/Category:Fedora_39_Test_Results 18:27:38 <adamw> we're missing real-world cloud testing 18:27:43 <adamw> anyone feel like firing up ec2 quick 18:28:24 <adamw> we don't have every single test for aarch64 workstation, but i think we have enough to be fine (given we covered all the missing ones on x86_64) 18:28:52 * nirik can try a ec2 18:28:53 <Son_Goku> I don't exactly have monies 18:28:57 <dcavalca> adamw: if you have an AMI ID to test I can fire an instance up 18:29:26 <adamw> dcavalca: they're on the page 18:29:36 <adamw> https://fedoraproject.org/wiki/Test_Results:Fedora_39_RC_1.5_Cloud 18:29:47 <adamw> there are tables of AMIs for x86_64 and aarch64 18:29:54 <adamw> sorry, i usually do this but was just spread too thin... 18:31:28 <adamw> we're missing the USB UEFI tests for the aarch64 ISO images, too, if anyone has hardware that can run those 18:31:48 <dcavalca> ok spinning up ec2 instances for x86_64 and aarch64 now 18:31:56 <adamw> other than that we're looking good 18:32:05 <adamw> thanks 18:32:41 <adamw> you should be able to blow through startup , logging , identification , services_start and selinux easily right after boot 18:32:57 <adamw> then you can do package_install_remove and update_cli and reboot_umount quite easily after that 18:33:15 <adamw> service_manipulation takes forever but we can live without that since it works in openqa and everywhere else, no reason it'd be busted on ec2 18:33:30 <nirik> [fedora@ip-172-16-0-212 ~]$ uname -r 18:33:31 <nirik> 6.5.6-300.fc39.x86_64 18:33:56 <adamw> lruzicka: pwhalen do you have UEFI-capable arm64 hardware by any chance? i don't 18:34:30 <lruzicka> We have fried the PINEBOOK today and the only thing we have is a Raspberry 400. 18:34:49 <Southern_Gentlem> m2 mac? 18:35:15 <neil> I might be able to help with uefi arm 18:35:26 <Son_Goku> Southern_Gentlem: I think adamw would kill me if we used that for testing right now :) 18:35:58 <adamw> .fire everyone whose nick starts with 's' 18:35:58 <zodbot> adamw fires everyone whose nick starts with 's' 18:36:18 <neil> *technically* they start with 'S' 18:36:19 <Son_Goku> also we don't yet have a uefi iso boot mode in AS Macs 18:36:40 <adamw> neil: if you're in a position to test that quickly it'd be great 18:36:49 <neil> adamw: pgreco tells me he can test, but doesn't have a graphical interface, only serial.. 18:36:54 <dcavalca> adamw: everything looks good on ec2 18:37:15 <dcavalca> on aarch64 I have 18:37:15 <adamw> neil: the test is just that the installer image boots and you can run an install... 18:37:17 <dcavalca> Nov 02 18:36:17 localhost kernel: CPU features: detected: Hardware dirty bit management 18:37:26 <adamw> dcavalca: this is more about the firmware, AIUI. 18:37:33 <dcavalca> in the log but that's totally fine, it's just the regexp on the testcase that's greedy 18:37:40 <dcavalca> yup 18:37:46 <nirik> yeah, confirm ec2 ok here. 18:37:50 <dcavalca> I also have access to GCP if you need me to test something there 18:37:54 <adamw> oh, yeah, the test case mentions the grep is a bit greedy 18:38:18 <neil> adamw: have a link to the test case I can share? 18:38:25 <adamw> neil: https://fedoraproject.org/wiki/QA:Testcase_Boot_default_install 18:38:27 <neil> ty! 18:38:56 <adamw> but it's basically 'grab https://kojipkgs.fedoraproject.org/compose/39/Fedora-39-20231031.1/compose/Server/aarch64/iso/Fedora-Server-netinst-aarch64-39-1.5.iso , dd it to a USB stick, boot from that USB stick, install" 18:39:11 <adamw> needs to be on hardware that supports this kind of PC-style 'UEFI boot from generic medium', though. that's the tricky part. 18:40:31 <pwhalen> adamw: I can if someone else doesn't already have it 18:41:00 <adamw> if you don't mind firing it up in parallel, sure - the more the merrier :P 18:41:19 <pwhalen> sure thing :) 18:41:35 <nirik> nothing like last minute testing with time pressure. :) 18:42:52 <Son_Goku> I'm sure it'll work fine :D 18:43:14 <Son_Goku> if we broke arm uefi, people would be screaming, right? ... right? 18:43:21 <adamw> heh 18:44:27 <adamw> oh, shoot. you know what. we kinda missed a blocker. 18:44:36 <adamw> let's circle back while pwhalen tests 18:44:36 <neil> too late. no going back 18:44:38 <neil> #yolo 18:44:42 <adamw> #topic Oops, we missed a blocker 18:44:53 <adamw> #topic (2242759) dnf system-upgrade fails on some RPi4 due to system boot date that pre-dates gpg key 18:44:53 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=2242759 18:44:53 <adamw> #link https://pagure.io/fedora-qa/blocker-review/issue/1392 18:44:54 <adamw> #info Accepted Previous Release Blocker, distribution, NEW 18:44:56 <neil> pgreco is on the arm test too. so we'll see who finishes the race D: 18:45:02 <adamw> so, yeah, this one's hanging around like a bad smell 18:45:03 <adamw> woohoo, races! 18:45:15 <nirik> oh yeah, this ol one. 18:45:16 <neil> ouch. yeah that one does smell :( 18:45:17 <amoloney> cash prize for the winner! 18:45:20 <amoloney> :-D 18:45:21 <Son_Goku> bleeeeech 18:45:31 <adamw> so. ultimately i don't think we should block the release on this 18:45:34 <geraldo_simiao> Race condition? ;) 18:45:36 <neil> Son_Goku: s/ec/eac/ 18:45:48 <Son_Goku> lol 18:45:52 <Son_Goku> adamw: I agree 18:45:59 <nirik> adamw: I agree... it's not even clear the right answer here. ;( 18:46:01 <neil> agreed there 18:46:03 <Son_Goku> also isn't there a systemd update that will make this mostly go away? 18:46:11 <adamw> Son_Goku: not afaik, no. 18:46:23 <Son_Goku> meh 18:46:29 <Son_Goku> it's also mostly unfixable 18:46:31 <adamw> it turns out the 'just update systemd and the problem goes away' plan doesn't work if *anything at all* gets built after systemd 18:46:37 <Son_Goku> ugh 18:46:49 <mattdm_not_sus> ๐ข 18:46:57 <nirik> depending on network here also seems wrong to me. 18:47:02 <adamw> i mean, i think it's not unfixable in theory, just any practical way to fix it is too radical to do in f39 at least 18:47:29 <pgreco> neil: my sata-usb adapter seems to have left the realm of the living, so it's gonna take me a few minutes to clear up a disk ;) 18:47:53 <adamw> we might be able to come up with a plan to avoid this for upgrades from f40 onwards, or something. but unless someone has a genius plan, i don't see really a safe way to hack up our existing releases and f39 to make it work :/ 18:48:20 <nirik> so the critera here is upgrades have to work right? 18:48:22 <adamw> yeah 18:48:28 <adamw> this is a conditional violation for systems with no RTC 18:48:44 <nirik> which is all the pis? 18:49:00 <dustymabe> i think pi5 (brand new) has an RTC 18:49:10 <dustymabe> older ones don't 18:49:22 <nirik> so, I think this is probibly a blocker, but we should waive it also under the 'sorry, too late/hard to fix' thing. 18:49:25 <adamw> maybe other cheap SBCs too? i dunno 18:49:31 <geraldo_simiao> What's an RTC? 18:49:33 <adamw> the problem with waiving it, i think, is we get in a shim situation 18:49:36 <adamw> real time clock 18:49:37 <nirik> realtime clock 18:49:39 <Son_Goku> geraldo_simiao: real-time clock 18:49:48 <geraldo_simiao> Thanks :) 18:49:52 <adamw> hardware thingy that keeps track of what time it is, so you don't always boot in 1973 18:50:01 <adamw> (and rely on the OS to find the real time) 18:50:09 <nirik> well, we could also adjust critera later and say 'if you have a no rtc system... ' 18:50:17 <Son_Goku> we probably should 18:50:25 <adamw> so for me the problem with waiving it is, we'd wind up in a kinda shim situation 18:50:27 <Son_Goku> unless we're going to implement some kind of fallback fake rtc 18:50:47 <adamw> we'd have to waive it for, like, a *long* time. at least until two releases after whatever fix we came up with, unless the fix was sufficiently 'safe' to be applied to stable releases (which i doubt) 18:50:50 <nirik> well, if we don't accept it as a blocker, we are still in the same situation no? 18:51:12 <adamw> we're in the same situation in *practical* terms, i mean in "blocker process" terms 18:51:39 <adamw> it would just kinda suck having to keep waiving this every beta and final for several cycles 18:51:43 <nirik> I would hope we could come up with a fix that would work upgrading to 40+ 18:51:45 <Southern_Gentlem> rock and a hard place 18:51:57 <adamw> nirik: i'm not sure we can? all the ideas seem too radical to change in a stable release 18:52:03 <adamw> and i don't think you can fix it from the target release 18:52:07 <adamw> but maybe? 18:52:27 <adamw> maybe we can waive it at least once and see how it goes 18:52:47 <nirik> yeah... and if it's going to be a longer term thing, we change critera? 18:53:34 <nirik> oh, I think I misread... systemd-timesyncd doesn't need network to fix this? 18:53:49 <dustymabe> I kind of feel like the behavior that systemd-timesyncd has where it writes out a file on shutdown and then sets the time to that value on boot should be default in systemd and not only if systemd-timesyncd is enabled 18:53:59 <adamw> right. but to me, it would be rather a big change to just rip out chrony and replace it with timesyncd on existing stable releases 18:54:14 <dustymabe> adamw: correct 18:54:20 <thebeanogamer> Is adding chrony to the required services during an update a more acceptable change? 18:54:21 <dustymabe> it can be documented as a workaround for this 18:54:31 <adamw> i suppose we could try and do some hinky thing of enabling timesyncd in the upgrade environment only, but...i'm not sure if that would even work? maybe it needs to be running *before* the reboot as well as *after* to wrok correctly? 18:54:34 <dustymabe> thebeanogamer: I don't think so because we don't have network? 18:54:37 <nirik> or perhaps chrony could do the same thing ? 18:54:43 <adamw> thebeanogamer: *that* would rely on network, i think 18:54:44 <Son_Goku> chrony could be enhanced to do the same thing 18:54:45 <thebeanogamer> dustymabe: Urgh good point 18:54:51 <adamw> that was my suggestion, but it didn't go down terribly well 18:54:52 <dustymabe> adamw: yeah I don't think that would work 18:55:04 <adamw> anyhow 18:55:07 <dustymabe> I think timesyncd has to be running before so on the previous shutdown it writes out the file 18:55:08 <adamw> we don't have to fix it in this meeting :D 18:55:13 <Son_Goku> we could ask the chrony folks about thie problem and see if they can make a fix for f40 18:55:27 <Son_Goku> because we do ship chrony everywhere right now 18:55:28 <nirik> or even f38/39 18:55:56 <adamw> so...shall we go with waiving it? 18:55:59 <pjones> it seems like the better fix is to teach rpmlib to be able to tell if there's an rtc and disable the validity window check if there isn't. 18:56:25 <geraldo_simiao> +1 waive 18:56:34 <Southern_Gentlem> +1 waive 18:56:41 <pjones> (or maybe if there isn't /and/ there's no time synchronization happening... but that's complicated) 18:56:52 <thebeanogamer> +1 waive 18:56:54 <nirik> yeah, I think we should accept it, waive and try and deal with it somehow before f40 beta. ;) 18:57:07 <dustymabe> pjones: yeah, I mean this is kind of a generic problem with systems with no RTC 18:57:25 <nirik> if you have no rtc, just do fresh installs. problem solved. ;) 18:57:27 <dustymabe> this happens to be one symptom, but it's not the only problem systems with no RTC face 18:57:34 <adamw> proposed #agreed 2242759 is waived to Fedora 40 Beta as "Difficult to fix blocker bugs" - it turns out to be rather harder to fix than we expected, and we do not want to try and rush something in for the release window. we plan to document the possible workarounds for this and keep trying to come up with a good fix post-release 18:57:47 <nirik> +1 18:58:02 <lruzicka> +1 18:58:04 <lruzicka> ack 18:58:05 <thebeanogamer> ack 18:58:13 <neil> ack 18:58:15 <geraldo_simiao> ack 18:58:29 <Son_Goku> ack 18:58:35 <Son_Goku> +1 18:58:37 <adamw> #agreed 2242759 is waived to Fedora 40 Beta as "Difficult to fix blocker bugs" - it turns out to be rather harder to fix than we expected, and we do not want to try and rush something in for the release window. we plan to document the possible workarounds for this and keep trying to come up with a good fix post-release 18:58:41 <Son_Goku> ack 18:59:40 <nirik> ack 19:00:00 <adamw> okay, so, back to 19:00:01 <adamw> #topic Current status - test matrices 19:00:06 <adamw> sorry for the interruption, amoloney 19:00:26 <adamw> how's the testing going, pwhalen/pgreco ? 19:00:27 <amoloney> not a problem, it was a fairly important blocker to discuss 19:00:53 <pwhalen> adamw: 43% 19:01:38 <pgreco> adamw: found a sata-usb that works, erasing a disk to start from scratch 19:02:20 <neil> pwhalen is off to a convincing lead, but, anything can happen! 19:02:22 <adamw> pwhalen: hey, if it's got all the way there it's probably fine! 19:02:28 <adamw> ...nah, we should probably wait for bootloader. :P 19:03:20 <geraldo_simiao> pgreco++ 19:03:43 <geraldo_simiao> pwhalen++ 19:04:15 * nirik has a mustang around somewhere on a shelf. I should set that up someday for testing. 19:04:48 <geraldo_simiao> Humm doesn't work on web, or this username 19:04:57 <pwhalen> 508/681 19:05:08 <pwhalen> almost there. 19:05:23 <adamw> excitement! 19:06:04 <nirik> ๐ญ the mustard indicates progress. 19:07:41 <geraldo_simiao> ๐ 19:08:01 <adamw> so, what did everyone dress as for halloween 19:08:27 <Southern_Gentlem> grumpy old horn dog 19:08:30 <Son_Goku> the scariest thing I could think of: myself :) 19:08:31 <lruzicka> ๐ 19:08:39 <lruzicka> The banana peel indicates it, too 19:09:05 <lruzicka> ๐ท๏ธ 19:09:09 <neil> i dressed up as a human 19:09:17 <neil> scariest monster tbh 19:10:51 <amoloney> you people really do have dark humour :-D 19:10:58 <amoloney> I feel at home ha 19:11:37 <neil> :) 19:11:39 <neil> amoloney++ 19:11:39 <zodbot> neil: Karma for amoloney changed to 3 (for the release cycle f39): https://badges.fedoraproject.org/tags/cookie/any 19:13:36 <amoloney> its been way too long since I was at a fancy dress party for halloween (which is a costume party for those who are in North America as apparently that description is not a thing?) 19:16:32 <Son_Goku> fancy dress party in north america is quite different 19:16:36 * nirik goes to a halloween thing every year. I always go as the devil... it's tradition. ;) 19:16:40 * thebeanogamer has decided to celebrate the potential for an approved RC by smashing their head into the server rack they're working on and needs to lie down for a few 19:16:45 <Son_Goku> that usually means... literally fancy dresses rather than costumes 19:17:15 <amoloney> yeah its like tuxedos and stuff! 19:17:22 <amoloney> which would also be funny 19:17:33 <Southern_Gentlem> or a tux tshirt 19:17:35 * sgallagh_ has actually shown up at a European-style fancy-dress party in a suit and tie. 19:17:47 <adamw> a tuxedo works for both! 19:17:55 <lruzicka> one guy once went dressed up as a "wounded guy" with artificial blood all over, but then got stoned, fell asleep on the street and some nice lady called an ambulance and the police. 19:17:55 <adamw> if it turns out to be a costume party just say you came as james bond 19:18:01 <nirik> In the US at least we confusingly have also "white tie" and "black tie" 19:18:02 <sgallagh_> I covered my mistake with a pair of sunglasses, becoming the MiB 19:18:05 <amoloney> thebeanogamer are you still ok? 19:18:13 <adamw> lruzicka: ...it was you, wasn't it 19:18:24 <lruzicka> adamw, I do not drink that much :D 19:18:33 <geraldo_simiao> Go dressed as Tux... 19:18:34 <adamw> nirik: pfft, that's not confusing at all. (and those are british terms originally) 19:18:35 <lruzicka> adamw, I ask for a friend 19:18:43 * thebeano1amer is ok, other than suddenly talking in the third person 19:19:14 <amoloney> head trauma will do that to you... 19:19:19 <nirik> it's always the british! 19:19:24 <adamw> white tie is a stiff collar, white bow tie and evening tailcoat. black tie is a soft collar, black bow tie and a dinner jacket. everyone knows THAT. pfft. 19:19:33 <adamw> (....everyone knows that, right?) 19:19:37 <amoloney> we do now 19:20:10 <amoloney> Adam always learns us 19:20:35 <Southern_Gentlem> https://m.media-amazon.com/images/I/61ItOgu-CiL._AC_SX466_.jpg 19:20:51 <adamw> that works for all occasions 19:21:28 * nirik notes adamw didn't say anything about pants. 19:21:30 <adamw> https://genius.com/The-hold-steady-t-shirt-tux-lyrics 19:21:35 <adamw> nirik: always optional 19:21:35 <sgallagh_> All outfits work for all occasions if you just don't give a damn 19:21:40 <adamw> (no, actually, the pants are the same for both.) 19:22:03 <nirik> so, hows that image looking? ;) 19:22:21 <amoloney> I hope youre referring to the tests ... 19:22:31 <pgreco> downloading rpms, 21% 19:22:34 <pwhalen> nirik: like it heard you, rebooting 19:22:50 <neil> Southern_Gentlem: hey.. I have one of those.. but it says "SELF" on it ;) 19:23:01 * nirik goes to look for any progress on the framework suspend thing 19:23:39 <pwhalen> success! 19:23:54 <adamw> woohoo! 19:24:01 <adamw> ...and with that i guess we can say the matrix coverage is good 19:24:06 <nirik> excellent! 19:24:14 <pwhalen> updated a couple tests on the matrix 19:24:16 <pgreco> adamw: intslling 275/432 19:24:20 <adamw> we were hoping to have azure testing done for final, but davdunc isn't around so i've no idea how to do that 19:24:40 <adamw> #info matrix coverage is acceptable 19:24:57 <neil> oh. i can do azure.. 19:24:58 <dustymabe> FWIW FCOS `next-devel` and `next` boots and passes tests on Azure 19:25:11 <geraldo_simiao> Acceptable is great 19:25:17 <neil> will do that asap.. 19:25:28 <adamw> neil: do you know where there are images? 19:25:36 <adamw> davdunc was supposed to provide a list, but i don't think it happened 19:25:43 <lruzicka> Does it take long? I guess I should be going ... 19:25:44 <neil> ah.. i was hoping that had happened 19:25:45 <adamw> you know, whatever the azure equivalent of AMIs is. 19:25:59 <adamw> dustymabe: great, that's a good indicator 19:26:03 <neil> I do know how to hack and slash a qcow/vhd into azure, though 19:26:11 <adamw> eh, we can probably live without it 19:26:22 <neil> ack 19:26:23 <adamw> would be great to do it async from this meeting and let us know if there's any trouble though 19:26:26 <neil> can do 19:26:32 <adamw> amoloney: wanna move on to next topic? 19:26:39 <amoloney> indeed 19:26:54 <amoloney> #topic Fedora CoreOS / IoT check-in 19:27:03 <amoloney> dustymabe: pbrobinson: coremodule: are CoreOS and IoT prepared for release? 19:27:36 <coremodule> aye, IoT looks good from my most recent testing, we can figure out which release will be Gold later this week/early next 19:27:48 <adamw> note, i've sent the stable push request for the one thing that wasn't stable already (firefox and nss) 19:27:58 <pwhalen> IoT is ready for the final compose. We'll do it tomorrow so its ready for next week 19:28:00 <adamw> so that should be pushed soon and nightlies after today should match RC-1.5 19:28:05 <Southern_Gentlem> neil yep with tux coming out of the pocket 19:28:06 * nirik can do that here after this meeting, if it ever ends 19:28:30 <adamw> note we may want to push linux-firmware stable for f39 very soon to fix https://bodhi.fedoraproject.org/updates/FEDORA-2023-1069778c50#comment-3266891 ... 19:29:16 <nirik> hurm 19:29:58 <nirik> I don't think we want to push that now... just make sure it's pushed as soon as we stage the release (ie, in the first 0 day batch) 19:30:37 <dustymabe> amoloney: yep. I think we're good 19:31:06 <adamw> nirik: yes, but my point is, we need to do that *ASAP* or else all f38->f39 upgrades will fail (including openqa tests, which is a problem) 19:31:09 <Southern_Gentlem> yeah i hit that today updating the respin builder to f39 19:31:20 <adamw> i guess i can add the update to the openqa workarounds to fix update tests at least 19:31:29 <pgreco> amoloney: sorry for the noise ;). adamw: installed and rebooted without issues. absolute success 19:31:33 <amoloney> Then once I info that both CoreOS and IoT are good I think we might be ready to poll? 19:31:40 <nirik> well, we can't until after we stage the release... so tomorrow. ;( 19:31:44 <adamw> we can't push it before we freeze the 'final' repo, but i'm saying maybe we need to do that quicker than usual 19:31:54 <adamw> amoloney: i believe so 19:31:58 <amoloney> exciting! 19:32:17 <amoloney> ok any more 'oops we should check xx'? 19:32:35 <adamw> i can't think of any... 19:33:24 <amoloney> righteo Ill info 19:34:04 <amoloney> #info Both Fedora CoreOS and Fedora IoT are in good condition and ready 19:34:22 <amoloney> #topic Go/No-Go decision 19:34:28 <amoloney> Tis time 19:34:32 <adamw> yaaay 19:34:36 <neil> <3 19:34:40 <neil> ๐ฅณ 19:34:49 <amoloney> I will poll each team. Please reply 'go' or 'no-go' 19:34:54 <amoloney> FESCo? 19:35:07 <nirik> go 19:35:13 <amoloney> Releng? 19:35:18 * nirik puts on another hat 19:35:19 <nirik> go 19:35:28 <amoloney> QA? 19:35:32 <lruzicka> go 19:35:59 <amoloney> #agreed Fedora Linux 39 Final is GO 19:36:09 <adamw> yaaay 19:36:12 <geraldo_simiao> ๐ 19:36:17 <thebeanogamer> wooooo 19:36:21 <alciregi> \o/ 19:36:23 <neil> WOOOO! ๐ 19:36:23 <lruzicka> Jubilation, she loves me again .... 19:36:25 <adamw> good job, everyone 19:36:35 <nirik> great! 19:36:37 <amoloney> whats our release date? 19:36:44 <adamw> 7th 19:36:46 <amoloney> I am thinking its 7th? 19:36:47 <lruzicka> Happy Release, I need to rush. Take care. 19:36:47 <nirik> 2023-11-07 19:36:59 <amoloney> excellent. Wouldnt be the first time I recorded it wrong 19:37:01 <neil> thank you lruzicka :) you take care too 19:37:12 * nirik sadly has another topic before we close tho... or I guess we can discuss out of meeting if desired. 19:37:14 <decathorpe> so we can announce release is GO and the release party together? ;) 19:37:16 <amoloney> #info Fedora Linux 39 Final will release on the early target date (2023-11-07) 19:37:22 <adamw> we can do an open floor 19:37:25 <amoloney> thank you all! 19:37:26 <adamw> er 19:37:27 <decathorpe> early? :) 19:37:30 <adamw> that's not the early target date :D 19:37:39 <nirik> it's early for xmas! 19:37:39 <amoloney> aw sweet baby 19:37:40 <adamw> i think we're at target date #3 19:37:42 <neil> hehe 19:37:46 <amoloney> #undo 19:37:46 <zodbot> Removing item from minutes: INFO by amoloney at 19:37:16 : Fedora Linux 39 Final will release on the early target date (2023-11-07) 19:37:55 <geraldo_simiao> Yes target #3 19:38:09 <neil> if we've learned nothing from the rpi rtc issue, is it not that time is relative, adamw? 19:38:15 <amoloney> info Fedora Linux 39 Final will release on target date #3 (2023-10-17) 19:38:29 <amoloney> one more try 19:38:44 <adamw> bingo 19:38:49 <adamw> neil: hehe 19:38:52 <nirik> next new release date: 1970-01-01! 19:38:56 <Son_Goku> lol no 19:38:59 <neil> err -- 10-07 , not 17 19:39:02 <amoloney> #info Fedora Linux 39 Final will release on target date #3 (2023-11-07) 19:39:07 <adamw> nirik: dangit, you beat me to the joke 19:39:08 <neil> woo! 19:39:09 <neil> nice 19:39:12 <Son_Goku> woot 19:39:28 <amoloney> nearly failed at the last info hahaha 19:39:30 <adamw> amoloney: can you do an open floor for nirik's thing? 19:39:34 <amoloney> I can indeed 19:39:47 <amoloney> #action amoloney to announce decision 19:39:53 <amoloney> #topic Open Floor 19:39:56 <nirik> ok, so, RC-1.5, which is "go" for release... is missing some images. These failed to compose for basically 3 reasons: 19:40:12 <nirik> 1. Some weird transient 502 errors when making ostree installers that we haven't been able to track down 19:40:27 <nirik> 2. Some weird aarch64 python SIGILL's on live media composes 19:40:35 <nirik> 3. Dep issues on Astronomy 19:40:50 <nirik> For the record, here's the missing ones: 19:40:56 <adamw> the problem with 1 and 2 is, well, we haven't fixed them. so any compose we do is likely to be missing *something* 19:41:01 <nirik> (pardon the long paste) 19:41:04 <nirik> missing ostree install images: 19:41:04 <nirik> Onyx - x86_64 19:41:05 <nirik> Sericea - x86_64 19:41:05 <nirik> Silverblue - aarch64 19:41:05 <nirik> Kinoite - ppc64le 19:41:05 <nirik> missing Live images: 19:41:07 <nirik> Security - x86_64 19:41:11 <nirik> Astronomy_KDE - x86_64 19:41:13 <nirik> LXQt - aarch64 19:41:15 <nirik> KDE - aarch64 19:41:17 <nirik> Workstation - aarch64 19:41:21 <Son_Goku> holy crap 19:41:29 <nirik> So, we could do a few things about this... we could: 19:41:35 <Son_Goku> that's a lot of missing stuff 19:41:45 <nirik> 1. nothing 19:41:45 <nirik> 2. try and rebuild with another name (1.6?) 19:41:45 <nirik> 3. ship the nightly versions from tomorrow (if available) 19:41:59 <Son_Goku> I'd really want to try for 2 19:42:18 <Son_Goku> we have a content lock for GA now anyway 19:42:25 <nirik> 2 and 3 are more work for sure... 19:42:45 <decathorpe> does option 1) mean that those images will just not be available for Fedora 39? 19:42:46 <nirik> the name will be different from all the rest in both cases and websites will need changes, we will have to distribute them from some other place, etc. 19:42:53 <nirik> decathorpe: yep. 19:42:54 <sgallagh_> Worth noting: Silverblue aarch64 was also missing from F38, so there would be no official media from which to install AT ALL 19:43:18 <sgallagh_> For F38, people have been given the F37 media and told to update from there 19:43:26 <decathorpe> that's not good 19:43:40 <decathorpe> wasn't Onyx just added in F39? so there would be no way to install it at all? 19:43:43 <sgallagh_> So I would also recommend a no-changes rebuild 19:43:49 <nirik> The live ones we likely cannot fix right now... we don't have any fix in hand anyhow 19:44:17 <adamw> we cannot, by policy, rebuild and ship a different compose 19:44:24 <nirik> right. 19:44:24 <adamw> we signed off 1.5. not a 1.6 that doesn't exist yet. 19:44:26 * dustymabe kind of feels like we should just deliver one install media and make it netinstall (pull from remote OSTree repo) 19:44:59 <nirik> if we do 3... the name will be YYYYMMDD.n.0 instead of 1.5... but it should be the same content. 19:45:12 <adamw> i think 3 is probably the best option 19:45:14 <adamw> we've done that before 19:45:14 <nirik> (and if they compose and don't hit the sporadic issue) 19:45:40 <adamw> i know it's work for releng to decide where to put the bits and websites team to link to them :/ 19:46:08 <nirik> yeah, but I think we should try and do something (aside from fixing the issues...) 19:46:08 <neil> yeah i think 3 makes the most sense (despite it being a pain logistically) 19:47:18 <nirik> One advantage of 2 is that we can just keep trying those. ;) 19:47:35 <nirik> but I agree 1.6 is bad on naming because it shouldn't exist. I guess we could call it something else? 19:48:00 <adamw> i don't really like the idea of inventing some new label name 19:48:16 <adamw> we have conventions about those, i think they're even enforced in productmd to some extent 19:48:29 <nirik> oh, how about... 20231102.n.1 ? 19:48:40 <nirik> well, no, 20231103.n.1 19:48:46 <nirik> (after the final compose) 19:48:50 <adamw> well, no, it wouldn't be a nightly. 19:48:57 <adamw> and i don't think that's a valid label name... 19:49:06 <nirik> to be clear.. I am not talking about a entire new compose. 19:49:28 <adamw> oh 19:49:31 <nirik> I'm talking about specifically doing a compsoe of image X named whatever we want with the final repos 19:49:31 <adamw> what are you talking about? 19:49:44 <adamw> oh, okay. like a scratch build? 19:49:45 <nirik> just each image as a one off. 19:49:57 <adamw> well, i guess i'd be fine with that, if we can do it 19:49:57 <nirik> yes, but I can make them official builds so they stay around 19:50:20 <nirik> it would have to be after the final nightly tho 19:50:27 <nirik> or else it would be missing firefox/nss 19:51:24 <nirik> so, how about we try 3... and if something fails again we fall back to doing those specific ones as method 3 with .n.1 19:51:48 <Son_Goku> I don't see what's wrong with 1.6? 19:51:51 <adamw> if we're doing non-pungi image builds i'd kinda like the name *not* to look like it came out of a nightly compose, but meh, it's not too important 19:51:58 <adamw> i prefer names that don't lie 19:52:09 <adamw> '1.6' looks like it came out of a candidate compose with label 1.6 19:52:10 <nirik> I'm open to whatever name... 19:52:21 <adamw> which never actually happened 19:52:31 <nirik> 1.5-respin ? 19:52:32 <adamw> we can bikeshed the name tomorrow, i guess :D 19:52:34 <Son_Goku> why couldn't we do a 1.6 based on the same content set? 19:52:48 <Son_Goku> that's just kevin running the process no? 19:52:49 <nirik> because by policy there's no cause to do another compose after we are go. 19:52:59 <adamw> Son_Goku: because that'd be needlessly confusing. we couldn't ship it. so it would exist but not actually be the released compose but we'd release some images from it. wat. i don't like that 19:53:03 <adamw> (and yes, that) 19:53:47 <nirik> well, we can talk about it more, I just wanted to bring it up... 19:54:17 <adamw> yeah, i think we have a viable plan anyhow 19:54:20 <adamw> we can work on the details later 19:54:53 <nirik> hum so nightly will still have the 'prerelease' thing? 19:55:21 <Son_Goku> it would :( 19:55:39 <nirik> oh no it won't... 19:55:46 <Son_Goku> no? 19:55:48 <Son_Goku> why not? 19:55:52 <nirik> it's keying off fedora-release 19:56:00 <adamw> um 19:56:00 <dustymabe> #proposed let nirik use his best judgement and everyone be happy with that 19:56:04 <adamw> it depends on the image 19:56:20 <adamw> i think lives key off fedora-release but traditional installers depend on an argument passed at build time? but i have to relook this up every time 19:56:32 <nirik> yeah, I can never recall 19:56:33 <adamw> anyhow, maybe we can close out the meeting now? 19:56:38 <adamw> and work this all out over on matrix 19:57:16 <nirik> yep. +1 19:57:29 <thebeanogamer> Sounds good 19:57:33 <adamw> amoloney: still awake? :D 19:57:42 * thebeanogamer is looking forward to the change proposal for moving all this to Matrix 19:57:49 <mattdm_not_sus> where on matrix? 19:58:14 <amoloney> I am :) the benefit of DST is its only 8pm now, rather than 9pm so yay I guess! 19:58:31 <amoloney> does this need to be info-ed or is it ok to wrap as is? 19:59:21 <adamw> #info several images are missing from RC-1.5 due to known bugs in the compose process, releng will work on a plan to try and provide builds of those images from nightly composes or special build tasks 19:59:27 <adamw> there 19:59:34 <nirik> mattdm_not_sus: releng channel? 19:59:36 <adamw> mattdm_not_sus: releng channel i guess 19:59:42 <mattdm_not_sus> wfm! 19:59:49 <amoloney> thanks Adam! 19:59:57 <amoloney> fin 20:00:01 <amoloney> #endmeeting