15:00:09 <pwhalen> #startmeeting Fedora ARM and AArch64 Status Meeting
15:00:09 <zodbot> Meeting started Tue Oct 26 15:00:09 2021 UTC.
15:00:09 <zodbot> This meeting is logged and archived in a public location.
15:00:09 <zodbot> The chair is pwhalen. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
15:00:09 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:09 <zodbot> The meeting name has been set to 'fedora_arm_and_aarch64_status_meeting'
15:00:09 <pwhalen> #chair pwhalen pbrobinson coremodule jlinton
15:00:09 <zodbot> Current chairs: coremodule jlinton pbrobinson pwhalen
15:00:09 <pwhalen> #topic roll call
15:00:21 <pwhalen> Good morning folks, who's here today?
15:00:27 * coremodule is here, good morning pwhalen
15:01:33 * jlinton waves
15:02:58 <pwhalen> Morning gents, hope all is well
15:03:21 <pwhalen> Lets get started
15:03:27 <pwhalen> #topic 1) ==== Userspace Status  ====
15:03:39 <pwhalen> Any new user space issues to discuss?
15:03:52 <pbrobinson> nothing of note from me on userspace
15:04:11 <pwhalen> #info Nothing reported.
15:04:23 <pwhalen> #topic 2) ==== Kernel Status ====
15:04:30 <pwhalen> #info kernel-5.15.0-0.rc7.53.fc36
15:04:30 <pwhalen> #link https://koji.fedoraproject.org/koji/buildinfo?buildID=1848907
15:05:12 <pwhalen> RC7 is building now, has anyone seen any issues with RC6?
15:06:06 <pwhalen> I did some testing last week but hampered by an infrastructure issue that is still unresolved.
15:06:14 * nirik waves
15:06:23 <nirik> there's a kernel problem in rawhide. :)
15:06:44 <pwhalen> nirik: the armhfp issue?
15:06:50 <nirik> https://bugzilla.redhat.com/show_bug.cgi?id=2016782
15:06:56 <nirik> no, aarch64.
15:07:19 <nirik> I guess we will see how rc7 looks. :)
15:09:09 <pwhalen> Thanks. I'll try some hardware with that kernel
15:09:31 <pbrobinson> yea, I saw that bug
15:09:32 <jlinton> Yah, i've been hoping my hyper-v issues (random lockups) go away since its a pain to pin down.
15:09:43 <pwhalen> #info Bug 2016782 - kernel-5.15.0-0.rc6.20211021git2f111a6fd5b5.49.fc36 doesn't boot on aarch64 vm's
15:09:44 <jlinton> I wonder if its related.
15:11:37 <pwhalen> #info kernel-5.14.14-300.fc35
15:11:37 <pwhalen> #link https://koji.fedoraproject.org/koji/buildinfo?buildID=1847211
15:12:16 <pwhalen> Any new issues with 5.14?
15:13:59 <pwhalen> #info Please test and report any issues to the list or #fedora-arm.
15:14:15 <pwhalen> #topic 3) ==== Bootloader Status ====
15:14:44 <pwhalen> #info uboot-tools-2021.10-2.fc35
15:14:44 <pwhalen> #link https://koji.fedoraproject.org/koji/buildinfo?buildID=1846324
15:15:31 <pwhalen> Any issues with uboot?
15:16:33 <pwhalen> #info No reported issues.
15:16:45 <pwhalen> #topic 4) ==== Fedora 35 ====
15:16:56 <pwhalen> #info Latest Fedora 35 RC1.2
15:16:56 <pwhalen> #link https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test
15:17:07 <pwhalen> #info Blockers
15:17:07 <pwhalen> #link https://qa.fedoraproject.org/blockerbugs/milestone/35/final/buglist
15:17:53 <pbrobinson> anything of note for arm?
15:18:47 <pwhalen> #info WS GIS aarch64 doesn't accept input from wireless keyboard
15:18:47 <pwhalen> #link https://bugzilla.redhat.com/show_bug.cgi?id=2017043
15:19:11 <pwhalen> This is the only new one, seems to affect Logitech wireless keyboards.
15:19:31 <pwhalen> and only on the RPis, nano worked OK for me
15:20:03 <pwhalen> keyboard works for everything but GIS text input
15:20:15 <pbrobinson> what's "WS GIS"?
15:20:26 <pwhalen> Workstation Gnome initial setup
15:20:32 <pbrobinson> oh
15:20:49 <pwhalen> I tested another wireless keyboard and it was fine
15:20:53 <pbrobinson> weird, like it works elsewhere, it's a device and appears like a wired keyboard
15:21:15 <jlinton> (screams HW problem to me)
15:21:19 <pbrobinson> is eldest back at school or still sick?
15:21:20 <pwhalen> works for everything, setting wifi password etc
15:21:31 <pwhalen> Just not creating a user
15:22:03 <jlinton> unless there are XHCI messages in the log?
15:22:08 <pbrobinson> Is this the rpi4?
15:22:31 <coremodule> rpi3 and rpi4
15:22:32 <pbrobinson> if so there's no accelerated graphics so I wonder if it's the same of fbdev issue for wayland
15:23:14 <pbrobinson> I'm guessing it's something around fbdev + wayland, jlinton what's your thoughts, you've dug into that more than me
15:24:00 <jlinton> But its all in the setup menus, no? Its not changing X->wayland between those menus?
15:24:20 <pbrobinson> not at that stage
15:25:23 <jlinton> and the initial setup is always X, no?
15:26:15 <pbrobinson> jlinton: gnome has it's own I-S and it should be wayland
15:26:16 <jlinton> Its an odd one, but and it reminds me of the "don't use a camera flash around your rpi" kind of bug.
15:26:33 <pwhalen> I'll get some logs to add to the bug, very weird. Keyboard works for moving forward in gnome-initial-setup, can input the wireless password,  but the text box remains greyed out until another keyboard is plugged in
15:26:50 <pwhalen> text box for adding a user
15:27:47 <pwhalen> And not all wireless keyboards are affected
15:28:00 <jlinton> yah, I would look and see if the thing is getting reset at some point (due to clock changes?) I've seen it likes to reset the USB with a mouse I have, and that causes "glitching" but it usually comes back.
15:28:37 <pwhalen> Ok, will do
15:28:50 <pwhalen> The other arm bug
15:29:00 <pwhalen> #info It is possible to go through the initial setup without creating a root and without adding a user to the wheel group
15:29:00 <pwhalen> #link https://bugzilla.redhat.com/show_bug.cgi?id=2015490
15:29:52 <pwhalen> I checked F34 and its the same. On that basis it was denied blocker but we'll get it fixed for F36
15:30:07 <pbrobinson> they keyboard bug looks like its rpi4, I don't see mention of rpi3
15:30:26 <pwhalen> coremodule: also hit it on the rpi3
15:30:52 <pwhalen> iirc, I think he left a comment
15:32:52 <pwhalen> The last issue is the cloud/btrfs bug
15:32:57 <pwhalen> #info Fedora 35 aarch64 cloud image based openstack VM hangs
15:32:57 <pwhalen> #link https://bugzilla.redhat.com/show_bug.cgi?id=2011928
15:33:03 <pbrobinson> anyway it's some weird issue with that app only
15:33:43 <pbrobinson> yeah, that btrfs bug is weird, but we saw it on HW too, so I'm not convinced it's not hiding some horrible btrfs/compression bug
15:34:11 <pwhalen> Agreed, definitely concerning
15:35:36 <pbrobinson> jlinton: there was a comment around the arm relaxed memory model (I think), what are your thoughts there?
15:39:00 <pwhalen> I think that's it for arm issues. Does anyone have anything else to mention?
15:40:18 <jlinton> I don't see that comment, and only looked at that defect in passing a couple times. The oops is on the front side of an IO submission, so unless its migrated CPU's... hard to imagine a problem. But that is why digging into this isn't a 5 min thing, I have to study the code path to understand if there could be a problem, and since i've not looked at
15:40:18 <jlinton> btrfs before its going to take a bit.
15:41:31 <jlinton> Is it just TX1's having problems still?
15:41:49 <pbrobinson> jlinton: I'll see if I can see where I saw it or maybe I dreamed it
15:42:45 <pbrobinson> we've seen it on the QCom Amberwing in F-34 timeframe, pwhalen can put in more details there
15:43:31 <pwhalen> Right, Amberwing is still an issue - https://bugzilla.redhat.com/show_bug.cgi?id=1949334
15:43:55 <pwhalen> That's the only hardware I've seen it on.
15:44:00 <jlinton> there are no more amberwings, right, they got recalled... lol
15:45:01 <pbrobinson> jlinton: right, but if it's on TX1 and Amberwing I would guess it's a more wider problem and no uarch specific, but it could be any number of things so it's hard!
15:45:10 <pbrobinson> as is all filesystem stuff
15:45:37 <jlinton> yah, and btrfs fails the KISS test.
15:45:50 <jlinton> which is bad for persistent storage.
15:49:04 <pwhalen> Anything else for F35? Issues not mentioned?
15:50:02 <pwhalen> RC 1.2 is available now, help testing is appreciated!
15:51:01 <pwhalen> #topic 5)  == Open Floor ==
15:51:27 <pwhalen> Anything for open floor?
15:51:42 <nirik> There was a plea from the glibc maintainer to try and get a better 32bit arm builder env... https://bugzilla.redhat.com/show_bug.cgi?id=1920183#c15
15:52:26 <nirik> if anyone had time to play with that it would be welcome. ;)
15:53:25 <jlinton> I wonder if we could gradually move
15:53:46 <nirik> I guess best possible solution would be koji/mock just handling it... and doing 32bit containers on 64bit builder.
15:54:24 <jlinton> Yes, thats what i'm alluding to, but I suspect that at least some of the packages won't work, so being able to mark them "legacy" or something might be helpful.
15:54:47 <jlinton> but the question really is, how long is armhfp going to last?
15:54:52 <nirik> yeah, I really don't know. I think it's gonna take a lot of tinkering. ;(
15:59:25 <pwhalen> indeed. Thanks for joining everyone!
15:59:36 <pwhalen> #endmeeting