16:31:30 <dustymabe> #startmeeting fedora_coreos_meeting
16:31:30 <zodbot> Meeting started Wed Mar 30 16:31:30 2022 UTC.
16:31:30 <zodbot> This meeting is logged and archived in a public location.
16:31:30 <zodbot> The chair is dustymabe. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:31:30 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:31:30 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:31:35 <dustymabe> #topic roll call
16:31:36 <bgilbert> .hi
16:31:37 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:31:37 <dustymabe> .hi
16:31:40 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:31:51 <miabbott> .hello miabbott
16:31:52 <zodbot> miabbott: miabbott 'Micah Abbott' <miabbott@redhat.com>
16:31:54 <ravanelli> .hi
16:31:55 <zodbot> ravanelli: ravanelli 'Renata Renata Andrade Matos Ravanelii' <renata.ravanelli@gmail.com>
16:32:07 <skunkerk> .hello sohank2602
16:32:08 <zodbot> skunkerk: sohank2602 'Sohan Kunkerkar' <skunkerk@redhat.com>
16:32:08 <jlebon> .hello2
16:32:11 <zodbot> jlebon: jlebon 'None' <jonathan@jlebon.com>
16:32:15 <saqali> .hi
16:32:16 <zodbot> saqali: saqali 'Saqib Ali' <saqali@redhat.com>
16:32:59 <aaradhak[m]> .hi
16:33:00 <zodbot> aaradhak[m]: Sorry, but user 'aaradhak [m]' does not exist
16:33:35 <aaradhak> .hi
16:33:36 <zodbot> aaradhak: aaradhak 'Aashish Radhakrishnan' <aaradhak@redhat.com>
16:34:11 <dustymabe> #chair bgilbert miabbott ravanelli skunkerk jlebon saqali aaradhak
16:34:11 <zodbot> Current chairs: aaradhak bgilbert dustymabe jlebon miabbott ravanelli saqali skunkerk
16:34:39 <davdunc> .hello
16:34:39 <zodbot> davdunc: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1".
16:34:42 <davdunc> .hello2
16:34:42 <dustymabe> #chair davdunc
16:34:42 <zodbot> Current chairs: aaradhak bgilbert davdunc dustymabe jlebon miabbott ravanelli saqali skunkerk
16:34:43 <zodbot> davdunc: davdunc 'David Duncan' <davdunc@amazon.com>
16:34:47 <dustymabe> good to see everyone today!
16:35:35 <dustymabe> as always - if there are topics you want to discuss during the meeting please do add the `meeting` label to them or ping me and let me know
16:35:56 <dustymabe> #topic Action items from last meeting
16:36:04 <dustymabe> * davdunc to put a package review in for ec2-net-utils and brainstorm on how we can use that for #601
16:36:07 <dustymabe> * ravanelli and mnguyen and miabbott to work with dustymabe to organize the test day for f36. miabbott will attempt to document the process for future iterations
16:36:09 <dustymabe> * miabbott to send an email to the list about planned updates to the OVA
16:36:11 <dustymabe> * jlebon bgilbert travier to discuss potential solutions for #392 and update the ticket and present the outcome at a future meeting.
16:36:20 <davdunc> so..
16:36:25 <davdunc> can start with mine.
16:36:30 <dustymabe> +1
16:36:48 <davdunc> I had a meeting with Noah Meyerhans who maintains the ec2-net-utils for AWS.
16:37:00 <davdunc> we discussed using the packaging for that and the ec2-utils.
16:37:27 <davdunc> he said that he doesn't think that the udev rules are going to support what we need and that it should be used as a reference onl.
16:37:29 <davdunc> only*
16:38:20 <davdunc> so now I am thinking that whatever needs to be done, should be just handled as a separate code group.
16:38:34 <dustymabe> just noting here.. this is in relation to https://github.com/coreos/fedora-coreos-tracker/issues/601 - correct?
16:38:47 <davdunc> yes dustymabe
16:38:55 <davdunc> should probably link it.
16:39:11 <davdunc> #link https://github.com/coreos/fedora-coreos-tracker/issues/601
16:39:59 <davdunc> so while they have all of the component parts, Noah's assessment was that we can collaborate on something that fits the FCOS boot process.
16:40:59 <dustymabe> ok this is getting a little bit long but i'm inclined to dig in just a bit
16:41:08 <davdunc> dustymabe shoot!
16:41:43 <dustymabe> so CL used to symlink /dev/xvd* to the /dev/nvme - and we're trying to get that functionality back, but we (FCOS) have desired to not own the udev rules because they would be useful to more than just FCOS
16:41:52 <dustymabe> is that an accurate representation of this issue?
16:42:06 <davdunc> yes.
16:42:33 <dustymabe> ec2-net-utils owns code that does this, but we shouldn't use it?
16:42:44 <davdunc> that's definitely accurate, the cloud-base images would benefit and there are a number of other distributions that could benefit.
16:42:51 <davdunc> ec2-utils specifically.
16:43:45 <dustymabe> and you approached the ec2-net-utils maintainer about moving them into ec2-net-utils?
16:43:57 <davdunc> no.. it's the same maintainer.
16:44:04 <dustymabe> ahh ok
16:44:10 <davdunc> #link https://github.com/amazonlinux/amazon-ec2-utils
16:44:56 <dustymabe> but the outcome is that we should come up with our own udev rules for this?
16:45:31 <davdunc> Noah stated that neither package (and this is why I confused you with additional information) was in a state that was suited to be used for other distribution work.
16:45:38 <davdunc> yes.
16:46:22 <dustymabe> I see, but did he think that some pieces of those packages should be factored out into something that could be used by other distros, or did he think that wouldn't be useful
16:46:29 <davdunc> the outcome is that there are udev rules in the ec2-utils that will make a good reference, but the package as a whole is not expected to continue to provide the best practice for FCOS.
16:46:45 <davdunc> yes.
16:46:48 <jlebon> would Amazon be open to maintaining a new separate package that is fit for distro inclusion?
16:46:52 <davdunc> to answer you question simply.
16:47:20 <davdunc> jlebon: we can definitely get Amazon engineers to submit PRs to one we maintain.
16:47:52 <jlebon> and obviously, we could help adding rules too as we work through these tickets
16:47:53 <dustymabe> one original idea was to just get these udev rules into systemd: https://github.com/coreos/fedora-coreos-tracker/issues/601#issuecomment-672894089
16:47:56 <davdunc> Noah is a big contributor to Debian and maintains work over there.
16:48:06 <bgilbert> davdunc: "we maintain" - "we" is Amazon?
16:48:18 <dustymabe> bgilbert: :)
16:48:21 <davdunc> bgilbert: I'm here for Fedora. ;)
16:48:24 <bgilbert> ah
16:49:22 <davdunc> So we can get some additional guidance from Noah on how he separates them out.
16:49:32 <bgilbert> udev upstream seems like the next place to try?
16:49:47 <davdunc> bgilbert: that would be ideal.
16:50:05 <dustymabe> +1
16:50:17 <dustymabe> any actions we should assign/take on that front?
16:50:52 <dustymabe> #info for #601 after davadunc talked with Amazon engineers the code in ec2-utils and ec2-net-utils might be a good reference for other distributions, but is not promised to work. They encourage us to write our own with those as a reference.
16:51:02 <davdunc> the assignment scripting is the part we would need. looking at the metadata from the nvme and then associating the symlink to the device name (from the config) and determining it as ephemeral or not.
16:51:04 <dustymabe> let me know if I need any corrections ^^
16:51:29 <davdunc> that looks right to me.
16:51:56 <dustymabe> davdunc: are you overlapping this with https://github.com/coreos/fedora-coreos-tracker/issues/1122 ?
16:52:06 * dustymabe sees them as related but not exactly the same thing
16:52:44 <davdunc> i see them as too related to talk about one without including the other. the information is used in the same way.
16:53:02 <dustymabe> ok I wasn't sure if one required extra information embedded than the other
16:53:08 <dustymabe> fair enough
16:53:18 <dustymabe> let's get to the remaining action items
16:53:27 <dustymabe> * ravanelli and mnguyen and miabbott to work with dustymabe to organize the test day for f36. miabbott will attempt to document the process for future iterations
16:53:30 <dustymabe> * miabbott to send an email to the list about planned updates to the OVA
16:53:32 <dustymabe> * jlebon bgilbert travier to discuss potential solutions for #392 and update the ticket and present the outcome at a future meeting.
16:53:44 <jlebon> #action jlebon bgilbert travier to discuss potential solutions for #392 and update the ticket and present the outcome at a future meeting.
16:53:48 <jlebon> :)
16:53:55 <dustymabe> :)
16:54:09 <bgilbert> :)
16:54:32 <miabbott> dustymabe aaradhak ravanelli mnguyen and i met today to lay out the todo items for f36 test day - https://github.com/coreos/fedora-coreos-tracker/issues/1147
16:54:38 <miabbott> #link https://github.com/coreos/fedora-coreos-tracker/issues/1147
16:54:51 <dustymabe> miabbott: want to paste that again with #info in front?
16:55:00 <miabbott> #info dustymabe aaradhak ravanelli mnguyen and i met today to lay out the todo items for f36 test day - https://github.com/coreos/fedora-coreos-tracker/issues/1147
16:55:03 <dustymabe> yay
16:55:19 <dustymabe> one last one
16:55:28 <miabbott> #info miabbott sent an email to coreos-status for ovf changes and posted to the fedora discussion site
16:55:36 <miabbott> #link https://lists.fedoraproject.org/archives/list/coreos-status@lists.fedoraproject.org/thread/OC2FRKNF6LGHJG5G5G2XYHZ6VKCDPEVW/
16:55:42 <miabbott> #link https://discussion.fedoraproject.org/t/changes-to-vmware-ovf-metadata/37792
16:55:46 <dustymabe> thanks miabbott
16:55:48 <dustymabe> moving on
16:55:53 <dustymabe> #topic supportability of non-EOL VMware platforms
16:55:57 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1146
16:56:22 <dustymabe> I think this came up during a PR discussion
16:56:23 <miabbott> this is something that bgilbert mentioned during the review of the OVF docs PR
16:56:48 <miabbott> i just wanted to make sure the community was comfortable with the statement before committing to git  ;)
16:57:22 <dustymabe> miabbott: can you write the statement in sentence form and we can debate it
16:58:30 <miabbott> FCOS will not make any changes to the hardware version in the OVF metadata of the VMware OVA that would exclude non-EOL platforms from being supported
16:59:29 <dustymabe> IOW if the platform is not EOL FCOS image should work on it?
16:59:38 <miabbott> correct
16:59:52 <miabbott> (i have to attend another meeting now, but i can follow along here)
17:00:03 <bgilbert> I think this is reasonable.  predictability is good, and we generally shouldn't need super-advanced-latest-gizmo features from VMware
17:00:46 <bgilbert> if we do, we have a process for one-off breaking changes
17:01:08 <dustymabe> #proposed FCOS will generally try to support any non-EOL VMWare platform and we won't take steps to artificially remove support of any platform before it is EOL.
17:01:17 <dustymabe> does that capture the sentiment of the moment ^^?
17:02:02 <jlebon> should it be a bit stronger?
17:02:08 <bgilbert> I'd go for a bit stronger, yeah
17:02:09 <dustymabe> jlebon: go for it
17:02:36 <jlebon> what miabbott said SGTM
17:02:49 <jlebon> #proposed FCOS will not make any changes to the hardware version in the OVF metadata of the VMware OVA that would exclude non-EOL platforms from being supported.
17:03:05 <dustymabe> ack +1
17:03:09 <bgilbert> +1
17:03:27 <jlebon> #agreed FCOS will not make any changes to the hardware version in the OVF metadata of the VMware OVA that would exclude non-EOL platforms from being supported.
17:04:00 <dustymabe> #topic F36 FCOS Test Day TODO List
17:04:06 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1147
17:04:41 <dustymabe> So miabbott created this ticket out of our "test day organizing" meeting this morning
17:05:17 <dustymabe> We've identified work items that could be done to improve our test day coverage
17:05:49 <dustymabe> (test day is generally useful to us to help exercise our documentation and find issues there, and also gaps in our automated test coverage)
17:06:24 <dustymabe> does anyone see an item they would like to volunteer for (either creating a test case or missing documentation)?
17:06:25 <jlebon> nice!
17:06:51 <dustymabe> obviously if you have access to either nutanix or azure stack you're going to be popular here :)
17:06:57 <jlebon> happy to pick up a couple of checkboxes
17:07:23 <davdunc> I can do the testing FCOS tutorials.
17:07:32 <skunkerk> I can pick the nutanix one
17:07:34 <dustymabe> thanks davdunc!
17:08:02 <dustymabe> skunkerk++ - we'll try to find someone else to execute the nutanix test case after you write it all up
17:08:35 <fifofonix> i'd like to volunteer for something simple this go around.  open to suggestions for a newbie to test day.
17:08:38 <skunkerk> dustymabe: sounds good!
17:09:02 <aaradhak[m]> I can take up adding tests for nutanix
17:09:19 <dustymabe> aaradhak[m]: do you have access to nutantix ?
17:09:32 <dustymabe> anybody own a rpi4?
17:09:34 <aaradhak[m]> i would need to request for access
17:09:37 <ravanelli> I can go with debugging kernel crashes
17:09:42 <dustymabe> ravanelli++
17:09:53 <davdunc> should we add updates to the issue?
17:10:05 <dustymabe> davdunc: i'm noting things down and will try to update the description
17:10:20 <davdunc> dustymabe: I have an rpi4 and I can test that one too.
17:10:23 <dustymabe> fifofonix: how about something like `configuring SwapOnZRam`
17:10:35 <aaradhak[m]> i can take up debugging with toolbox
17:10:41 <dustymabe> davdunc: to be clear - in this case you are volunteering to write a test case for it
17:10:54 <dustymabe> which is mostly just linking to our documentation
17:11:00 <dustymabe> it's easy - don't worry!
17:11:09 <davdunc> ah. well in that case. I'll still do it.
17:11:11 <davdunc> :)
17:11:14 <dustymabe> aaradhak[m]: +1
17:11:15 <fifofonix> dustymabe: put me down for the ZRAM thing.  thanks.
17:11:20 <dustymabe> will do
17:11:50 <dustymabe> ok provisioning on vultr - I can handle that one
17:12:05 <dustymabe> customizing nic name -> /me makes jlebon do that one
17:12:14 <jlebon> yup sure
17:12:29 <jlebon> feel free to give me 2-3
17:12:33 <dustymabe> configuring wireguard -> I can try it out - will ring jdoss if it doesn't work
17:12:48 <dustymabe> thanks jlebon - i think that should cover most of them
17:12:50 * jdoss waves
17:13:04 <jdoss> I like WireGuard
17:13:10 <dustymabe> i think the biggest thing left was azurestack maybe (don't recall if someone had access and was willing to write docs for it)
17:13:36 * dustymabe will make mike do it :)
17:13:46 <dustymabe> he said he just got access to azurestack
17:14:26 <dustymabe> well that was super productive - everyone who signed up for something if you don't mind pinging me in #fedora-coreos with your GH username (so i don't have to track it down) that would be useful
17:14:58 <dustymabe> #info we have most work items that we identified assigned to volunteers!
17:15:19 <dustymabe> ok one other thing from the meeting this morning
17:16:21 <dustymabe> the organizers of the test day are going to get together in an informal session next Tuesday (April 5) to strategize about what test cases are left to execute and divide them up
17:16:42 <dustymabe> we'll send out an announcement to the list with invite details - all are welcome to join
17:17:08 <dustymabe> #info the organizers of the test day are going to get together in an informal session next Tuesday (April 5) to strategize about what test cases are left to execute and divide them up. All are welcome to join.
17:17:43 <dustymabe> anything else on this topic? any questions?
17:18:48 <dustymabe> #topic tracker: Fedora 36 changes considerations
17:18:52 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/918
17:19:12 <dustymabe> just wanted to circle back on this to see if there was anything new to discuss
17:19:29 <dustymabe> #info FCOS `next` stream is now based on Fedora 36 beta content
17:20:03 <jlebon> woohoo!
17:20:14 <dustymabe> #info an issue found so far with Fedora 36 is RPi4 networking seems to not work: https://github.com/coreos/fedora-coreos-tracker/issues/1145
17:21:16 <dustymabe> #info another issue found (not filed), but NM dispatchers are blocked by SELinux: https://bugzilla.redhat.com/show_bug.cgi?id=2065940
17:21:36 <dustymabe> those were mostly #info pieces
17:21:49 <fifofonix> that's a long list you're tracking dusty.  thanks again to you and the team for everything.
17:22:12 <dustymabe> actual remaining #changes discussion - I think the biggest one was related PPC64 in https://github.com/coreos/fedora-coreos-tracker/issues/1120
17:22:34 <dustymabe> ravanelli: was there any news on that front (doesn't affect us too much right now, but it's nice to know)
17:23:41 <dustymabe> fifofonix: thanks :)
17:24:22 * dustymabe moves to open floor
17:24:30 <dustymabe> #topic open floor
17:24:35 <dustymabe> anyone with anything for open floor today?
17:25:26 <fifofonix> random one.  you introduced node counting a while ago.  how many nodes do you think are out there?
17:25:57 <ravanelli> dustymabe: not yet, I will try to ping the gcc team again today
17:26:22 <dustymabe> fifofonix: it's not a perfect metric because a lot of nodes can be set up so they never talk to the internet, but last I looked the DB was at something like 15K nodes
17:26:40 <dustymabe> though the zincati data about unique checkins was much higher IIUC
17:26:58 <dustymabe> luca has a dashboard for that somewhere
17:27:00 <fifofonix> What does the zincati number look like?
17:27:23 <dustymabe> anybody remember what his last report was on that?
17:28:33 <dustymabe> I wanted to say it was above 50k - but I think that might be how many nodes have checked in over time (i.e. the countme metrics restart every week)
17:28:41 <dustymabe> let's ask him next time :)
17:29:09 <dustymabe> one thing for open floor - next week is our first meeting of the new month - anyone want to organize a video meeting?
17:29:20 <dustymabe> or know of any good topics or things to do during a video meeting?
17:29:29 <dustymabe> demos or guests to invite?
17:29:53 <jlebon> demos would be cool
17:29:57 <dustymabe> I know there was an engineer doing a bunch of work to improve our selinux story - a demo on that could be cool
17:30:31 <jlebon> yeah, still trying to test his work, but that might be a good one
17:30:46 <dustymabe> nemric was adding user systemd unit support to ignition - that would be cool to see
17:31:24 <dustymabe> ok we're out of time - let me know if anyone wants to organize that and I'll mentor you through the process
17:31:40 <dustymabe> #endmeeting