16:30:26 <dustymabe> #startmeeting fedora_coreos_meeting
16:30:26 <zodbot> Meeting started Wed Sep 22 16:30:26 2021 UTC.
16:30:26 <zodbot> This meeting is logged and archived in a public location.
16:30:26 <zodbot> The chair is dustymabe. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:30:26 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:30:26 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:30:31 <dustymabe> #topic roll call
16:30:33 <bgilbert> .hi
16:30:33 <dustymabe> .hi
16:30:33 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:30:36 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:30:37 <jbrooks> .hello jasonbrooks
16:30:42 <zodbot> jbrooks: jasonbrooks 'Jason Brooks' <jbrooks@redhat.com>
16:31:16 <travier> .hi siosm
16:31:17 <zodbot> travier: Sorry, but user 'travier' does not exist
16:31:23 <travier> .hi2 siosm
16:31:32 <travier> .hello2 siosm
16:31:33 <zodbot> travier: Sorry, but user 'travier' does not exist
16:31:34 <lorbus> .hi
16:31:35 <zodbot> lorbus: lorbus 'Christian Glombek' <cglombek@redhat.com>
16:31:36 <travier> .hello siosm
16:31:38 <zodbot> travier: siosm 'Timothée Ravier' <travier@redhat.com>
16:31:40 <dustymabe> #chair bgilbert jbrooks travier lorbus
16:31:40 <zodbot> Current chairs: bgilbert dustymabe jbrooks lorbus travier
16:31:49 <jschinta> .hi (from 46.5.175.*)
16:31:50 <zodbot> jschinta: Sorry, but user 'jschinta' does not exist
16:32:56 <jlebon> .hello2
16:32:57 <zodbot> jlebon: jlebon 'None' <jonathan@jlebon.com>
16:33:02 <mnguyen> .hello mnguyen
16:33:03 <zodbot> mnguyen: mnguyen 'Michael Nguyen' <mnguyen@redhat.com>
16:33:55 <dustymabe> thanks all for coming today
16:34:23 <dustymabe> #chair jschinta jlebon mnguyen
16:34:23 <zodbot> Current chairs: bgilbert dustymabe jbrooks jlebon jschinta lorbus mnguyen travier
16:34:28 <dustymabe> did I miss anyone
16:34:33 <saqali> .hi saqali
16:34:34 <zodbot> saqali: saqali 'Saqib Ali' <saqali@redhat.com>
16:34:45 <dustymabe> #chair saqali
16:34:45 <zodbot> Current chairs: bgilbert dustymabe jbrooks jlebon jschinta lorbus mnguyen saqali travier
16:34:55 <mnguyen> jschinta travier
16:35:14 <dustymabe> hmm
16:35:16 <mnguyen> wait nevermind i see it
16:35:17 <jschinta> .hi  (from 46.5.175.*)
16:35:18 <zodbot> jschinta: Sorry, but user 'jschinta' does not exist
16:36:03 <dustymabe> #topic Action items from last meeting
16:36:11 <dustymabe> looks like we only have one action item
16:36:18 <dustymabe> * lucab to follow up with the reporter in order to get clarity on the
16:36:20 <dustymabe> environment and on the actual problem encountered
16:36:39 <dustymabe> this was for https://github.com/coreos/fedora-coreos-tracker/issues/963 IIRC
16:36:45 <dustymabe> oh wait, not that one
16:37:05 <dustymabe> this one https://github.com/coreos/fedora-coreos-tracker/issues/947
16:37:24 <dustymabe> looks like luca did follow up and the OP responded, waiting on further discussion
16:37:39 <dustymabe> ok, will move to meeting topics
16:37:49 <jlebon> +1
16:37:57 <dustymabe> #topic New Package Request: fcoe-utils
16:38:01 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/956
16:38:08 <dustymabe> walters want to intro this one?
16:38:19 <dustymabe> (if around)
16:38:24 <walters> .hi
16:38:25 <zodbot> walters: walters 'Colin Walters' <walters@redhat.com>
16:38:28 <dustymabe> #chair walters
16:38:28 <zodbot> Current chairs: bgilbert dustymabe jbrooks jlebon jschinta lorbus mnguyen saqali travier walters
16:38:40 <walters> sure, this came from an internal chat; I don't have much more information than that
16:38:46 <walters> (that isn't in the issue)
16:39:00 <walters> just filed it as a placeholder for public discussion
16:39:12 <jaimelm> .hello2
16:39:13 <zodbot> jaimelm: jaimelm 'Jaime Magiera' <jaimelm@umich.edu>
16:39:29 <dustymabe> right - usually we'll recap the issue briefly
16:39:41 <dustymabe> which could be seen as a waste of time, but generally sets the stage for discussion nicely
16:39:46 <jaimelm> ^^
16:39:49 <jaimelm> recaps are good
16:40:19 <travier> The idea is pre-installing Fibre Channel tools on FCOS
16:40:27 <travier> Because those are needed to setup the network
16:40:46 <dustymabe> travier: but the real goal is disk access?
16:41:02 <dustymabe> i.e. remote storage
16:41:18 <walters> yeah, i think so
16:42:09 <travier> In general we have network setup tools in FCOS because we can not rpm-ostree install them if we don't have network
16:42:23 <dustymabe> walters: you had a good point about "That said, I think fcoe may not support being set up from the initramfs. If true, that weakens significantly the integration story,"
16:42:44 <dustymabe> travier: i'm not quite sure it's the same thing
16:42:59 <travier> We definitely need further information from actual FCoE users
16:43:19 <dustymabe> IIUC FCOE is a layer on top of your existing network that allows you to access fiberchannel SAN
16:44:39 <dustymabe> I think if you *can* use fibre channel + fcoe to have your root device be on a remote SAN then we should definitely include it
16:45:05 <dustymabe> if not then it's not as compelling (basically what walters said)
16:45:32 <jlebon> that's assuming some configuration isn't needed in the initramfs to find the rootfs
16:45:42 <jlebon> that would turn it into another multipath situation
16:45:42 <walters> i'm not sure about root device; that's a whole other thing akin to NFS or iSCSI root
16:46:16 <walters> thinking more about things like `/var/srv/mydatabase`
16:46:38 <jlebon> this could be a good candidate for day-1 pkglayering (where we `apply-live` or something smarter before even switching root)
16:46:54 <travier> jlebon: +1
16:47:38 <walters> possibly, but that also means one can't use ignition to make filesystems on top
16:48:03 <walters> OTOH maybe most FCoE users expect to be mounting an already (externally created) filesystem
16:49:07 <dustymabe> walters: both valid points
16:49:08 <jlebon> having Ignition make filesystems on top would require either teaching Ignition about FCoE or having something akin to rd.multipath
16:49:39 <jlebon> i don't know enough about FCoE to determine how hard either of those things are
16:51:09 <jlebon> just grepping dracut at least, `rd.fcoe` is a thing
16:51:25 <dustymabe> open questions:
16:51:32 <dustymabe> - can FCoE be used as a root device (set up in the initramfs)
16:51:37 <dustymabe> - what would need to be done to use FCoE to access a block device in the initramfs and allow ignition to create filesystems on it?
16:51:51 <jlebon> so if it's as simple as that to have the device show up, then it's plausible: the Ignition config could specify it via kargs, and then the disks stage could access it
16:53:27 <dustymabe> maybe we should propose getting more information in the ticket and see if someone wants to step up and test rd.fcoe with a provided dev image
16:53:38 <dustymabe> if no one answers the question, we don't add it
16:54:00 <jschinta> dusty: +1 (from 46.5.175.*)
16:54:16 <jlebon> seems reasonable
16:54:39 <walters> the rootfs case seems the hardest, not clear to me it's desired even
16:55:30 <dustymabe> #proposed We'll ask more questions in the ticket about how fcoe is planned to be used and offer a development image with fcoe installed to anyone to see if they can get it to work with rd.fcoe. If no one answers the call for more information then we'll leave it there.
16:56:01 <travier> +1
16:56:14 <jschinta> +1 (from 46.5.175.*)
16:56:27 <travier> (we need to ask the original reported which did not open this issue unfortunately)
16:56:45 <dustymabe> i imagine walters can find them
16:57:22 <walters> oh I'd missed that https://github.com/dracutdevs/dracut/blob/2d83bce21bfc874b29c1fb99e8fabb843f038725/modules.d/95fcoe/parse-fcoe.sh exists now
16:58:07 <dustymabe> #agreed We'll ask more questions in the ticket about how fcoe is planned to be used and offer a development image with fcoe installed to anyone to see if they can get it to work with rd.fcoe. If no one answers the call for more information then we'll leave it there.
16:58:14 <dustymabe> ETIMEOUT
16:58:23 <dustymabe> moving on to the next topic...
16:58:46 <dustymabe> #topic tracker: Fedora 35 changes considerations
16:58:51 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/856
16:59:08 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/labels/F35-changes
16:59:22 <dustymabe> looks like we have travier with one and jlebon with one ticket left
16:59:27 <travier> Working on https://github.com/coreos/fedora-coreos-tracker/issues/875 (SSSD change)
16:59:42 <jlebon> I closed the libvirt one earlier
16:59:52 <dustymabe> travier: ack thanks for picking that one up
16:59:56 <jlebon> and ravanelli tackled the LUKS one
17:00:53 <travier> https://fedoraproject.org/wiki/Changes/rpmautospec > and this one is next after SSSD
17:01:03 <jlebon> updated tracker comment
17:01:24 <dustymabe> jlebon: the other one that had your name on it was "F35: CHANGE: Remove authselect-compat package" https://github.com/coreos/fedora-coreos-tracker/issues/933
17:01:47 <dustymabe> which you apparently volunteered for (or I'm a jerk)
17:02:10 <jlebon> heh, yup I did. will tackle that one this week
17:02:13 <dustymabe> travier: thanks - should I put your name on the autospec one?
17:02:17 <dustymabe> jlebon: awesome!
17:02:30 <travier> dustymabe: pleas do
17:02:55 <dustymabe> ok sweet! thanks all - it's not fun, but is good to get those tracked down and looked at
17:03:10 <dustymabe> thanks to everyone who has volunteered over the course of f35 changes review
17:03:19 <dustymabe> next    topic    soonn
17:03:47 <dustymabe> #topic Kubernetes v1.22+ container runtime on Fedora CoreOS
17:03:53 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/767
17:04:13 <dustymabe> ok - i added the meeting label to this
17:04:40 <dustymabe> basically it's an old dicsussion about how to handle kube 1.22+ after the deprecation of docker-shim
17:04:54 <dustymabe> and in general dovetails more into our story for kube+fcos
17:05:11 <dustymabe> we now have solved the problem of getting cri-o installed without issues into FCOS
17:05:28 <dustymabe> thanks jlebon/team for adding modularity support to rpm-ostree
17:05:53 <jlebon> thanks! :)
17:06:05 <dustymabe> what are our next steps on this particular issue
17:07:34 <jlebon> i think next steps are getting together with container folks to agree on what will be supported (which versions), and how (at the modular level)
17:08:30 <dustymabe> jlebon: is the end goal to try to bake cri-o in to FCOS (i.e. containerd was chosen by dghubble because it's already there in FCOS, also it doesn't suffer the same version constraints cri-o does)
17:09:34 <jlebon> dustymabe: i think the end goal is a supported path to obtaining the cri-o they need for k8s distributions
17:09:47 <jlebon> so not baking anything in
17:10:09 <lorbus> Last I checked there was a suggestion to make a Kubernetes module that includes cri-o (at least as a sub-module)
17:10:37 <dustymabe> ok so next steps: get together with containers folks to agree what will be supported (which versions), and how (at the modular level)
17:10:38 <lorbus> I’d like that because it would ease things a bit for OKD
17:11:16 <dustymabe> lorbus: is anybody keeping kubernetes up to date in fedora ?
17:11:55 <jlebon> not currently, it's on Peter's radar
17:12:06 <lorbus> I don’t think so, but haircommander said he would look at maintaining the module nonetheless IIRC
17:12:09 <jbrooks> looks like no https://bodhi.fedoraproject.org/updates/?packages=kubernetes
17:12:29 <dustymabe> it would be nice to have someone other than me drive/own this discussion/collaboration with the containers team
17:12:35 <dustymabe> any volunteers?
17:12:58 <dustymabe> if we don't put any effort towards it we might as well close the issue with "use containerd"
17:13:07 * lorbus raises hand
17:13:16 <dustymabe> ❤️❤️❤️❤️
17:13:32 <jlebon> I can definitely help on the host side of things
17:13:52 <dustymabe> awesome - lorbus and jlebon will start a discussion with the containers folks.
17:14:02 <dustymabe> jbrooks: are you interested just from a high level perspective?
17:14:49 <dustymabe> it would be great if as part of this process we added tests for kubernetes to our CI
17:14:49 <jbrooks> dustymabe, My interest, such as it is, has shifted more to openshift
17:15:05 <dustymabe> got ya
17:15:09 <dustymabe> ok I'll update the ticket
17:15:13 <dustymabe> thanks lorbus and jlebon
17:15:47 <dustymabe> #action lorbus and jlebon to reach out to the containers team to discuss what cri-o versions will be supported and how at the modular level to pull it off
17:15:55 <dustymabe> #topic open floor
17:15:55 <jbrooks> I think what's ideal for us is making it easier for people to choose their own path w/ that stuff -- to me, package layering has always seemed like a decent option
17:16:20 <dustymabe> anything for open floor today?
17:16:23 <jschinta> I had one topic for the open floor reagrding s390x pipeline for fcos (from 46.5.175.*)
17:16:33 <dustymabe> jschinta: go for it
17:17:00 <jschinta> Basically the Problem is finding a publicly accessible LPAR (Bare Metal) Machine to run the Pipeline on (from 46.5.175.*)
17:17:41 <jschinta> My Question is if it would be possible to have the Machine behind a Firewall instead and push the resulting build back to the jenkins (from 46.5.175.*)
17:18:10 <jschinta> Because we have Machines in our Lab, but for Cloud you can only get KVM/zVM (from 46.5.175.*)
17:18:31 <jschinta> And it takes about 8h to run a build + kola in nested zVM/KVM (from 46.5.175.*)
17:18:47 <dustymabe> jschinta: maybe - though we should definitely include fedora infra/releng in any discussion
17:19:00 <dustymabe> jschinta: maybe our next step is to set up a meeting with kevin/mohan and see what our options are
17:19:14 <dustymabe> maybe we can re-use the existing s390x setup that's already being used for koji?
17:19:38 <jlebon> really hoping we follow the same SSH model as aarch64 for additional arches rather than introducing another model
17:19:52 <jschinta> i'm not involved in the setup for koji, but my proposal would be to host the LPAR inside IBM Infra (from 46.5.175.*)
17:20:15 <jschinta> jlebon: I would like to use it as well, but i can't find a public LPAR (from 46.5.175.*)
17:20:52 <dustymabe> jschinta: but maybe there is an LPAR that fedora has that we can re-use - wouldn't that be ideal?
17:21:24 <jschinta> dutymabe: That would be ideal, if you can point me to the right people to ask i can follow up on that (from 46.5.175.*)
17:21:43 <dustymabe> jschinta: will do
17:21:51 <jlebon> +1
17:21:53 <dustymabe> let's set up some time with then later this week
17:22:04 <dustymabe> and also might as well include ppc64le in the discussion - cc ravanelli
17:22:10 <jschinta> dustymabe: thanks (from 46.5.175.*)
17:22:49 <dustymabe> anything else for open floor?
17:23:26 <anthr76[m]> .hello anthr76
17:23:27 <zodbot> anthr76[m]: anthr76 'Anthony Rabbito' <hello@anthonyrabbito.com>
17:23:49 <dustymabe> hi anthr76[m]
17:23:53 <lorbus> I wanted to shout out and say thank you to everyone involved in getting the FCOS AArch64 AMIs released on AWS. Thanks Dusty et al!!!
17:24:02 <walters> I still think investing in OSBS medium term makes sense
17:24:21 <dustymabe> lorbus: ❤️
17:24:30 <dustymabe> lorbus: is OKD picking up and using them?
17:25:24 <lorbus> OKD will be, yes! Hopefully we’ll be able to do arm64 CI image builds, and then it’s just a matter of wiring things up to create OKD payloads :)
17:25:38 <lorbus> *soon
17:25:50 <lorbus> Is missing from that sentence ^
17:26:03 * dustymabe will close meeting in 60s if no new topics come up
17:26:51 <anthr76[m]> Good day! I'm a little late today, but I have two things.
17:26:51 <anthr76[m]> 1. Are there any blockers revolving this issue? https://github.com/coreos/fedora-coreos-tracker/issues/258
17:26:51 <anthr76[m]> Stable streams seem to be rolling with aarch64 now and it seems UEFI Firmware is playing nice as well. I remember a ipxe gripe early on was the gunzipped kernel. Anyone aware if that's still an issue? I was provisioning kubic but I'd like to provide a guide to getting started with k8s on Pis soon...
17:26:51 <anthr76[m]> 2. A follow up of https://github.com/coreos/fedora-coreos-tracker/issues/258 from a couple minutes. This is fully opinionated though it would be really nice to have cri-o as a first class citzen in coreos. I know the versioning makes it tough. But I think it can be manageable if we follow  k8s major version releases. I personally think moby should be switched out for crio
17:27:47 <anthr76[m]> I've raised some questions in fedora devel to also ensure the kubernetes package is staying well up to date as well. Though I like the pattern of running the kubelet in podman
17:28:10 <dustymabe> hey anthr76[m] - i'm running FCOS on my pi4 - but I don't know if it works in all cases
17:28:23 <jlebon> anthr76[m]: 2 was discussed earlier on in this meeting. TL;DR is we're working on it :)  you can track progress at https://github.com/coreos/fedora-coreos-tracker/issues/767
17:28:42 <dustymabe> I'm using https://github.com/pftf/RPi4 on an sdcard and I'm installing to a USB disk
17:28:51 <dustymabe> that model at least I know works ^^
17:28:59 <dustymabe> havne't had a chance to test out other iterations
17:29:18 <anthr76[m]> dustymabe: Awesome. Are you providing ignition from a network source?
17:30:06 <dustymabe> anthr76[m]: yeah, I was just booting from ISO (usb stick) and then running coreos-installer by hand with --ignition-url https://
17:31:13 <dustymabe> #endmeeting