16:30:21 <cverna> #startmeeting fedora_coreos_meeting
16:30:21 <zodbot> Meeting started Wed Aug 26 16:30:21 2020 UTC.
16:30:21 <zodbot> This meeting is logged and archived in a public location.
16:30:21 <zodbot> The chair is cverna. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:30:21 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:30:21 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:30:30 <bgilbert> .hello2
16:30:33 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:30:38 <cverna> #topic roll call
16:30:42 <dustymabe> .hello2
16:30:43 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:31:11 <lorbus> .hello2
16:31:12 <zodbot> lorbus: lorbus 'Christian Glombek' <cglombek@redhat.com>
16:31:22 <darkmuggle> .ello2
16:31:22 <cverna> #chair bgilbert dustymabe lorbus
16:31:22 <zodbot> Current chairs: bgilbert cverna dustymabe lorbus
16:31:25 <darkmuggle> .hello2
16:31:26 <zodbot> darkmuggle: darkmuggle 'None' <me@muggle.dev>
16:31:32 <davdunc> .hello2
16:31:34 <zodbot> davdunc: davdunc 'David Duncan' <davdunc@amazon.com>
16:31:59 <jlebon> .hello2
16:32:00 <zodbot> jlebon: jlebon 'None' <jonathan@jlebon.com>
16:32:27 <cverna> #chair darkmuggle davdunc jlebon
16:32:27 <zodbot> Current chairs: bgilbert cverna darkmuggle davdunc dustymabe jlebon lorbus
16:32:50 <aoei> .hello2
16:32:51 <zodbot> aoei: aoei 'Joanna Janet Zaitseva-Doyle' <jjadoyle@gmail.com>
16:32:53 <skunkerk> .hello sohank2602
16:32:53 <cyberpear> .hello2
16:32:54 <zodbot> skunkerk: sohank2602 'Sohan Kunkerkar' <skunkerk@redhat.com>
16:32:55 <cverna> let's give another minutes to see if anyone else will be joining
16:32:57 <zodbot> cyberpear: cyberpear 'James Cassell' <fedoraproject@cyberpear.com>
16:33:36 <cverna> #chair skunkerk cyberpear  aoei
16:33:36 <zodbot> Current chairs: aoei bgilbert cverna cyberpear darkmuggle davdunc dustymabe jlebon lorbus skunkerk
16:34:33 <cverna> #topic Looking at what's next for FCOS
16:34:53 <lucab> .hello2
16:34:54 <zodbot> lucab: lucab 'Luca Bruno' <lucab@redhat.com>
16:35:25 <cverna> So today's meeting is a bit particular we wanted to take the opportunity to look at what are the issues we would like to focus on
16:36:12 <cverna> you can find a list of the some of the issues in this hackmd https://hackmd.io/aVJiL9DUQpCDSafSBcs6ZQ?edit
16:36:18 <cverna> #chair lucab
16:36:18 <zodbot> Current chairs: aoei bgilbert cverna cyberpear darkmuggle davdunc dustymabe jlebon lorbus lucab skunkerk
16:37:08 <dustymabe> Note the items are labeled with a letter so we can easily refer to them here in discussion
16:37:13 <cverna> and this meeting is an opportunity to discuss these tickets and try to come up with a rough ideas of priority
16:37:27 <miabbott> .hello miabbott
16:37:28 <zodbot> miabbott: miabbott 'Micah Abbott' <miabbott@redhat.com>
16:37:44 <imcleod> .hello imcleod
16:37:45 <zodbot> imcleod: imcleod 'Ian McLeod' <imcleod@redhat.com>
16:37:51 <cverna> ideally we would have this kind of meeting on a regular basis and re-evaluate these priorities
16:38:04 <cverna> #chair miabbott imcleod
16:38:04 <zodbot> Current chairs: aoei bgilbert cverna cyberpear darkmuggle davdunc dustymabe imcleod jlebon lorbus lucab miabbott skunkerk
16:38:25 <cverna> any questions ? or anything not clear with what we trying to achieve today ?
16:38:51 <dustymabe> cverna: if I see an item missing from the list, what should I do as a user?
16:40:12 <cverna> dustymabe: add it to the hackmd :) then we will discuss each category (In progress, To do, etc ..)
16:40:47 <dustymabe> +1
16:41:31 <cverna> ok then let's look at the In Progress tickets
16:41:56 <cverna> in this list is there anything we want to discuss in particular ?
16:42:08 <cverna> anything/any ticket ?
16:42:28 <jlebon> i think they're all important and it's good they're in progress :)
16:42:39 <cverna> I assume that if these tickets are in progress it would make sense to complete them before starting something else
16:43:05 <jlebon> +1
16:43:08 <dustymabe> no questions from me.. I will point out that the mulit-arch thing is a POC. really hoping it yields good results and we can add other architectures to our output
16:43:35 <miabbott> i haven't looked at the related ticket in a while, but what's the priority of the multi-arch item?  is this something we need to get done sooner than later or, is it more best effort?
16:44:20 <dustymabe> miabbott: we have people asking it for us often and parallel efforts within the community to produce multi-arch artifacts
16:44:25 <jlebon> i don't think there's any hard timeline. but the sooner the better :)
16:44:39 <dustymabe> bringing it under the production pipeline umbrella mostly helps us to save everyone a lot of wasted effort
16:45:05 <cverna> dustymabe: what happen once the POC is completed ? do we have another ticket to implement the solution if viable ?
16:45:40 <dustymabe> cverna: correct. once we have sufficient investigation there we'll know what it will take to implement
16:45:49 <dustymabe> it might be a lot of work, it might be a little
16:45:57 <cverna> +1
16:46:01 <dustymabe> so we'll have to re-evaluate in a bit
16:46:04 <jdoss> .hello2
16:46:05 <zodbot> jdoss: jdoss 'Joe Doss' <joe@solidadmin.com>
16:46:19 <jdoss> Sorry I am late. Double booked hard on Wednesdays.
16:46:20 <cverna> #chair jdoss
16:46:20 <zodbot> Current chairs: aoei bgilbert cverna cyberpear darkmuggle davdunc dustymabe imcleod jdoss jlebon lorbus lucab miabbott skunkerk
16:46:51 <dustymabe> 📖 📖
16:47:07 <cverna> ok so let's move to the issues that are in the "To Do" category and try to find which one are the top 2 or 3
16:47:46 <cverna> Do we want to talk about item E - Move `next` stream to Fedora 33 https://github.com/coreos/fedora-coreos-tracker/issues/609
16:47:57 <cverna> since there is a time constraint there ?
16:48:14 <dustymabe> I think it's just something we'll have to do. Not sure if there is much to discuss
16:48:26 <aoei> I think F and G sound potentially quite nice for users- tho perhaps not super high priority
16:48:34 <aoei> but nice things are also good
16:49:03 <jlebon> yeah, agreed we should prep for f33 soon.  the mechanical work shouldn't be hard.  likely the hard part will be dealing with whatever fallout
16:49:17 <dustymabe> jlebon: indeed
16:49:54 <dustymabe> maybe we should make concrete proposals
16:50:03 <cverna> do we have anything that we thing should come before F and G ?
16:50:03 <dustymabe> i propose we drop H out of consideration.
16:50:17 <dustymabe> having a rawhide stream will be useful, but not necessary right now
16:50:36 <cverna> dustymabe:  we can move it to the bottom of the list :)
16:50:41 <lucab> I think F is still blocked on rpm-ostree supporting that in a declarative way, no?
16:50:45 <aoei> I think K is valuable though also
16:50:50 <jlebon> i reworded F a bit to be about the larger issue instead of focusing on the final bit
16:50:56 <aoei> anything that helps you troubleshoot and fix stuff is a Good Thing
16:51:12 <jlebon> lucab: indeed, there's a lot of design work there
16:51:12 <dustymabe> lucab: right. it's kind of a high level "goal" - plubming required
16:51:26 <bgilbert> K and L are valuable.  M would help us.
16:51:41 <cyberpear> rawhide stream seems like it'd be very low maintenance, and be a "super sensitive canary"
16:51:53 <bgilbert> IMO G is not a priority, since we've consistently discouraged layering.
16:52:09 <jlebon> N would be nice, but is probably not in the top 3
16:52:44 <aoei> bgilbert, good point. I like F but G is not necessary
16:52:48 <dustymabe> bgilbert: I think G will be useful (layering sugar), but I think the kargs sugar is way more useful short term (F)
16:52:53 <cyberpear> G is important, IMO
16:53:19 <cverna> so let's talk about F, lucab raised a point about a dep on rpm-ostree is that blocking that ticket ? or part of the ticket ?
16:53:23 <dustymabe> cyberpear: let's say you had to choose between F and G
16:53:43 <cyberpear> F is more time-pressing, I'd think
16:53:49 <jlebon> cverna: i'd say that's part of the ticket. i rephrased it
16:53:49 <aoei> cverna, yes i think I saw that, someone is waiting on a fix in external software I think
16:54:13 <cverna> jlebon: cool just saw that :)
16:54:20 <cyberpear> but B and G are almost the same
16:54:49 <dustymabe> I see F and G as part of the same "pipeline" so I think it would be safe to eliminate G and keep F in consideration for now
16:55:13 <cverna> I feel that we have a strong interest about F and it would make sense to keep it as the first ticket on the list
16:55:36 <cverna> +1 to move G down for now ?
16:55:39 <aoei> +1
16:55:41 <bgilbert> +1
16:55:42 <miabbott> +1
16:55:47 <jlebon> SGTM
16:56:05 <skunkerk> +1
16:56:06 <lorbus> +- 0
16:56:31 <cverna> then after F we have K how does that sounds ?
16:56:33 <jlebon> bgilbert: i think eventually there'll be extensions (layered pkgs) which we'll consider reasonable. but first we need to make it solid
16:56:38 <dustymabe> one sec
16:56:44 <dustymabe> cyberpear: yeah B and G have overlap
16:56:45 <cyberpear> the most important IMO would be B, C, E, F, K, I
16:56:47 <aoei> I'm all for K
16:56:47 <jdoss> -1 because missing packages is currently a big pain point for me
16:57:04 <dustymabe> right now I'm limiting B to "the ability to package layer even on older content"
16:57:13 <dustymabe> and I'm real close to having that problem solved
16:57:22 <dustymabe> G is more the "now let's make it ergonomic" step
16:57:34 <jdoss> but if B handles G in the short term then +1
16:57:39 <jdoss> sorry catching up.
16:58:00 <jlebon> i think R is really important
16:58:04 <lorbus> would G also need changes to the Ignition config spec?
16:58:26 <lorbus> as in fcct sugar translated to what exactly?
16:58:27 <dustymabe> lorbus: not sure exactly.. it needs to go through some design
16:58:48 <lorbus> in MCO this is done with a field on the MachineConfig right now.
16:58:54 <jlebon> we need to nail down our promise re. root disk size otherwise we risk making people angry in the best case, or lose data in the worse case
16:58:59 <aoei> true its a somewhat vague proposal as it currently stands
16:59:01 <lorbus> sorry, don't want to diverge here. keep going all :)
16:59:15 <bgilbert> lorbus: units, presumably
16:59:19 <dustymabe> current discussion topic is R Stable root image size
16:59:34 <cverna> jlebon: would you consider R to be in the top 3 ?
16:59:38 <aoei> earlier jlebon suggested I just use envsubst which works pretty well for some things
16:59:47 <dustymabe> jlebon: I think I agree and I think the work effort there is pretty light.. bgilbert can confirm
16:59:48 <aoei> no fcct sugar needed
16:59:49 <cverna> top 3 of the TO Do
16:59:51 <lorbus> bgilbert: ah yeah that should work
17:00:03 <jlebon> cverna: yes, i think so
17:00:23 <bgilbert> effort for R might be a bit more substantial if we need Ign spec changes for partition resizing
17:00:27 <bgilbert> not sure if we do
17:00:49 <aoei> how much headroom do your images typically have?
17:01:25 <dustymabe> proposal: drop M from consideration, since Q was created
17:01:33 <jlebon> bgilbert: would you agree we should address it Very Soon though?
17:01:39 <bgilbert> jlebon: yes
17:01:45 <dustymabe> when I created M I was mostly referring to the work described by Q
17:01:49 <cverna> dustymabe: +1
17:02:05 <bgilbert> aoei: headroom isn't the issue, it's that we set the size dynamically at build time, which means we risk clobbering user data when reprovisioning
17:02:10 <lorbus> +1 to axing M
17:02:14 <aoei> bgilbert, ahh okay
17:02:16 <jlebon> SGTM re. moving M down
17:02:42 <aoei> bgilbert, could you add headroom to the build time size to mitigate?
17:02:57 <cverna> So it looks like we have a top 4 for now with F, K, R and L
17:03:14 <jlebon> aoei: probably not worth getting into it right now. let's chat in the ticket
17:03:15 <cverna> anyone disagree with that top 4 ?
17:03:20 <bgilbert> aoei: see https://github.com/coreos/fedora-coreos-tracker/issues/586 for details
17:03:21 <aoei> jlebon, ok fair
17:03:29 <lucab> I've been chopping at K in the last two sprints together with abai
17:04:13 <jlebon> lucab: how would you evaluate K and L?
17:04:15 <dustymabe> lucab: nice. how far is that from being over the finish line
17:05:12 <darkmuggle> fwiw, once Q is done, jlebon has a WIP that would make M done, I think
17:05:17 <lucab> jlebon: I think K is way simpler than L
17:05:28 <cyberpear> L sounds nice to have, but it's not a deficiency relative to regular Fedora
17:05:35 <lorbus> is L really that high of a prio?
17:05:42 <jlebon> darkmuggle: indeed, that'll be fun :)
17:05:53 <dustymabe> proposal replace L with P in consideration
17:06:07 <dustymabe> for P we have a cloud env ready to tap into - we just need to wire things up
17:06:17 <aoei> that makes sense dusty
17:06:25 <cyberpear> agreed, P sounds much easier than L to solve
17:06:25 <dustymabe> and hope our openstack provider in mantle works (not sure how long it's been since it was touched)
17:06:33 <aoei> K is both simpler and arguably more important than P or L
17:06:34 <lorbus> fwiw greenboot is now getting some real testing with Fedora IoT and RHEL Edge, and I suspect it will play a part in L, but I'd leave it to bake for a little longer
17:06:41 <lucab> dustymabe: not too far, but there is still a bit of design work to be done
17:06:51 <dustymabe> lucab: +1
17:07:32 <lucab> my biggest unknown on K is: how to setup rpm-ostree to use offline repos
17:07:35 <jlebon> L is one of those things that sound super cool, but it's not clear how impactful it'll actually be in practice
17:07:37 <dustymabe> ok so F.K.R.P. are the top contenders right now?
17:07:54 <dustymabe> lucab: update the ostree remote ?
17:08:03 <miabbott> so close to W.K.R.P
17:08:10 <dustymabe> it's pretty easy to do
17:08:43 <lorbus> N has been an issue that has been punted due to low prio for ages....would be great to move that forward not too far into the future
17:08:50 * dustymabe assumes "offline" means "disconnected"
17:08:52 <jlebon> it depends on the definition of "offline", but yeah there are different approaches
17:09:00 <lucab> dustymabe: well yes, the whole end-to-end thing, both for repo-mirroring and client-setup
17:09:26 <dustymabe> I can share some info on that. The rpm-ostree side is easy (IIUC).
17:09:38 <cverna> ok coming back to having the next 3-4 most wanted items in the To Do section
17:09:54 <cverna> it looks like we have F, K, R and P
17:10:30 <dustymabe> sounds right to me
17:10:38 <jlebon> i like it too :)
17:10:48 <cverna> I don't think it is worth looking at the rest now, we should probably do this exercise in a month or so and see if we want to update that order or not
17:10:51 <dustymabe> as much as I hate to kick the systemd-sysusers stuff down the road (again) :)
17:11:01 <lucab> K is actually two very different subpoints, which one we care the most?
17:11:02 <lorbus> +1 to FKRP as top 4
17:11:07 <bgilbert> +1 to FKRP
17:11:08 <aoei> aw poor systemd
17:11:11 <aoei> +1 FKRP
17:11:35 <lorbus> aoei :P
17:11:42 <dustymabe> lucab: so Mirrored/Proxied vs Air-gapped ?
17:12:00 <dustymabe> should it be Proxied vs Mirrored/Air-Gapped ?
17:12:13 <lucab> dustymabe: yes. I've been mostly working on the former so far
17:12:32 <cyberpear> agreed, proxied v/ mirror/air-gapped seems more accurate
17:12:32 <dustymabe> I guess I'd need to understand the nuances between the two
17:12:55 <jlebon> maybe let's chat about that at some point? interested as well
17:13:08 <cyberpear> generally, I've needed a mirror to use in the air-gapped environment
17:13:11 <lucab> uhm no, air-gapped is basically "no network, somebody manually goes with an update disk"
17:13:34 <dustymabe> ahh. hmm
17:13:37 <aoei> that sounds like something you want to never happen
17:13:42 <cverna> yeah I don't think we need to go into the details now, let's keep K there and we can look at it further when we come close to implement it
17:13:51 <cyberpear> in my world, air-gapped is "a network of systems not connected to the internet or any other network"
17:13:53 <cverna> even tho lucab is already working on it :)
17:14:10 <dustymabe> lucab: definitely want to dig in to that soon so we can define the individual use cases and choose the ones that are highest priority
17:14:12 <cyberpear> (but sometimes could be "a single standalone system with no network at all")
17:14:29 <lucab> the former is "auto-update on, but from local infra", the latter is "auto-updates off, manual intervention with disks"
17:14:43 <aoei> cyberpear, more than one otherwise isolated systems sounds substantially less painful than a single unetworked machine lol
17:14:53 <misc> air gapped sound fancier than "my network card driver is not working"
17:15:00 <dustymabe> :)
17:15:04 <lucab> cyberpear: https://github.com/coreos/fedora-coreos-tracker/issues/261 is the latter
17:15:08 <aoei> misc, xD
17:15:22 <dustymabe> lucab: maybe we can try to define this topic in more detail and go over it next meeting?
17:15:31 <lucab> misc: apparently there are people with purposedly broken network cards ;)
17:15:31 <cverna> dustymabe: +1
17:15:56 <cyberpear> misc: agreed... I don't often have to deal with one-off standalone systems, but networked non-internet systems are quite common
17:15:59 <aoei> lucab, :o
17:16:04 <dustymabe> pending that discussion we can break out K into the one we care most about at this point
17:16:04 <lucab> dustymabe: sure, it's already two separate tickets on -tracker
17:16:34 <cverna> I think we have done a pretty good job at prioritizing these issues :) do we want to discuss anything else today ?
17:17:03 <aoei> yeah I vote to change all the .ign files to .json
17:17:15 <aoei> i keep spelling it wrong D:
17:17:21 <dustymabe> cverna: one sec
17:17:53 <cverna> aoei: :D
17:17:58 <dustymabe> #info We have determined that F.K.R.P are the items with the most interest in completing first
17:18:10 <dustymabe> #info - F. Add coherent support for modifying kernel arguments, and provide fcct sugar
17:18:18 <dustymabe> #info - K. Updates in offline/disconnected environment
17:18:28 <dustymabe> #info - R. [Stable root image size](https://github.com/coreos/fedora-coreos-tracker/issues/586)
17:18:39 <dustymabe> #info - P. [Adding OpenStack to our CI](https://github.com/coreos/fedora-coreos-tracker/issues/612)
17:19:03 <bgilbert> #info (after the items already in progress)
17:19:09 <dustymabe> correct
17:19:16 <aoei> should we open F and K for discussion as tickets as well?
17:19:42 <bgilbert> aoei: both already have tickets?
17:19:42 <dustymabe> aoei: it looks like it would be useful for a high level ticket to summarize the goals of F
17:19:43 <jlebon> yeah, it's on my plate to open a top-level tracker ticket for F
17:20:14 <dustymabe> for K it looks like we already have two tickets and we'll use next week's meeting to try to focus on which one to work on
17:20:14 <aoei> bgilbert, I assumed there was a reason they were some of the few without links to the tracker
17:20:24 <aoei> ah ok i see
17:21:00 <bgilbert> jlebon: F: https://github.com/coreos/fedora-coreos-tracker/issues/318
17:21:32 <jlebon> bgilbert: nice, thanks for adding that
17:21:47 <jlebon> will add a comment there
17:22:27 <cverna> #topic Open Floor
17:22:55 <lorbus> thanks for kicking off this discussion, cverna!
17:23:02 <bgilbert> PXE boot on the `next` stream now requires the separate rootfs image
17:23:05 <lorbus> cverna++
17:23:05 <zodbot> lorbus: Karma for cverna changed to 21 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:23:08 <cverna> I have a little question, how did you find that type of session ?
17:23:18 <aoei> yes thank you cverna and everyone else, it has been educational
17:23:22 <bgilbert> cverna++
17:23:23 <zodbot> bgilbert: Karma for cverna changed to 22 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:23:26 <aoei> cverna++
17:23:28 <cyberpear> re K: 240 seems to be about offline mirroring, to be also used with disconnected networks - 261 seems to be about a literally standalone system, bringing updates to that system on a disc or similar, or "how to ship an update for an FCOS-based appliance"
17:23:47 <cverna> lorbus: np, dustymabe has been helping a lot behind the scene :)
17:23:55 <cverna> dustymabe++
17:24:00 <miabbott> cverna: i'm a bit biased based on my $DAYJOB, but prioritizing the future work is super helpful
17:24:02 <dustymabe> 🤗
17:24:08 <lorbus> dustymabe++
17:24:20 <aoei> cverna, I think I'm in agreement with yuo that it's a good idea to do fairly often
17:24:23 <dustymabe> cyberpear: if that's the case then I agree with Luca that they are very different :)
17:25:09 <jlebon> cverna: yeah, it was definitely helpful to see it all laid out like this!
17:25:21 <aoei> good to have an overview
17:25:27 <bgilbert> cverna: yup, super helpful IMO
17:25:31 <cverna> cool, let's try to do this maybe once a month and see how that goes, if it is too often we can slow down the cadence
17:25:32 <lucab> cyberpear: yes, I think we should update 240 to avoid the term air-gapped
17:25:58 <dustymabe> #info PXE boot on the `next` stream (as of 32.20200824.1.0) now requires the separate rootfs image
17:26:39 <bgilbert> dustymabe++
17:26:53 <dustymabe> cverna: +1 for once a month.. or maybe 6 week cadence
17:27:07 <dustymabe> either way.. we'll figure it out as we go.. like everything else
17:27:29 <cverna> +1
17:27:36 <dustymabe> I can't wait to knock some of these items out
17:27:42 <aoei> dustymabe, :D
17:28:46 <cverna> dustymabe: that's the hard part, planning is easy :P
17:29:07 <aoei> feels good when you close the things tho
17:30:48 <dustymabe> Thanks cverna for running the meeting today
17:30:48 <cverna> ok thanks everyone for joining and participating
17:30:56 <cverna> I really enjoyed it
17:31:01 <cverna> #endmeeting