16:00:31 <sgallagh> #startmeeting Server SIG Weekly Meeting (2016-01-26)
16:00:31 <zodbot> Meeting started Tue Mar  8 16:00:31 2016 UTC.  The chair is sgallagh. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:31 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:31 <zodbot> The meeting name has been set to 'server_sig_weekly_meeting_(2016-01-26)'
16:00:31 <sgallagh> #meetingname ServerSIG
16:00:31 <sgallagh> #chair sgallagh mizmo nirik stefw adamw simo danofsatx mhayden jds2001
16:00:31 <sgallagh> #topic roll call
16:00:31 <zodbot> The meeting name has been set to 'serversig'
16:00:31 <zodbot> Current chairs: adamw danofsatx jds2001 mhayden mizmo nirik sgallagh simo stefw
16:00:37 * simo sine-waves
16:00:39 <adamw> ahoyhoy
16:00:46 <adamw> .hello adamwill
16:00:46 <zodbot> adamw: adamwill 'Adam Williamson' <awilliam@redhat.com>
16:00:48 <simo> .hello simo
16:00:49 <zodbot> simo: simo 'Simo Sorce' <ssorce@redhat.com>
16:00:50 <sgallagh> .hello sgallagh
16:00:50 <danofsatx> .hello dmossor
16:00:52 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
16:00:52 <nirik> .hello kevin
16:00:53 <bconoboy> .hello blc
16:00:55 <zodbot> danofsatx: dmossor 'Dan Mossor' <danofsatx@gmail.com>
16:00:58 <zodbot> nirik: kevin 'Kevin Fenzi' <kevin@scrye.com>
16:01:00 <danofsatx> greetings and salutations, folks
16:01:00 <sgallagh> mizmo: you are back? Welcome!
16:01:00 <zodbot> bconoboy: blc 'None' <blc@redhat.com>
16:01:06 <langdon> .hello langdon
16:01:07 * nirik is here, but also in another meeting, will try and pay attention too tho
16:01:07 <zodbot> langdon: langdon 'Langdon White' <langdon@fishjump.com>
16:01:21 <mizmo> .hello duffy
16:01:21 <sctw> .hello sct
16:01:21 <zodbot> mizmo: duffy 'Máirín Duffy' <fedora@linuxgrrl.com>
16:01:24 <zodbot> sctw: sct 'Stephen Tweedie' <sct@redhat.com>
16:01:26 <sgallagh> bconoboy: You may want to add your real name to FAS :)
16:01:54 * langdon thinks adamw needs to update zodbot to allow "ahoy" as an alias to hello
16:01:56 <jwb> hi
16:02:03 <adamw> it's 'ahoyhoy'
16:02:04 <nirik> welcome back mizmo!
16:02:07 <adamw> not a mere 'ahoy'
16:02:12 <jzb> .hellomynameis jzb
16:02:13 <zodbot> jzb: jzb 'Joe Brockmeier' <jzb@redhat.com>
16:02:17 <langdon> adamw, that is too excited for this early in the morning
16:02:31 <mizmo> thanks nirik :)
16:02:40 <adamw> langdon: if it's good enough for montgomery burns, it's good enough for me
16:02:41 <bconoboy> sgallagh: I have, I don't know why it does that.
16:02:46 <langdon> adamw, lol
16:02:56 <sgallagh> bconoboy: You must have set your info private, then
16:03:01 <mattdm> .hello mattdm
16:03:02 <zodbot> mattdm: mattdm 'Matthew Miller' <mattdm@mattdm.org>
16:03:05 <stefw> .hello stefw
16:03:07 <zodbot> stefw: stefw 'Stef Walter' <stefw@redhat.com>
16:03:27 <mattdm> mizmo hello!
16:03:47 <bconoboy> .hello blc
16:03:48 <zodbot> bconoboy: blc 'Brendan Conoboy' <blc@redhat.com>
16:03:52 <bconoboy> sgallagh: woot
16:04:04 <mizmo> hey mattdm
16:04:32 <langdon> hi mizmo!
16:05:00 <sgallagh> OK, let's get started.
16:05:05 <sgallagh> #topic Agenda
16:05:05 <sgallagh> #info Agenda Topic: Modularization
16:05:35 <sgallagh> We're going to reserve the entire meeting for a single topic, unless there is a five-alarm (or more) fire.
16:05:46 <jds2001> .hello jstanley
16:05:47 <zodbot> jds2001: jstanley 'Jon Stanley' <jonstanley@gmail.com>
16:05:55 <sgallagh> /me pauses for 30s to listen for sirens.
16:06:17 <sgallagh> OK, I hear two sirens, tops.
16:06:26 <sgallagh> #topic Modularization
16:06:26 <sgallagh> #info Guest Speaker: Langdon White
16:06:26 <sgallagh> #chair langdon
16:06:26 <zodbot> Current chairs: adamw danofsatx jds2001 langdon mhayden mizmo nirik sgallagh simo stefw
16:06:34 <sgallagh> This will be an informational meeting.
16:06:34 <sgallagh> We intend to avoid deep implementation discussions today and focus on high-level goals.
16:06:34 <sgallagh> I will be moderating the discussion to keep us on topic; if we get too far off course, please respect it if I ask you to take your thoughts to the mailing list.
16:06:41 * langdon notes in downtown boston, that isn't even worth remarking on
16:06:46 * danofsatx listens, saves questions for the end
16:07:08 <sgallagh> And with that, I'll pass the mic over to langdon to give his overview.
16:07:37 <langdon> ok.. despite having considered typing this all in advance decided to do real time.. hopes fingers are up to iit
16:08:17 <sgallagh> I will attempt to #info key points as we go
16:08:21 <langdon> so.. modularization.. in essence, open source has won.. and how does a distro keep up..
16:09:57 <langdon> modularization is the idea that we can start to work with modules as a larger granular unit than a package.. this does not mean "packages" go away.. but rather that we start to worry about the things at a higher level.. thereby allowing us to recognize some of the changes to the development world and packaging world.. and allow the tradeoffs between them as we goo
16:10:18 <mattdm> "as we goo", yes.
16:10:37 <langdon> and.. to danofsatx's point.. please don't wait til the end of qs.. else my monolugue will make my fingers hurt
16:10:50 <danofsatx> roger, ball
16:10:52 <langdon> s/goo/go
16:10:56 <langdon> :P
16:10:58 <mattdm> ohhhhhhhh :)
16:11:03 <sgallagh> langdon: It might be useful to define some stopping points for questions, though.
16:11:07 * danofsatx thought the pun was intended
16:11:13 <sgallagh> Lest we jump in right before you were about to cover the answer :)
16:11:42 <langdon> sgallagh, meh.. i can always delete or mistime things a al my siren joke above
16:11:48 <langdon> ill try to use ... as well
16:12:01 <sgallagh> OK, please continue.
16:12:32 <langdon> so.. to get started.. we want to consider a "core" module that is useful for physical, vm, and container images installs.. key being "binaries available but not all installed in all cases"
16:12:52 <langdon> but we also want to consider a few simple modules that are basically just "applications"..
16:13:39 <langdon> currently thinking httpd (in fedora but a desire for updates on their own cycle) .. howdoi ( a little python app not in fedora but fun and useful, also changes a lot)
16:13:41 <mizmo> langdon, would the core be the same for each use case or would it be tuned specifically via configs / etc for each case?
16:14:00 <simo> .oO(roles)
16:14:08 <langdon> mizmo, the "module" would be the same... but the installation on demand would be different
16:14:14 <jwb> mizmo: the core is a set of packages, which may or may not all be installed for each usecase
16:14:32 <langdon> jwb, ack
16:14:36 <mizmo> okay so a module is just the software, the specific config is layered on top?
16:14:41 <jwb> like... containers don't need a kernel.  but the kernel is probably going to be in the core module
16:14:43 <sgallagh> To visualize it, think about a DNF repo specifically for core that passes repoclosure.
16:14:44 <nirik> do you install multiple modules? or just one module is created with all your needs?
16:14:53 <sgallagh> Not necessarily for buildrequires, but just for runtime.
16:14:58 <jwb> nirik: multiple.
16:15:12 <langdon> nirik, what jwb said.. with, likely, dependencies..
16:15:17 * nirik nods.
16:15:18 <langdon> in some ways very like rpms..
16:15:21 <sgallagh> If you want to build something and call it Fedora, the foundational pieces would come from that repo
16:15:26 <nirik> metarpms. ;)
16:15:51 <langdon> nirik, or groups even.. we have many analogues.. but they are not "first order citizens"
16:16:03 <langdon> in fact ... backing up a bit...
16:16:05 <mattdm> sgallagh: passes repoclosure on its own, or on top of some other defined set?
16:16:30 <sgallagh> mattdm: Let's stay out of the implementation details. I was trying to use that as a handy visualization, rather than an implementation suggestion
16:16:33 <langdon> i kinda glossed over some of the motivations.. basically.. look at puppet (which is in my forthcoming blog post)..
16:16:44 <sgallagh> Also I was specifically talking about Core there
16:17:08 <langdon> important to fedora .. but countless problems about maintaining the "correct" version of ruby for it (and I am sure many other examples)..
16:18:00 <langdon> one of the main goals here is to allow for things to operate on a separate pace from the core OS.. in other words.. less like a "distro is released at point x" rather "an os is released at point x" and a bunch of apps are released on their own lifecycles..
16:18:35 <langdon> another motivation is "quality" a la rings.. but that isn't "quite" right.. as in the quality of a beta isn't "bad" it just isn't ready yet..
16:18:52 <langdon> the "quality" of puppet isn't bad.. it just hasn't updated to latest ruby yet..
16:19:13 <langdon> and the latest python isn't "bad" just the rest of the distro isn't ready for it yet..
16:19:27 <langdon> so .. these things need their own lifecycles.. modularity is attempting to support that
16:19:34 <stefw> So different lifecycles this implies that some modules are incompatible with others, right?
16:19:49 <stefw> So *given* different lifecycles ...
16:20:15 <jwb> yes
16:20:22 <langdon> stefw, that is a great question.. i think interactions between them might be "not supported" (in a sense).. but i think the long term goal is that they can coexist without hurting each other.. aside from deps...
16:20:34 <jds2001> stefw: unless you use a packaging mechanism that supports that???
16:20:45 <langdon> as in httpd-24-module deps core-f23 & core-f24 .. but not core-f22
16:20:53 <jds2001> stefw: not trying to go into implementation, but something like SCL's?
16:21:21 <langdon> scls are another good example of a "solution" that was never a first-order citizen.. just a way to solve a problem
16:21:41 * stefw will ask about testing later :)
16:21:55 <langdon> in other words.. we have lots of things that deal with this lifecycle symptom.. but not at the core (sorry to reuse the word) of the distro
16:22:57 <mizmo> okay another q.... modules == roles?
16:23:10 <langdon> another motivation is the i included this package for my app to work but i am only supporting it for my app.. but, now a few apps use it.. and now we have another lifecycle problem.. does app-a or app-b get to decide on an update? what if we could declare a dep as a shared one or a private-to-app one..
16:23:51 <sgallagh> mizmo: roles are possibly one implementation or a subset of modules
16:23:58 <langdon> mizmo, roles, as i understand them, are about getting something "running".. a role may require / use one or more modules. but modules are more about the logical set of things.. rather than the use of them..
16:24:00 <simo> langdon: how do we avoid balkanization and having to maintain 15 different versions of the same package ?
16:24:41 <jds2001> and importantly, keeping things around that have known security issues?
16:24:43 <mattdm> simo: What are the problems with that?
16:24:53 <sgallagh> simo: That's an open question and one that can *only* lead to complicated implementation discussions. Let's hold that for later.
16:24:58 <simo> (how do we even maintain 2 version in Fedora, given currently you cannot have more than one version per package in repos ?)
16:25:06 <sgallagh> Or I guess we can discuss the high-level maintenance burden issues.
16:25:20 <simo> I am not asking about the technical solution
16:25:23 <langdon> simo, righto.. that is a problem.. but one we currently have.. i think it is just not as explicit.. we need to still foster a community working together... but, enforcement via fiat seems to cause workarounds not solutions
16:25:26 <mattdm> simo: figuring out an implementation for that is part of the immediate work to do
16:25:28 <simo> but what is the plan to deal with thes issues
16:25:31 <stefw> yes, a guess a question of "who is responsible"?
16:25:42 <stefw> are modules "supported" (as in security) by Fedora itsealf?
16:25:59 <sgallagh> I'd suggest that each module should probably have its own SIG maintaining it.
16:26:01 <langdon> the "fedora modules" are .. but their would also be a copr concept as well
16:26:20 <sgallagh> And if the SIG falls into a lack of maintenance, the module is retired.
16:26:22 <langdon> sgallagh, maybe sets of modules.. otherwise there may be a LOT of sigs
16:26:30 <mattdm> People who need it for their module are responsible. If a version maintained to a high standard by someone else exists, great. If not, don't stop them from doing it as long as bar for security responsiblity is met
16:26:35 <stefw> so therte is a strong incentive to Fedora to keep the modules it supports on the same schedule?
16:26:43 <stefw> at least until/if tooling makes it a non-issue?
16:26:48 <mattdm> sgallagh: SIG is too big a construct
16:26:57 <mattdm> something smaller than SIG
16:26:58 <langdon> stefw, not nec....i am not seeing the logic leap there
16:27:03 <langdon> stefw, ohhh
16:27:03 <sgallagh> mattdm: Perhaps, but we don't have a smaller one today.
16:27:08 <nirik> maintainers/co-maintainers, like packages probibly
16:27:16 <mattdm> sgallagh: Sure we do. .... yeah what nirik said
16:27:17 <sctw> stefw: Yes, very much so.  Although we would at least have the ability to have differing schedules in special cases where we want it ---
16:27:28 <langdon> ok.. lets be clear.. a REQUIREMENT of modularity is significant tooling change/automation/improvement..
16:27:29 <sgallagh> OK, I suppose that's true.
16:27:40 <sgallagh> #info a REQUIREMENT of modularity is significant tooling change/automation/improvement
16:27:58 <sctw> stefw: eg. if Fedora atomic host wants to move kube+docker faster, then that would be possible if we have a distinct "container support" module.
16:28:22 <sctw> stefw: But still keep the core on the same schedule.
16:28:23 <langdon> we need to have automated rebuilds of modules.. we need basic testing of a module on security updates automatically.. we need to build all that out for modules.. to a much greater degree than we do with rpms..
16:28:23 <stefw> It seems like another requirement (or perhaps consequence) is "more maintainers" or more interested parties?
16:28:41 <langdon> because we don't want to update a module on the end system if it hasn't been tested as a unit
16:28:42 <mattdm> stefw: On "supported" — I'd like to have levels, ranging from "very centrally important -- critpath" to "gets security attention and help" to "copr-like edge" to "third party"
16:28:58 <stefw> interesting
16:29:41 <langdon> we also need to improve the "app -> module in fedora" path.. like really lower the barrier (of effort not quality) in an app developer maintaining their own module..
16:30:06 <sgallagh> On the tooling side: I'd say a prerequisite for this is going to be Continuous Deployment-based testing of modules right from the start.
16:30:08 <sctw> +1, and to relate that to Simo's question earlier ---
16:30:14 <simo> langdon: do we need a new packaging system for that ?
16:30:20 <mattdm> stefw: A high-level hope is to move the focus of a lot of contributor activity away from working on individual tiny dep rpms to larger chunks.
16:30:20 <langdon> but that is longer term.. for now.. we are just playing with modules as rpm repos.. the simplest case.. so a repo ~= module.. with some extra metadata and some client side tooling
16:30:23 <sgallagh> sctw: What were you +1-ing?
16:30:52 <sctw> langdon's app-to-module path.  If a dependency is brought in purely to support one particular app, and the maintainer has no real interest in the dependency itself,
16:30:55 <sgallagh> simo: Maybe new tooling, but there's some work underway to help with automating the creation of RPMs
16:30:59 <mattdm> make dealing with libraries and dependent modules less time consuming, freeing effort for applications people really care about
16:31:11 <simo> sgallagh: I am thinking of delivery
16:31:20 <sctw> then it may actually be just _fine_ to support multiple instances of that.  Mark those dependencies as belonging just to the app.
16:31:43 <nirik> there probibly should be levels of importance inside modules...
16:31:45 <sctw> And if somebody wants to maintain the dependency as an independent thing in its own right, then it belongs in some module of its own, not in the app module.
16:31:48 <simo> the problem with rpms is that we'll end up with app-a requiring foo-1.1 and app-b requiring foo1.2
16:31:51 <sgallagh> simo: In my ideal world, I'd like to see this handled with RPMs and groups/metapackages. So we don't lose the flexibility we have with all the little pieces.
16:31:55 <langdon> sctw, that is kinda what i am thinking (marking as "for that app")
16:32:01 <sgallagh> They should just be the "expert-mode", not the standard way of doing things
16:32:04 <jwb> the word everyone is avoiding here is bundling.  there will be a form of bundling within modules
16:32:05 <simo> and if they are both build with standard rpms the 2 modules wll conflict
16:32:10 <sgallagh> (But that's just my 2 cents)
16:32:15 <simo> so you won't be able to install both app-a and app-b
16:32:28 <simo> unless you allow relocation/embedding of packages
16:32:35 <sgallagh> simo: No: figuring out how to cleanly parallel-install deps is a key point.
16:32:45 <jwb> nirik: i don't follow your level of importance comment
16:32:52 <langdon> jwb, well.. its bundling in the sense of being declarative about your deps.. but not in the sense an end system owner can't update the deps directly.. even if it may break an app on their machine
16:32:53 <sgallagh> And yes, bundling/relocation are on the table here, with tooling to reduce the problems inherent there.
16:33:16 <jwb> langdon: right.  but module a and module b might both contain package C, at differing verions
16:33:18 <sctw> simo: Indeed. So we may need to be able to package those bundled dependencies in a path that belongs to the app that wants them, and not have them share the system path.
16:33:22 <sgallagh> (On the table != fait accompli, for the record)
16:33:24 <simo> jwb: I was not avoiding thee word, I just gave it for granted
16:33:26 <simo> :)
16:33:28 <nirik> ie, the 'cockpit' module would have cockpit itself as critical, but 'libfoo' thats pulled in for something as much less so...
16:34:02 <jwb> nirik: i dunno.  if cockpit doesn't function without libfoo, then it's important.  if it does, why is it in the cockpit module?
16:34:08 <jds2001> but if cockpit wont work without libfoo.....
16:34:13 <sgallagh> jwb: I think he means "level of maintainer care"
16:34:13 <langdon> also.. to simo's point (sorta) earlier.. i would like the language packaging mechanisms to be more directly supported.. reducing app dev effort, simplifying packaging, not creating a new packaging system, relying on app language unbundling, etc
16:34:22 <jwb> sgallagh: still seems pretty high to me.
16:34:34 <sgallagh> jwb: They care that it works *for their app*
16:34:37 <nirik> jwb: well, it may only need some small part of libfoo and not care about anything else it does...
16:34:47 <sgallagh> They may not care at all if it works for anyone else's app
16:34:52 <nirik> so it could be upgraded and as long as that small part works, great
16:34:58 <jwb> sgallagh: yes.  but he said within modules.
16:35:01 <sgallagh> Right, or only a subset of functionality, etc.
16:35:01 <simo> langdon: I am not against, that's why I was thinking a new module-evel packaging system may be required
16:35:12 <sgallagh> jwb: I think he meant in terms of when to share and when not to
16:35:19 <sgallagh> But now I'm putting words in his mouth, so I'll stop
16:35:20 <simo> or may be we simply build a big RPM with the contents fo all the RPMs used to generate the module
16:35:24 <nirik> yeah, that.
16:35:25 <jwb> nirik: oh, yes.  but that doesn't mean we need to mark importance levels
16:35:33 <simo> a meta-rpm of sort
16:35:36 <langdon> nirik, actually.. that is very much what i mean.. essentially a module only tests its deps for the api it needs.. if a dep becomes popular enough, it might get its own api test
16:35:43 <nirik> importance might have been the wrong word...
16:35:48 <simo> that exposes only the deps you want to make public, and not every singlle package
16:35:56 <simo> but that would require some relocation/smarts
16:36:00 <langdon> simo, i really don't want a new packaging method.. i would much rather embrace all the existing ones
16:36:30 <simo> langdon: it doesn't matter what packaging tool you use underneath, but you have to deliver a module and be able to coherently introspect it
16:36:53 <sgallagh> simo: +1
16:36:56 <simo> call it a meta-package, you need info on all files, where they go, how to verify if they are still what was delivered, etc...
16:37:00 <sgallagh> That *will* need new tooling.
16:37:01 <langdon> simo, ahh i see what you are saying.. in that sense.. yes..
16:37:39 <langdon> but.. this "packaging" would not actually package the binary like rpm does.. we are imagining a split channel.. metadata that maps to binaries through two channels
16:38:13 <bconoboy> simo: You might run into the issue that some "modules" have pieces that don't need to be installed (Like the base distribution module) whereas on the app spectrum it's more likely all the packages in the module need to be installed.
16:38:16 * langdon is impressed with his typing speed
16:38:18 <sgallagh> I think it's best if we discuss only what information we need and stay away from the exact mechanism that writes bits to disk
16:38:40 <jwb> i don't mean to discourage discussion, but does everyone understand the motivations here?
16:38:42 <sgallagh> bconoboy: Well, that's not firm either.
16:38:48 <sgallagh> Consider an Eclipse module.
16:38:48 <jwb> if not, we should go back up a few thousand feet
16:38:54 <bconoboy> sgallagh: Ah, true
16:39:01 <bconoboy> jwb: +1
16:39:08 <jds2001> jwb: somewhat
16:39:20 <jds2001> jwb: but backing up might not be a bad idea
16:39:33 <sgallagh> Yeah, let's enumerate the problems we believe we can solve
16:39:35 <mizmo> so motivations:
16:39:49 <mizmo> 1) allow software to operate at a difference pace from OS
16:39:58 <jds2001> let me try and articulate what I've got: We need a way for applications to move indepentently of Fedora
16:40:04 <mizmo> 2) allow for the latest version of foo to be available more easily
16:40:11 <sgallagh> #info Motivation 1) allow software to operate at a difference pace from OS
16:40:27 <sgallagh> mizmo: 2) I think is incomplete
16:40:30 <mizmo> 3) solve the problem where someone pulls in some kind of dep and only that app uses it, then suddenly another app needs it, who gets to decide when to update
16:40:30 <langdon> clarification on foo?
16:40:47 <langdon> 3) reworded:
16:40:52 <mizmo> langdon, eg puppet hasn't updated to latest ruby
16:41:23 <langdon> 3) make an explicit promise about whether a dep is for an app or meant to be shared
16:41:23 <sgallagh> Reworded 2) Fedora needs to be more flexible about having newer or older versions of libraries, frameworks or applications available.
16:41:27 <mizmo> langdon, i guess the wording there was poor - 2) allow latest version of foo to be available without breaking apps that rely on older versions?
16:41:39 <mizmo> much better 3 :)
16:41:45 <mizmo> anyway, that's what i got re the motivations
16:41:50 <langdon> ok.... trying 2 as well
16:42:34 <mattdm> 4 (?) - move testing from component level to application level
16:42:39 <langdon> 2) allow an application to have an independent lifecycle, including its dependencies, from other applications and the OS but only branch on deps when necessary
16:42:48 <langdon> might be a bit of 2&1
16:43:01 <langdon> mattdm, 4 = module level
16:43:15 <sgallagh> mattdm: Maybe 4) Make testing at a module level rather than individual or whole-distro level
16:43:32 <langdon> 4) move testing from implicit every package to explicit exposed public apis of a module
16:44:10 <langdon> on 4) right now we implicitly promise an api/abi for every rpm for the lifecycle of the release.. but we don't really mean it
16:44:29 <mattdm> langdon: sure, module level. But that seems circular :)
16:44:33 <langdon> i propose moving it to more explicit, limited api/abis at a less granular level
16:44:45 <nirik> so how do modules map? or is that up to the maintainer(s)? ie, if there's fedora22/23/24 core modules, would there be just one httpd module that has 3 versions depending on which core?
16:45:15 <langdon> nirik, hopefully up to the maintainer.. but in the short term, 1-1 .. moving very quickly to maintainer choice
16:45:51 * langdon hopes no one points out have more than 1 core on one system ;)
16:45:59 <langdon> s/have/having
16:46:14 <bconoboy> nirik: ideally a module can have a single source base that works on multiple versions of fedora
16:46:36 <langdon> bconoboy, i am not sure i followed that...
16:47:18 <bconoboy> Rather than having separate sources for fc22, and fc23, and fc24 you instead have sources for your module that get built on fc22, fc23, fc24.
16:47:25 <nirik> right, so there might be httpd22 httpd24 and they would be built against all the various core/modules that they have as deps...
16:47:30 <bconoboy> one source, many binaries
16:47:37 <langdon> right.. yes
16:47:40 <mattdm> write once, run everywhere!
16:47:41 <langdon> but the other way too
16:47:50 * nirik hands mattdm a cup of java
16:47:56 <mizmo> ohhh that would be nice
16:47:59 <masta> so versioned cores/bases... but rawhide modules all things?
16:48:05 * mattdm drinks that down. burns tongue.
16:48:13 <langdon> as in .. core source may offer f22, f23, f24 apis .. even though it is only one source..
16:48:33 <simo> langdon: do we have a plan on where to start and where we'll see the first artifacts ?
16:48:35 <langdon> but... i think that is a ways away
16:48:51 <langdon> simo, typing...
16:49:00 <bconoboy> yes, definitely a ways off
16:49:37 <langdon> simo, getting there.. we are updating the wiki pages right now (wiki/modularization), renewing the objective in a couple weeks, some prelim initial module attempts are getting written up now..
16:49:45 <langdon> also posting some blog posts..
16:49:54 <langdon> if i finish it today.. today.. else this week
16:50:19 <jzb> langdon & mattdm happy to help with a user-facing post on Fedora Magazine.
16:50:25 <jzb> if needed/desired
16:50:38 <langdon> jzb, actually doing commblog.. as it seemed the more appropriate
16:50:51 <langdon> i think when we have something "working" .. it would go to fed-mag
16:50:56 <jzb> groovy
16:51:06 * nirik thinks this all sounds awesome. The devil is always in the details tho.
16:51:13 <langdon> pshaw
16:51:34 <jwb> nirik: sympathy for the devil will be required
16:51:47 <langdon> well.. yes.. exactly.. thats in some ways why we have had trouble getting started and why we are starting with "simple" .. rpm-repo ~= module
16:52:08 <sgallagh> nirik: Right, the major intention of this meeting was to get a feel for "Does this sound like something we want to do?" vs "This is a terrible idea, go home and leave us alone"
16:52:13 <stefw> So is Fedora the first distro to try this with such a broad target (ie: not just "make everything into a container")?
16:52:14 <mattdm> langdon yeah I was just saying to jzb we could do a user-focused magazine post pointing to your commblog posts
16:52:22 <mattdm> or just link it in 5tftw
16:52:33 <sgallagh> stefw: As always :-P
16:52:40 <langdon> stefw, i think so.. and i think 'make it all a container' is not gonna work..
16:52:43 * nirik wonders about some kind of rpm namespacing to support this, but thats way too into the details weeds
16:52:57 * langdon mutters relocatable rpms..
16:53:23 <langdon> ok.. come on.. more questions?
16:53:33 <langdon> we haven't even used an HOUR
16:53:34 <mizmo> i have a summary i've almost got written up of this meeting, post to server-sig list?
16:53:52 <sgallagh> We talked about automatically building the modules for each base, but is there value in trying to avoid doing that entirely when possible?
16:53:57 <jds2001> mizmo: you're too fast :D
16:54:01 <jwb> langdon: so which WGs/SIGs are we going to ask to spearhead this?
16:54:18 <sgallagh> For example, if we have modules with no shared deps, it makes sense to have the same exact module just tested atop F24 and F25 cores.
16:54:21 <langdon> mizmo, mattdm or fedora-devel? or wait for the blog posts?
16:54:22 <jds2001> sgallagh: what about FTBFS against some base but not another?
16:54:31 <mattdm> mizmo++
16:54:57 <mattdm> langdon: got that first blog post ready to go?
16:55:03 <sgallagh> jds2001: Unclear today.
16:55:08 <mattdm> post it right now and add the url to mizmo's summary :)
16:55:13 <langdon> sgallagh, i think it is too early to say about optimizing builds.. i think we just "build it all" until we can tell we can short circuit.. i know sct had some thoughts on where can do that... but i would punt for now
16:55:14 <sgallagh> Needs investigation. I expect that's going to emerge quickly
16:55:31 <mattdm> I think this is important enough it should go to devel rather than just server
16:55:33 <stefw> I think posting this to a broad audience without concrete examples is going to be premature
16:55:45 <mizmo> jds2001, what is ftbfs?
16:55:46 <mattdm> stefw: okay, that's a point too :)
16:55:54 <jds2001> mizmo: fail to build from source
16:56:00 <mizmo> thanks :)
16:56:28 <sctw> langdon: Right.  The main thing with builds is that I would like automation to try all builds and tell us what breaks;
16:56:29 <sgallagh> langdon: Fair point.
16:56:36 <langdon> jwb, part of the objective-renewal for the council is to ask base and e&s to make some changes to org and goals.. and then they would spearhead it
16:56:52 <mizmo> mattdm, langdon - i was thinking just server list since its the folks who were here today and isn't broader, and the notes could be used for anyone prepping something for a broader audience
16:56:55 <sctw> regarding FTBFS across all bases, it may be ok if new packages fail to build on old cores,
16:56:57 <sgallagh> Yeah, I'm going to reiterate that:
16:56:57 <sgallagh> #info Automation is the linchpin for all of this
16:57:09 <mattdm> mizmo: sounds good
16:57:11 <sgallagh> BTW, did we ever come up with a clean version of motivations 2+?
16:57:19 <sgallagh> I only have 1 in the logs at the moment
16:57:31 <langdon> i thought i had one.. but it really did 1&2 combined.. /me looks at scrollback
16:57:36 <sctw> but the build system should tell us it fails and reject the build until we waive older builds. Ie. accepting failures should always require manual choice, the tools should tell us exactly what fails by default.
16:57:49 <langdon> 2) allow an application to have an independent lifecycle, including its dependencies, from other applications and the OS but only branch on deps when necessary
16:58:08 <mizmo> yeh i can paste all the cleans in
16:58:08 <mizmo> sec
16:58:09 <langdon> sctw, +1 ..
16:58:15 <nirik> sctw: yeah, and at that point you could decide that you don't want to keep supporting that old core or fix things so it works or whatever.
16:58:18 <mizmo> 1) allow software to operate at a difference pace from OS
16:58:22 <mizmo> 2) allow an application to have an independent lifecycle, including its dependencies, from other applications and the OS but only branch on deps when necessary
16:58:29 <mizmo> 3) make an explicit promise about whether a dep is for an app or meant to be shared
16:58:33 <mizmo> 4) move testing from implicit every package to explicit exposed public apis of a module
16:59:11 <sgallagh> #info Motivation 2) allow an application to have an independent lifecycle, including its dependencies, from other applications and the OS but only branch on deps when necessary
16:59:19 <sgallagh> #info Motivation 3) make an explicit promise about whether a dep is for an app or meant to be shared
16:59:31 <sgallagh> #info Motivation 4) move testing from implicit every package to explicit exposed public apis of a module
16:59:34 <mizmo> i'm visualizing some kind of dashboard to be able to figure out what things ftbfs against which cores across different modules...
16:59:46 <sgallagh> adamw can rewrite relval again :-P
17:00:00 <langdon> a point i am not sure was clearly made.. we don't want to ship anything that hasn't passed testing as a module.. so for example, app-a builds successfully w/ dep-d.sec-patch but app-b does not.. so app-a ships an update.. but app-b does not..
17:00:01 * mattdm is imaginging that dashboard all dark red :)
17:00:45 * adamw shoots sgallagh
17:00:51 <mattdm> langdon: as an end user, how do I know that my system is safe from the vulnerability fixed in sec-patch?
17:00:54 <langdon> so app-b is still vulnerable to --dep-d.sec-patch but.. we need client tooling to allow 1) disable app-b 2) force patch app-b 3) ignore w/ risk
17:01:02 <langdon> mattdm, ha.. was typing
17:01:13 * mattdm looks for more softballs to lob
17:01:15 <mizmo> from a UX pov, i think overall this is going to be a big win for users, who always want new hotness but can't without an OS update. i'm hoping that making it so you can build against older cores would make it easier to make new apps available for older cores?
17:01:37 * mizmo <= who builds inkscape from source to get new hotness on occasion
17:01:49 <sgallagh> mizmo: Probably, but as Time Passes, more and more stuff will likely end up bundled.
17:01:57 <langdon> app-b w/ risk seems like an ok idea on inward facing apps.. like .. what we also are supporting here is the end user system owner making explicit judgements about what risks they want to take..
17:02:04 <adamw> not to be awkward, but has anyone actually convincingly defined a 'core / apps' boundary yet? /me tends to find one person's core is another person's app. which is GTK+?
17:02:08 <sgallagh> So it will result in an increasing maintenance burden over time. But with automation, maybe not too much
17:02:31 <nirik> and modules likely will grow over time if they support older cores I bet.
17:02:32 <stefw> it'll result in a lot more maintenance burden
17:02:34 <sgallagh> adamw: We've been kind of actively avoiding talking about the desktop stack, because it's the hardest problem.
17:02:50 <stefw> supporting against the versions of Fedora is already really insane ... eg: without automation Cockpit would be impossible
17:02:51 <mattdm> mizmo: I'd like to eventually get to the point where os core and modules can be mixed and match betweeen Fedora and CentOS/RHEL
17:02:54 <sgallagh> It's one of the reasons we came to Server SIG first: the headless case is a lot more direct.
17:03:07 <adamw> sgallagh: fine, which is python-requests ?
17:03:08 <stefw> supporting against N modules * M cores will take the already insane state to the next level
17:03:11 <langdon> adamw, i think this has been a huge problem.. so we are somewhat taking the approach of "pull all the things out we can, and what is left is the core"
17:03:22 <mattdm> GTK+ is "stacks" :)
17:03:34 <langdon> stefw, but CI fixes ALL THE THINGS!
17:03:57 <stefw> if only it was a requirement before pushing *any* update to *any* package
17:04:10 <sgallagh> stefw: That's on the table for this effort.
17:04:18 <langdon> stefw, that is, i think, the plan... if i am following you correctly
17:04:19 <mizmo> mattdm, im not following - frankenlinux?
17:04:41 <sgallagh> mizmo: I think it's more Katamari Linux
17:04:43 <mattdm> four basic kinds  of modules: base system; system services (e.g. logging); infrastructure (web, database, etc.); and then actual applications
17:04:45 <mattdm> sgallagh++
17:04:49 * adamw looks forward to seeing the container truck fleet and helicopters turn up to deliver the CI cluster
17:04:57 <stefw> i guess my concern is that we are forcing a tree of packages
17:05:03 <stefw> er modules
17:05:06 <stefw> and making a graph impossible
17:05:16 <stefw> if fragmentation happens at each/many levels of the tree
17:05:21 <mattdm> mizmo: no, because the modules are big, clear deliniation points, rather than organs all sewn together
17:05:29 <stefw> therhe is no hope for tools and projects that try to make the result usable, integrated, etc.
17:05:47 <stefw> so there needs to be a balance found, and perhaps very strict CI would be that balance
17:05:48 <sgallagh> stefw: If we can get to a place where the API/ABI boundary is really what we care about rather than the package versions, I think it gets more straightforward
17:05:54 * mattdm remembers something about voltron
17:06:09 <stefw> sgallagh, indeed but in reality you have things constantly deprecating and then removing API
17:06:19 <stefw> a good example is docker
17:06:30 <stefw> dropping things out of the API (with fair warning mostly)
17:06:30 <sgallagh> Docker is not a good example of anything -_-
17:06:32 <stefw> from version to version
17:06:34 <langdon> stefw, i guess i hope for automation to help with this.. as in, module owners have deps suggested to them when they work for them.. so the automation is actually trying to minimize the number of versions of deps
17:06:37 <stefw> a good example of the problem
17:06:55 <simo> sgallagh: it is a good example of how people do things unfortunately
17:07:17 <sgallagh> simo, stefw: Yes, I was being glib.
17:07:27 <stefw> i guess if we allow modules to require certain packages or versions of API
17:07:33 <mizmo> mattdm, but i'm thinking the cores for centos vs fedora etc would be different? so how could you voltron modules
17:07:33 <stefw> and as a result make them incompatible with other versions
17:07:40 <stefw> then we'll at least model the issue quite nicely
17:07:44 <stefw> and force fixes rather than ignore them
17:07:47 <sgallagh> stefw: Yes, that's part of motivation 1, I think.
17:08:34 <mattdm> mizmo: bring along the appropriate runtime
17:09:00 <mizmo> mattdm, :( you lost me
17:09:04 <jwb> mizmo: bundling
17:09:08 * stefw has gotta run
17:09:13 <langdon> stefw, if we are constantly scanning for where modules' deps overlap, and testing versions for simplification, i think this will be better than you think.. software is much more sophisticated in recent years.. with much better backward/forward compat in libraries..
17:09:14 <mattdm> This is easier in container world....
17:09:18 <mattdm> in concrete terms:
17:09:27 <mattdm> with docker, use the fedora or centos base image
17:09:28 <sgallagh> mizmo: I think he's talking about something similar to Docker, where you carry the entire user-space runtime with you
17:09:30 <mizmo> mattdm, jwb - but doesn't the runtime have to be built against a different core? or when you say module, you mean higher level not the binaries
17:09:33 <mattdm> with xdg-app, use the centos or fedora runtimes
17:09:40 <stefw> langdon, yes it may have been worse ... but from working on Cockpit ... it's pretty bad
17:09:45 <stefw> people are constantly pushing broken stuff into Fedora
17:09:47 <langdon> stefw, lol
17:09:48 <stefw> regressions
17:09:56 <stefw> from systemd, to docker, to kubernetes, to NetworkManager, to lvm
17:10:12 <langdon> stefw, right.. but we block that from coming in.. as in all the modules fail with new dep.. so no one picks it up
17:10:17 <stefw> it's hard to think of a high level API that hasn't regressed at some point over the last couple years
17:10:32 <mattdm> mizmo: the other approach is rebuild, as people were talking about for f23/f24/f25 -- could be centos too
17:10:33 <stefw> langdon, yes, that would be one way to fix this
17:10:33 <sgallagh> stefw: nsswitch?
17:10:35 <stefw> should be a requirement
17:10:46 <mattdm> mizmo: a la EPEL, really
17:10:48 <langdon> stefw, ahh yes.. platform-api problem there.. i think that will be a harder case.. sorta..
17:10:50 <stefw> it's 2016 ... to me "API" means something remotable
17:11:05 <stefw> like DBus, REST, spawning a process, etc.
17:11:14 <sgallagh> stefw++
17:11:15 <zodbot> sgallagh: Karma for stefw changed to 1 (for the f23 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:11:16 <langdon> stefw, yeah.. i just mean the libraries that you use.. local or remote
17:11:22 <sgallagh> We need to harp on that wherever we can.
17:11:51 <stefw> langdon, right, so you're right that this is a requirement of modularization
17:11:55 <stefw> it should be a hard requirement
17:12:09 <stefw> as in CI
17:12:21 <stefw> otherwise we'd be building something that will get out of hand rapidly
17:12:29 <langdon> stefw, i almost think it is a by product.. no module is updated unless it passes its tests.. so if you push a broken shared library.. no module will pick it up
17:12:39 <jwb> mizmo: tbh, i don't think a module spanning across fedora and centos is an immediate target.  but mattdm's explanations are one approach to doing that.
17:12:54 <stefw> currently most of the tests are done by things that integrate other things
17:13:07 <stefw> the libraries and modules themselves don't do near the amount of testing required to make this work
17:13:08 <sgallagh> jwb: I agree. Let's try to keep the ocean at a low simmer for now.
17:13:09 <mizmo> mattdm, jwb: so by a module spanning fedora / centos you dont literally mean the same module, but the functionality / definition of the module
17:13:11 <langdon> stefw, yeah... glossing over the current lack of tests ;)
17:13:29 <mattdm> I am partly Dreaming Big, here, not suggesting an immediate target, but I do want it kept in mind as we design
17:13:37 <langdon> stefw, but, as with all things software these days, need to start fixing recipes not cakes
17:13:43 <mattdm> mizmo: with container approaches, maybe literally the same binary
17:13:52 <jwb> mizmo: something like that, yes.  it might be easier to think of it as "the source code for a module can be rebuilt on top of fedora and centos cores"
17:14:01 * stefw has really gotta run now ...
17:14:07 <langdon> stefw, sorry ;)
17:14:08 <jwb> mizmo: and then with containers, it might map to the same binaries too.
17:14:48 <jwb> but again, that is all an eventuality we may not even try
17:15:15 <langdon> although, if we can, it means we really did disconnect the os from the apps.. and honestly, for many apps built on scripting languages.. i bet running on multiple cores is not that hard even today
17:16:06 <sgallagh> Or even running on golang and other bundle-the-world languages
17:16:08 <mattdm> mizmo are jwb and I making sense? :)
17:16:47 <mizmo> mattdm, not 100%, i mean, i'm filling in specific examples in my head and not quite getting it in terms of what it means for end use.
17:17:18 <mizmo> mattdm, eg, if inkscape is an app module - i can install that on epel or fedora today. the diff i guess is the epel version tends to be an older version of inkscape than available in fedora (and the newest version of fedora gets the newest version of inkscape)
17:17:32 <mizmo> so the big gain here is that you could have the latest version of inkscape on either centos or fedora?
17:17:43 <mattdm> mizmo: check this out: https://wiki.gnome.org/Projects/SandboxedApps/NightlyBuilds
17:17:52 <mattdm> includes inkscape as actual example :)
17:18:00 <sgallagh> mizmo: I think the idea is that we would build tooling to bundle as much as you needed to from Fedora Latest into the EPEL module so that it would run
17:18:06 <langdon> mizmo, from an end user perspective it is just, you have all current versions of an app avail for either cent or fedora..
17:18:07 <mizmo> mattdm, yeh i have that inkscape and gimp running now
17:18:21 <mizmo> langdon, okay cool, that was the piece i was missing
17:18:26 <mizmo> \o/
17:18:36 <mattdm> and yeah, latest build of inkscape *or* old version your institution has decided to standarize on
17:18:38 <sgallagh> But they would be separate builds, not the same as installing inkscape.fc24.rpm on EPEL
17:18:45 <langdon> but.. an app will start to have to seriously think about LTS style versions.. because the end user will start to care about their version stickiness more than the target OS
17:19:46 <langdon> sgallagh, well.. from an end user perspective.. it doesn't matter.. dnf install inkscape.stable, inkscape.beta, inkscape.lts or whatever.. and you will get the right one..
17:20:04 <sgallagh> Well, hopefully :)
17:20:04 <langdon> and if a patch comes out.. inkscape gets tested with the patch before the patch is shipped..
17:20:07 <nirik> (provided there are modules for all those maintained/available)
17:20:13 <langdon> nirik, right
17:20:15 <mizmo> langdon, yes!
17:20:16 <nirik> but there could be. ;)
17:20:36 <mizmo> langdon, i love how that would integrate with the systems that already exist rather than be yet another tool/system (which the currently nightly builds are)
17:20:52 <langdon> mizmo, so.. inkscape may have the patch but openoffice may not.. because oo failed the new sec patch test...
17:21:00 <nirik> could also help with the 'can you duplicate this bug against the latest upstream git snapshot?' problem... :) sure, let me install that module, yes, here's the error.
17:21:48 <sgallagh> nirik: I hadn't thought of that. That would be brilliant.
17:22:12 <langdon> nirik, OH YES.. i forgot that one!
17:22:14 <langdon> ...
17:22:14 <nirik> (provided the git snapshot module existed and passed builds/tests)
17:22:20 <mizmo> nirik, yes thats already helped with inkscape / gimp bugs
17:22:33 <mizmo> using alex's builds
17:22:39 <sgallagh> Probably not in the first blush, but tools to hook up a git repo and do automated nightly builds would be fantastic too
17:22:47 <langdon> we can also show to the app/module provider exactly what the end user is using when the bug was filed.. including force-patched fixes..
17:23:07 <mattdm> sgallagh: aiming big: forget nightly. do it on each commit!
17:23:11 <sgallagh> langdon: What is a force-patched fix?
17:23:29 <sgallagh> mattdm: Probably unachievable for huge projects like kernel or glibc
17:23:40 <langdon> sgallagh, that is my idea from above.. where oo failed the rebuild but the end user decided they wanted the security patch enough to force it on the module
17:23:55 <nirik> sgallagh: pshaw, it's only hardware. ;)
17:24:07 <sgallagh> ...
17:24:29 <jwb> the kernel already has CI that does builds and testing on every commit
17:24:37 <sgallagh> langdon: I think we'll have to come back to that one. I have concerns, but they're too low-level for this discussion
17:24:56 <jwb> unfortunately, the kernel also supports so much diverse hardware that it is impossible to test all those CI builds on all the hardware, so bugs still happen a lot.
17:25:00 <langdon> but.. suffice to say.. an end user can exactly report what binaries are in use in a particular module, you know that the module has been approved to use with those binaries (or are warned), and so bug reports are rarely "what the h*@&*@k did they have installed besides my app"
17:25:17 <sgallagh> jwb: Doesn't building an armv7hl kernel take days? I know it took 22 hours for glibc on armv7hl this cycle...
17:25:34 <jwb> sgallagh: the CI infrastructure is run upstream.  it isn't a fedora/Red Hat thing
17:25:39 <sgallagh> /me nods
17:25:51 <jwb> sgallagh: iirc, it's Intel.  i don't think ARM is tested, but i think it is built
17:26:17 <jwb> sgallagh: and no, it doesn't take days even in fedora.  it takes around 4-6 horus
17:26:21 <sgallagh> OK, well it's a solvable problem if we get there.
17:26:30 <mizmo> okay guys i got a hard stop
17:26:32 <mizmo> we good we good?
17:26:43 <mizmo> ill send my summary to server list after i eat my food
17:27:04 * langdon jealous of mizmo with all her fancy "lunch"
17:27:34 <mizmo> heh all i have right now is hunger, i gotta figure out what "lunch" is before next meeting
17:27:59 <sgallagh> I hear that.
17:28:07 <langdon> mizmo, +1
17:28:09 <sgallagh> OK, anyone have other questions?
17:28:25 <sgallagh> Otherwise, we'll call it a day. Thank you very much for the participation.
17:28:32 <mizmo> this was really useful, thank you
17:29:47 <sgallagh> #endmeeting