20:01:06 <sgallagh> #startmeeting Server Working Group Weekly Meeting (2017-04-25)
20:01:06 <zodbot> Meeting started Tue Apr 25 20:01:06 2017 UTC.  The chair is sgallagh. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:01:06 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:01:06 <zodbot> The meeting name has been set to 'server_working_group_weekly_meeting_(2017-04-25)'
20:01:06 <sgallagh> #chair nirik sgallagh mhayden dperpeet smooge jds2001 vvaldez adamw mjwolf
20:01:06 <sgallagh> #topic roll call
20:01:06 <sgallagh> .hello sgallagh
20:01:06 <zodbot> Current chairs: adamw dperpeet jds2001 mhayden mjwolf nirik sgallagh smooge vvaldez
20:01:07 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
20:01:15 <vvaldez> .hello vvaldez
20:01:16 <zodbot> vvaldez: vvaldez 'Vinny Valdez' <vvaldez@redhat.com>
20:01:22 <mjwolf> .hello mjwolf
20:01:23 <zodbot> mjwolf: mjwolf 'Michael Wolf' <mjwolf@us.ibm.com>
20:02:20 * nirik waves
20:02:27 <tmoney_> sgallagh, i'm here
20:02:28 <tmoney_> typo
20:02:39 <sgallagh> tmoney_: Welcome
20:02:46 <sgallagh> .fas tmoney
20:02:46 <zodbot> sgallagh: chartmoney 'mohd naushad' <chartmoney.in@gmail.com>
20:02:47 <tmoney_> thanks!
20:02:51 * tmoney_ waves
20:02:52 <sgallagh> oops, that's wrong
20:03:21 <sgallagh> We'll wait two more minutes to see if anyone else shows up, then get straight to it.
20:03:36 <sgallagh> Sorry I've been unavailable to run the meeting the last couple weeks.
20:03:45 <tmoney_> sgallagh, how does this work? irc only?
20:04:11 <sgallagh> tmoney_: Yes, IRC only. If we have need of a more visual session, we can schedule one for a future meeting.
20:04:28 <vvaldez> sgallagh: np, same with me. and I’ll be out next week for Red Hat Summit
20:04:32 <sgallagh> #topic Server Roles and Ansible
20:04:36 <sgallagh> vvaldez: As will I
20:04:44 <sgallagh> (I was going to mention that in Open Floor)
20:05:40 <sgallagh> So it's been about a month since our last meeting by my estimation and a lot has happened in that time.
20:06:04 <sgallagh> I've invited tmoney_ (Terry Bowling) here today to talk about what is going on with Fedora and Ansible.
20:07:10 <sgallagh> As a quick level-set, Terry is working on formalizing a set of Ansible modules and roles that will form essentially a supportable framework for deploying and managing services.
20:07:37 * vvaldez waves at tmoney_
20:07:42 <sgallagh> tmoney_: Please provide a more detailed explanation :)
20:07:46 <tmoney_> Hi vinny!
20:07:51 <tmoney_> thanks sgallagh
20:08:05 <tmoney_> I'll expand on that a bit.  Conceptually, we were thinking...
20:08:15 <tmoney_> how can we make RHEL easier to manage
20:08:22 <tmoney_> simple and consistent accross releases
20:08:35 <tmoney_> of course, everyone loves Ansible
20:08:48 <sgallagh> (anyone who doesn't is clearly lying)
20:08:50 <tmoney_> but each admin has to do the version integration themselves
20:08:53 <tmoney_> lol
20:08:54 * nirik nods. :)
20:09:18 <tmoney_> I did meet a grumpy person who just called it fancy, glorified shell scripting
20:09:28 <tmoney_> but i don't think he had *really* used it
20:09:33 <tmoney_> anyway...
20:09:58 <tmoney_> we started thinking of how we could use ansible to make something conceptually like a "rhel/linux system api"
20:10:21 <tmoney_> so we've been working with RHEL engineers to create a framework of roles and modules
20:10:25 <tmoney_> to manage the subsystems
20:10:34 <tmoney_> sorry, the rhel/linux subsystems
20:10:42 <tmoney_> and we want to share this to the community
20:10:46 * tmoney_ grabs linky
20:11:13 <tmoney_> so...
20:11:17 <nirik> cool. What form does this take? just a wrapper over those ? or ?
20:11:18 <tmoney_> here's our POC effort  https://github.com/cockpit-project/system-api-roles
20:11:23 * sgallagh notes that his suggestion to call this effort the Open Linux Management Infrastructure was soundly rejected ;-)
20:11:42 <nirik> ha ha
20:11:45 <tmoney_> only after much laughter and affection
20:12:15 <tmoney_> we are in the process of breaking each subsystem role/module into its own repo under  https://github.com/linux-system-roles
20:12:35 <tmoney_> and our Ansible brethren are working with us to expose each of these in Ansible Galaxy
20:12:53 * dperpeet points out that this isn't something "Cockpit" tries to own, the GitHub organization is just where this happened
20:13:24 <sgallagh> dperpeet: Right, I heard today that this will get moved out of the cockpit organization in the near future
20:13:26 <tmoney_> YES!  the Cockpit team was EXTREMELY helpful and gracious in providing a place for us to incubate this idea
20:13:47 <tmoney_> I was torn on that.  I was tempted to suggest that we stay put
20:14:07 <dperpeet> thanks - and there are many more people involved beyond the Cockpit team :)
20:14:10 <tmoney_> but others wisely suggested that on its own, it my get more community contributions
20:14:18 <tmoney_> to expand to other distros
20:14:56 <tmoney_> so, i'm a slow and verbose typist
20:15:04 <tmoney_> did i explain it well enough?
20:15:07 <tmoney_> any questions?
20:16:10 <sgallagh> I have many, but I'll open the floor first :)
20:16:25 <sgallagh> I suspect jds2001 would have plenty to ask, but I think he didn't make it this week
20:16:28 <nirik> and the idea is that cockpit (for one) will use this ?
20:16:34 <dperpeet> I just wanted to point out (I didn't contribute) that https://github.com/cockpit-project/system-api-roles/tree/master/roles shows a nice cross section of different roles already
20:18:30 <dperpeet> tmoney_, correct me if I'm wrong, but Cockpit isn't necessarily the "primary" or even first consumer here
20:18:36 <sgallagh> tmoney_: This might be a good crowd to ask opinions about the "immutable roles" concept.
20:19:05 <sgallagh> I suspect mjwolf and nirik at least would have opinions from an end-user perspective.
20:19:14 <dperpeet> nirik, but it's definitely something cockpit could use
20:19:20 <tmoney_> our first 5 we are focusing on stabilizing are Networking, email/postfix, selinux, kdump, and timesync
20:19:29 <nirik> right. I can see lots of uses... ;)
20:19:39 <tmoney_> dperpeet, correct, we have a big financial partner who has provided feedback
20:19:56 <tmoney_> and tons of input from some Ansible folks who helped us fix a lot of issues early on
20:20:21 <tmoney_> we hope that other layered products will become consumers of our efforts
20:20:33 <sgallagh> dperpeet: While not a primary consumer, is it safe to say that it is at least a *blocking* consumer, given that it's basically the front door to the OS?
20:20:38 <tmoney_> such as cockpit, cloudforms/manageIQ, openstack, satellite and more
20:21:25 <dperpeet> I would venture that this is a generalization of what Cockpit and other tools have taught us we were missing before
20:21:31 <tmoney_> yes, the support of roles is a challenging topic we are discussing with the ansible team
20:21:51 <tmoney_> the whole enterprise support discussion just complicates things.  we should stop that :-P
20:22:30 <tmoney_> seriously, "immutable roles" would be helpful way for vendors to be able to keep some appropriate logic
20:22:51 <tmoney_> within the role, keeping the module smaller and more self contained
20:23:39 <tmoney_> my understanding is that currently, the logic for IF rhel DO THIS, IF debian DO THAT would live in the role
20:23:42 <sgallagh> tmoney_: I think you need to provide some background information
20:23:52 <tmoney_> but given that users are expected to modify roles
20:23:53 <sgallagh> Because your audience is lacking context.
20:24:16 <tmoney_> it is very confusing on how to say what is supported, without supporting/debugging user modified roles
20:24:21 <tmoney_> which ansible currently does not
20:24:59 <tmoney_> so if a Vendor said, "here is my roles that we have tested and support", it would be easier to treat them as a "stable interface"
20:25:33 <tmoney_> currently, we can only ask the users not to modify our vendor provided role.  instead, make a copy to do you edits
20:25:50 <tmoney_> but for support, reproduce issues using our "stable, tested role"
20:25:51 <sgallagh> tmoney_: My concern is this: if the only part of Ansible that can be considered fully stable is the module, then we need to treat the module as the API. The roles then can only be examples.
20:26:10 <sgallagh> If we want to consider certain roles to be API, then there needs to be a way for the system to consider them effectively unchangeable.
20:26:33 <tmoney_> yes, that is a great (and shorter) summary
20:26:47 <sgallagh> (and that's where it starts me wondering if we need to move more of the logic into the modules in order to ensure supportability)
20:27:14 <tmoney_> currently, I have been pointed to systemd as an example of separating distro provided unit files versus admin custom unit files
20:27:37 <tmoney_> look at me, using systemd as an example for others to follow :)
20:28:23 <sgallagh> tmoney_: I think that's actually a great example, because it's completely different from the way Ansible wants us to do things :)
20:29:07 <nirik> you can do that same thing with rolepath in ansible if you wanted (but of course it's not the default at all)
20:29:12 <sgallagh> systemd provides a declarative configuration for which you can edit some options but cannot fundamentally change the logic that is executed (without considerable effort)
20:29:54 <adamw> .hello adamwill
20:29:55 <zodbot> adamw: adamwill 'Adam Williamson' <awilliam@redhat.com>
20:30:06 <adamw> sorry, i was busy reorganizing my filing cabinet. (no, really.)
20:30:11 <sgallagh> Whereas with Ansible, the roles themselves provide most of the logic. So allowing the end-users to modify them is granting FAR more leeway to break things.
20:30:37 <sgallagh> adamw: welcome
20:31:51 <tmoney_> correct.  and from a vendor's perspective, makes it almost impossible and commercially unreasonable to support
20:32:12 <tmoney_> a good analogy is support for Shell or Kickstart files
20:32:23 <tmoney_> we support the runtime for executing those
20:32:47 <tmoney_> but supporting complex shell code or a kickstart post install script is out of scope for support
20:32:48 <sgallagh> Anything that can be modified by the end-user (without recompiling) is (to my mind) not "API".
20:33:23 <tmoney_> yes,  no matter how much I *want* reality to be different, sgallagh is correct
20:33:30 <sgallagh> tmoney_: Right, unless of course we stick the shell script into /usr/bin and treat it as a formal application :)
20:33:50 * tmoney_ groans
20:34:01 <tmoney_> do we still do that?
20:34:14 <sgallagh> tmoney_: Let's leave it at "we can"
20:34:52 <sgallagh> But in that situation, we're talking about treating the inputs and the outputs of the script as API.
20:35:09 <sgallagh> And it's placed in a portion of the filesystem that only the package manager is supposed to modify.
20:35:31 <sgallagh> Which leads me somewhat back towards the concept of "immutable roles".
20:36:18 <tmoney_> has anyone discussed the idea of immutable roles before?
20:36:38 <sgallagh> tmoney_: No, to the best of my knowledge, I made the term up during our conversation this morning :)
20:37:13 <tmoney_> lol
20:37:37 <sgallagh> The idea of API is a little fuzzier if we had a location into which only package-manager-owned roles could be placed, and these roles were essentially reserved names for playbook import.
20:38:22 <tmoney_> yeah, we still need to work out a few details
20:38:25 <sgallagh> i.e. end-users can either use these roles *exactly as written* or else they have to change the name and assume ownership.
20:38:43 <sgallagh> And that would be enforced at the "interpreter" level.
20:38:45 <tmoney_> such as, could /etc/ansible/roles have a vendor structure
20:39:00 <tmoney_> or distro
20:39:02 <sgallagh> tmoney_: I'd stay away from /etc for this example.
20:39:19 <tmoney_> hmmm
20:39:20 <sgallagh> /opt/$VENDOR would be better if the vendors wanted their own immutable versions
20:39:35 <sgallagh> /etc is *really* meant to be reserved for local administrator changes
20:39:42 <tmoney_> do you remember where tower puts its roles?
20:39:52 <sgallagh> Things that are *expressly* not provided by the vendor
20:40:02 <sgallagh> I have no idea. nirik ?
20:40:10 * tmoney_ looks
20:41:19 <nirik> I have 0 idea about any tower thing
20:41:22 <tmoney_> /var/lib/awx/projects
20:41:23 <nirik> never used it
20:41:28 <tmoney_> weird location
20:41:42 <nirik> I would expect something like /usr/share/whatever/ would be best?
20:41:49 <tmoney_> and this is coming down the pipe https://github.com/ansible/ansible/pull/23038
20:41:57 <nirik> thats ro arch independent area
20:42:16 <tmoney_> so, let me rephrase...
20:42:35 <tmoney_> so, if we had a distro structure like:
20:42:40 <sgallagh> Well, if we consider Tower as if it was a real third-party (not providing content through our RPM repos), it probably belongs in /opt/share/ansible,
20:42:55 <tmoney_> actually, not distro
20:43:01 <tmoney_> the roles can handle distros
20:43:03 <tmoney_> vendors...
20:43:12 <sgallagh> Maybe I am misremembering the details of /opt layour
20:43:14 <sgallagh> *layout
20:43:32 <tmoney_> /$rolepath/RedHat/rhel-system-roles/networking/
20:43:50 <tmoney_> /$rolepath/linux-system-roles/networking/
20:44:13 <tmoney_> the first would where our rpm's could install our supported copies
20:44:37 <tmoney_> or if galaxy supports signed roles, we could support that as well
20:44:57 <tmoney_> and the second example would be the community version pulled from galaxy the normal way
20:45:18 <tmoney_> or maybe the proper syntax would be /$rolepath/linux-system-roles.networking/
20:45:42 <sgallagh> I think you skipped a step in explaining what the difference is between those two things.
20:45:51 <tmoney_> but having a vendor directory would allow a way for a vendor to provide role bundles for various products
20:46:16 <sgallagh> Because from my perspective, the latter sounds like something a user might fiddle with, but the former would be something that you'd expect to get via dnf
20:46:36 <tmoney_> oh, the idea would be that vendor dirs would not be modified, so that we could support it
20:46:40 <sgallagh> tmoney_: I suspect we're probably agreeing, but your examples are insufficiently descriptive
20:46:47 <linuxmodder> .hello linuxmodder
20:46:48 <zodbot> linuxmodder: linuxmodder 'Corey W Sheldon' <sheldon.corey@openmailbox.org>
20:46:50 <tmoney_> yeah, i'm a lazy typer
20:47:17 <tmoney_> /$rolepath/RedHat/rhel-system-roles/networking/
20:47:32 <tmoney_> /$rolepath/RedHat/some-other-product-role-bundle
20:47:47 <tmoney_> so, ^^ those would be vendor supported and not modified by users
20:48:04 <tmoney_> users could consume the community versions from galaxy
20:48:10 <sgallagh> ok...
20:48:13 <tmoney_> /$rolepath/linux-system-roles/networking/
20:48:27 <tmoney_> ^ would be the community supported galaxy version that users could tinker with
20:48:28 <sgallagh> So I think the problem here is you're using $rolepath in both places.
20:48:32 <sgallagh> Which can't work :)
20:48:47 <tmoney_> yes, i'm mixing aren't I
20:48:54 <sgallagh> Either that path is managed by a package manager, or it's open to user editing, but it shouldn't be both
20:49:55 <tmoney_> well... everything is modifiable.  the users will simply su
20:49:56 <sgallagh> (Yes, I realize I may be ratholing into technical detail, but it's significant enough that it's impacting my ability to be sure I am following along)
20:50:17 <sgallagh> tmoney_: No, because paths owned by the package manager will get overridden without notification when the package updates.
20:50:30 <sgallagh> And that's right and proper for supported, "immutable" roles.
20:50:40 <tmoney_> true, but that is not the same as immutable
20:50:53 <sgallagh> Hence the quotes around it :)
20:51:07 <tmoney_> we could make the file immutable in the filesystem, but users who want to tinker will tinker still
20:51:07 <sgallagh> Nothing is ever truly immutable, but we can make a reasonable effort
20:51:08 <tmoney_> yeah
20:51:28 <tmoney_> and that does provide a good warning sign not to proceed
20:51:50 <tmoney_> and we document well, "this is how you modify them..."
20:52:27 <sgallagh> no, we document: "you do not modify them. If they don't meet your needs, you can try to copy them and write your own over HERE"
20:52:30 <tmoney_> So, I know we are discussing these topics with the Ansible team and community
20:52:48 <tmoney_> yeah, that's what I meant.
20:53:23 * nirik thinks this isn't too bid a problem...
20:53:25 <tmoney_> I think what we need most right now is user feedback on how to improve the existing roles/modules
20:53:26 <nirik> big
20:53:34 <tmoney_> and expanding to other subsystems
20:53:58 <nirik> yeah, the way you fix something or improve it... is to get the thing upstreamed... not by hacking it locally (except for testing perhaps)
20:54:03 <tmoney_> i don't think we can solve the "immutable role" topic without the ansible folks in the discussion
20:54:11 <vvaldez> it comes down to the supportability as discussed earlier
20:54:37 <vvaldez> similar to how CloudForms automation is treated: it’s usually ruby code, we can’t support all customer hacking on that, but we recommend they make copies and tweak if desired
20:55:06 <tmoney_> vvaldez, how is CF or OCP sharing their ansible roles/modules?
20:56:20 <vvaldez> I’m not as close to CF, but they do have the upstream repo for manageiq.org. for OCP, the primary location is openshift-ansible-contrib https://github.com/openshift/openshift-ansible-contrib
20:58:19 <smooge> sorry
20:58:49 <sgallagh> We are just about at the top of the hour now. tmoney_ Have you any closing remarks?
20:59:22 <sgallagh> Please make sure to keep server@lists.fp.o in the loop on any discussions around this, as I think we'll all find it useful (and want to provide feedback)
20:59:43 <tmoney_> Nothing else from me at this time
20:59:56 <tmoney_> except watch those urls for changes this week
21:00:07 <tmoney_> and we would love test feedback!
21:00:15 <sgallagh> tmoney_: Then once again, thank you for joining us this week. It was very interesting (and it's great to see that this is moving forwards)
21:00:15 <tmoney_> we'll talk about it at summit next wee
21:00:18 <tmoney_> week
21:00:25 <sgallagh> #topic Open Floor
21:00:26 <tmoney_> https://rh2017.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=102399
21:00:36 <sgallagh> #link https://rh2017.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=102399
21:00:40 <tmoney_> as well as during the RHEL Roadmap presentation
21:00:41 <sgallagh> Come see more at Summit ^^
21:00:50 <tmoney_> thanks everyone!
21:00:52 <tmoney_> sgallagh++
21:00:58 <smooge> sgallagh++
21:00:59 <zodbot> smooge: Karma for sgallagh changed to 11 (for the f25 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
21:01:01 <sgallagh> I will be manning a booth at the Open Hardware Lab next week during this meeting as well
21:01:02 <smooge> thanks you
21:01:03 <vvaldez> thanks tmoney_
21:01:25 <sgallagh> Does anyone want to cover this meeting or shall we pre-emptively cancel it?
21:01:32 <smooge> I would say cancel
21:01:34 <vvaldez> and I’ll be fantically trying to make sure my 2 labs are functioning properly next week
21:01:47 <dperpeet> I vote for cancel
21:01:55 <smooge> I don't think people are going to be 'free' enough even if they aren't summiting
21:01:56 <mjwolf> I vote cancel
21:02:05 <sgallagh> #info No Server SIG meeting next week.
21:02:11 <vvaldez> ack
21:02:23 <sgallagh> tabowling++
21:02:23 <zodbot> sgallagh: Karma for tabowling changed to 1 (for the f25 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
21:02:34 <sgallagh> (had to look up your FAS account)
21:02:51 <sgallagh> Thank you everyone for coming this week.
21:03:05 <sgallagh> tmoney_: Many thanks
21:03:20 <dperpeet> thanks everyone
21:03:25 <nirik> thanks
21:03:27 <sgallagh> #endmeeting