19:06:00 <bcoca> #startmeeting core public irc meeting
19:06:00 <zodbot> Meeting started Tue Jul  7 19:06:00 2020 UTC.
19:06:00 <zodbot> This meeting is logged and archived in a public location.
19:06:00 <zodbot> The chair is bcoca. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:06:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:06:00 <zodbot> The meeting name has been set to 'core_public_irc_meeting'
19:06:27 * shertel waves
19:06:27 <bcoca> #topic https://github.com/ansible/proposals/issues/181
19:06:33 <sdoran> \o
19:06:47 <apollo13> I am here :)
19:07:12 <bcoca> first point, 2.10 beta has shipped so no feature changes will make it in for 2.10 .. its too late at this point
19:07:19 <bcoca> any changes will now be looking at 2.11
19:07:28 <apollo13> fine with me
19:07:44 <sivel> In concept I have no arguments with the proposal
19:08:04 <sivel> But I don't think you are going to get any more of an agreement than that in this meeting for now
19:08:04 <cyberpear> agreed w the concept
19:08:23 <sivel> We need to draft a complete product requirements doc to outline the behaviors
19:08:26 <bcoca> main reason roles were not updatable was 'no role version scheme'
19:08:44 <apollo13> bcoca: right, I never realized that, but I am fine with doing that just for collections
19:08:45 <sivel> and then likely assign this work to an internal team to spend a release or part of a release, addressing this
19:09:21 <sivel> we also didn't have the ability to even compare versions early in the release
19:09:23 <cyberpear> my only concern would be to keep a hash of the installed role or collection and only overwrite if not locally modified
19:09:47 <sdoran> It would be nice to have `install --upgrade` or just `upgrade` as a subcommand.
19:09:48 <sivel> but as I outline above, I think it's premature to really go over this topic now
19:09:50 <bcoca> also we just decided collections require a semantic version for ansible-base also, that opens this up, but also explains why the feature was not thoguht of for 2.10
19:10:17 * bcoca votes for `upgrade/downgrade`
19:10:18 <cyberpear> otherwise, require --force to avoid stepping on local ad-hoc changes
19:10:36 <apollo13> bcoca: the reason against another subcommand would be the semantics
19:10:40 <bcoca> cyberpear: we already hase 'verify' so detecting 'dirty' collection should be simple
19:10:44 <sdoran> So we're in favor of the upgrade option, but a lot more design work needs to happen.
19:11:16 <cyberpear> I think with -r it should upgrade or downgrade as appropriate
19:11:20 <apollo13> ie install asd:123 would install a collection if not existant, but you'd need update asd:123 to actually update to it?
19:11:29 <cyberpear> to match the requirements
19:11:40 <apollo13> cyberpear: same can be said if you include the version on the cmd line
19:11:50 <cyberpear> agreed
19:11:53 <apollo13> or how do you see a difference between -r and …
19:11:54 <apollo13> ok
19:11:56 <cyberpear> same way pip works
19:12:42 <sivel> This is the only topic on the agenda, so I won't stop you all from going to town on this. But in concept we are agreed, but I'd ask that no one go run off and work on this
19:12:53 <cyberpear> `pip install 'ansible < 2.9'` will downgrade me to 2.8, IIRC
19:12:56 <sivel> So that we can properly scope this and create a PRD to ensure we cover everything
19:13:01 <apollo13> okay, can you help me a little bit with regard to the next steps
19:13:08 <sivel> apollo13: just gave them to you
19:13:10 <bcoca> sivel: run off and work on design in proposal ticket .. should be fine
19:13:25 <sivel> sure, just not actually develop it.
19:13:32 <bcoca> toola .. er nmvd
19:13:38 <apollo13> sivel: sorry, not up2date with the acronyms. PRD is what?
19:13:50 <sivel> We'll need to put together a team that will work with the community to create a complete product requirements doc
19:14:04 <sivel> apollo13: ^ PRD == Product Requirements Doc
19:14:16 <shertel> +1 for proper scoping
19:14:18 <sivel> But we do understand that this is needed
19:14:29 <sivel> But it will also not be limited to simply the `install` option
19:14:38 <sivel> we need to go back and do it completely for the whole CLI
19:15:21 <sivel> So it's likely that in a future release, we will assemble a project team of employees that will spend a release or part of a release addressing this
19:15:33 <apollo13> sivel: to what extend are you thinking? there appears only to be the install command that seems to be affected by that
19:15:38 <apollo13> (re whole CLI)
19:16:13 <bcoca> apollo13: we also install roles
19:16:18 <sivel> apollo13: no one sat down and scoped this from the beginning really.  Taking different user stories into account
19:16:27 <sivel> There are also missing commands that people want
19:16:28 <bcoca> and it has more implications than just at upgrade
19:16:36 <apollo13> also do you think this should be a whole kitchensink thing (ie introduce all at once) or could it be split in smaller subtasks, ie relax the requirement of force if a version is specified
19:16:40 <bcoca> specially with deps
19:17:00 <apollo13> mhm, right, which brings me to the next question: are we going to reimplement pip?
19:17:00 <sdoran> Further design proposals and discussion can (and probably should) happen in the ticket.
19:17:11 <sivel> apollo13: when we scope this work for people to actual work on, it will be broken down into smaller tasks
19:17:31 <apollo13> ok, so in the proposal ticket I can focus on the whole thing. that already helps a lot
19:18:13 <sivel> Yes, so for now we are in "pre-design" phase, we'll take the proposal and turn it into the actual design and scope before we work on it, and then we'll break it into smaller deliverables
19:18:34 <apollo13> perfect, how much do you think that roles should be part of the design consideration
19:18:48 <apollo13> ie are roles something that is already on the way to beeing deprecated or…
19:18:50 <sivel> I wouldn't focus too much on roles for now
19:19:08 <apollo13> noted
19:19:24 <sivel> we can always apply the same ideas to roles later if that is desireable
19:19:35 <bcoca> but would require 'versioning' to work
19:19:50 <apollo13> at least for the situation where you want --upgrade to actually work
19:20:01 <bcoca> which i think is a good compromise and using --force for any 'non versioned/other versioned' role scheme
19:20:06 <apollo13> if you do role install some_role=some_version it wouldn't really matter
19:20:25 <sdoran> The lack of semver for roles is what makes it really tricky.
19:20:34 <apollo13> Personally I think that split would complicate the differences between roles and collections even further
19:20:44 <sdoran> Best to focus on collections for now because roles have other issues that need to be addressed first.
19:21:15 <apollo13> ok, that sounds all good and well and it's great to know that this proposal at least has a chance of surviving :)
19:21:23 <apollo13> thank you for talking the time!
19:21:27 <apollo13> taking*
19:22:03 * cyberpear expects roles to stay around forever
19:22:03 <sdoran> I'm not disagreeing, just saying that role versioning is the wild west because we never set a standard or enforced anything, which is why we never had an update mechanism for roles: no good way to compare arbitrary version strings.
19:22:38 <sdoran> Yes, the general consensu is we want a better update mechanism for collections.
19:22:41 <bcoca> collections are now REQUIRED to use semantic versioniong .. eliminating that problem
19:22:43 <sdoran> We just want to plan and scope it correctly.
19:24:22 <apollo13> sounds good, will it help (or would it be okay) if I write down user stories on the proposal? do you have any examples of how you'd like those to look?
19:25:03 <sdoran> I'm not sure we have examples of how they should look, but you can add them to the proposal in any way you feel is legible.
19:25:23 <shertel> that would be helpful
19:26:36 <apollo13> ok, that is all I needed to know at this stage I think. Is there anything else you'd need from me?
19:28:17 <bcoca> k, we can add more to the proposal while we wait for a more formal project, so going to stop the discussion on this now and open the floor to other topics
19:28:24 <bcoca> #topic open floor
19:30:31 <apollo13> mhm if open floor means I can throw thoughts around, then I have something else: As a follow up to my proposal I was wondering if there was a way to ensure that depedencies are satisfied when a playbook is run. Image I am using "xxx.yyy" collection in my playbook and the requirements file has it "pinned" to version 10. Now if a user has version 9 installed I want to be able to tell them about that somehow. Do you have any
19:30:31 <apollo13> ideas/thoughts about that (or is that out of scope for here)
19:31:30 <apollo13> currently requirements.yaml is connected to the ansible/ansible-playbook CLI in any way. And I am not really asking for a way to auto download/update dependencies but mostly for a way to prevent dangerous situations where users forgot to upgrade
19:32:17 <sivel> apollo13: 3rd party or external tooling could be written to do that. We have no plans of adding that functionality to playbook execution
19:34:07 <bcoca> awx/tower already provides a 'install deps and execute' faciliity
19:36:02 <sivel> runtime would be complicated anyway, especially with includes
19:36:17 <sivel> static analysis is what I have been playing with lately
19:36:39 <apollo13> sivel: oh I'd be fine with requiring a requirements.yaml in those cases
19:36:52 <bcoca> he, been helping other teams with static analysis ..  fun with includes is just the start ` action: module='{{varfromcustomvarsplugin}}'`
19:36:57 <apollo13> Ie just iterate through that and the dependency metadata
19:37:20 * cyberpear happy `meta/requirements.yml` will finally be honored in 2.10
19:37:33 <bcoca> action: module={{ lookup('vars', onevar + lookup('random' ....}}
19:37:40 <apollo13> cyberpear: oh? where does that go, in a role/collection or playbook level?
19:37:46 <cyberpear> in roles
19:37:57 <apollo13> to bad :D
19:38:02 <cyberpear> but only for `ansible-galaxy install` process
19:38:03 <bcoca> collections have galaxy.yml
19:38:23 <bcoca> ^ looking at making that work as requirements.yml also for cli, so not just on install
19:38:54 <apollo13> I think part of our problem (when we don't use tower) is that we work with multiple ansible "projects" and all having different requirements.txt. The default method of installing collections globally for the user isn't helpful in that regard either. But maybe we are just doing it wrong?
19:39:16 <bcoca> -p and use per project/enviornment directories
19:39:27 <bcoca> the default is useful for most cases, not all
19:39:37 <bcoca> impossible for it to be so
19:39:58 <apollo13> well we currently put ansible.cfg in the project and customize roles and collection paths there to install into a per env directory
19:40:11 <apollo13> Not sure if that is a good solution in the long term though
19:40:39 <apollo13> seems to work at least and is what you suggest with -p without having to actually specify -p
19:40:50 * cyberpear uses an `activate`-like script to set the options via env vars, such as roles_path, etc
19:40:55 <apollo13> (apparently ansible-galaxy installs into the first location in that path)
19:41:07 <cyberpear> yes, galaxy installs to the first writable location in the path
19:41:14 <bcoca> first writable
19:41:41 <cyberpear> was fun when RHEL shipped with no writable location in their downstream roles_path
19:43:03 <apollo13> bcoca: when you said "<bcoca> ^ looking at making that work as requirements.yml also for cli, so not just on install" what does that mean exactly?
19:43:53 <bcoca> ansible-galaxy install -r galaxy.yml
19:44:19 <apollo13> ah, I got confused by "so not just on install" :D
19:44:20 <cyberpear> I see
19:44:45 <apollo13> because that is "on install" :D
19:45:11 <bcoca> it runs when you install the collection, but not when you 'unpack/git clong/etc'
19:45:23 <bcoca> so ansible-galaxy install my.coll1
19:45:35 <bcoca> will run galaxy.yml, but we are also workign so you can 'run directly'
19:45:50 <apollo13> ah neat
19:47:27 <bcoca> also, 2 question on upgrade, a) upgrade current collection? b) upgrade dependencies?
19:47:41 <bcoca> i would only use --force on downgrade
19:50:44 <shertel> `--force` doesn't affect deps by default
19:51:04 <apollo13> bcoca: depends how the upgrade is triggered imo, ie if you say "galaxy install --upgrade collection" then do as little work as possible to satisfy constraints
19:51:41 <apollo13> uhm that example was probably not the best
19:52:47 <shertel> I think I kind of prefer b. `--force-with-deps` could just be for downgrading deps then too.
19:53:43 <apollo13> I guess it depends on how dependencies are specified
19:54:13 <apollo13> if dependencies are specified with a version (if possible), then imo galaxy install --upgrade collection should ensure that the dependency is satisfied
19:54:16 <apollo13> even if that means downgrade
19:54:28 <bcoca> ev3en more fun, what do you do when you installed collection B as dep of collection A, but the upgraded B is incompatible?
19:55:05 <apollo13> that can only happen if A specifies a dependency to B without a version right?
19:55:23 <apollo13> in which case you are imo asking for pain
19:55:52 <bcoca> no, can happen if you upgrade B directly
19:55:59 <bcoca> but yes, also in that case
19:56:21 <bcoca> or a > 1.x version, but 2.x didnt exist at that time and now is backwards incompat ...
19:56:25 <apollo13> well even if you update B manually I wouldn't overcomplicate it
19:56:34 <bcoca> many ways to tie noose around neck using package management
19:56:38 <apollo13> ie some failures are to be expected (like with pip)
19:56:46 <bcoca> too many, imho ...
19:57:09 <apollo13> maybe, but then again ansible-galaxy invents yet another package manager XD
19:57:29 <bcoca> yes .. i've had 7yrs to make my peace with that ... will get there .. one day ...
19:58:00 <apollo13> soon *scnr* ;)
19:58:22 * bcoca can point to many rants on 'dont make another package manager'
19:58:46 <misc> no rant on "do not make too much rants on not making a package manager" ?
19:59:02 <agaffney> that's a bit meta
19:59:16 <bcoca> misc: no, im pro rant
19:59:36 <bcoca> k, this is going waay off... and its time
19:59:39 <bcoca> #endmeeting