env-and-stacks
LOGS
13:08:40 <hhorak> #startmeeting Env and Stacks (2015-02-12)
13:08:40 <zodbot> Meeting started Thu Feb 12 13:08:40 2015 UTC.  The chair is hhorak. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:08:40 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
13:08:52 <hhorak> #meetingname env-and-stacks
13:08:52 <zodbot> The meeting name has been set to 'env-and-stacks'
13:08:56 <hhorak> #chair bkabrda hhorak juhp ncoghlan vpavlin sicampbell walters ttomecek phracek
13:08:56 <zodbot> Current chairs: bkabrda hhorak juhp ncoghlan phracek sicampbell ttomecek vpavlin walters
13:10:10 <hhorak> #topic Env&Stacks Governance Charter Review as per https://fedoraproject.org/wiki/Env_and_Stacks/Governance_Charter#Changing_These_Rules
13:10:15 <hhorak> #link https://fedoraproject.org/wiki/Env_and_Stacks/Governance_Charter#Changing_These_Rules
13:10:36 <hhorak> I guess this is not much to discuss for today
13:11:23 <hhorak> I just wanted to ask you to read at it and if there are some things we need to change, we can add some specific issues to the agenda for some of the next meetings.
13:11:28 <hhorak> seems fine?
13:12:01 * vpavlin nods
13:12:15 <phracek> #agreed
13:12:29 <hhorak> #action all to read Governance charter and if there are any issues that should be revisited, let's bring them on ML or add them to agenda for some of the next weeks
13:12:41 <bkabrda> so just an update about devpi - I'm not sure if I've already shared this. upstream wanted us to create a formal proposal for the feature we need. I passed the work on to Matej Stuchlik and we now have the proposal for upstream https://groups.google.com/forum/#!topic/devpi-dev/py-B9kwaK5Y and are waiting for comments
13:12:55 <hhorak> #topic Follow-ups -- what has changed since last meeting
13:13:32 <hhorak> #info update about devpi - upstream wanted us to create a formal proposal for the feature we need. I passed the work on to Matej Stuchlik and we now have the proposal for upstream https://groups.google.com/forum/#!topic/devpi-dev/py-B9kwaK5Y and are waiting for comments
13:14:16 <hhorak> bkabrda: so is this the main blocker and we can see some PoC working with this feature implemented? or are there any other blockers?
13:15:00 <bkabrda> hhorak: this is the only blocker (that I know of). once it's done, we can go on and deploy devpi testing instance
13:15:28 <bkabrda> upstream seems to be very busy, but we'll keep poking them and hopefully they'll be able to approve the proposal soon
13:15:35 <phracek> bkabrda: only for my understanding devpi is a UI for downloaded and installing PyPi packages to Fedora? Without RPM Packaging?
13:15:50 <walters> is there a link for more information on this devpi proposal?
13:16:06 <ncoghlan> it's the pilot for https://fedoraproject.org/wiki/Env_and_Stacks/Projects/LanguageSpecificRepositories
13:16:17 <hhorak> #info the devpi feature seems to be the only blocker. once it's done, we can go on and deploy devpi testing instance
13:16:22 <phracek> or is the devpi connected to DevAssistant
13:16:24 <ncoghlan> if we decide it's a good idea, we'd switch to the Pulp based repos
13:16:25 <hhorak> #link https://fedoraproject.org/wiki/Env_and_Stacks/Projects/LanguageSpecificRepositories
13:16:40 <phracek> thanks
13:16:47 <ncoghlan> #link http://pulp-python.readthedocs.org/en/latest/
13:16:51 <bkabrda> phracek: well it's not a UI, it's a transparent PyPI caching server with some nice features :)
13:17:15 <walters> ah right, thanks.  Excellent wiki pages I have to say
13:17:31 <ncoghlan> but the Pulp plugins are really new, so we didn't want to be debugging them will working out if we liked the overall workflow
13:17:57 <hhorak> walters: if only the other projects have the same quality pages :)
13:18:16 <bkabrda> walters: I still need to explicitly state what we want to do (like what to upload, what to prebuild, etc etc)
13:18:23 * ncoghlan has probably written more PEPs than is healthy
13:18:55 <walters> i think my main question is - the package would get into fedora, then its upstream would be mirrored?
13:19:14 <ncoghlan> I think that's the way to go for the time being
13:19:23 <walters> makes sense
13:19:29 <ncoghlan> longer term, we'd like to explore the idea of "staged inclusion"
13:19:37 <ncoghlan> where the upstream packaging gets mirrored first
13:19:52 <ncoghlan> then we get a non-policy compliant RPM into COPR
13:20:03 <ncoghlan> (potentially non-policy compliant)
13:20:22 <ncoghlan> and then a package can graduate to the main repos once it's packaging is actually right
13:20:55 <ncoghlan> that's a lot more working though, since it means proposing changes to the package review process
13:21:10 <bkabrda> walters: ncoghlan: I'll try to work with Matej on these and we'll come up with a proposal that tries to answer all of this (including workflows around that etc)
13:21:17 <ttomecek> would nice to have a tool, which would autopackage stuff from that devpi instance to rpm
13:21:37 <bkabrda> then we'll at least have something to discuss :)
13:21:46 <walters> what I've always wanted is *bidirectional* sync between upstream and packages
13:21:52 <ncoghlan> ttomecek: that becomes feasible once we sort out PEP 426 upstream
13:22:14 <ncoghlan> at the moment, it isn't really feasible because the upstream metadata is missing stuff Fedora needs
13:22:19 <walters> if you autopackage and then git commit the spec file, then you have {Build,}Requires drift over time, etc
13:22:23 <ncoghlan> and there's no extension system to stash it
13:22:36 <walters> has anyone looked at this from the Maven perspective?
13:22:38 <hhorak> #info the plan is the package would get into fedora, then its upstream would be mirrored, at least for the time being
13:22:38 <hhorak> #info longer term, the upstream packaging may be mirrored first and a potentially non-policy compliant RPM got into COPR
13:23:03 <ncoghlan> walters: I've been nudging some of the Java folks here in BNE about it, but no takers so far
13:23:31 <bkabrda> walters: theoretically, you should be able to regenerate the specfile automatically on every rebase, assuming the automatic generation works in the first place, right?
13:23:54 <ncoghlan> that's one spectacular assumption at this point, though :)
13:24:02 <walters> bkabrda, but that means if another packager comes by and adds a missing Requires (e.g. on some external /usr/bin/foo), that gets lost
13:24:35 <ttomecek> ncoghlan, what a long pep
13:25:01 <ncoghlan> ttomecek: and that's *after* I moved big chunks of it out to PEP 440 and PEP 459 :)
13:25:10 <bkabrda> walters: true, but I'm sure we could solve these automatically somehow
13:25:48 <ncoghlan> bkabrda, walters: for Python, this is actually one of my intended use cases for the metadata extensions
13:26:16 <hhorak> #info PEP 426 should allow autopackage stuff from devpi instance to rpm
13:26:16 <hhorak> #link https://www.python.org/dev/peps/pep-0426/
13:27:40 <ncoghlan> my hope is that if we can get at least *one* language ecosystem playing nice with distro packaging, we might be able to encourage others to follow suit
13:27:58 <bkabrda> ncoghlan: I have to admit that I haven't read the whole pep yet, but I'm still assuming that it won't be possible to just say "generate specfile from foo.tar.gz" - there'll still be a need for some tool that'll unpack the tarball, generate dist_info and then parse it (and actually dist info can theoretically be different for different runtimes, if I'm not mistaken)
13:28:12 <ncoghlan> I have at time been accused of being overly optimistic :)
13:28:52 <ncoghlan> bkabrda: aim will be to have a stable JSON description of the source tarball
13:29:17 <ncoghlan> bkabrda: hence the environment marker support in the dependency specifiers
13:30:03 <bkabrda> ncoghlan: ah, ok. not wanting to dive into details, but where will this description live?
13:30:15 <bkabrda> either way, pyp2rpm is looking forward to this ;)
13:30:21 <ncoghlan> bkabrda: in the sdist, so it will still need to be unpacked
13:30:38 <ncoghlan> although ideally we'd have it available direct from PyPI as well
13:30:53 <bkabrda> ncoghlan: ok, thanks. I'll read the whole pep and then continue having stupid questions :)
13:30:59 <ncoghlan> Donald's currently working on getting the PyPI rewrite into shape
13:31:18 <phracek> what about with dependencies to another PyPi packages. Is there a plan how to build a dependencies?
13:31:31 <ncoghlan> https://github.com/pypa/interoperability-peps if you spot any major issues
13:31:47 <walters> related to bridging upstream and packaging is https://people.gnome.org/~walters/docs/build-api.txt
13:32:21 <ncoghlan> although I also have a bunch of open issues at https://bitbucket.org/pypa/pypi-metadata-formats/issues?status=new&status=open
13:33:13 <ncoghlan> walters: one that came up at LCA2015 last month was Meson
13:33:35 <ncoghlan> walters: apparently that has a JSON build description format for IDE integration
13:34:16 <hhorak> #link https://people.gnome.org/~walters/docs/build-api.txt
13:34:33 <ncoghlan> #info it would be nice to have a tool independent "How do we build this?" description format
13:34:41 <ncoghlan> #link https://github.com/jpakkane/meson/wiki/IDE-integration
13:35:09 <phracek> bkabrda: pyp2rpm is going to generate devpi stuff to RPM? What about devpi2rpm?
13:35:10 <ncoghlan> we have a related problem in Python upstream, where we'd like to make distutils/setuptools optional
13:35:15 <walters> declarative JSON is good but there are some important things like the "meta-build" system (rpm/dpkg/etc) being able to specify --enable-foo --disable-bar, set CFLAGS etc
13:35:16 <hhorak> ncoghlan: thanks for helping with #infoing :)
13:35:55 <ncoghlan> walters: yeah, that's why we lobbed it into the "too hard" basket upstream for now, and are sticking with setup.py
13:36:11 <bkabrda> phracek: devpi will have the same source package format as PyPI, it's just a mirror
13:36:13 <phracek> bkabrda: But it is not related to this discussion. THe name is not important
13:36:23 <phracek> bkabrda: ok
13:36:32 <ncoghlan> walters: but the core design of distutils was laid down 17 years ago, and it has not aged well :)
13:36:40 <bkabrda> the "pyp" in pyp2rpm is for "PYthon Package" :)
13:36:47 <phracek> make sense to me. thanks for explanation.
13:39:44 <hhorak> either I got offline or we're done with this topic :)
13:40:02 <ncoghlan> ... for now :)
13:40:13 <hhorak> ncoghlan: right
13:40:37 <hhorak> #topic DevConf legacy -- something interesting from DevConf we should mention here?
13:41:27 <hhorak> Btw. it was very nice to meet you in Brno!
13:42:32 <walters> Nice to meet you guys too!
13:42:44 * vpavlin nods
13:42:46 * langdon now lurking here
13:43:17 <ncoghlan> I didn't make it, but hopefully at least some folks got to meet dcallagh from the Beaker team
13:43:38 <ncoghlan> I know he delivered some Python propaganda to bkabrda for me :)
13:44:21 <bkabrda> ncoghlan: actually he delivered it to a guy who delivered it to me :) and thanks
13:45:36 <ncoghlan> this isn't DevConf related, but the Beaker connection reminded me: https://bugzilla.redhat.com/show_bug.cgi?id=1183913
13:46:18 <ncoghlan> we're aiming to add a standard task to check that a batch of SRPMs rebuild properly
13:46:42 <ncoghlan> the intended use case is to have a way to check that a mass rebuild is going to work before actually doing it
13:47:04 <ncoghlan> and use Beaker's failure reporting to highlight the SRPMs with problems
13:47:20 <walters> aren't there like 5 other implementations of that?
13:47:41 <walters> from Koschei to what matt domsch used to do to ...
13:47:48 <vpavlin> f.e. Koschei
13:48:06 <vpavlin> ah..my irc behaves weirdly..
13:48:11 <ncoghlan> walters: probably, but very few of them will let us automatically do it across every arch the distro supports
13:48:57 <ncoghlan> more importantly, that's only the initial iteration
13:49:18 <ncoghlan> the next step will be to turn it into a regression test for the build toolchain itself
13:49:38 <ncoghlan> where we can swap that out (or even just configure it differently) and see what breaks
13:50:40 <ncoghlan> walters: although if you wanted to send links my way, or add them to the ticket, that would be useful background info
13:53:34 <walters> with gnome-continuous when someone introduces a build break we generally fix it to the previous working version; we're constantly *trying* new things but quickly reverting if they break, for a fast iteration cycle.  See https://git.gnome.org/browse/gnome-continuous/commit/?id=42187d2e8d5060bbb0ba0d3c82b3e2c1bf25a8ee
13:54:03 <walters> but that's not possible with rpm/yum/koji due to the newer-is-always-better semantic
13:54:40 <ncoghlan> walters: yeah
13:55:16 <ncoghlan> I think we're getting off-topic for Env & Stacks though :)
13:55:29 <ncoghlan> (as near and dear as CI is to my heart)
13:55:38 <walters> this is CD, not CI
13:56:16 <walters> Koschei and your Beaker proposal are CI - after a build the binaries are garbage collected, all you have is the logs
13:56:45 <walters> gnome-continuous is CD - after a build, you can atomically upgrade to the result and boot it
13:58:05 <ncoghlan> walters: interesting, and OSTree based so upgrade/downgrade can be atomic
13:58:32 <walters> anyways from Devconf I just got the perspective of a *lot* going on...
14:00:01 <hhorak> As for SCLs in fedora -- I talked with pingou and denis.. we agreed that it makes things simple and more clear to have scls in current dist-git in special branches.. it will be easier to implement in rcm and pkgdb.
14:00:09 <hhorak> But first we need to have the guidelines if this way is fine, some agreement for branches naming, etc..
14:00:16 <hhorak> So, that's my plan to look at for the following weeks.
14:00:26 <hhorak> #info it makes things simple and more clear to have scls in current dist-git in special branches.. it will be easier to implement in rcm and pkgdb.
14:00:26 <hhorak> #info First, we need to have the guidelines if this way is fine, some agreement for branches naming, etc..
14:00:35 <walters> you're working on SCLs in CentOS too?  is that matching Fedora?
14:01:07 <hhorak> walters: yes, I'm involved in anything which is connected to SCLs :)
14:01:45 <hhorak> walters: I think we should use the same approaches in centos as in fedora or the the other way round (providing centos gets scls sooner)
14:02:34 <hhorak> there is also epel in the game, where I'm not actually sure if we're fine with SCLs from centos or we need something more
14:03:02 <hhorak> that's what needs to be cleared
14:04:48 <hhorak> #topic Open Floor
14:05:22 <hhorak> Sorry guys, somebody goes to throw me out, which means I'll lose connection soon. Could somebody close the meeting instead of me after you guys go through Open Floor? (good time for arbitrary questions not only from new members.. basically about anything)
14:06:01 * hhorak going offline.. Thanks everybody!!!
14:07:07 <walters> anyone have anything for the open floor?
14:08:45 <bkabrda> I guess not :)
14:08:49 <walters> i just had one question - my experience (from the rpm-ostree side) is that doing anything that involves touching the fedora releng side is quite difficult.  For example with the devpi work, would the mirror set be versioned and released with Fedora X under releng?
14:09:29 <bkabrda> walters: we'll run the pilot ourselves and we haven't really thought beyond that, yet
14:10:35 <ncoghlan> I also think the rel-eng side of things has been identified as an area needing more hands
14:11:00 <ncoghlan> hence stickster's recent post about hiring into that area
14:12:21 <walters> there is an alt.fedoraproject.org which is a staging area of sorts
14:13:02 <walters> not very formal but it might make sense to create an intermediate stage like COPR between not-fedora and official-releng
14:13:21 <walters> anyways that's all I had myself
14:14:45 <walters> closing in 10...
14:15:08 <walters> #endmeeting