13:01:07 <amoloney> #startmeeting CPE-PO-OH
13:01:07 <zodbot> Meeting started Thu May 14 13:01:07 2020 UTC.
13:01:07 <zodbot> This meeting is logged and archived in a public location.
13:01:07 <zodbot> The chair is amoloney. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:01:07 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
13:01:07 <zodbot> The meeting name has been set to 'cpe-po-oh'
13:01:29 <amoloney> #meetingname CPE Project Owner Office Hours
13:01:29 <zodbot> The meeting name has been set to 'cpe_project_owner_office_hours'
13:01:31 <siddharthvipul> hehe, it's funny, po-oh
13:01:44 <siddharthvipul> amoloney: hellow
13:01:46 <amoloney> hahaha!
13:01:51 <amoloney> #info About the CPE team: https://docs.fedoraproject.org/en-US/cpe/
13:01:55 <pingou> ó/
13:01:59 <nils> \o
13:02:08 <amoloney> I better change that abbreviation :)
13:02:11 <siddharthvipul> pingou: nils \o/
13:02:22 <nils> amoloney, it's perfect!
13:02:29 <siddharthvipul> amoloney: everything is funny to me, ignore that XD
13:02:31 <amoloney> #topic Open Floo
13:02:40 <Defolos> .hello
13:02:40 <zodbot> Defolos: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1".
13:02:41 <amoloney> #info Agenda is at: https://board.net/p/cpe-po-oh
13:02:49 <nils> .hello nphilipp
13:02:50 <zodbot> nils: nphilipp 'Nils Philippsen' <nphilipp@redhat.com>
13:02:54 <siddharthvipul> .hello siddharthvipul1
13:02:55 <zodbot> siddharthvipul: siddharthvipul1 'Vipul Siddharth' <siddharthvipul1@gmail.com>
13:02:58 <Defolos> .hello defolos
13:02:59 <zodbot> Defolos: defolos 'Dan Čermák' <dan.cermak@cgc-instruments.com>
13:03:58 <amoloney> Hi Everyone!
13:04:14 <Defolos> o/
13:04:22 <siddharthvipul> Defolos: how is it going?
13:04:29 <mboddu> .hello mohanboddu
13:04:30 <zodbot> mboddu: mohanboddu 'Mohan Boddu' <mboddu@bhujji.com>
13:04:41 <siddharthvipul> mboddu: hellow \o
13:04:43 <pingou> \ó/
13:04:50 <mboddu> siddharthvipul: Hello
13:04:51 <|King_InuYasha|> .hello ngompa
13:04:52 <zodbot> |King_InuYasha|: ngompa 'Neal Gompa' <ngompa13@gmail.com>
13:04:54 <mboddu> Hello everyone :)
13:04:55 <|King_InuYasha|> gah
13:04:59 <pingou> amoloney: Ben is going to hate you!
13:05:35 <amoloney> Ill join all of his too to even it up :)
13:06:04 <mboddu> amoloney: You wont be sleeping for the next foreseeable future :P
13:06:20 <amoloney> Theres no agenda today, its open floor so so if there is something you want to talk about/ask - go for it
13:06:46 <amoloney> mboddu: what is this thing you speak of?
13:06:50 <King_InuYasha> .hello ngompa
13:06:51 <zodbot> King_InuYasha: ngompa 'Neal Gompa' <ngompa13@gmail.com>
13:06:53 <King_InuYasha> there we go
13:06:55 <Defolos> any updates on the missing pagure features?
13:07:01 <cverna> hello
13:07:32 <Defolos> (ftr: I asked about that last week)
13:07:57 <cverna> when I read CPE-PO I can only think of C-3PO :D
13:08:17 <amoloney> Defolos: yep I remember and I said I would be posting them t o the wiki this week...and its still this week :)
13:08:18 <pingou> cverna: amoloney is worth 3PO :-p
13:08:33 <Defolos> amoloney: sure, no problem, just asking
13:08:42 <Defolos> not like I would've had time to touch pagure…
13:08:43 <nils> pingou, ... in gold even :)
13:08:43 <amoloney> no seriously though there will be updates on friday on the wiki :)
13:08:54 <Defolos> cool, thanks!
13:08:57 <siddharthvipul> amoloney: thank you :)
13:09:16 <amoloney> You're welcome!
13:10:54 <Defolos> cverna: you are driving the communication with gitlab?
13:12:02 <amoloney> Just to set expectation the feature list is not really a list verbatim, it is a text discussing the decision considerations of the three options
13:12:50 <amoloney> and yes cverna and I are discussing with GitLab, Clement from a technical side and I from an administrative side
13:13:34 <mboddu> Defolos: He actually shared the gitlab ticket with devel@ list as well - https://gitlab.com/gitlab-org/gitlab/-/issues/217350 (if in case you missed it)
13:14:34 <mboddu> You can let him know if the ticket is missing any features
13:15:40 <cverna> yeah what amoloney1 and mboddu said :)
13:16:14 <Defolos> mboddu: already subscribed there
13:16:35 <King_InuYasha> same :)
13:18:34 <cverna> I think there 2 quite high priority issue so far, the fact that you can't have projects with '+' in the name and the lack of support for a messaging bus
13:18:41 <cverna> there are *
13:19:36 <mboddu> Defolos: +1
13:20:09 <King_InuYasha> I'm not optimistic that either of those will be resolved anytime soon in GitLab
13:20:27 <mboddu> cverna: No message bus support is considerably very big.
13:20:42 <King_InuYasha> the former has come up before in previous OSS gitlab migratinos (e.g. GNOME) and the latter is very anti-GitLab philosophically
13:21:11 <Defolos> the cynic in me want's to throw in "it's open source, you can implement that yourself!!11eleven"
13:21:15 <King_InuYasha> having a message bus makes it easier for things to integrate around GitLab rather than be *in* GitLab, and GitLab's philosophy is to be "the tool that does it all"
13:21:26 <Defolos> indeed
13:21:37 <Defolos> gitlab tries to be the swiss-army knife
13:21:52 <Defolos> or let's say the swiss-army knife monolith
13:21:53 <King_InuYasha> Defolos: of course we can attempt to integrate it ourselves, but at what layer? what aspect? and can we get it right and maintain that going forward?
13:22:07 <Defolos> King_InuYasha: I wasn't exactly serious
13:22:15 <nils> Defolos, you mean it tries to be the Wenger Giant?
13:22:21 <King_InuYasha> sure, but it's a very important question
13:22:37 <King_InuYasha> three years of maintaining two major GitLab instances has given me an unusually sharp understanding of GitLab's architecture
13:22:52 <King_InuYasha> I can honestly say I don't want to be the guy maintaining an out of tree patch for GitLab of that type
13:23:25 <King_InuYasha> the size and scope of that patch would be half the size of the entire pagure codebase :(
13:23:40 <cverna> the message bus is indeed a problem that also impact CentOS so I would be keen on considering this as a blocker. And I am not sure we want to maintain the abstraction layer GitLab webhooks to message bus
13:23:44 <King_InuYasha> and worse, it'd be in Go and Ruby across multiple projects
13:24:01 <King_InuYasha> gitaly, gitlab-rails, gitlab-workhorse, etc.
13:24:31 <King_InuYasha> you *could* do a subset of stuff just in gitlab-rails, but that's still a massive patch
13:24:33 <nils> yeah, if we replace our current distgit-on-Pagure workload with something similar in size, just different, little is won
13:25:14 <King_InuYasha> imo, nothing is won and we'd be in worse shape because we couldn't even control our own destiny anymore
13:26:30 <King_InuYasha> also, I don't think this is the first time message bus stuff has come up... GitLab's response has largely been "use webhooks"
13:26:42 <King_InuYasha> which are primitive and actually somewhat errorprone on firing
13:27:06 <King_InuYasha> e.g., webhooks do not guarantee sequential ordering of anything (when they're executed, what's in the payload, etc.)
13:27:24 <King_InuYasha> this is something we have now with fedora-messaging
13:28:10 <King_InuYasha> cverna, nils: and of course, I'm ignoring the fact that GitLab is in Ruby, C, and Go
13:28:37 <King_InuYasha> and uses paradigms that are largely unfamiliar to both CPE and most of the Fedora contributor community
13:28:59 <Defolos> King_InuYasha: gitlab has C in its codebase?
13:29:07 <King_InuYasha> in a few places, not much
13:29:22 <King_InuYasha> mostly from gems and go things that they've forked and taken over
13:29:24 <nils> heh, C would be the language of the three I'm most familiar with
13:29:51 <King_InuYasha> it's not a *blocker* per se, but it makes it difficult for the Fedora community to actively contribute to it for Fedora-fit-for-purpose changes
13:29:57 <Defolos> nils: same here…
13:30:10 <Defolos> Ruby is arcane to me and didn't touch go in half a decade
13:30:23 <King_InuYasha> nils, Defolos: C is the language they use the least of :)
13:30:32 <Defolos> yay
13:30:34 <King_InuYasha> mostly code they inherited when they forked and took over gems or go stuff
13:30:39 <pingou> King_InuYasha: in all fairness, we do have some ruby dev in the community
13:30:43 <King_InuYasha> we do
13:30:58 <King_InuYasha> I'm not saying it's a blocker, it is just more difficult
13:31:04 <pingou> to some
13:31:06 <King_InuYasha> we have Go devs in the community too
13:31:07 <pingou> (like me)
13:31:12 <pingou> but not others (like Vít)
13:31:15 <King_InuYasha> right
13:31:35 <King_InuYasha> but it is a consideration if CPE is going to be porting stuff over from pagure to gitlab
13:31:45 <pingou> but I don't think "pingou should be able to hack on it" is in the requirements, or cverna  is it? :)
13:31:56 <nils> amoloney, cverna, would the hackmd doc be the place to jot down the high points of the convo?
13:32:02 <Defolos> I think it should be ;-P
13:32:06 <nils> There are some details I haven't seen mentioned elsewhere
13:32:27 * pingou trying to parse nils' question
13:32:38 <nils> hah sorry
13:32:43 * nils pedals back
13:33:44 <nils> amoloney, cverna, King_InuYasha mentioned a couple of things, reservations, I haven't seen tracked anywhere yet (not in that detail), not in the GitLab ticket nor in the related hackmd document. Is the latter the place where to put them?
13:33:51 <nils> pingou, ^^ there, better?
13:34:06 <pingou> nils: much :)
13:34:16 <King_InuYasha> there's also scaling issues, but if CPE doesn't run it, then that's _less_ of a problem
13:34:21 <pingou> I think that's a question for cverna
13:34:28 <King_InuYasha> but philosophically, I think not running our core infra is probably a bad idea
13:34:42 <nils> pingou, also: "pingou should be able to hack on it" correlates well with many sane sets of requirements ;)
13:34:44 <cverna> the concerns about the language stack ?
13:34:50 <Defolos> King_InuYasha: scaling problems? I though gitlab.com is pretty huge
13:35:05 <King_InuYasha> Defolos: gitlab.com is a *very* special deployment
13:35:14 <King_InuYasha> there's a lot in there that no other GitLab instance has
13:35:24 <Defolos> oh jolly
13:35:27 <nils> cverna, for instance the drawbacks of webhooks, e.g. no ordering guarantees
13:35:29 <King_InuYasha> it doesn't even currently use the GitLab Geo or HA features that everyone else has
13:35:30 <cverna> I in purpose did not put all the concern, minor points in the ticket before I had a clear answer to the things that I consider being blockers
13:35:51 <cverna> nils: yeah that's covered by the need to have a messaging bus interface
13:36:04 <King_InuYasha> Defolos: to be fair to GitLab, they are trying to fix this by beefing up those features to handle gitlab.com, but it's going to be a while
13:36:14 <nils> cverna, yeah I thought we don't want to put this in the ticket directly, hence the hackmd doc
13:36:48 <King_InuYasha> but gitlab.com's philosophy of features over functionality is scary too
13:36:50 <nils> cverna, if the messaging bus interface is a req, we may need to phrase this more explicitly
13:36:57 <King_InuYasha> s/gitlab.com/gitlab's/
13:37:05 <cverna> IMO if we don't have the message bus interface that's a big bummer and we should consider carefully the way forward
13:37:09 <nils> right now it reads a bit like "this is how we do it now, but can webhooks do it instead?"
13:37:17 <King_InuYasha> webhooks _definitely_ cannot
13:37:25 * King_InuYasha has been trying for years :(
13:38:11 <nils> King_InuYasha, that's why I want this tracked, i.e. somewhere where the people involved look for it
13:38:23 <nils> otherwise you'll be sad for no good reason and we can't have that
13:38:57 <King_InuYasha> nils: I think most people are okay with that these days :P
13:39:08 <cverna> my hope is that they agree to maintain a bridge between their system hooks and a messaging bus. I think we can even offer to develop that app as long as they take over the maintenance
13:39:28 <pingou> cverna: the ordering remains a question though
13:39:40 <nils> ^^
13:40:21 <cverna> so what is exactly the ordering issue ? compared to what we currently have ?
13:40:34 <cverna> so that I can note that down :)
13:40:59 <nils> AIUI messages on our bus come "in order", their webhooks don't
13:41:10 <nils> That's how I read King_InuYasha's comments above
13:41:15 <pingou> same here
13:42:01 <King_InuYasha> webhooks in gitlab fire based on processing by sidekiq
13:42:06 <nils> And AIUI (again), many of our things rely on that order, otherwise we wouldn't have to care
13:42:13 <King_InuYasha> and sidekiq parallelizes these events
13:42:22 <King_InuYasha> so they *definitely* don't happen in order
13:42:28 <King_InuYasha> and there's no guarantee of that
13:42:42 <King_InuYasha> (I've been bitten by this at $DAYJOB at least twice because of it)
13:43:41 <cverna> when do we care about the order of messages currently ? (so that I can have an example of why it is important)
13:44:14 <pingou> CI started, CI finished
13:44:33 <pingou> commit A, commit B
13:45:03 <nils> CI failed, PR updated, CI succeeded
13:45:12 <nils> We definitely don't want to have the OOO
13:45:18 <nils> *that
13:45:54 <cverna> what system is consuming this ?
13:47:12 <nils> I don't think it's a single system.
13:47:28 <King_InuYasha> it's more like we rely on it for pipelining across our infra
13:47:50 <nils> I could write a message bus consumer doing stuff here at home and let it act on changes in say my package repos.
13:48:15 <cverna> I am asking because I think we will want an exact case when it matters to put weight on the problem
13:49:48 <King_InuYasha> cverna: pagure -> copr
13:50:02 <King_InuYasha> that ordering is essentially mandatory for automatic rebuilds from dist-git to work properly
13:50:18 <cverna> thanks
13:50:35 <King_InuYasha> cverna: it's not public, but I use it to kick rebuilds of the fedora kernel downstream with some flags changed to add debug features locally
13:50:46 <King_InuYasha> and losing that guarantee would seriously mess things up for me
13:51:27 <Defolos> I know my word doesn't have a lot of weight: but we definitely *want* automatic rebuilds
13:51:36 <King_InuYasha> cverna: copr has a webhook thing but again, it's not as reliable because of gitlab's handling of webhooks
13:51:48 <Defolos> believe me, as someone who's worked with obs for quite some time now: we want them and we want them now
13:52:09 <Defolos> them being taken away because of the lack of a decent message bus would suck big time
13:52:31 <King_InuYasha> Defolos: one day I'll finish writing my auto-commit+rebuild service for my packages :D
13:53:12 <pingou> King_InuYasha: rpmautospec?
13:53:31 <King_InuYasha> pingou: that handles the ugly part: now we need to detect and trigger rebuilds :D
13:53:52 <pingou> koschei?
13:54:00 <King_InuYasha> koschei doesn't submit :(
13:54:06 <pingou> yet? :?)
13:54:16 <King_InuYasha> but koschei also demands ordered info from the source
13:54:29 <King_InuYasha> otherwise, the wrong things will happen
13:55:22 <King_InuYasha> pingou: well, perhaps with rpmautospec enabled packages, koschei could be configured to auto-submit successful rebuilds instead of just doing scratch ones
13:55:25 <amoloney> this is a great conversation ^^
13:55:47 <nils> King_InuYasha, what message bus topics do you use for this? I'm going through the list of topics but there's so much what looks like outdate cruft to me there...
13:55:56 <amoloney> cverna: we can use the notes from this meeting to make sure those requirements are captured in the hackmd doc and ticket
13:56:13 <nils> amoloney, I've added a note about the ordering thing to the hackmd doc
13:56:16 <King_InuYasha> nils: git.receive and pull-request.closed (with merged as the note)
13:56:23 <King_InuYasha> there's a few others, but those are the two major ones
13:56:27 <nils> King_InuYasha++
13:56:36 <King_InuYasha> nils: use my fas :P
13:56:38 <mboddu> ngompa++
13:56:41 <nils> of course this isn't your primary nick :)
13:56:41 <zodbot> mboddu: Karma for ngompa changed to 4 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
13:56:44 <nils> ngompa++
13:56:44 <zodbot> nils: Karma for ngompa changed to 5 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
13:56:56 <amoloney> nils: thank you kindly
13:57:00 <amoloney> nils: ++
13:57:09 <King_InuYasha> nils: when noggin comes online, all my nicks will be able to work :D
13:57:10 <amoloney> * nils:++
13:57:27 <amoloney> Nils im sending you karma but IRC hates me
13:57:31 <nils> the colon has to go
13:57:32 <pingou> nils++
13:57:34 <mboddu> nils++
13:57:34 <zodbot> mboddu: Karma for nphilipp changed to 8 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
13:57:47 <amoloney> nils++
13:57:47 <zodbot> amoloney: Karma for nphilipp changed to 9 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
13:57:49 <nils> which sounds like a line from MP's sense of life
13:57:55 <amoloney> yes!!!!!
13:57:57 <nils> thanks :)
13:58:25 <amoloney> 2 minute warning!
13:58:36 <King_InuYasha> amoloney: so... when noggin replace fas? :D
13:58:45 <amoloney> thanks again for coming to this meeting, I will be moving it to this time from now on
13:58:54 <cverna> thanks all for coming, I have to go afk :)
13:58:55 <mboddu> King_InuYasha: I learned a great deal of things today, thanks
13:59:12 <amoloney> King_InuYasha: are you asking me a timframe?
13:59:16 <King_InuYasha> amoloney: yes :D
13:59:30 <cverna> King_InuYasha: one day maybe :P
13:59:35 <amoloney> What an awful thing to as a product owner:)
13:59:41 <King_InuYasha> :P
13:59:51 <rbowen> amoloney: Oh, did I get the time wrong? I thought office hours were *starting* in 2 minutes, not ending.
14:00:01 <King_InuYasha> rbowen: hey Rich :)
14:00:04 <amoloney> <King_InuYasha ":P"> Were hoping to deploy in Q3 which is around the October timeframe
14:00:06 <pingou> rbowen: got rescheduled due to a conflicting meeting :(
14:00:31 <amoloney> sorry, switch in October, deploy a little earlier :)
14:00:33 <rbowen> Well, shoot. I just tweeted about it, based on the note in the CPE Weekly email. So it'll be an hour earlier next week, too?
14:00:51 <King_InuYasha> rbowen: going forward, yes :)
14:01:01 <rbowen> ok. Fixed tweet for next week.
14:01:13 <rbowen> Was trying to get some CentOS folks here to air their grievances. :)
14:01:16 <King_InuYasha> at least it'll be easier for me to attend :)
14:01:20 <amoloney> oh sorry Rich! I only realized I had a conflict yesterday but this is the new time going forward.
14:01:26 <amoloney> Thanks everyone for participating!
14:01:28 <King_InuYasha> the previous time collided with many things :D
14:01:39 <amoloney> #endmeeting