15:00:06 <tflink> #startmeeting fedora-qadevel 15:00:06 <zodbot> Meeting started Mon Jan 15 15:00:06 2018 UTC. The chair is tflink. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:06 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:00:06 <zodbot> The meeting name has been set to 'fedora-qadevel' 15:00:06 <tflink> #topic roll call 15:00:13 * kparal is here 15:00:21 <tflink> #chair kparal lbrabec 15:00:22 <zodbot> Current chairs: kparal lbrabec tflink 15:00:37 * lbrabec is here 15:01:29 * tflink is hoping that a josef is going to show up 15:02:07 <tflink> #chair jskladan 15:02:07 <zodbot> Current chairs: jskladan kparal lbrabec tflink 15:02:08 * jskladan a josef joins 15:02:29 <jskladan> (the josef is smelling glue right now, though...) 15:02:38 <tflink> cool, lets get this party started 15:03:22 <tflink> https://cdn.meme.am/instances/500x/67493334/looks-like-i-picked-the-wrong-week-to-stop-sniffing-glue.jpg 15:03:39 <tflink> #topic Announcements and Information 15:03:59 <tflink> #info taskotron production redeployed - tflink 15:03:59 <tflink> #info phabricator is dead - tflink 15:04:00 <tflink> #info fixed artifacts not accessible on taskotron-dev -- kparal 15:04:00 <tflink> #info converted task-abicheck and task-python-versions to ansiblized taskotron -- kparal 15:04:15 <tflink> do folks have anything to add? 15:04:16 <jskladan> Phabricator is dead! All hail the Phabricator! 15:04:32 <tflink> I still need to remove the dns entries but I should be able to get that done today 15:04:38 <kparal> long live phabricator 15:04:53 <kparal> err, rest it peace 15:04:55 <tflink> I also forgot to backup the docs before I blew the vm away so those will need to be regenerated 15:04:57 <kparal> *in 15:05:41 <kparal> no questions 15:05:43 <tflink> if there's nothing to add, any questions/comments? 15:06:37 * tflink takes that as a "no" 15:06:49 <tflink> #topic Ansiblize Branch Progress 15:07:17 <tflink> I think that most of this was covered earlier but anything else to add? 15:07:29 <kparal> 4 tasks converted, I'm working on rpmdeplint now 15:07:34 <tflink> cool 15:07:36 <kparal> it's going to be a bit more difficult 15:08:35 <kparal> that's all from me, unless you have some specific questions 15:08:41 <tflink> does that indicate that we might want/need custom modules to deal with parts of rpmdeplint? 15:09:15 <kparal> that would be great, but for the moment, I simply write a python script, import libtaskotron and call the directives directly 15:09:44 <kparal> in the future, it would be nice to convert at least some of those directive to ansible modules and distribute them in fedora 15:09:48 <tflink> did I ever post the custom modules I wrote? 15:09:55 <tflink> this is going to be embarrassing if I didn't 15:10:02 <kparal> there's a koji module bundled in libtaskotron 15:10:07 <kparal> I don't know of anything else 15:10:15 <tflink> I think that's the only one I finished 15:10:35 <kparal> either way, it needs to be packaged and available in fedora, which is somewhat more work 15:10:43 <tflink> it sounds like this is going to be a good conversation to have once the porting is done 15:10:56 <kparal> sure, we can talk about this as the next step 15:11:06 <tflink> since I wasn't sure what modules would even be needed 15:11:49 <tflink> any other questions/comments here? 15:12:11 <jskladan> nope 15:12:19 <tflink> ok, moving on 15:12:23 <tflink> #topic Cloud Plans 15:12:45 <tflink> Not much of an update here - I started working on generating images for openstack again last week but didn't get very far 15:13:24 * kparal is looking forward to it 15:13:45 <tflink> I don't remember if I've said this yet but moving to openstack will solve a good chunk of our non-x86 arch issues 15:14:09 <tflink> as it already has ppc instances available and will have aarch64 instances available before long 15:14:18 <jskladan> cool! 15:14:40 <tflink> my current plan is to generate images and probably keep them on the box we were using for phab 15:14:55 <tflink> and use rsync to keep the client hosts up to date for now 15:16:05 <tflink> I'm hoping that will solve the publicly available image problem while we're at it 15:16:32 <tflink> other than that, I don't have much of an update here - was working on other stuff last week 15:16:38 <tflink> any questions/comments? 15:16:43 <kparal> nope, thanks for update 15:17:17 <tflink> ok, moving on to a topic that I remembered about an hour ago and isn't on the agenda :) 15:17:29 <tflink> #topic ResultsDB Deployment and Hosting 15:17:59 <tflink> one of the things that came up during the production redeployment is that there are new things relying on resultsdb 15:18:26 <tflink> specifically, greenwave is the brand new thing but AFAIK, bodhi is relying on resultsdb more than it used to 15:19:12 <tflink> one of the questions asked when the redeployment happened was if we could build the new instance and just change dns to resolve to the second host instead of destroying and rebuilding during an outage 15:19:51 <tflink> another question I have is whether it might be wise for us to move production resultsdb over to RHEL instead of Fedora 15:20:20 <jskladan> tflink: so we don't need to upgrade the machine every six months? 15:20:28 <jskladan> or for any other benefits? 15:20:48 <tflink> decreased downtime, moving it closer to other infra services 15:21:27 <tflink> if we can do the "hot swap" upgrade method, I'm not sure it's so much of an issue, though 15:21:54 <eseyman> /win 19 15:22:12 <tflink> also, are there any reasonable options to start doing HA for resultsdb? 15:22:39 <tflink> I suspect that having multiple instances with a load balancer would work without issue 15:22:46 * jskladan has no experience with Hight Availability whatsoever 15:23:43 <tflink> resultsdb seems to be on the path to be a rather critical piece of infra for Fedora, I think it'd be wise to start exploring this sooner than later :) 15:23:55 <tflink> hence the topic 15:23:59 <jskladan> are we talking multiple 'frontends' (as in the resultsdb wsgi apps) with single DB backend, or multiple DB backends too? 15:24:16 <tflink> multiple frontends to start 15:24:42 <tflink> IIRC, the change to multiple db backends is a very non-trivial process 15:25:04 <jskladan> tflink: I would think so, sounds like real-time data replication :) 15:26:05 <tflink> but that's outside my experience so it'd likely be good to talk to others 15:26:15 * jskladan agrees 15:26:36 <tflink> I don't know what happened with the postgres bdr proposal that infra had 15:26:54 * tflink makes note to follow up on that and ask questions about HA etc. 15:27:52 <tflink> also, having a load balancer in front of the wsgi apps would let us do upgrades with little to no downtime 15:28:00 <jskladan> yup 15:28:06 <tflink> any other questions/comments on this? 15:28:41 * tflink takes that as a no 15:28:44 <jskladan> do you know if there's already something like that in place (koji/bodhi would be my candidates) for other apps in infra, so we can get "creatively insipired"? 15:28:44 <tflink> #topic Tasking 15:28:50 <tflink> sorry 15:28:52 <tflink> #undo 15:28:52 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x29d0bd90> 15:29:03 <tflink> jskladan: blockerbugs is HA 15:29:21 <tflink> that would be my first thought 15:29:25 <jskladan> tflink: cool, that makes it even simpler, I guess 15:30:06 <tflink> it's mostly a matter of getting multiple instances, setting up haproxy and reworking the proxy settings so that the traffic flows the way it's supposed to 15:30:18 <jskladan> makes sense 15:30:40 <tflink> it'll require some downtime to set up but I think it'd be a worthwhile thing to do 15:30:50 * jskladan agrees 15:31:02 * tflink will talk to infra about it 15:31:09 <tflink> anything else? 15:31:24 <jskladan> nope 15:31:39 <jskladan> cccccccdbcihvffhghuvrjntljvvkfjkjttcfeutdfld 15:31:43 <jskladan> *sigh... 15:31:45 <tflink> actually, this is kinda related - did we ever figure out more details on what kind of scripts would be required for us to decommission most of taskotron-stg? 15:31:52 <tflink> ooh, an OTP 15:32:19 * jskladan should really re-configure that token to not send enter... 15:32:44 <tflink> eh, I think we've all done it at some point :) 15:33:15 <tflink> adamw hasn't gotten very loud about the box I promised him but that could change :) 15:33:23 <tflink> loud/pester-y 15:33:44 <kparal> I haven't done any work on looking at taskotron-stg mock replacement 15:34:05 <kparal> so I believe the answer is no 15:34:12 <kparal> unless someone else did 15:34:16 <tflink> that's what I thought but figured I would ask 15:34:32 * tflink takes that as a "no" 15:34:40 <tflink> moving on to the next topic 15:34:43 <tflink> #topic Tasking 15:34:53 <tflink> Do folks have things to work on? 15:36:04 <kparal> I'm converting the remaining tasks 15:36:14 <tflink> or need/want tasks? 15:36:21 <tflink> kparal: sounds good 15:36:35 <tflink> has taskotron-dev been behaving for the most part? 15:36:56 <jskladan> I'm going to be working on code to make passing a structured file with arg-data from trigger to the task - talked with kparal last week, and it seems like the sooner we can do this, the better 15:37:17 <jskladan> but if there's something else I could help with, shoot! 15:37:23 <tflink> did I miss the conversation/ticket(s) there? 15:37:39 <kparal> -dev has been working fine so far 15:38:05 <jskladan> tflik - this is something we were discussin on and off for quite some time (since the "git commit" item type IIRC) 15:38:58 <tflink> yeah, I recall talking about it in the past but wasn't quite sure what brought the topic up recently 15:39:22 <jskladan> kparal's recent experience with converting the tasks :) 15:39:31 <kparal> it's kind of hard to split triplets of information (like the git commit item) in ansible playbook 15:39:38 <kparal> without writing a custom module 15:39:53 <kparal> so that's why we really need structured input information soon 15:40:21 <kparal> jskladan can tell, according to his task-taskotron-ci 15:40:58 <tflink> ok - do you have any implementation details/plans? 15:41:35 <kparal> I don't, really 15:41:48 <kparal> I'm hoping jskladan has :) 15:41:55 * jskladan plans to have a look at buildbot's capabilities 15:42:11 <jskladan> hopefully it will be able to pass a file down to the worker 15:42:24 <tflink> what about generating a file on task invocation? 15:43:01 <tflink> ie - convert the cli args to a file prior to starting any ansible execution 15:43:21 * tflink realizes that would only solve part of the problem 15:43:42 <tflink> the problem that having a file with data would solve, rather 15:45:01 <tflink> we can continue this conversation later, I guess 15:45:21 <jskladan> tflink: generating a file using a ShellCommand and some echo magic was my backup plan - depending on whether there is no more elegant way 15:46:02 <tflink> I suppose that would probably be easier 15:47:14 <tflink> lbrabec: do you have enough tasks to keep you busy? 15:47:19 <jskladan> and for executor - my plan is to just add a "--file" (or something like that) parameter, which will load the file (yaml/json) and override/add more stuff to the 'arg data', so we can pass it to the ansible using the means already in place 15:47:56 <lbrabec> I guess so, there's still plenty of TODOs and FIXMEs in the code 15:48:14 <tflink> lbrabec: anything in particular that you're planning to look at? 15:49:05 * jskladan has to go afk for a moment 15:49:19 <lbrabec> mainly the SI parts 15:49:57 <tflink> the parts that can/will run SI stuff without the need for any porting? 15:50:09 <lbrabec> yep 15:50:58 <tflink> cool. I'm looking forward to hearing what all (if anything) is still needed to make that happen 15:51:29 <tflink> do you think you could get that list together this week? 15:51:48 <tflink> it'll be a good topic for conversation around qecamp/devconf 15:51:51 <lbrabec> sure 15:52:02 <tflink> great, thanks 15:52:11 <tflink> I think this brings us to ... 15:52:15 <tflink> #topic Open Floor 15:52:37 <tflink> is there anything that folks want to bring up before closing the meeting out? 15:52:55 <lbrabec> nothing here 15:53:01 <kparal> nope 15:53:54 <tflink> ok, I'll end the meeting momentarily, then 15:54:00 <tflink> thanks for coming, everyone 15:54:04 <kparal> thanks 15:54:08 * tflink will send out minutes shortly 15:54:11 <tflink> #endmeeting