15:01:10 <mkonecny> #startmeeting Infrastructure (2020-08-13)
15:01:10 <zodbot> Meeting started Thu Aug 13 15:01:10 2020 UTC.
15:01:10 <zodbot> This meeting is logged and archived in a public location.
15:01:10 <zodbot> The chair is mkonecny. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:10 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:10 <zodbot> The meeting name has been set to 'infrastructure_(2020-08-13)'
15:01:10 <mkonecny> #meetingname infrastructure
15:01:10 <zodbot> The meeting name has been set to 'infrastructure'
15:01:10 <mkonecny> #chair nirik pingou smooge cverna mizdebsk mkonecny abompard siddharthvipul mobrien
15:01:10 <mkonecny> #info Agenda is at: https://board.net/p/fedora-infra
15:01:10 <zodbot> Current chairs: abompard cverna mizdebsk mkonecny mobrien nirik pingou siddharthvipul smooge
15:01:11 <mkonecny> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
15:01:12 <mkonecny> #topic aloha
15:01:17 <nirik> ahoy
15:01:18 <siddharthvipul> \o
15:01:23 <mkonecny> .hello zlopez
15:01:24 <zodbot> mkonecny: zlopez 'Michal Konečný' <michal.konecny@psmail.xyz>
15:01:25 <mboddu> Hola :D
15:01:28 <nasirhm> .hello2
15:01:29 <zodbot> nasirhm: nasirhm 'Nasir Hussain' <nasirhussainm14@gmail.com>
15:01:30 <siddharthvipul> goood morning people
15:01:38 <siddharthvipul> or the timezone, you know what I mean
15:02:13 <mobrien> hello
15:02:22 <mkonecny> Hi everyone, I welcome you to your weekly meeting
15:02:45 <nirik> siddharthvipul: haven't you heard of UGT? :)
15:02:50 <mkonecny> Let's start with new people
15:02:51 <mkonecny> #topic New folks introductions
15:02:51 <mkonecny> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
15:02:51 <mkonecny> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
15:03:05 <siddharthvipul> nirik: nothing like aoife time zone.. it's a new thing in market ;)
15:03:09 <mkonecny> Anyone new, who wants to say hello?
15:03:09 <nirik> http://www.total-knowledge.com/~ilya/mips/ugt.html
15:03:15 <nasirhm> I'm new here.
15:03:40 <mkonecny> hi nasirhm, thanks for calling oncall with the badges issue :-)
15:03:50 <siddharthvipul> also, nirik congratulations on getting the Linux Engineering excellence award
15:04:00 <nasirhm> mkonecny: Happy to help.
15:04:01 <mobrien> welcome nasirhm
15:04:02 <nirik> welcome nasirhm
15:04:10 <nirik> siddharthvipul: thanks.
15:04:15 <nasirhm> May Congratulations nirik 👏
15:04:44 <darknao> .hello2
15:04:45 <zodbot> darknao: darknao 'Francois Andrieu' <naolwen@gmail.com>
15:05:02 <darknao> o/
15:05:04 <nasirhm> I am trying to find a good first issue to get started with Infra and apply for Infrastructure Apprenticeship.
15:05:27 <mkonecny> darknao: Welcome
15:05:38 <nirik> hey darknao.
15:05:45 <siddharthvipul> darknao: \o
15:05:46 <nils> .hello nphilipp
15:05:47 <zodbot> nils: nphilipp 'Nils Philippsen' <nphilipp@redhat.com>
15:05:50 <nils> hey all!
15:05:53 <siddharthvipul> oh, hello nils :D
15:05:54 <nasirhm> darknao: Welcom o/
15:06:19 <mkonecny> Do we have any low trouble ticket that could be fixed by nasirhm?
15:07:05 <nasirhm> I'm comfortable with working on Python and Go as well if there's something that needs a patch to be fixed.
15:07:14 <jednorozec> .hello humaton
15:07:15 <zodbot> jednorozec: humaton 'Tomáš Hrčka' <thrcka@redhat.com>
15:07:50 <amrmzaki> .hello amrmostafazaki
15:07:51 <zodbot> amrmzaki: amrmostafazaki 'Amr Mostafa Zaki' <amr.mostafa.zaki@gmail.com>
15:08:01 <mkonecny> nasirhm: you can look at the tickets marked as dev, if you want to work with Python
15:08:02 <nirik> well, there's adding monitoring to registry...
15:08:11 <nirik> https://pagure.io/fedora-infrastructure/issue/9231
15:08:25 <nirik> (which we will talk about in the standup later today I am sure)
15:08:33 <siddharthvipul> mkonecny: we should set some time to flesh out good first issues
15:09:14 <mkonecny> We could probably mark them during our daily meeting
15:09:14 <mboddu> I totally agree
15:09:15 <siddharthvipul> nasirhm: yep, what nirik said is a good place I think.. should be easy
15:09:30 <nirik> yeah, they are hard... often it's easy to just fix them... or if you file a ticket and no one does it, they get stale.
15:09:35 <nasirhm> nirik: I'm planning to work on the upstream registry maintained by jessfraz too.
15:09:36 <siddharthvipul> mkonecny: I am not sure if what we discuss in meetings are for new comers
15:10:01 <siddharthvipul> it would be more like nitpicks that we can't get to work because of other issues not for new comers
15:10:14 <mkonecny> siddharthvipul: Yeah, but we could mark them with good_first_issue label
15:10:27 <siddharthvipul> nirik: we can set a time (like say 2 weeks or a month )
15:10:29 <mkonecny> If there is any
15:10:32 <siddharthvipul> if they cross that, we take care of it
15:10:43 <siddharthvipul> mkonecny: oh sure, if they are, let's try to mark them
15:10:57 <nirik> we have 'easyfix' for that
15:11:09 <nasirhm> I would like to work on improving Monitoring for Registry, Thank You nirik
15:11:12 <nasirhm> I saw the easyfix tag on it.
15:11:14 <mkonecny> Forgot the easyfix label
15:11:38 <nirik> https://fedoraproject.org/easyfix/
15:11:42 <mkonecny> Ok, let's move on for now
15:11:46 <mkonecny> #topic Next chair
15:11:46 <mkonecny> #info magic eight ball says:
15:11:46 <mkonecny> #info 2020-08-20 - siddharthvipul
15:11:46 <mkonecny> #info 2020-08-27 - ???
15:11:55 <nirik> some of that is pretty out of date. ;(
15:12:11 <mkonecny> We are looking for the volunteer for 2020-08-27
15:12:12 <nirik> new easyfix bug: update contacts for easyfix tickets. :)
15:12:30 <nirik> I can do that one...
15:12:56 <mkonecny> nirik: This should be also done for plenty of docs, but I don't want to bother newcomers with docs
15:12:58 <nasirhm> nirik: There's some work needs to be done on the easyfix site, I'm thinking of working on that too with the join SIG.
15:12:59 <siddharthvipul> since I am doing it next week, I will rather have someone else take it (just so everyone gets to do it)
15:13:09 <siddharthvipul> if no one takes it, we can leave it and get to it next week
15:13:28 <mkonecny> nirik: This was about volunteering?
15:13:43 <nirik> yeah, but the easyfix site lists contacts that are not around anymore at all. ;( so we need to fix that
15:13:49 <nirik> yes, I can run the 08-27 meeting
15:13:55 <siddharthvipul> nirik: I can update the contacts
15:14:01 <nirik> (hopefully I remember, I am on pto next week)
15:14:04 <siddharthvipul> may bug to you to get them
15:14:05 <mkonecny> #info 2020-08-27 - nirik
15:14:33 <nirik> +1
15:15:06 <mkonecny> #topic announcements and information
15:15:06 <mkonecny> #info CPE Sustaining EU-hours team has a Monday through Friday 30 minute meeting going through tickets at 0830 UTC in #centos-meeting
15:15:06 <mkonecny> #info CPE Sustaining NA-hours team has a Monday through Friday 30 minute meeting going through tickets at 1800 UTC in #fedora-admin
15:15:06 <mkonecny> #info Fedora Infrastructure will be moving in 2020-06 from its Phoenix Az datacenter to one near Herndon Va. A lot of planning will be involved on this. Please watch out for announcements on changes.
15:15:08 <mkonecny> #info Fedora Communishift move has started but will take longer than expected. Current estimate for bringing back into production is TBD
15:15:11 <mkonecny> #info https://fedoraproject.org/wiki/Infrastructure/2020-post-datacenter-move-known-issues has a list of known items from post datacenter move
15:15:21 <mkonecny> Any other thing we want to announce?
15:15:38 <nirik> #info nirik will be out next week on pto
15:15:51 <mkonecny> nirik: Well deserved
15:15:53 <nirik> #info many people will be out tomorrow for a Red Hat recharge day
15:16:23 <nirik> I'll still be reachable, but will try and avoid non fun work things. ;)
15:16:26 <siddharthvipul> nirik: take the day off for real real :D
15:16:36 <siddharthvipul> week*
15:17:05 <mkonecny> Just relax
15:17:31 <mboddu> I am still not very sure if nirik is gonna be away for a week not looking at his laptop/emails/irc
15:17:34 <siddharthvipul> we all should try to take tomorrow absolute off
15:17:41 <mkonecny> Anything else?
15:17:53 <siddharthvipul> mkonecny: oh one thing
15:18:00 <mkonecny> siddharthvipul: I will lock my laptop somewhere :-D
15:18:21 <siddharthvipul> mkonecny: 1 minute
15:18:22 <siddharthvipul> please hold on
15:18:53 <siddharthvipul> #info please take a minute to fill CPE survey https://fedoraproject.limequery.com/696793?lang=en
15:18:58 <siddharthvipul> mkonecny: that's all :)
15:19:12 <mkonecny> Very important, thanks siddharthvipul
15:19:27 <mkonecny> So let's look on the oncall
15:19:29 <mkonecny> #topic Oncall
15:19:29 <mkonecny> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
15:19:29 <mkonecny> #info mkonecny is oncall for  2020-08-06 to 2020-08-13
15:19:29 <mkonecny> #info siddharthvipul is oncall for 2020-08-13 to 2020-08-20
15:19:29 <mkonecny> #info ??? is oncall for 2020-08-20 to 2020-08-27
15:19:32 * nirik has plans to catch up on his packages and other fun things, so yeah, I will still be on, just not doing directly work things. :)
15:20:07 <siddharthvipul> haha, I was expecting something like that .. narrator: nirik worked more on week off than usual
15:20:17 <mkonecny> Do we have any volunteer for 2020-08-20 to 2020-08-27
15:20:28 <mkonecny> ?
15:20:42 * nirik would, but will be off part of that, so nope. :)
15:21:08 <mkonecny> nirik: Give chance to someone else for once :-D
15:21:33 <siddharthvipul> I would but back to back 2 weeks are not favorable because sharing is caring
15:21:43 <siddharthvipul> if no one wants, I can take care of it next week
15:21:46 <mobrien> I'll take it
15:21:59 <siddharthvipul> mobrien: you are awesome.. also congrats on sysadmin-main
15:22:04 <mkonecny> #info mobrien is oncall for 2020-08-20 to 2020-08-27
15:22:07 <mobrien> Thanks
15:22:19 <siddharthvipul> .oncalltakeeu
15:22:19 <zodbot> siddharthvipul: Kneel before zod!
15:22:29 <mkonecny> mobrien: You would at least be able to solve the issues by yourself :-D
15:22:37 <siddharthvipul> mkonecny: XD
15:22:51 <siddharthvipul> mobrien: no one should pass your barrier..
15:23:09 <mkonecny> #info Summary of last week: (from current oncall )
15:23:13 <mobrien> I may have access but I still lack a lot of knowledge :p
15:23:30 <mkonecny> The week was quiet till today
15:23:37 <mkonecny> I got two pings today
15:23:52 <mkonecny> One was registry.fp.o missing entries
15:24:12 <mkonecny> It was fixed by mizdebsk and it was just failing mount
15:24:34 <mkonecny> Second one was badges not being resolved by DNS resolver
15:24:58 <mkonecny> This was also fixed by mizdebsk and it was typo in address in resolv.conf
15:25:15 <nirik> sorry about that mistake there... :( was my typo. I should have run the playbook instead
15:26:00 <mkonecny> nirik: Nothing happened, it was fixed rather quickly, when someone reported it
15:26:03 <mkonecny> #topic Monitoring discussion [nirik]
15:26:04 <mkonecny> #info https://nagios.fedoraproject.org/nagios
15:26:04 <mkonecny> #info Go over existing out items and fix
15:27:21 <nirik> so, we have a bunch of things... but nothing unexpected.
15:27:46 <nirik> There's some down arm builders. We have access to the power now to reboot them, but no map as to what port is on what machine yet.
15:28:06 <nirik> The pdc-backend instances show down, but will be removed by pingou this week.
15:28:24 <nirik> There's a number of proxy33/34 stuff, but mobrien is building those out still.
15:29:01 <nirik> the bastion mail queue is apparently due to a problem on the redhat.com side.
15:29:34 <mkonecny> According to smooge it's some issue with virus checker
15:29:56 <nirik> and some misc unknown ones we should investigate and fix. It would be nice to have this clean when we go into beta freeze.
15:30:10 <mobrien> looking at those as we speak
15:30:28 <nirik> (beta freeze is the 25th)
15:30:43 <nirik> thats about it, unless there's some specific one anyone wants to talk about more
15:31:23 <mkonecny> Let's move on
15:31:25 <mkonecny> #topic Data-Center Move update
15:31:36 <mkonecny> Do we have any update?
15:31:48 <smooge> we moved?
15:31:54 <nirik> we can probibly remove this item after this week. :) There's still things to do, but they will become their own projects
15:32:03 <nirik> we still need to bring up more staging
15:32:15 <nirik> we still need to bring up the rest of the openqa alternative arch stuff
15:32:21 <siddharthvipul> smooge: let me ask you this. did you get to sleep in a last couple of months :P
15:32:27 <nirik> we need to bring up stuff in RDU2.
15:32:36 <mkonecny> Do we also want to remove the item from announcements?
15:33:16 <nirik> speaking of RDU2, smooge and I talked about this the other day and hopefully came up with a short term plan that would let us get retrace/copr stuff/arm maintainer test boxes online... pending a bigger network reorg for the rest
15:33:42 <smooge> yup
15:33:45 <nirik> mkonecny: yeah, I'd say so...
15:34:24 <smooge> mkonecny, yes. I think it is probably time to remove from announcements
15:34:41 <nirik> also, we should remove it from topic in admin
15:34:58 <mkonecny> What about communishift and post move issues?
15:36:19 <nirik> well, we could put in something, but we will only know once we know... so seems silly to have a meeting section for it where we say "we don't know yet"
15:36:47 <mkonecny> I'm talking about announcements section
15:36:48 <nirik> communishift is pending a network reorg in RDU2... thats going to take some time... we have the process started
15:36:58 <nirik> ah, I see.
15:37:31 <nirik> How about changing it to: Datacenter move is over, but some items still need to be done: see https://fedoraproject.org/wiki/Infrastructure/2020-post-datacenter-move-known-issues
15:37:51 <mkonecny> nirik: sounds good
15:38:45 <jlanda> +1
15:39:42 <mkonecny> Updated in the agenda
15:39:48 <mkonecny> #topic Open Floor
15:39:57 <mkonecny> Last topic of the day
15:40:05 * nirik updates the wiki
15:40:27 <mkonecny> If you want to say something, say it now or stay silent forever :-D
15:41:01 <misc> so I have a question
15:41:16 <misc> is there a plan already for datanommer/datagrepper and fedora-messaging ?
15:41:34 <misc> (snehal, who work on badges, asked me and I wasn't able to find anything for that)
15:41:37 <mkonecny> misc: There are CPE initiatives for this
15:41:56 <mkonecny> https://tree.taiga.io/project/amoloney1-cpe-team-projects/kanban?epic=null
15:42:28 * nirik nods
15:42:36 <mkonecny> Fedora messaging is currently in progress and datanommer/datagrepper is scoped
15:42:42 <misc> ok
15:43:52 <mkonecny> I will wait for a few minutes if there is any question
15:45:33 <darknao> about the mail queue on bastion
15:46:05 <darknao> i've made a small script a few days ago to help me to understand why the queue was full
15:46:27 <smooge> cool
15:46:32 <darknao> it show how the mail queues are spread over time period, much like qshape tool
15:46:44 <smooge> even cooler
15:47:07 <darknao> actually, if qshape is available on bastion already, then this tools is kind of useless :p
15:48:00 <darknao> but anyway, if you think it can be useful for anyone, i can work a little bit more on it to make it prettier
15:48:16 <smooge> well i have not heard of qshape before this so it isn't
15:48:30 <smooge> it would be useful for me
15:50:01 <darknao> ok, i'll work on that then
15:52:45 <mkonecny> Let's give a few minutes of your life back and end this meeting
15:52:46 <mkonecny> #endmeeting