16:02:51 <mobrien> #startmeeting Infrastructure (2021-12-02)
16:02:51 <zodbot> Meeting started Thu Dec  2 16:02:51 2021 UTC.
16:02:51 <zodbot> This meeting is logged and archived in a public location.
16:02:51 <zodbot> The chair is mobrien. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:02:51 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:02:51 <zodbot> The meeting name has been set to 'infrastructure_(2021-12-02)'
16:02:57 <mobrien> #meetingname infrastructure
16:02:57 <zodbot> The meeting name has been set to 'infrastructure'
16:03:02 <mobrien> #chair nirik siddharthvipul mobrien zlopez pingou bodanel dtometzki jnsamyak computerkid
16:03:02 <zodbot> Current chairs: bodanel computerkid dtometzki jnsamyak mobrien nirik pingou siddharthvipul zlopez
16:03:07 <mobrien> #info Agenda is at: https://board.net/p/fedora-infra
16:03:13 <mobrien> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:03:18 <mobrien> #topic greetings!
16:03:20 <darknao> .hi
16:03:21 <mobrien> .hi
16:03:21 <zodbot> darknao: darknao 'Francois Andrieu' <darknao@drkn.ninja>
16:03:24 <dtometzki> .hi
16:03:24 <zodbot> mobrien: mobrien 'Mark O'Brien' <markobri@redhat.com>
16:03:26 <nirik> morning
16:03:27 <zodbot> dtometzki: dtometzki 'Damian Tometzki' <linux@tometzki.de>
16:03:36 <mobrien> Sorry everyone I am a few mins late today
16:03:38 * nirik has to step away for just a few min, back soon.
16:03:48 <dtometzki> no problem
16:03:53 <petebuffon> .hello petebuffon
16:03:54 <zodbot> petebuffon: petebuffon 'Peter Buffon' <pabuffon@gmail.com>
16:04:04 <VipulSiddharth[m> .hello siddharthvipul1
16:04:05 <zodbot> VipulSiddharth[m: siddharthvipul1 'Vipul Siddharth' <siddharthvipul1@gmail.com>
16:04:34 <austinpowered> .hi
16:04:35 <zodbot> austinpowered: austinpowered 'T.C. Williams' <fedoraproject@wootenwilliams.com>
16:04:41 <dtometzki> hi
16:04:56 <mobrien> #topic New folks introductions
16:05:03 <mobrien> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:05:03 <Leo[m]> .hello Leo
16:05:03 <zodbot> Leo[m]: leo 'Leo Puvilland' <leo@craftcat.dev>
16:05:04 <mobrien> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:05:18 <mobrien> Any new folks around this week?
16:05:58 <mobrien> Don't see any so far but I will give it a couple of mins
16:06:00 <VipulSiddharth[m> mobrien: define new
16:06:05 <VipulSiddharth[m> :P
16:06:10 <dtometzki> :-)
16:06:28 <nils> .hello nphilipp
16:06:29 <zodbot> nils: nphilipp 'Nils Philippsen' <nphilipp@redhat.com>
16:06:46 <dtometzki> the first time in the meeting
16:06:59 <mobrien> Vipul Siddharth: produced, introduced, or discovered recently or now for the first time; not existing before
16:07:01 <pmoura_> .hello phsmoura
16:07:02 <zodbot> pmoura_: phsmoura 'Pedro Moura' <moura.pedro123@gmail.com>
16:07:12 <VipulSiddharth[m> mobrien: set a 15 minutes timer XD
16:07:20 <mkonecny> .hello zlopez
16:07:21 <zodbot> mkonecny: zlopez 'Michal Konecny' <michal.konecny@psmail.xyz>
16:07:49 <Leo[m]> Good morning! :)
16:07:52 <nirik> ok, back
16:08:01 <mobrien> Ok looks as though nobody new so I will move to next topic
16:08:07 <mobrien> #topic Next chair
16:08:07 <dustymabe> 👋
16:08:15 <mobrien> #info magic eight ball says:
16:08:17 <mobrien> #info chair 2021-12-02 - mobrien
16:08:25 <dustymabe> ha
16:08:32 <mobrien> who would like to chair next
16:08:37 <dtometzki> Ok will do next
16:08:45 <mobrien> very fancy with the wave dustymabe
16:09:02 <nirik> that brings up the holidays... ;) which meetings shall we cancel.
16:09:06 <dtometzki> mobrien,  add me
16:09:22 <mobrien> #info chair 2021-12-09 - dtometzki
16:09:28 <Leo[m]> I won’t be here starting the 18th :o
16:09:38 <Leo[m]> France vacation!! :)
16:09:38 <mobrien> done dtometzki
16:09:39 * nirik is out after the 11th.
16:09:44 <dtometzki> thanks
16:09:52 <mobrien> I would say 16th as the last for the year?
16:10:05 <mkonecny> +1
16:10:08 <mobrien> Resume next year on 6th Jan?
16:10:09 <VipulSiddharth[m> mobrien: yeah, makes sense
16:10:11 <VipulSiddharth[m> +1 as well
16:10:13 <VipulSiddharth[m> yep
16:10:14 <Leo[m]> +1
16:10:14 <dtometzki> +1
16:10:14 <nirik> definitely cancel 23rd and 30th... but not sure about 16th
16:10:38 <petebuffon> sounds good to me
16:10:41 <VipulSiddharth[m> 16th as the last which means there would be a meeting on 16th, right?
16:10:49 <mobrien> yes
16:10:52 <dtometzki> right
16:10:55 <Leo[m]> I can attend the 16
16:11:01 <VipulSiddharth[m> me too
16:11:01 <mobrien> any volunteer to chair 16th?
16:11:08 <VipulSiddharth[m> I can
16:11:09 <Leo[m]> probably
16:11:50 <mobrien> #info chair 2021-12-16 - siddharthvipul
16:11:56 <mobrien> Sold!
16:11:57 <VipulSiddharth[m> mobrien: thank you :)
16:12:21 <mobrien> ok now that is resolved lets move to next topic
16:12:26 <mobrien> #topic announcements and information
16:12:32 <mobrien> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting
16:12:37 <mobrien> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
16:12:43 <mobrien> #info If your team wants support from the Fedora Program Management Team, file an isssue: https://pagure.io/fedora-pgm/pgm_team/issues?template=support_request
16:12:52 <mobrien> any other info we should add?
16:13:23 <mobrien> #info no infra weekly meeting 23rd or 30th December
16:14:06 <mobrien> undo
16:14:31 <mobrien> oops, meant to type that elsewhere
16:14:38 <copperi[m]> :)
16:16:12 <mobrien> Ok looks like no further announcements, lets move on
16:16:22 <mobrien> #topic Oncall
16:16:30 <mobrien> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:16:36 <mobrien> #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/
16:16:46 <mobrien> # .oncalltakeeu .oncalltakeus
16:16:55 <mobrien> #info lenkaseg on call from 2021-11-25 to 2021-12-02
16:17:14 <mobrien> Looks like we need a volunteer to take it from today for the week
16:17:31 <VipulSiddharth[m> I can do it
16:17:51 <mobrien> Sold!
16:17:52 <mobrien> #info siddharthvipul on call from 2021-12-02 to 2021-12-09
16:18:25 <dtometzki> i can do it from 12-09 ...
16:18:39 <mobrien> Thanks dtometzki
16:18:46 <nirik> perhaps...
16:18:59 <mobrien> #info dtometzki on call from 2021-12-09 to 2021-12-16
16:19:05 <mobrien> go ahead nirik
16:19:06 <nirik> for 16-6th we should make no one oncall... ie, have it just ask people to file tickets.
16:19:17 <Leo[m]> +1
16:19:22 <copperi[m]> +1
16:19:22 <mobrien> +1
16:19:23 * nirik will be watching tickets and around, but working on projects and fun things.
16:19:33 <dtometzki> *1
16:19:40 <Leo[m]> fun things?
16:19:41 * mobrien will be here until 24th
16:19:43 <VipulSiddharth[m> +1
16:19:45 <Leo[m]> cool
16:19:52 <nirik> alternately, I could take it, but I would rather not be bothered unless it's important. ;)
16:20:15 <nirik> Leo[m]: I hear that some things in computing are fun... so they tell me.
16:20:36 <Leo[m]> have fun :)
16:20:47 <mobrien> nirik leave it as nobody
16:21:05 <nirik> yeah, that seems best
16:21:14 <mobrien> Vipul Siddharth: feel free to take on call whenever you want
16:21:25 <VipulSiddharth[m> let's try, I am using matrix
16:21:32 <VipulSiddharth[m> I might have to verify
16:21:37 <VipulSiddharth[m> .oncalltakeeu
16:21:37 <zodbot> VipulSiddharth[m: Error: Missing "]".  You may want to quote your arguments with double quotes in order to prevent extra brackets from being evaluated as nested commands.
16:21:40 <mobrien> you will need to verify I would say
16:22:16 <mobrien> anyway...
16:22:17 <siddharthvipul> .oncalltakeeu
16:22:17 <zodbot> siddharthvipul: Error: You don't have the alias.add capability. If you think that you should have this capability, be sure that you are identified before trying again. The 'whoami' command can tell you if you're identified.
16:22:17 <mobrien> #info Summary of last week: (from current oncall )
16:22:34 <Leo[m]> .whoami
16:22:34 <zodbot> Leo[m]: I don't recognize you. You can message me either of these two commands: "user identify <username> <password>" to log in or "user register <username> <password>" to register.
16:22:44 <mobrien> lenkaseg: do you have an update from the last week?
16:23:22 <siddharthvipul> mobrien: can you remind me the command
16:23:23 <nirik> there were some oncall pings last night by time for copr being down... and one this morning about odcs.
16:23:28 <siddharthvipul> it was
16:23:35 <siddharthvipul> identify username password to zodbot, right?
16:24:29 <mobrien> siddharthvipul: identify <user> <pass> to zodbot
16:24:31 <mkonecny> siddharthvipul:  user identify <name> <password>
16:25:22 <mobrien> your zodbot pass is likely different to your matrix one
16:25:45 <siddharthvipul> mobrien: yeah, it's the libera chat one I think
16:25:50 <siddharthvipul> I am using IRC for now..
16:25:54 <siddharthvipul> .whoami
16:25:54 <zodbot> siddharthvipul: I don't recognize you. You can message me either of these two commands: "user identify <username> <password>" to log in or "user register <username> <password>" to register.
16:26:26 <lenkaseg> mobrien: sorry, was on a kid duty, but I'm back now
16:26:36 <mobrien> no problem lenkaseg
16:27:00 <lenkaseg> So, all the week no oncall, only now like 2 hours ago
16:27:01 <siddharthvipul> .oncalltakeeu
16:27:01 <zodbot> siddharthvipul: Kneel before zod!
16:27:27 <lenkaseg> it was about koji and copr being down, or ome part
16:27:53 <lenkaseg> I contacted mobrien and I saw nirik was also solving the issue
16:28:20 <mobrien> Thanks lenkaseg
16:28:34 <nirik> yeah, I happened to see the copr one before heading to bed... lighttpd failed on copr backend, I am not sure why. ;( I restarted it.
16:28:41 <lenkaseg> nirik: ok, I must have missed the night one.
16:29:46 <mkonecny> lenkaseg:  You are not supposed to respond outside your work time
16:30:14 <mkonecny> Nobody expect that
16:30:36 <mobrien> +1
16:30:40 <nirik> yeah, if you are not around, they should file tickets.
16:30:48 <nirik> and they did. :)
16:31:26 <mobrien> ok I think we can move to
16:31:33 <mobrien> #topic Monitoring discussion [nirik]
16:31:40 <mobrien> #info https://nagios.fedoraproject.org/nagios
16:31:44 <mobrien> #info Go over existing out items and fix
16:32:43 <nirik> lets see...
16:33:16 <nirik> huh... os-control01.stg is down?
16:33:36 <mobrien> the os-control-stg one should be back up in a min, forgot to switch it back on after trying and failing to migrate it earlier
16:33:44 <nirik> otherwise, some aarch64 builders. Might be able to power cycle one or two
16:33:46 <mobrien> firewall issues
16:33:49 <nirik> ah.
16:34:07 <nirik> mdapi is still down/broken. :(
16:34:34 <nirik> some copr checks for rsyslog are altering... probibly we need to adjust nagios to ignore that there.
16:35:18 <nirik> there's a queue alert for eln-next queue (which they aren't using yet)
16:35:39 <nirik> it would sure be nice to get mdapi fixed.
16:35:46 <nirik> otherwise thats all... just small stuff.
16:36:28 <nirik> who was signed up for POC on mdapi?
16:37:16 <mkonecny> Only pingou
16:37:32 <mobrien> t0xic0der: is down to learn it
16:37:59 <mkonecny> Yeah, he is trying to hack on it this quarter
16:38:18 <nirik> ah cool.
16:38:37 <nirik> well, basically we need https://pagure.io/mdapi/pull-request/120 or something like it in and a few release pushed to openshift clusters.
16:41:01 <mobrien> Ok so lets move to the next topic
16:41:06 <mobrien> thanks nirik
16:41:16 <mobrien> #topic Fedora Infra backlog refinement
16:41:22 <mobrien> #info Refine oldest tickets on https://pagure.io/fedora-infrastructure/issues
16:42:30 <mobrien> Ok so if I am doing this correctly I ref the ticket that has been the longest without comment?
16:42:38 <nirik> yeah
16:42:43 <mkonecny> Yes, exactly
16:42:53 <mobrien> ok here we go
16:43:03 <mobrien> .ticket 9846
16:43:04 <zodbot> mobrien: Issue #9846: openid_login() from python-fedora to id.fedoraproject.org fails with 500 when you forget 2nd factor - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9846
16:43:22 <nirik> this is pending a ipsilon release I think.
16:43:32 <nirik> (along with some other bits)
16:43:43 <nirik> abompard was working on this.
16:43:50 * nirik can ask status in ticket?
16:44:00 <mobrien> Sounds good to me
16:44:20 <nirik> and/or I can email abompard about it. ;)
16:44:27 <abompard> I'm here
16:44:32 <pingou> nirik: toxicoder made an update PR which I need to finish reviewing and merge
16:44:40 <pingou> (for mdapi, sorry I'm late)
16:44:44 <abompard> I'm still waiting for an ipsilon release to get a separate field for OTP
16:45:07 <mkonecny> I thought that this is already in place on staging
16:45:13 <nirik> pingou: ah, cool.
16:45:25 <abompard> Zlopez: yes, that's where I tested stuff
16:45:26 <nirik> welcome abompard and pingou. ;)
16:45:29 <abompard> but it's not released
16:45:51 <nirik> abompard: can you do a release? can we grant you powers to? ;)
16:45:52 <Leo[m]> pingou: fixed changes you wanted on the mdapi pr
16:45:52 <abompard> that said there are a few tickets about ipsilon erroring out instead of giving good messages back to the app or the user, and I haven't looked at those yet
16:45:59 <Leo[m]> also squashed into 1 commit
16:47:43 <pingou> Leo[m]: thanks!
16:47:46 <mobrien> Ok I think we can move to the next ticket as that one is just waiting on ipsilon release
16:47:55 <mobrien> .ticket 9831
16:47:56 <zodbot> mobrien: Issue #9831: Patches from lists.fedoraproject.org contain trailing ^M which cause the failure of git am - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9831
16:47:58 * nirik nods.
16:48:25 <nirik> This one needs investigation... it may also need newer mailman release if it's a bug in mailman.
16:48:39 <nirik> I've not had any time to look into it... would someone like to? ;)
16:48:42 <Leo[m]> pingou: one more commit squashed, added those comments too
16:48:53 <mobrien> ah newer mailman, if only
16:49:23 <nirik> it seems odd to me that git wouldn't handle this.
16:49:28 <nirik> or that mailman wouldn't.
16:49:33 <nirik> but like I said, I haven't looked
16:50:13 <nirik> oh, I did ask the reporter a question...
16:50:49 <nirik> perhaps we should ask if it's still happening?
16:50:50 <mobrien> ping the reporter again and ask for an update?
16:51:48 <nirik> yeah. +1
16:52:03 <nirik> shall I, or you got it?
16:52:56 <mobrien> I got it
16:53:30 <nirik> cool. One more? or open flood?
16:53:39 <mobrien> ok we'll go one more
16:53:56 <mobrien> .ticket 9648
16:53:57 <zodbot> mobrien: Issue #9648: IAD2: PDU bring up/configuration - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9648
16:54:28 <nirik> I keep not getting any status on this. I can ask again...
16:54:59 <mobrien> Ya, hard to do anything if you are getting no reply
16:56:38 <mobrien> ok I guess open floor time
16:56:43 * nirik added a note there about that
16:56:59 <mobrien> #topic Open Floor
16:57:12 <mobrien> let the chat commence
16:57:21 <Leo[m]> yay
16:57:27 <eddiejennings> <-- been on a work call for now almost two hours.  Nothing to contribute here :(
16:57:41 <austinpowered> Can I ask if there is an open ticket I can try?
16:57:58 <mobrien> Have a very needy dog looking for attention
16:58:02 <Leo[m]> same ^^
16:58:06 <austinpowered> Ruff Ruff
16:58:12 <Leo[m]> the ticket not the dog hah
16:58:21 <Leo[m]> lmk if there’s anything I can work on too :)
16:58:33 <austinpowered> I learned aome things when I worked on the last one.
16:58:50 <austinpowered> That's why I'm here. Trying to learn more about devops and ansible.
16:58:58 <mkonecny> Anything from these https://pagure.io/fedora-infrastructure/issues?status=Open&tags=easyfix&priority=0&close_status=
16:59:17 <mobrien> This one maybe? https://pagure.io/fedora-infrastructure/issue/10377
17:00:27 <Leo[m]> austin if you want to take it go ahead, if not I can :)
17:00:59 <mobrien> anyhow looks like we have run out of time
17:01:21 <mobrien> Thanks to all who attended and thanks to everyone who contributes
17:01:23 <mobrien> #endmeeting