16:00:54 <dtometzki> #startmeeting Infrastructure (2022-02-10) 16:00:54 <zodbot> Meeting started Thu Feb 10 16:00:54 2022 UTC. 16:00:54 <zodbot> This meeting is logged and archived in a public location. 16:00:54 <zodbot> The chair is dtometzki. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions. 16:00:54 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:54 <zodbot> The meeting name has been set to 'infrastructure_(2022-02-10)' 16:00:54 <dtometzki> #meetingname infrastructure 16:00:54 <zodbot> The meeting name has been set to 'infrastructure' 16:00:54 <dtometzki> #chair nirik siddharthvipul mobrien zlopez pingou bodanel dtometzki jnsamyak computerkid 16:00:54 <dtometzki> #info Agenda is at: https://board.net/p/fedora-infra 16:00:54 <dtometzki> #info About our team: https://docs.fedoraproject.org/en-US/cpe/ 16:00:54 <zodbot> Current chairs: bodanel computerkid dtometzki jnsamyak mobrien nirik pingou siddharthvipul zlopez 16:00:55 <dtometzki> #topic greetings! 16:01:09 <dtometzki> .hi 16:01:10 <zodbot> dtometzki: dtometzki 'Damian Tometzki' <linux@tometzki.de> 16:01:11 <mobrien> .hi 16:01:13 <zodbot> mobrien: mobrien 'Mark O'Brien' <markobri@redhat.com> 16:01:18 <mkonecny> .hello zlopez 16:01:19 <zodbot> mkonecny: zlopez 'Michal Konecny' <michal.konecny@psmail.xyz> 16:01:43 <phsmoura> .hi 16:01:44 <zodbot> phsmoura: phsmoura 'Pedro Moura' <moura.pedro123@gmail.com> 16:01:48 <petebuffon> .hello petebuffon 16:01:49 <zodbot> petebuffon: petebuffon 'Peter Buffon' <pabuffon@gmail.com> 16:03:02 <dtometzki> hello together :-) 16:03:06 <nirik> morning 16:03:15 <dtometzki> morning nirik 16:03:50 * nirik 🌊s 16:04:00 <dtometzki> #topic New folks introductions 16:04:00 <dtometzki> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves 16:04:00 <dtometzki> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted 16:04:30 <dtometzki> Any newcomers here ? 16:05:59 <dtometzki> please say hello and give us a short intro 16:07:21 <dtometzki> ok no one , let´s go to the next topic 16:07:32 <dtometzki> #topic Next chair 16:07:32 <dtometzki> #info magic eight ball says: 16:07:32 <dtometzki> #info chair 2022-02-10 - dtometzki 16:07:32 <dtometzki> #info chair 2022-02-17 - mkonecny 16:07:32 <dtometzki> #info chair 2022-02-24 -petebuffon 16:08:08 <dtometzki> Anyone for 2022-03-03 ? 16:08:48 <dtometzki> come on :-) 16:09:01 <mobrien> If we don't have a volunteer I think 2 weeks ahead is good 16:09:17 <dtometzki> ok then let us go forward 16:09:26 <dtometzki> #topic announcements and information 16:09:26 <dtometzki> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting 16:09:26 <dtometzki> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3 16:09:26 <dtometzki> #info If your team wants support from the Fedora Program Management Team, file an isssue: https://pagure.io/fedora-pgm/pgm_team/issues?template=support_request 16:09:28 <dtometzki> #info matrix sig is forming, see https://discussion.fedoraproject.org/t/bi-weekly-meeting-for-matrix-sig-or-matrix-team/35947/3 if interested 16:09:31 <dtometzki> #info Mass rebuild for F36 finished 16:09:41 <dtometzki> any new news to add ? 16:10:08 <dtometzki> i think the mass rebuild we can remove or ? 16:10:16 <nirik> #info branched (f36) composes running fine 16:10:18 <nirik> dtometzki: yep. 16:10:31 <nirik> #info rawhide composes (f37) waiting on signing to finish to resume. 16:11:23 <dtometzki> thanks nirik 16:11:53 <mkonecny> #info API breaking change in bugzilla will happen 28th February 16:12:11 <nirik> yeah, we really need to get going on that, not much time. ;( 16:12:18 <nirik> #info mass update/reboot next week 16:12:22 <dtometzki> yeah 16:12:31 <nirik> #info Red Hat recharge day next friday 16:13:02 <dtometzki> thanks 16:13:18 <dtometzki> #topic Oncall 16:13:18 <dtometzki> #info https://fedoraproject.org/wiki/Infrastructure/Oncall 16:13:18 <dtometzki> #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/ 16:13:18 <dtometzki> ## .oncalltakeeu .oncalltakeus 16:13:18 <dtometzki> #info nirik on call from 2022-01-27 to 2022-02-03 16:13:19 <dtometzki> #info mobrien on call from 2022-02-04 to 2022-02-10 16:13:21 <dtometzki> #info dtometzki on call from 2022-02-11 to 2022-02-17 16:13:33 <mkonecny> @nirik I already prepared the-new-hotness for it, but need new API key 16:13:36 <mobrien> oh I need to send the mail on those reboots.. 16:14:16 <dtometzki> Anyone for the week from 2022-02-18 ? 16:15:20 <mkonecny> I can take it 16:16:09 <dtometzki> #info mkonecny on call from 2022-02-18 to 2022-02-24 16:16:23 <dtometzki> great mkonecny thanks 16:17:18 <dtometzki> #info Summary of last week: (from current oncall ) 16:17:27 <mobrien> I had a couple of pings 16:17:50 <dtometzki> a short summary = 16:17:52 <dtometzki> ? 16:17:55 <mobrien> Friday night there was an issue with mirrorlist which nirik fixed 16:18:11 <mobrien> Tuesday morning pagure was down, a httpd reload fixed it 16:18:38 <mobrien> There was also one earlier today about dns but was answered before I could respond 16:18:44 <mobrien> self solved 16:18:51 <dtometzki> yes 16:19:09 <dtometzki> ok 16:19:20 <mkonecny> I noticed that the pagure prod instance is not in nagios 16:20:04 <nirik> it actually is... it's just called pagure02.fedoraproject.org 16:20:24 <nirik> but we aren't monitoring any specific pages or anything, just the normal disk space/etc... 16:21:06 <mkonecny> I tried to filter services by name in nagios and it only found stg instance 16:21:53 <dtometzki> its a good point to to the next topic 16:21:54 <nirik> https://nagios.fedoraproject.org/nagios/cgi-bin//status.cgi?navbarsearch=1&host=pagure02 16:22:05 <dtometzki> #topic Monitoring discussion [nirik] 16:22:05 <dtometzki> #info https://nagios.fedoraproject.org/nagios 16:22:05 <dtometzki> #info Go over existing out items and fix 16:22:32 <mkonecny> nirik: Strange, not sure why I didn't found it :-/ 16:23:01 <nirik> so, it's pretty much standard... the usual few arm boxes down, and misc other things... nothing too out of the ordinary. 16:23:13 <mkonecny> So if you are searching for pagure, it doesn't find pagure02 :-D 16:23:22 <nirik> I did make some changes to try and reduce alerts recently, might keep doing that... the alerts about low swap are kind of useless. 16:23:33 <nirik> mkonecny: nice. ;( 16:24:02 <mkonecny> For next time I need to search for pagure* :-) 16:24:04 <nirik> we have some certs that are expiring soon... but I had to switch back to an old cert for a rawhide compose bug. ;( 16:24:36 <nirik> mobrien: SSL WARNING - Certificate 'apps.ocp.fedoraproject.org' expires in 20 day(s) (2022-03-02 23:26 +0000/GMT). needs renewing 16:25:08 <nirik> othewise pretty much as usual.... we cna move on unless folks have any questions. 16:25:26 <dtometzki> any questions ? 16:25:27 <mobrien> I will update that cert 16:26:03 <dtometzki> For today we have no learning topic 16:26:28 <dtometzki> next week docs pipline 16:26:38 <nirik> cool. 16:27:27 <mkonecny> We can do backlog refinement then 16:27:32 <dtometzki> yes 16:27:35 * nirik nods 16:27:35 <dtometzki> #topic Fedora Infra backlog refinement 16:27:35 <dtometzki> #info Refine oldest tickets on https://pagure.io/fedora-infrastructure/issues 16:28:04 <mkonecny> .ticket 9691 16:28:05 <zodbot> mkonecny: Issue #9691: STG: Move vmguests from vmhost-x86-01/02/03 to vmhost-x86-11/12 - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9691 16:28:31 <nirik> still needs doing.... just haven't had a chance due to so many other things going on. 16:28:47 * nirik can update the ticket 16:29:00 <nirik> perhaps we could do this next week when we do stg updates/reboots. 16:29:58 <mobrien> Oh yes, I went to do that before christmas but ran into firewall issues 16:30:01 <mkonecny> Could phsmoura do this? 16:30:20 <mobrien> They have since been fixed but I haven't had time to get back to it 16:30:40 <dtometzki> .tcket 9692 16:31:03 <dtometzki> is in the same area 16:31:22 <nirik> mkonecny: it needs perms and following the sop, but otherwise sure. 16:32:12 * nirik waits for that ticket... hello pagure? 16:32:12 <mkonecny> He can at least shadow you :-) 16:32:28 <nirik> definitely. 16:32:32 <mkonecny> .ticket 5478 16:32:33 <zodbot> mkonecny: Issue #5478: Establishing a new FAS group for Python-SIG - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/5478 16:32:48 <nirik> oh yeah, I never got back to this. It did get unblocked I think. 16:33:27 <mkonecny> It seems like everything was solved, it was just forgotten 16:33:40 <nirik> so far. ;) But there's blockers on many other things. 16:33:57 <nirik> anyhow, I will try and just do this one too... it's the oldest open ticket we have I think. 16:34:06 <phsmoura> mkonecny: sure, Id like to work on something new and learn more about fedora infra. Just need to talk to you guys what I should prioritize cause I started working in another issue and still stuck on it and I yes need perms 16:34:07 <nirik> 5 years. sheesh 16:35:16 <mkonecny> It brings memories :-D 16:35:31 <dtometzki> mkonecny, which order do you use in the list ? 16:35:59 <mkonecny> Last modified date 1->9 16:36:23 <mkonecny> It should be described in the agenda 16:36:55 <nirik> 9892 is next... 16:37:08 <mkonecny> nirik: Do you want to comment on 5478? 16:37:26 <nirik> yeah, I will do that. 16:37:48 <mkonecny> Ok 16:37:51 <mkonecny> .ticket 9892 16:37:52 <zodbot> mkonecny: Issue #9892: Centralise and update Fedora infra and releng SOPs - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9892 16:38:26 <mkonecny> Infrastructure documentation is already moved to docs.fedoraproject.org 16:38:33 <nirik> we did move the infra stuff, but releng isn't right? 16:38:36 <mkonecny> RelEng documentation is being migrated 16:38:51 <mkonecny> @pbokoc is working on that 16:38:55 <mkonecny> I will update the ticket 16:39:31 <nirik> ah, ok. Are mboddu and jednorozec aware? :) 16:40:33 <mkonecny> We were talking about this on planning meeting 16:40:55 <mkonecny> At least jednorozec should be aware 16:41:21 <mboddu> Yeah, I heard about it, but jednorozec was more involved than me in that 16:41:27 <nirik> ok. In any case I think it needs doing... ;) 16:41:56 <jednorozec> I need to ping Petr 16:42:14 <jednorozec> because there was bunch of updates to the SOPs in last few weeks 16:42:41 <mkonecny> jednorozec: I pinged him around this two days ago and he replied that he is working on it 16:42:54 <mkonecny> s/around/about/ 16:43:39 <mkonecny> Anything else regarding this ticket? 16:43:40 <jednorozec> mkonecny, that is fine, but we want to dropt some sops and update others 16:44:11 <mkonecny> He will probably ping you as well 16:44:30 <mkonecny> So next ticket 16:44:39 <mkonecny> .ticket 8213 16:44:40 <zodbot> mkonecny: Issue #8213: fedmsg -> fedora-messaging migration tracker - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8213 16:45:04 <nirik> datagrepper is done now right? 16:45:27 <mkonecny> Let me check 16:45:38 <nirik> but all the rest in the last comment are still to be done. ;( 16:46:49 <mkonecny> It seems like the badges-backend is done as well 16:47:19 <nirik> oh? when was that done? 16:47:21 <mkonecny> Hm, the datagrepper still has fedmsg/base in playbook 16:47:54 <mkonecny> Not sure, it's just not showing on the grep in playbooks/group 16:48:38 <nirik> yeah, I guess datagrepper is still fedmsg. I thought it was changed, but I guess that was just the db... 16:48:47 <mobrien> Datagrepper is in openshift now so that playbook may be out of date 16:49:02 <nirik> and badges-backend01 is still definitely fedmsg 16:49:13 <mkonecny> Ok, badges-backend is still there, it's just not on the top 16:49:14 <nirik> mobrien: oh right, we should nuke it then. 16:49:46 <mkonecny> I see resultsdb as well, which is not on the list 16:50:00 <mobrien> We might want to confim with Aurélien B just to be 100% 16:50:10 <nirik> yeah, thats moving to openshift really soon I hope. 16:50:14 <mkonecny> I will reply to the ticket then 16:50:16 <nirik> mobrien: yeah. 16:50:56 <mkonecny> We skipped resultsdb on purpose on the list? 16:51:24 <nirik> well at the time I think it wasn't clear if qa was going to keep running it or not... 16:51:27 <mobrien> datagrepper servers are commented out in the ansible inventory too 16:51:53 <mkonecny> I see that it was marked as done in the past 16:51:56 <nirik> leor is working on the openshift version... 16:52:36 <nirik> ah it is using fedora-messaging 16:52:42 <nirik> even the vm one thats live now. 16:53:02 <nirik> so, resultsdb doesn't need to be on the list. 16:53:25 <mkonecny> It has the fedmsg/base which just confused me 16:53:47 <nirik> it shouldn't need it... 16:54:01 <mkonecny> I see that it has both 16:54:27 <nirik> it's only running fedora-messaging. 16:54:47 <mkonecny> So it's just a leftover in the playbook 16:54:54 <nirik> yeah. 16:55:13 <mkonecny> I see that the datagrepper has rabbit/user role and fedmsg/base as well 16:55:59 <mkonecny> Commented on the ticket 16:56:19 <abompard> Hey folks! 16:56:30 <nirik> huh, where do you see it for datagrepper? 16:56:33 <abompard> Just finished a meeting, I can answer questions if needed 16:56:36 <mkonecny> I would say that we can go to open floor now 16:56:54 <nirik> abompard: datagrepper is fedora-messaging now right? 16:56:55 <dtometzki> yep for the last 4 minutes 16:57:03 <abompard> nirik: it is 16:57:09 <dtometzki> #topic Open Floor 16:57:12 <abompard> as well as datanommer 16:57:12 <mkonecny> nirik: In playbooks/groups/datagrepper.yml 16:57:32 <mkonecny> Let me update the comment 16:57:35 <abompard> playbooks/groups/datagrepper is probably the old one, the new one is running in openshift 16:57:44 <abompard> as well as datanommer 16:58:00 <nirik> yeah, thats the old vm one, we need to clean it up. 16:59:16 <mkonecny> I updated the ticket 16:59:41 <mkonecny> We still have 16 services to migrate 17:00:10 <nirik> :( 17:00:28 <mkonecny> And I'm wondering why FMN isn't on the list 17:00:42 <nirik> it is... its just called notifs there. 17:01:27 <dtometzki> ok we have now 6:00 o´clock and i will close the meeting shortly. many thanks for joining 17:01:53 <mkonecny> This is why I'm struggling to find it each time :-D 17:02:11 <mkonecny> dtometzki: Thanks for running this :-) 17:02:17 <nirik> yeah, no idea why it was called that... ;) 17:02:22 <dtometzki> #endmeeting