16:00:40 <dtometzki> #startmeeting Infrastructure
16:00:40 <zodbot> Meeting started Thu Feb 25 16:00:40 2021 UTC.
16:00:40 <zodbot> This meeting is logged and archived in a public location.
16:00:40 <zodbot> The chair is dtometzki. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:40 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:40 <zodbot> The meeting name has been set to 'infrastructure'
16:00:59 <smooge> hello
16:01:06 <dtometzki> hello together :-)
16:02:12 <dtometzki> is my first meeting as chair. I think we will wait a minute
16:02:19 <smooge> i am updating the https://board.net/p/fedora-infra
16:02:44 <nirik> morning
16:02:55 <siddharthvipul> .hello siddharthvipul1
16:02:56 <zodbot> siddharthvipul: siddharthvipul1 'Vipul Siddharth' <siddharthvipul1@gmail.com>
16:03:08 <dtometzki> morning nirik
16:03:14 <siddharthvipul> ooh, early :D
16:03:24 <t0xic0der> .hello t0xic0der
16:03:25 <zodbot> t0xic0der: t0xic0der 'Akashdeep Dhar' <akashdeep.dhar@gmail.com>
16:03:31 <siddharthvipul> dtometzki, no issues, let me know if you need help
16:03:38 <dtometzki> yes
16:04:02 <dtometzki> ok we will start with the first topic
16:04:06 <dtometzki> #topic aloha
16:04:17 <mobrien> .hi
16:04:18 <zodbot> mobrien: mobrien 'Mark O'Brien' <markobri@redhat.com>
16:04:54 <siddharthvipul> .hello siddharthvipul1
16:04:54 <dtometzki> #topic New folks introductions
16:04:54 <zodbot> siddharthvipul: siddharthvipul1 'Vipul Siddharth' <siddharthvipul1@gmail.com>
16:05:07 <siddharthvipul> I know t0xic0der is new here :)
16:05:15 <t0xic0der> Hey everyone! :)
16:05:29 <dtometzki> welcome to the infrstructure
16:06:31 <t0xic0der> .hello t0xic0der
16:06:31 <t0xic0der> Thanks @dtometzki
16:06:31 <dtometzki> Anyone new here ?
16:06:31 <zodbot> t0xic0der: t0xic0der 'Akashdeep Dhar' <akashdeep.dhar@gmail.com>
16:06:39 <mrreboot> Hello --> Luke Rawlins Sysadmin at Ohio State University. Just curious at the moment.
16:06:50 <dtometzki> hello
16:07:15 <dtometzki> please introduce yourself if you are new ?
16:08:21 <rparimi> Hello!
16:08:27 <dtometzki> hello
16:08:29 <mobrien> welcome mrreboot, t0xic0der, rparimi  let us know if you want any info or have any questionss
16:08:45 <siddharthvipul> rparimi, mrreboot, t0xic0der \o welcome :) do let us know if you want to know who we are or what we do
16:09:07 <t0xic0der> Hey, Akashdeep here from India. I am an SWE intern at UBS and I mostly develop web applications, networking services and container management. o/
16:09:31 <siddharthvipul> welcome t0xic0der :D
16:09:46 <dtometzki> welcome mrreboot, t0xic0der, rparimi a short introduction ?
16:09:53 <nirik> welcome all. ;)
16:09:56 <dtometzki> yourself ?
16:10:08 <rparimi> I am Ramya Parimi, I have been part of Fedora since 8 or so months. I am into webdev and stuff. I would love to know more about this team as well!
16:10:16 <t0xic0der> Yes, @siddharthvipul1. I'd love to know about what infrastructure team does.
16:10:21 <siddharthvipul> rparimi, \o/ welcome
16:10:43 <rparimi> siddharthvipul: thank you !
16:10:53 <smooge> welcome all
16:11:00 <dtometzki> t0xic0der ?
16:11:01 <mrreboot> Luke Rawlins, Linux Systems Administrator at OSU. Red Hat Certified Engineer, mostly doing server deployments, firewalls, config management.
16:11:17 <smooge> I have put in some low-level tickets for people to look at
16:11:31 <siddharthvipul> smooge, yesss
16:11:55 <t0xic0der[m]> @dtometzki yes?
16:12:09 <rparimi> .hello ramyaparimi
16:12:10 <zodbot> rparimi: ramyaparimi 'Ramya Parimi' <rlp.parimi@gmail.com>
16:12:13 <dtometzki> ok then we will go to the next topic
16:12:16 <dtometzki> #topic Next chair
16:12:37 <dtometzki> i dont know how i get the actual list
16:12:54 <siddharthvipul> dtometzki you can copy paste the existing chairs for coming week
16:13:05 <siddharthvipul> then we know which week we need a volunteer for :)
16:13:07 <dtometzki> #info chair 2021-03-04 - siddharthvipul
16:13:07 <dtometzki> #info chair 2021-03-11 - mobrien
16:13:07 <dtometzki> #info chair 2021-03-18 - ???
16:13:18 <siddharthvipul> right, so we need one for March 18th :)
16:13:38 <bodanel> what's the chair supose to do ?
16:14:00 <bodanel> for me it's not quite clear
16:14:03 <nirik> bodanel: run the meeting, like dtometzki is doing today. :)
16:14:05 <siddharthvipul> bodanel, run this meeting (follow agenda) :)
16:14:05 <mobrien> what dtometzki is doing now, run through the board
16:14:20 <bodanel> put me
16:14:28 <siddharthvipul> \o/
16:14:30 <siddharthvipul> bodanel++
16:14:37 <siddharthvipul> you can let anyone of us know if you need help
16:14:38 <dtometzki> you can find the agenda  https://board.net/p/fedora-infra here
16:15:35 <dtometzki> anyone for the meeting 2021-03-25
16:15:37 <dtometzki> ?
16:15:39 <nirik> it's fun and a good way to help out. :)
16:15:50 <mobrien> It mostly copy paste from the board then update the board with whatever is needed. i.e. put in the next chair for the date
16:15:53 <siddharthvipul> dtometzki, I think we can skip this for now
16:16:00 <bodanel> put me also
16:16:18 <bodanel> I'll have the chance to practice what I've did the week before
16:16:26 <siddharthvipul> bodanel, let's wait for next meeting I will say
16:16:30 <bodanel> sure
16:16:36 <siddharthvipul> we can have more volunteers, if we can't, we will assign it to you :D
16:16:38 <dtometzki> ok then we will go forward
16:16:45 <siddharthvipul> we are 3 +, so okay to move ;)
16:16:47 <t0xic0der> Sounds good. Looking forward to chairing the future meets. :)
16:16:54 <dtometzki> #topic announcements and information
16:17:27 <dtometzki> nay news or information for team ?
16:17:30 <dtometzki> any
16:17:34 <nirik> #info We are in Fedora 34 Beta Freeze
16:17:56 <siddharthvipul> #info CPE Infra&Releng EU-hours team has a Monday through Friday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting
16:17:56 <siddharthvipul> #info CPE Infra&Releng NA-hours team has a Monday through Friday 30 minute meeting going through tickets at 1800 UTC in #fedora-admin
16:17:56 <siddharthvipul> #info Datacenter move is over, but some items still need to be done: see https://fedoraproject.org/wiki/Infrastructure/2020-post-datacenter-move-known-issues
16:17:56 <siddharthvipul> #info work is being done on getting COPR proper powerPC systems
16:19:14 <dtometzki> which datacenter was moved ?
16:19:52 <nirik> we moved from our old PHX2 (phoenix, az, us) to IAD2 (ashburn, va, us) last year.
16:20:02 <nirik> we can probibly drop that line from the board now.
16:20:20 <dtometzki> good
16:20:46 <siddharthvipul> dropped
16:21:02 <nirik> thanks!
16:21:06 <siddharthvipul> we can move to next topic dtometzki (if no one else has anything else to add)
16:21:16 <dtometzki> I think the next topic is a discussion topics.
16:21:24 <dtometzki> or ?
16:21:34 <siddharthvipul> dtometzki, oncall
16:21:47 <dtometzki> #topic Oncall
16:22:17 <siddharthvipul> it's me it's me next this week
16:22:19 <dtometzki> #info siddharthvipul is on call for 2021-02-18 to 2021-02-25
16:22:19 <dtometzki> #info zlopez is on call for 2021-03-04 to 2021-03-11
16:22:19 <dtometzki> #info ??? is on call for 2021-03-11 to 2021-03-18
16:22:43 <siddharthvipul> right, we need a volunteer for 2021-03-11 to 2021-03-18
16:22:55 <dtometzki> ok will do it
16:23:20 <siddharthvipul> dtometzki, will you be oncall for the week?
16:23:24 <dtometzki> yes
16:24:10 <dtometzki> can you put me on the list ?
16:24:38 <siddharthvipul> sure
16:25:33 <dtometzki> for 2021-03-11 to 2021-03-18
16:25:34 <siddharthvipul> okay, let me take oncall this wekk
16:25:34 <siddharthvipul> week*
16:25:38 <siddharthvipul> dtometzki, also, make sure you have read about oncall (link with the topic)
16:25:46 <siddharthvipul> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:25:51 <siddharthvipul> .oncalltakeeu
16:25:51 <zodbot> siddharthvipul: Kneel before zod!
16:26:00 <dtometzki> yes i read it
16:26:14 <siddharthvipul> nice
16:26:18 <siddharthvipul> I think we can move to next meeting
16:26:22 <siddharthvipul> s/meeting/topic
16:26:30 <dtometzki> ok the next topic is
16:26:45 <t0xic0der[m]> <siddharthvipul "I think we can move to next meet"> Too early to do that I think :P
16:26:48 <dtometzki> #topic Learning topic
16:26:56 <smooge> did I miss my oncall items?
16:27:12 <dtometzki> ok
16:27:13 <bodanel> smooge, yes
16:27:14 <siddharthvipul> smooge, oh oh, summary was next agenda item in my mind
16:27:17 <siddharthvipul> #undo
16:27:17 <smooge> sorry
16:27:37 <siddharthvipul> oh, I am not chair
16:27:46 <dtometzki> #undo
16:27:46 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x7fd49edf3550>
16:27:57 <siddharthvipul> thank you dtometzki :)
16:28:48 <siddharthvipul> #info Summary of last week: (from current oncall )
16:28:52 <siddharthvipul> smooge, all yours :)
16:28:54 <smooge> So this weeks oncall was pretty light. There were a lot of direct pings
16:29:45 <smooge> we had a couple of outages of systems which came up as tickets and there were a few 'hey can I .. ' type requests to be answered.
16:29:51 <smooge> Nothing really hard to deal with.
16:30:15 * siddharthvipul missed the busy week then (from my perspective, outages are busy) :P
16:30:21 <siddharthvipul> haha
16:30:31 <smooge> we can move to the next toipic
16:30:37 <dtometzki> yeah
16:30:44 <dtometzki> #topic Learning topic
16:31:29 <dtometzki> ist there anyone where prepared anything ?
16:32:00 <dtometzki> about an application or setup that we have ?
16:32:01 <bodanel> the today's topic supposed to be about rabbitmq :)
16:32:24 <nirik> yeah, I guess I was gonna do a quick thing on rabbitmq
16:32:29 <siddharthvipul> \o/
16:32:50 <dtometzki> what is rabbitmq ?
16:33:04 <nirik> rabbitmq is a message broker, written in erlang. :)
16:33:24 <nirik> so it processes messages... queues them up for consumers, makes sure they are delivered, etc.
16:33:36 <nirik> we use rabbitmq for fedora-messaging, our new message bus.
16:33:40 <dtometzki> would anyone explain more ?
16:34:24 <dtometzki> is it a middleware for all messages or only mail
16:34:32 <nirik> so, when some application does something (say a koji build starts) it emits a message about that. The message goes to the rabbitmq cluster and anyone listening for that kind of message will get it.
16:34:52 <nirik> not email. Only fedora-messaging messages.
16:35:05 <bodanel> +
16:35:26 <copperi> like status messages ?
16:35:29 <nirik> So, messages are emitted by lots of things...
16:35:35 <bodanel> so if you are listening for a different type of message, let's say apache restart you would not get it ?
16:35:39 <Zlopez[m]> The consumer needs to acknowledge message, so no message is lost
16:35:50 <nirik> koji emits them, composes do, when a bug is opened, etc.
16:36:56 <nirik> you can see them in datagrepper (which stores them and lets you search them):
16:37:00 <nirik> https://apps.fedoraproject.org/datagrepper/raw?category=greenwave&delta=17280
16:37:13 <nirik> will show you all the greenwave messages since 17280 seconds ago
16:37:48 <dtometzki> and what is the greenwave ?
16:38:17 <nirik> greenwave is a service that tells if an update passes all it's tests/is ok to push
16:38:31 <nirik> https://apps.fedoraproject.org/datagrepper/raw?rows_per_page=1&delta=127800
16:38:39 <nirik> is the full feed you can see em coming in.
16:38:58 <nirik> There's also #fedora-fedmsg that has a irc bot that just prints (almost) all the messages to irc.
16:39:22 <nirik> So, to make this sadly a bit more complex, we also have an _old_ message bus... fedmsg.
16:39:52 <copperi> does it have duplicates ?
16:39:54 <dtometzki> is anyone who read all the messages ?
16:39:56 <nirik> We currently have an app that exchanges the messages, so both buses will see all the messages, but we are trying to move everything off fedmsg
16:40:36 <nirik> most things only listen for a small subset of messages, and thats why a message bus is useful:
16:41:02 <nirik> say a compose finishes, it emits a message, openqa listens for that, sees it and starts testing it.
16:41:27 <nirik> Or a new package is added on src.fedoraproject.org, a app sees that message and adds the new package to koji
16:41:33 <dtometzki> ok
16:41:37 <nirik> so it's really handy for coordinating events.
16:41:53 <nirik> We have 2 rabbitmq clusters... one in production and one in staging.
16:42:01 <nirik> Both clusters have 3 vm's...
16:42:37 <nirik> most of our queues are 'durable' meaning the 3 nodes make sure to keep all messages in sync with each other and keep them on disk in case all of them rebooted or something.
16:42:53 <nirik> It's a pretty solid application.
16:43:07 <nirik> https://www.rabbitmq.com/ is the upstream
16:43:28 <nirik> We also are starting to have a few apps that need queues for their internal needs...
16:43:49 <nirik> mbs (module build service) and odcs (on demand compose service)
16:45:09 <nirik> we make them seperate vhosts/queues for them on our cluster, since it's already there and working for fedora-messaging.
16:45:09 <nirik> Oh, you cna also see the messages on your home machine, we gateway (read only)
16:45:09 <nirik> so, fedmsg-tail or fedora-messaging will listen and show you those messages
16:45:46 <nirik> https://fedora-messaging.readthedocs.io/en/stable/ is the docs on that
16:46:16 <nirik> ok, I think thats the high level overview? any questions on any of that?
16:46:49 <dtometzki> nirik many thanks for the information
16:47:14 <mobrien> nirik++
16:47:16 <nirik> no problem, thanks for lisening. As always, the config is in our ansible tree also for the cluster.
16:47:24 <dtometzki> any questions ? when no then we go to the next topic
16:47:40 <copperi> nirik++
16:47:40 <zodbot> copperi: Karma for kevin changed to 27 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:48:08 <dtometzki> #topic Ideas for upcoming learning topics
16:48:34 <smooge> thanks nirik
16:48:52 <smooge> I don't have any ideas on future items..
16:48:56 <nirik> I think we should do one on our docs... all the places they are.
16:49:00 <dtometzki> new learning topics for the next week ?
16:49:06 <nirik> infra-docs, howtos, etc
16:50:11 <mobrien> I can have a go at doing that one. I should have most of the info
16:50:19 <bodanel> nirik++
16:50:21 <zodbot> bodanel: Karma for kevin changed to 28 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:51:06 <t0xic0der[m]> WebSockets?
16:51:41 <nirik> t0xic0der[m]: sounds interesting... can you give it? ;)
16:52:12 <t0xic0der[m]> Certainly :D
16:52:31 <dtometzki> can anyone add this to the next agenda ?
16:52:33 <t0xic0der[m]> nirik I would be able to.
16:53:18 <nirik> cool. I would love to know more about those...
16:55:15 <dtometzki> #topic Open Floor
16:55:40 <dtometzki> any open Floor to discuss ?
16:57:06 <mobrien> #info t0xic0der[m] [Introduction to WebSockets ] on 2021-03-04
16:57:06 <mobrien> #info ??? [???] on 2021-03-04
16:57:21 <mobrien> sorry didn't mean the second line there
16:57:31 <dtometzki> ahh ok mobrien
16:57:51 <t0xic0der[m]> mobrien thanks for adding :)
16:58:16 <geppetto> #undo
16:58:27 <dtometzki> so if for all ok we will close the meeting ?
16:58:40 <mobrien> thanks for chairing dtometzki
16:58:44 <mobrien> dtometzki++
16:58:44 <zodbot> mobrien: Karma for dtometzki changed to 3 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:58:56 <Zlopez[m]> dtometzki++
16:59:00 <t0xic0der[m]> dtometzki++
16:59:09 <Zlopez[m]> * dtometzki++
16:59:15 <rparimi> So glad to be part of this meeting
16:59:15 <dtometzki> It was my first meeting
16:59:26 <dtometzki> #endmeeting