20:01:26 <smooge> #startmeeting infrastructure
20:01:26 <zodbot> Meeting started Thu Dec  2 20:01:26 2010 UTC.  The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:01:26 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:01:36 <smooge> #meeting name infrastructure
20:01:43 <smooge> #meetingname infrastructure
20:01:43 <zodbot> The meeting name has been set to 'infrastructure'
20:01:51 <smooge> #chairs mmcgrath smooge
20:02:01 <smooge> #chair mmcgrath
20:02:01 <zodbot> Current chairs: mmcgrath smooge
20:02:32 <smooge> #topic Roll Call
20:02:33 * nirik is here.
20:02:36 <smooge> smooge is here
20:02:40 <goozbach> here
20:02:49 * waltJ is here
20:03:10 * ninjazjb here (kind of)
20:03:18 <abadger1999> here
20:04:50 <smooge> #topic Old Items
20:05:07 <smooge> nirik, this would probably cover old tickets
20:05:11 <nirik> ok.
20:05:31 <nirik> as I mentioned on the list, I'm going to try and dredge through old tickets.
20:05:38 <nirik> we have currently 323 tickets open.
20:05:50 <skvidal> hi
20:05:51 <skvidal> meeting
20:05:53 * skvidal is here
20:05:54 <skvidal> sorry
20:06:08 <nirik> If folks could take a few minutes to look at all the ones they are currently assigned to and close any that don't matter or are done, etc that would be great.
20:06:10 * mdomsch 
20:06:33 <nirik> otherwise I wil try and figure out how to route some, or get movement or the like on them over the next few weeks.
20:06:35 <goozbach> there's a few that go back to fedora 12 which should be closed now no?
20:06:53 <goozbach> nirik: I can help a bit on this project
20:06:54 <nirik> depends. Some might need to just be updated for current if they still matter.
20:06:56 <skvidal> link to the tickets?
20:06:58 <nirik> goozbach: cool. ;)
20:07:02 <skvidal> b/c I can never find the darn thing
20:07:06 <nirik> https://fedorahosted.org/fedora-infrastructure/report/1
20:07:08 <skvidal> thx
20:07:25 <goozbach> there's this query for meeting tickets too
20:07:29 <goozbach> https://fedorahosted.org/fedora-infrastructure/query?status=new&status=assigned&status=reopened&keywords=~meeting&keywords=~meet&order=priority
20:07:40 <goozbach> maybe we could add a new report with that query
20:07:43 * ke4qqq shows up a bit late
20:07:51 <nirik> or https://fedorahosted.org/fedora-infrastructure/report/8 might be helpfull. as that shows your assigned tickets first
20:07:55 <nirik> goozbach: yeah, good idea.
20:07:59 * mmcgrath is sort of here
20:08:18 <goozbach> I tried to make a report I guess I don't have access to that
20:08:37 <nirik> yeah, needs admin.
20:08:55 <smooge> I am trying to get a new report built but my sql sucked
20:09:11 <smooge> really... like I had puked perl
20:09:16 <nirik> I can add one if someone can add me to admin. I setup the fesco trac a while back to do that.
20:09:31 <goozbach> if I can get a sponsor to get needed access I can do that
20:09:34 <smooge> ok will do so after meeting
20:09:49 <smooge> I have to see what needs to be done and who can do what.
20:09:57 * nirik is happy to let goozbach do it, and assist them.
20:10:02 <smooge> I will sponsor goozbach to test after meeting
20:10:14 <goozbach> smooge: ty
20:10:56 <smooge> I would also like any tickets that look important to have a meeting tag added to them so that the front page will get them
20:11:07 <smooge> thank you nirik and goozbach for doing this
20:11:40 <nirik> no prob.
20:11:48 <goozbach> no problem I don't have much project time, but I can do administriva in short spurts during $DAYJOB
20:12:14 <smooge> I will go through the list of items in the queue from yesterday
20:12:21 <smooge> #topic Upgrading puppet
20:13:17 <CodeBlock> I'm here; sorry, completely forgot
20:13:19 <smooge> I am trying to get publictest01 rebuilt so it can be a test puppet server. We will then use publictest03 as the client
20:13:52 <smooge> Todays luck with a) KVM and b) xen have been shitty to say the least so it will take me a bit
20:14:09 <ke4qqq> does that mean our puppet config will be effectively out in the open?
20:14:48 <smooge> not that I think. ricky did this in the past so I am going to ask what he did
20:15:10 <smooge> if it looks like it might be then we will review and redo
20:15:28 <CodeBlock> we should lock down who all has access to pt01 then (limit it to groups with access to puppet01) if we're testing with prod configs
20:15:31 <smooge> or limit it to just a few people onto the box
20:17:18 <smooge> we will want a project plan of some sort and steps to deal with it before going much further.
20:17:53 <smooge> however due to puppet being only supported on the 2.6 tree it will need to be done at some point
20:18:09 <smooge> #topic Monthly upgrades.
20:18:20 <smooge> ok we are out of freeze and have a lot of upgrades to do.
20:18:30 <smooge> skvidal mentioned he would spearhead this
20:18:40 <skvidal> yes
20:18:43 <skvidal> upgrades
20:18:50 <skvidal> I just reran func-yum getinfo
20:18:56 <skvidal> and we have a lot of boxes which need a lot of love
20:19:03 <skvidal> my plan looks something like this at the moment
20:19:06 <smooge> yep pretty much everything
20:19:14 <ninjazjb> smooge: sorry, i have to take off
20:19:18 <skvidal> 1. hit the boxes which are not likely to be updated to rhel6 first - so that means the rhel6 boxes :)
20:19:44 <skvidal> 2. sift the rhel5 list and apply security updates asap
20:19:52 <skvidal> 3. everything else
20:20:14 <skvidal> sound exciting?
20:20:32 <smooge> ninjazjb, catch up later
20:20:37 <smooge> tomorrow morning
20:21:28 <smooge> skvidal, yeah the one item I found was that security list only showed EPEL security items.. do the EL5 repos not get that info/
20:21:30 <CodeBlock> skvidal: when are we looking to start that process?
20:21:38 <skvidal> smooge: they should - I download it
20:21:52 <skvidal> CodeBlock: right after i finish futzing around with the nagios xmpp notification thing
20:22:00 * nirik also notes that with epel6 still in beta, there's no security/bugfix/enhancement info there.
20:22:06 <skvidal> which probably translates into "I'll work on the low-hanging fruit soon"
20:22:20 <skvidal> nirik: for the rhel6 boxes I'm likely to just apply all updates
20:22:24 <nirik> yeah.
20:22:36 <smooge> skvidal, will double check after meeting
20:22:41 <skvidal> ok
20:22:41 * nirik would be happy to help if needed, but not sure how much time he can commit to it.
20:22:53 <smooge> skvidal, I still need to do a yum-compare
20:23:08 <CodeBlock> skvidal: well i'm happy to help, but I'll be off and on next week (finals week)...but yeah
20:23:09 <smooge> since most of the EL6 boxes are beta boxes upgraded to EL6 or such
20:23:15 <skvidal> nod
20:23:24 <smooge> CodeBlock, finals first then fun :)
20:23:39 <CodeBlock> smooge: :(
20:24:24 <smooge> just don't do what I did my freshman finals which was vodka shots per physics question
20:24:52 <smooge> ok skvidal so we can start next week on Monday?
20:25:05 <CodeBlock> smooge: naw, won't do that. Will just be on little sleep, and helping here, and doing some other projects, etc
20:25:20 <smooge> #topic Rebuilding RHEL6 systems
20:25:28 <CodeBlock> but yes, finals come firsrt. *shuts up now*
20:25:49 <smooge> This ties into updates and such. Most of the EL6 boxes are EL6beta upgraded
20:26:09 <smooge> I have been trying to install various pt El6 boxes today and having 0 luck
20:26:55 <smooge> so I am not sure where this stands at the moment
20:28:27 <smooge> anyone who wants to see me have a temper tantrum after the meeting can watch in #fedora-noc
20:28:41 <smooge> #topic Moving .devel and .stg systems to dedicated hardware.
20:29:08 <smooge> ok so one of mmcgrath's last wishes before he passed onto the great cloud in the sky was to move all the PT boxs to ibiblio
20:29:44 <smooge> I am almost done with it if I can play some musical chairs.
20:30:11 <smooge> currently ibiblio has publictest3 and osuosl1 has a couple of staging and .dev boxes.
20:30:38 <smooge> I am going to move one of the .dev boxes to ibiblio (when I can get EL6 building on a xen server) and that should fix that.
20:31:04 <smooge> [again see temper tantrum after meeting]
20:31:12 <CodeBlock> xD
20:31:12 <smooge> #topic Moving to newer bacula
20:31:32 <smooge> Ok we are currently using two different methods to do backups here
20:32:01 <smooge> 1) Bacula from EL5 to a tape deck and 2) rsync scripts of certain items to an encrypted set of drives.
20:33:11 <smooge> The bacula is very old and no longer supported upstream. The idea was to move to the one with EL6 BUT it is a Major undertaking... as we have terabytes of backed up data and it isn't an easy conversion.
20:33:55 <smooge> Again this is going to need a big project plan
20:34:20 <smooge> there have been some volunteers on the list. anyone here?
20:34:40 * ke4qqq is willing to help with this
20:34:58 <smooge> thanks. are you going to fudcon?
20:35:01 * ke4qqq is about to do a similiar move to bacula 5.x
20:35:03 <ke4qqq> smooge: yes
20:35:45 <smooge> I think it would be a good time to lay down the plans and what will be needed. Do I need to look at getting new hardware or dump everything to a petabyte disk etc etc :)
20:36:59 <smooge> I think karanip will be there so he would be a resource to lean on (buy expensive case of cigarettes as bribes)
20:38:02 <smooge> #action ke4qqq and others will lok at bacula upgrade at FUDcon
20:38:43 <smooge> #topic Mirror systems moving to Fedora hardware
20:38:53 <smooge> Ok this is going to be my beast for the next month
20:39:15 <smooge> we have new hardware in PHX2 and soon at RDU to be the frontends for Fedora mirrors.
20:39:31 <smooge> mdomsch, you around?
20:39:56 <mdomsch> barely :-)
20:40:02 <mdomsch> yeah, i saw that on th elist
20:40:04 <mdomsch> how can I help?
20:40:57 <smooge> well it will be mostly communication and making changes to mirrormanager I would guess. I am going to reach out to warthog9 on ideas also
20:41:22 <smooge> basically we will have 4 hosts in PHX2 and 2 in RDU for I2 and Gigapop
20:41:25 <mdomsch> MM changes should be minor if at all
20:41:42 <smooge> they will have new ips I am going to get RSN(TM).
20:43:12 <smooge> and then it would be pointing out to the mirrors that we have moved locations and hostnames for top tier rsyncs
20:43:22 <mdomsch> can we move the hostname too?
20:43:40 <mdomsch> I don't even know the current hostnames
20:43:44 <mdomsch> download*.fedora.redhat.com i suppose
20:43:49 <mdomsch> and rhm*.redhat.com ?
20:44:14 <mdomsch> minimize pain for the syncers if we can keep using the same names
20:44:38 <smooge> the hostnames will move as it will go from redhat.com -> fedoraproject.org
20:44:41 <mdomsch> we have a pile of mirrors who aren't active participants on lists etc, but things continue to work, so they don't have to be too active
20:44:50 <mdomsch> can redhat.com give us a CNAME ?
20:45:31 <smooge> not sure.. I will ask
20:45:57 <goozbach> if so that's a good idea. principle of least suprise
20:47:22 <mdomsch> smooge: can we also get an estimate of how much bandwidth is OK for those servers to provide?
20:47:39 <mdomsch> so we can watch them, and add or restrict people to match?
20:48:09 <smooge> I will do so.
20:48:34 <mdomsch> gigapop?  where's that?
20:49:23 <mdomsch> oh, you have a gigapop connection in RDU.  Nice.
20:49:48 <smooge> oh sorry I was thinking of the network that replaces I2. NCSU has either gigapop or some similar thing
20:50:21 <mdomsch> NLR
20:50:35 <smooge> Lambda rail thanks
20:50:38 * Elwell apologises for tardiness
20:51:04 <smooge> np
20:53:13 <smooge> mdomsch, we won't be getting a cname for rhm.redhat.com as it will still be there to mirror RH stuff .
20:53:28 <mdomsch> ugh
20:53:35 <mdomsch> will they continue to carry fedora content on it?
20:53:39 <mdomsch> e.g. they pull from us?
20:53:48 <mdomsch> and continue to serve it?
20:54:05 <mdomsch> we can ask people to switch, but I hate just cutting them off if they're not watching the lists...
20:55:24 <smooge> no it will be for RHEL src.rpms and jboss stuff
20:55:40 <mdomsch> bugger
20:56:24 <smooge> basically they will stop mounting the Fedora netapp and we will start mounting that netapp
20:57:12 <mdomsch> ok
20:57:49 <smooge> so I guess we will need to start announcing to the mirror list and all that
20:59:26 <smooge> alright I think I have killed this meeting
20:59:34 <smooge> #topic open floor
20:59:43 <smooge> anyone have anything.. anyone.. bueller?
20:59:52 <mmcgrath> nein
21:00:03 <skvidal> umm
21:00:05 * abadger1999 had a meeting ticket
21:00:15 <skvidal> let abadger1999 go first
21:00:22 <skvidal> mine is not important
21:00:34 <abadger1999> https://fedorahosted.org/fedora-infrastructure/ticket/2481
21:00:38 * rbergeron hates to bump in but cloud meeting is starting in like .... imminent fashion ;)
21:00:49 * rbergeron looks to see if any other cloud folks are poking and and they aren't yet so go ahead
21:01:16 <abadger1999> We can continue on #fedora-admin?
21:01:19 <abadger1999> smooge: ?
21:01:30 * rbergeron starts to wonder if she should move out the cloud mtg a bit so she's not interrupting y'alls flow every week
21:01:35 <smooge> ok will close off and continue in #fedora-admin
21:01:45 <smooge> #endmeeting