infrastructure
LOGS
19:00:01 <nirik> #startmeeting Infrastructure (2011-11-10)
19:00:01 <zodbot> Meeting started Thu Nov 10 19:00:01 2011 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:00:01 <nirik> #meetingname infrastructure
19:00:01 <nirik> #topic Robot Roll Call
19:00:01 <nirik> #chair smooge skvidal Codeblock ricky nirik abadger1999 lmacken dgilmore
19:00:01 <zodbot> The meeting name has been set to 'infrastructure'
19:00:01 <zodbot> Current chairs: Codeblock abadger1999 dgilmore lmacken nirik ricky skvidal smooge
19:00:08 * abadger1999 here
19:00:12 * lmacken 
19:00:18 <wsterling> Here
19:00:29 <wsterling> \me here
19:00:30 <p3ll0n> Here. And a new face.
19:00:33 * nirik waves at everyone.
19:00:35 * pingou 
19:00:39 <npmccallum> hello :)
19:00:51 * adimania is here
19:01:23 <dgilmore> hola
19:01:31 <Backdraft> hello everyone
19:01:38 * nirik will wait a minute or two for folks to wander in.
19:03:20 <nirik> ok, I guess lets go ahead and dive on in. ;)
19:03:23 <nirik> #topic New folks introductions and apprentice tasks/feedback
19:03:42 <nirik> welcome to new folks. ;) Could you introduce yourselves and what kind of things you are looking to work on?
19:04:00 <nirik> and if there are any apprentices who want to talk about tickets or tasks, chime right in. ;)
19:04:19 <adimania> !
19:04:28 <adimania> Should I go first?
19:04:39 <p3ll0n> I'm a sysadmin with a startup that hosts a SaaS app. Would like to work on anything from app dev to provisioning and maintaining boxes.
19:04:43 <smooge> here
19:04:46 <smooge> ish
19:05:36 <Backdraft> I'm currently as SysAdmin on RedHat and AIX, and I'm interested in learning more about the larger sysadmin world and more about Linux in general
19:06:14 <adimania> My name is Aditya. I am from India and currently working as system admin. I have some experience with puppet and setting up Koji and can do some python scripting as well.
19:06:26 <abadger1999> p3ll0n: What programming languages do you know?
19:06:29 <nirik> awesome. Welcome all.
19:07:08 <p3ll0n> abadger1999: Python, C/C++ very well, Ruby/Perl decent knowledge
19:07:14 <abadger1999> Excellent.
19:07:52 <abadger1999> p3ll0n, adimania: If either of you want to work on developing our web apps, I'll get you started sometime.
19:07:53 <nirik> please do take a look at https://fedoraproject.org/wiki/Infrastructure_Apprentice and http://fedoraproject.org/wiki/Infrastructure/GettingStarted and do hang out in #fedora-admin and #fedora-noc... do chime in there with questions or asking about tickets or the like. Folks who would like to be apprentices can see me over there after the meeting.
19:08:29 <npmccallum> Nathaniel McCallum here... I'm hoping to pimp a bof at FUDCon on krb5 and OTP authentication :)
19:08:43 <p3ll0n> nirik: will do thanks!
19:09:13 <nirik> npmccallum: looking forward to it. Will see if it can meet our needs.
19:09:25 <adimania> abadger1999, I am more into admin stuff like puppet than development. So I would prefer something on that line.
19:09:44 <abadger1999> adimania: No problem.  We have plenty of that work too :-)
19:10:00 <adimania> nirik, will catch you for sure.
19:10:05 * LoKoMurdoK here
19:10:28 <nirik> great.
19:10:47 * adimania was directed to https://fedorahosted.org/fedora-infrastructure/report/14 by ricky
19:11:02 <nirik> ok, look forward to seeing folks in #fedora-admin/#fedora-noc. Don't feel shy about asking questions in those channels...
19:11:14 <nirik> adimania: yes indeed.
19:11:20 * nirik just added a new one before the meeting.
19:11:45 <nirik> ok, any other new folks or apprentice questions? shall we move on to the next topic?
19:12:32 <nirik> #topic Release retrospective
19:12:41 <nirik> so, tuesday was the Fedora 16 release day.
19:12:53 <nirik> How did everyone feel it went? anything we should change or improve?
19:13:53 <nirik> The main thing I was thinking is that we could silence some specific nagios alerts before hand... ie, process limits on download servers always alert on release day.
19:14:44 <mdomsch> doh
19:14:47 <mdomsch> DST switch got me
19:14:54 <nirik> mdomsch: happens to us all. ;)
19:15:17 <pingou> we had a remark on the little amount of seeder for some spins, don't know if we can help on this though
19:15:28 <abadger1999> nirik: we'd almost want a "if processes fall below this limit" check on release day.
19:15:46 <nirik> pingou: yeah, I was seeding, and a few others, but perhaps we could coordinate it better.
19:16:01 <nirik> abadger1999: yeah.
19:16:19 <nirik> nagios: take this check and reverse it. :)
19:16:53 <mdomsch> p3ll0n: I'm always looking for people to help maintain MirrorManager, written mostly in python
19:16:58 <nirik> pingou: perhaps next cycle we could post to the test list when torrents were made and anyone with the last rc or otherwise had content could seed.
19:17:31 <pingou> nirik: that would be a way for sure
19:17:33 <mdomsch> nirik: there was a request from mirrors to get the torrent files into the master mirror tree, rather than posted separately
19:17:56 <nirik> mdomsch: hum. I suppose thats a thought. Although this is getting a bit more rel-engy. ;)
19:18:01 <mdomsch> yep
19:18:12 <p3ll0n> mdomsch: Okay cool, let's connect after the meeting.
19:18:34 <nirik> ok, if nothing more on release, will move on to the next topic.
19:19:01 <nirik> #topic Fudcon Hackfests/Workshops
19:19:11 <npmccallum> OTP!
19:19:15 <nirik> So, I started a thread on this on the infrastructure list...
19:19:25 <mdomsch> download* boxes saw a small spike in traffic on Tues and Wed, but nothing they couldn't handle
19:19:47 <nirik> I was going to go add the ones suggested to the wiki... did anyone have other ideas? or changes to those?
19:20:12 <mdomsch> nirik: MM needs some work, mostly in the TG2 migration
19:20:45 <nirik> how many folks here are going to make it to the next north american fudcon out of curiosity?
19:20:54 <mdomsch> could be broken up and lend itself to a hackfest
19:20:56 <nirik> https://fedoraproject.org/wiki/FUDCon:Blacksburg_2012
19:20:56 * abadger1999 raises his hand
19:21:05 * mdomsch is planning on attending
19:21:11 * nirik should be there.
19:21:11 * npmccallum will be there
19:21:26 * lmacken too
19:21:40 * dgilmore should be there
19:21:59 <nirik> cool. Hopefully lots of folks can make it...
19:22:22 <pingou> mdomsch: I'll try to get a bit more done on the model this week-end, no hard promise though
19:22:26 <nirik> #action nirik to add workshops/hackfests to the wiki.
19:22:41 <nirik> if anyone thinks of other good ones they want to run or the like, do add them.
19:22:44 <mdomsch> pingou: no worries - I haven't had any time for it lately either
19:22:59 <pingou> mdomsch: too many things in parallel :)
19:23:47 <nirik> #topic serverbeach machine changes
19:23:55 <nirik> So, I thought I would give an update on this.
19:24:09 <nirik> we currently have 5 machines at serverbeach, that we got many years ago.
19:24:18 <nirik> We are getting 5 new machines to replace them.
19:24:38 <smooge> wheee
19:24:38 <nirik> this gives us a chance to migrate things to rhel6 and redo stuff.
19:24:43 <nirik> The deadline for migration is 2011-12-11
19:25:03 * adimania claps
19:25:06 <dgilmore> nirik: is this set by them?
19:25:06 <nirik> So, we need to move pretty fast on this stuff.
19:25:14 <nirik> dgilmore: yeah, they are giving us a month to move.
19:25:19 <dgilmore> nirik: ok
19:25:21 <nirik> we could possibly ask for more, but I think we can do it.
19:25:34 <dgilmore> biggest effected things will be mailing lists, and fedorahosted
19:25:40 <nirik> yep.
19:26:00 <nirik> so, I'm thinking:
19:26:28 <nirik> serverbeach1 -> goes away, and we replace it with a download-ibiblio01 machine instance at ibiblio.
19:26:28 <smooge> thanksgiving break move?
19:26:45 <smooge> that sounds good
19:27:03 * dgilmore is taking off the whole week of thanksgiving
19:27:13 <nirik> We have space there, and that leaves us more room at sb for hosted stuff.
19:27:17 <dgilmore> not that me being away should change anything
19:27:46 <nirik> serverbeach2/3 (collab1/2 and ns1) -> migrate to new serverbeach06/07
19:27:56 <nirik> and I have been considering the idea of using drbd there.
19:28:03 <nirik> for collab01/02
19:28:58 <dgilmore> ok
19:29:16 <nirik> serverbeach4/5 (hosted1/2) -> migrate to sb08/09/10. We need to decide if we want to just migrate the pile to rhel6, work on our plans to distribute it more or what.
19:29:35 * nirik is still thinking on that one, but will come up with a plan.
19:30:09 <nirik> Any questions or thoughts on all that?
19:30:19 <smooge> that sounds good
19:30:45 <smooge> ns is going to be the pain
19:30:54 * nirik thinks he will be around thanksgiving, but isn't 100% sure.
19:31:08 <nirik> smooge: oh yeah, changing ip's on that will be anoying.
19:31:21 <smooge> It can take a while to get nameservices dealt with
19:31:32 * nirik can try and ask them if we can migrate that ip.
19:32:02 <smooge> nirik, I doubt it.. and with the problems we have had with special networks I would prefer to just do the change.
19:32:11 <nirik> yeah, probibly so.
19:32:30 <nirik> I think the collab migration should be pretty simple.
19:32:42 <nirik> hosted could be more difficult depending on how we do it.
19:32:47 <lmacken> 6
19:32:52 <lmacken> oop
19:33:24 <nirik> #action nirik to come up with a concrete plan for hosted by next week.
19:33:39 <nirik> ok, moving along...
19:33:46 <nirik> #topic Upcoming Tasks/Items
19:34:15 <nirik> flood:
19:34:17 <nirik> #info 2011-11-10 infra meeting.
19:34:17 <nirik> #info 2011-11-11 kevin out on PTO
19:34:17 <nirik> #info 2011-11-14 - send out nag about pass/keys change
19:34:17 <nirik> #info 2011-11-17 infra meeting.
19:34:17 <nirik> #info 2011-11-21 - send out nag about pass/keys change
19:34:18 <nirik> #info 2011-11-24 infra meeting.
19:34:20 <nirik> #info 2011-11-28 - send out nag about pass/keys change.
19:34:23 <nirik> #info 2011-11-30 Flag day for password reset/new ssh keys.
19:34:24 <nirik> #info 2011-12-01 nag fi-apprentice folks for december
19:34:26 <nirik> #info 2011-12-11 Serverbeach migration deadline.
19:34:30 <nirik> #info 2011-12-26 to 2011-12-30 - rh shutdown week.
19:34:39 <nirik> any other stuff coming up that folks would like to note?
19:34:59 <nirik> I will be out tomorrow. ;)
19:35:09 <nirik> Might be checking in from time to time, but hopefully everything is quiet.
19:36:02 <abadger1999> 2011-11-17 final fas upgrade
19:36:03 <nirik> At some point we need to get going on migrating ibiblio01 -> ibiblio02 guests.
19:36:13 <nirik> abadger1999: cool. Adding.
19:36:34 <abadger1999> We've already upgraded to the beta -- final will just bring a few small features (ssh key clearing) and translations.
19:37:06 <nirik> excellent work on that abadger1999. :)
19:37:17 <nirik> very nice to close all those hotfix tickets and get the new captcha.
19:37:31 <abadger1999> others did most of the work :-), I just put it together for release.
19:38:20 <nirik> yeah. ;) Kudos to all those folks as well.
19:38:27 <nirik> #topic Meeting time
19:38:43 <nirik> Do we want to stick with 19UTC ? or move due to no more daylight savings time?
19:39:23 * dgilmore is down with whatever ;)
19:39:47 * abadger1999 can do either... the meeting is nicely in the middle of his day
19:40:01 * nirik is also fine with whatever. ;)
19:40:10 * adimania later than 19UTC would be a big bummer for me.
19:40:21 <smooge> likes 1900 UTC
19:41:05 <nirik> ok, lets keep it then... and see if anyone complains?
19:41:10 * pingou fine for both theoratycally
19:41:16 * nirik notes skvidal isn't here, he may have a conflict.
19:41:42 * mdomsch is rarely here, but the 2pm US time (DST or not) works better for me.
19:41:52 <mdomsch> don't plan anything around me though
19:42:10 <smooge> how does earlier look?
19:42:12 <wsterling> This time works well for me
19:42:46 <mdomsch> today's time slot works for me only because another standing meeting got cancelled
19:43:01 <nirik> the irc-support-sig is at 17utc.
19:43:14 <nirik> we could do 18, but that would be a bit back to back
19:43:43 <smooge> yeah
19:44:00 <smooge> I would like to make it good for EU people to stay with us.
19:44:10 * nirik nods.
19:44:13 * StylusEater is a dolt and forgot about the time switch
19:44:48 <nirik> #action will stay at 19UTC for now, objections/other plans to the list.
19:44:53 <nirik> #topic Open Floor
19:45:00 <nirik> anyone have any topics for open floor?
19:45:47 <smooge> mediawiki looks like it will have 118 out soon which means our 116 will be EOL I think
19:46:05 <smooge> I need to make new sets of packages for review
19:46:12 <nirik> fun.
19:46:22 <nirik> we can migrate when we get rhel6 app servers.
19:47:03 <StylusEater> mdomsch: is the mirror manager trac ticket page up to date?
19:47:05 <smooge> cool. I am dropping EL4 support for mediawiki.. and will need to look at EL5 support changes anyway as the PHP it wants is newq
19:48:39 <nirik> any other items for open floor?
19:49:36 <mdomsch> StylusEater: mostly.  ticket for TG2 migration would be nice
19:49:40 * nirik will close things out in a minute if nothing else comes up.
19:50:03 <StylusEater> mdomsch: once I get more changes in for abadger1999 I'll take a look at mirror manager
19:50:16 <smooge> I don't have anything for open floor.
19:50:34 * dgilmore has nothing
19:51:04 <nirik> ok, thanks for coming everyone!
19:51:08 <nirik> #endmeeting