19:00:01 <nirik> #startmeeting Infrastructure (2012-11-08)
19:00:01 <zodbot> Meeting started Thu Nov  8 19:00:01 2012 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:00:01 <nirik> #meetingname infrastructure
19:00:01 <zodbot> The meeting name has been set to 'infrastructure'
19:00:01 <nirik> #topic Hey there!
19:00:01 <nirik> #chair smooge skvidal CodeBlock ricky nirik abadger1999 lmacken dgilmore mdomsch threebean
19:00:01 <zodbot> Current chairs: CodeBlock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge threebean
19:00:14 <marcdeop> hello everyone
19:00:26 * nirik waits for folks to wander in
19:00:58 <marcdeop> I have a question
19:01:02 * pingou 
19:01:08 <smooge> crowbot here
19:01:09 <marcdeop> what time are the meetings supposed to be?
19:01:16 <marcdeop> 18h utc?
19:01:19 <nirik> marcdeop: we were at 18UTC...
19:01:28 <marcdeop> ain't it 19h right now?
19:01:29 <nirik> but DST ended in the us, so we moved with it to 19UTC
19:01:51 * nirik needs to update pages.
19:01:53 * relrod_school here
19:01:54 <nirik> I hate Daylight savings time.
19:02:00 <pingou> nirik: try fedocal :p
19:02:12 <nirik> pingou: :)
19:02:35 <pingou> ok you'll still have to change it twice a year :]
19:02:38 <marcdeop> why do we move things with daylight savings time?
19:02:45 <marcdeop> I mean... utc doesn't change, why should we?
19:03:00 <pingou> marcdeop: because local time do
19:03:19 <nirik> marcdeop: we don't have to...
19:04:01 * nirik doesn't feel too strongly about it, but there's also normally meetings around us and they switch, so we either overlap or move channels.
19:04:39 <pingou> I find it easier if the meeting is at the same time 50 weeks/year
19:05:26 <marcdeop> pingou: yeah, I understand but... EU and US don't cahnge the time at the same time, do they?
19:05:32 <marcdeop> ;)
19:05:36 <nirik> marcdeop: nope. it's a few weeks apart.
19:05:45 <pingou> marcdeop: that's why I said 50 and not 52 ;)
19:05:52 <pingou> nirik: 1 week
19:05:55 <nirik> anyhow, shall we get started?
19:06:05 * marcdeop is all for it
19:06:21 <nirik> #topic New folks introductions and Apprentice tasks
19:06:31 <nirik> any new folks? or apprentice questions or comments?
19:06:54 <marcdeop> me
19:07:06 <marcdeop> I mean, I'm new. Sorta
19:07:32 <mrwizer> I've already done my into but still new. I am looking around but willing to take on any tasks if anyone has some.
19:07:48 <nirik> welcome back marcdeop
19:07:52 <nirik> welcome mrwizer
19:08:18 <nirik> there's a list of easyfix stuff and you're welcome to just look through all existing tickets and ask about them further too.
19:08:20 <marcdeop> thanks! I'll definitelly try to help quite a bit more this time :)
19:08:47 <codegerbil> it looks like since zodbot is kind of a fedora thing, that it would be better if a fedora person write a script.  at least that 's my impression from reading the documentation
19:09:30 <nirik> codegerbil: this is refering to the ticket about pruning channels? or whats the context?
19:09:36 <codegerbil> how much does it take to mod supybot, because i am also interested in looking into that, after a ticket is submitted, which i'll get on.
19:09:39 <codegerbil> nirik: yes
19:09:45 <codegerbil> pruning channels
19:10:21 * abadger1999 here now
19:10:23 <nirik> right, I'm not sure how to easily script that. It might be possible to come up with a plugin that reports how long since it was addressed in channels... but that doesn't exist that I know of.
19:10:50 <pingou> based on the .seen command maybe
19:11:19 <codegerbil> ok, then!
19:11:26 <codegerbil> thanks
19:11:55 <nirik> supybot is in python... there's I think some guides on the net how to write up plugins...
19:12:03 <nirik> #topic Applications status / discussion
19:12:13 <nirik> any new application news this week or upcoming?
19:12:27 <nirik> I'll note we are no longer in freeze due to the crazy f18 schedule. ;)
19:12:31 * pingou worked on fedocal
19:12:45 <pingou> and I started the ansible playbook for jenkins_master
19:12:59 <pingou> threebean: pushed apps.fedoraproject.org to prod
19:13:08 <relrod_school> I've been working on extracting the mediawiki PHP fas auth stuff out to a common PHP lib, since we're going to need PHP auth for gallery3, and presumably other PHP apps in the future. See https://github.com/CodeBlock/php-fas/ if you care about that project ;)
19:13:22 <lmacken> Oh, we're not frozen? awesome. So yeah, I'm going to do a full bodhi update then instead of just a hotfix
19:13:27 <nirik> also, I need to work with puiterwijk to get the smolt eol code up and running. hopefully soon.
19:13:41 <nirik> lmacken: :)
19:13:43 <lmacken> the new apps.fp.o is awesome :)
19:14:03 <nirik> #info https://apps.fedoraproject.org/ is cool
19:14:19 <nirik> #info need to get smolt eol code setup and done.
19:14:26 <nirik> #info lmacken to do a bodhi update.
19:14:42 <nirik> relrod_school: is it looking likely to work? or ?
19:14:42 <threebean> I'm gonna aim for pushing zeromq3 stuff into production on monday
19:14:51 <lmacken> has anyone tested how smoltClient acts when smolt is gone?
19:14:55 <pingou> https://209.132.184.106/fedocal/ and there is the start of timezone suport
19:14:57 <pingou> +
19:14:59 <pingou> p
19:15:21 <nirik> lmacken: yeah, we are setting up a thing that accepts the request and sends them a thing that can point to a 'retired' page.
19:15:27 <nirik> we tested it in stg a while back
19:15:35 <lmacken> nirik: oh ok, cool.
19:15:59 <relrod_school> nirik: I'm still early in the project, but yeah, what I've done so far is working. It will hopefully fix that you can't log in to mw via email address too (FAS lets you auth via email or username, the wiki only lets you via username)
19:16:16 <nirik> that would be a nice bug to fix too, yeah
19:16:59 <nirik> ok, anything else on the app side?
19:17:14 <nirik> #topic Sysadmin status / discussion
19:17:25 <lmacken> one last app thing. we had a couple more mailbombings this week :\
19:17:28 <nirik> #info Applied updates to all machines today since we are not frozen.
19:17:35 <lmacken> still not fully resolved
19:17:40 <nirik> lmacken: oh yeah, from tagger?
19:17:52 <lmacken> yep. beaker anonymous sessions creating lots of file descriptors
19:18:10 <smooge> pingou AWESOME SAUCE
19:18:22 <nirik> strange. :( well, at least it also happens in stg, so we can hopefully track it down...
19:18:39 <nirik> #info ansible work continues (I used it to apply updates today)
19:18:40 <lmacken> yeah, I'll work with threebean to try and hunt it down
19:18:45 * threebean nods
19:18:49 <pingou> smooge: thanks, trashy (fas trasher) works on the css :)
19:18:53 <threebean> lmacken: it's real weird.
19:19:31 * abadger1999 is hoping to have a new pyhton-fedora out before we re-freeze.
19:19:35 <nirik> we have some new machines coming in in the next few weeks probibly... a pair of machines to replace sign-vault02 and a virthost machine to test out (new vendor)
19:19:48 <abadger1999> Should be additions to the APIs so low risk of breakage.
19:19:56 <nirik> cool.
19:20:02 <lmacken> abadger1999: I've been doing a little bit of hacking on the fedora.client.wiki this week. Will try and wrap that up before then, but we'll see
19:20:12 <abadger1999> lmacken: Cool.  Let me know.
19:20:22 <abadger1999> If it's in tomorrow definitely can go into the next release.
19:20:30 <lmacken> ok
19:20:41 <nirik> oh, there is likely going to be a phx2 outage in december. (date not set in stone yet). It will be a 30min-1hr outage of network.
19:20:45 <abadger1999> If it's not, still likely :-)
19:20:58 <skvidal> in december, how nice
19:20:59 <nirik> we need to look at if we can migrate everything away from phx2 for that time or just say we are out then or what.
19:21:03 <skvidal> maybe it'll be on the 24th
19:21:05 <skvidal> that'd be nice
19:21:17 <skvidal> nirik: is ALL of phx2 out? is it colo-wide?
19:21:17 <abadger1999> or the 27th :-)
19:21:18 <nirik> looking like 12/15
19:21:24 <skvidal> abadger1999: yes
19:21:37 <nirik> skvidal: it's a router/firewall thing, so all net will be out.
19:21:44 <nirik> machines will still be up, etc.
19:21:44 <skvidal> :(
19:21:47 <nirik> just no network
19:22:04 <nirik> #info phx2 network outage in december, need to plan for it.
19:22:34 <nirik> I think it might be a nice test of seeing if we can keep things working with it down... or at least not too badly broken
19:22:35 <smooge> it should be 15 minutes
19:22:42 <smooge> last year it was about 10 minutes
19:23:02 <nirik> All the stuff I have been told is at least 30min. :)
19:23:16 <smooge> ah.. I misread
19:23:17 * skvidal plans for 10 to 15 hours
19:23:31 <smooge> skvidal, pager off and a trip to the bahamas?
19:23:41 <nirik> :)
19:23:53 <nirik> anyhow, thats a bit away still, but just wanted to mention it...
19:24:02 <skvidal> nirik: thx
19:24:15 <nirik> any other sysadminy things recently or upcoming?
19:24:29 <nirik> #topic Private Cloud status update
19:24:37 <skvidal> clouds!
19:24:49 <nirik> I re-installed out openstack cloudlet with folsom
19:24:50 <skvidal> I wrote up some docs on setting up a persistent or transient host in the cloud
19:24:51 <skvidal> using ansible
19:25:09 <nirik> I'm likely to reinstall the head node one more time to get it more ansibled and also to fix a problem with the install that has since been fixed. ;)
19:25:28 <nirik> I also got euca2ools working fine against it.
19:25:34 <skvidal> yay!
19:25:46 <nirik> so, in theory we should be able to use the exact same ansible stuff for it as euca...
19:26:18 <nirik> skvidal: side note, there's no way to name an instance when you start it command line is there? it gets whatever the server assigns it?
19:26:30 <skvidal> I believe that's correct
19:26:44 <skvidal> you can with the nova tools - so it's probably in the os api not the ec2 api
19:26:49 <nirik> euca you get something like 'i-2712763453' right?
19:26:54 <skvidal> yep
19:27:00 <skvidal> but to be fair
19:27:03 <nirik> for openstack you get things like: 'server-2bf83951-dd53-461b-b14f-cebbbef329c5'
19:27:06 <skvidal> the terminate-instance thing I wrote
19:27:07 <nirik> which is a bit wacky
19:27:15 <skvidal> for euca2ools
19:27:19 <skvidal> will take ip addresses
19:27:26 <nirik> yeah.
19:27:55 <nirik> so, after I reinstall the openstack side again I am going to make accounts for a bunch of people, and we can hammer on it.
19:28:09 <skvidal> nirik: vlans!
19:28:29 <nirik> I can try messing with that... but the network situation is... in flux.
19:28:32 * herlo wants an account :)
19:28:52 <skvidal> so on the euca side
19:29:04 <skvidal> I am building the coprs backend and builders in place on it now
19:29:22 <nirik> If anyone wants to help out with cloudlets, see: https://fedoraproject.org/wiki/Infrastructure_private_cloud#Moving_to_.22production.22 for a list of things we need to work on/document/decide.
19:29:22 <skvidal> and it's coming along swimmingly
19:29:39 <skvidal> I also setup 2 persistent instances for the twistedmatrix folks to run their buildbots on
19:29:45 <skvidal> one under el6 the other on f17
19:29:59 <nirik> cool.
19:30:17 <nirik> herlo: sure thing, will ping you when stuff is reinstalled.
19:30:19 <skvidal> I asked him about it yesterday he seemed fine w/them for the moment
19:30:23 <herlo> nirik: great
19:30:24 <herlo> thanks
19:30:53 <nirik> ok, any further cloud news?
19:31:27 <nirik> #topic Security FAD update
19:31:46 <nirik> abadger1999 and I brainstormed some of the fas stuff we need to look at beforehand.
19:32:01 <nirik> https://fedoraproject.org/wiki/FAD_Infrastructure_Security_2012/FAS_plan
19:32:34 <nirik> so, if everyone could look at that and add to it/discuss/etc we might get a head start on the fad. ;)
19:33:16 <nirik> #topic Upcoming Tasks/Items
19:33:29 <nirik> (get ready for info dump :)
19:33:33 <nirik> #info 2012-11-07 - switch smolt server to placeholder code.
19:33:33 <nirik> #info 2012-11-09 purge inactive apprentices
19:33:33 <nirik> #info 2012-11-13 to 2012-11-27 F18 Beta Freeze
19:33:33 <nirik> #info 2012-11-20 FY2014 budget due
19:33:33 <nirik> #info 2012-11-27 F18 Beta release
19:33:34 <nirik> #info 2012-11-22 to 2012-11-23 Thanksgiving holiday
19:33:35 <nirik> #info 2012-11-26 to 2012-11-29 Security FAD
19:33:37 <nirik> #info 2012-12-18 to 2013-01-08 F18 Final Freeze
19:33:39 <nirik> #info 2012-11-30 end of 3nd quarter
19:33:41 <nirik> #info 2012-12-01 nag fi-apprentices
19:33:43 <nirik> #info 2012-12-24 to 2013-01-01 Red Hat Shutdown for holidays.
19:33:45 <nirik> #info 2013-01-08 F18 release.
19:33:47 <nirik> #info 2013-01-18 to 2013-01-20 FUDCON Lawrence
19:33:50 <nirik> Any items we need to schedule or note?
19:34:44 * skvidal is really not looking forward to early/mid january
19:34:45 <abadger1999> Need to update teh freeze and release dates.
19:34:50 <smooge> smolt didn't happen yesterday?
19:35:18 <nirik> abadger1999: I did I thought...
19:35:28 <nirik> smooge: no, I need to get with puiterwijk and make it happen
19:35:29 <abadger1999> Oh -- I'll be taking a short vacation post-FAD.
19:35:39 <abadger1999> nirik: oops.  I totally flubbed in reading those dates.
19:35:39 <nirik> skvidal: gonna be a busy time. ;(
19:35:50 <skvidal> nirik: yah
19:36:04 <abadger1999> Flying back on Tuesday Dec 4
19:36:21 <nirik> Oh, one other thing I wanted to note: everyone should think about talks / sessions / hackfests for fudcon...
19:36:33 <nirik> abadger1999: cool. Gonna do some hiking?
19:36:35 <abadger1999> Likely I'll have 0 internet access between the FAD and then.
19:36:58 <abadger1999> nirik: possibly a few day hikes.  Going to visit friends and family around NC/VA
19:37:07 <smooge> I will be doing a short talk on passwords and their strength changes.
19:37:10 <skvidal> abadger1999: wherebouts?
19:37:14 <nirik> I was thinking a general infra hackfest as usual... perhaps a talk/session on our private clouds... ansible? fedmeg ?
19:37:17 <skvidal> abadger1999: I'll ask oob, sorry
19:37:26 <skvidal> nirik: at fudcon? or at the fad?
19:37:33 <nirik> skvidal: fudcon.
19:37:41 <skvidal> nirik: +1 I want to do 2 talks if I can
19:37:45 <skvidal> 1 on ansible and 1 on coprs
19:38:15 * herlo definitely wants to attend coprs talk
19:38:15 <skvidal> hmmm - and maybe I would need to be involved on the private cloud things, too
19:38:17 <pingou> skvidal: would be pretty cool if we could finish jenkins master by the fudcon :)
19:38:19 <skvidal> depending on where we will be
19:38:22 <skvidal> pingou: we will
19:38:28 <skvidal> pingou: this I know :)
19:38:30 <pingou> skvidal: awesome :)
19:38:38 <nirik> we could do a bunch of lightning talks on infra stuff too...
19:38:55 <pingou> nirik: just put all the webapp dev on the list ;)
19:39:06 <skvidal> nirik: +1
19:39:54 * relrod_school could maybe look into presenting the fedorahosted web app I was working on, if I get that finished up/tested more/etc
19:40:22 <pingou> relrod_school: if I get some cycles I'd like to help on that one as well
19:40:46 <nirik> also, might be good to do something on our security work from the fad if it works out nicely.
19:40:50 <relrod_school> pingou: cool, always appreciated
19:41:20 <nirik> anyhow, a ways away, just something to be thinking about.
19:41:26 <nirik> #info think about fudcon talks
19:41:34 <nirik> #topic Open Floor
19:41:45 <nirik> anything for open floor? questions? comments? suggestions?
19:42:39 <skvidal> as before
19:42:47 <codegerbil> are apprentices allowed to go to fudcon?
19:42:47 <skvidal> anyone who wants to look at some scripting/playbooking tasks
19:43:02 <codegerbil> yes
19:43:03 <skvidal> come talk to me - I have a couple of things that could use eyes/testing
19:43:09 <pingou> codegerbil: anyone can go to FUDCon
19:43:13 <smooge> skvidal, I want to get to that after I get these things ordered
19:43:15 <codegerbil> pingou: yay!
19:43:23 <nirik> codegerbil: everyone is welcome. ;)
19:43:39 <nirik> https://fedoraproject.org/wiki/FUDCon:Lawrence_2013
19:43:39 <codegerbil> nirik: that's great :)
19:43:42 <smooge> codegerbil, if you need assistance the ticket system for FUDcon is open.
19:43:44 <nirik> codegerbil: where are you located?
19:44:03 <codegerbil> memphis.
19:44:21 <codegerbil> not too far away
19:44:25 <herlo> codegerbil: oh, you could drive!
19:44:26 <nirik> cool.
19:44:33 * pingou will have his tickets somewhere this week
19:44:38 <skvidal> codegerbil: a short 8 hour drive through desolate country
19:44:39 <skvidal> how nice
19:44:40 <nirik> fudcon is always a great time. Lots of good info, good folks, etc.
19:44:41 <codegerbil> :)
19:44:50 <herlo> +1 nirik
19:44:56 * herlo lubs the fudcon
19:44:58 <skvidal> oh one more thing
19:44:59 <codegerbil> nothing like rt 40 west -_-
19:45:04 * nirik is also going to drive most likely.
19:45:05 <skvidal> do not bring any lurgy to fudcon
19:45:07 <codegerbil> it does look fun
19:45:09 <skvidal> leave your lurgy at home
19:45:12 <codegerbil> ....lurgy?
19:45:19 <smooge> flu cold plague
19:45:29 <skvidal> what smooge said
19:45:33 <codegerbil> oh oh!  that makes sense.
19:45:37 <pingou> skvidal: I can bring one from Europe, no ?
19:45:40 * abadger1999 crosses "black death" off his packing list
19:45:43 <smooge> I am bringing black plague this year
19:45:45 <skvidal> pingou: nope., leave it at home
19:45:51 <smooge> so I got you covered abadger1999
19:45:51 <pingou> skvidal: Ooh :(
19:46:07 <nirik> heh
19:46:07 <pingou> smooge: I'm bringing 'French accent' with me
19:46:15 <smooge> oooh even worse
19:46:15 <pingou> and that's whatever skvidal says...
19:46:15 <skvidal> pingou: I bet you are!
19:46:23 <relrod_school> oh, sidenote/reminder for FAD stuff - RH people if you have to get your badge photo in -- do that ;) I did so last week
19:46:27 <pingou> skvidal: what's your t-shirt size btw ?
19:46:42 <skvidal> pingou: it's the "I don't wear tshirts really, ever" size
19:46:45 <nirik> relrod_school: yeah, send my pic in the other day, should have new badge next week.
19:47:01 <smooge> pingou, get him a french button down shirt
19:47:02 <pingou> skvidal: not even white with blue strips and beret ?
19:47:03 <skvidal> pingou: tshirts end up being used as grease rags in the bike shed (the bike shed is gray)
19:47:10 <skvidal> pingou: hahah - that's for spot
19:47:18 <skvidal> pingou: I lack the wine and the cigarettes
19:47:25 * marcdeop remembers very well his trip to Lawrence
19:47:30 <smooge> pingou if you bring the mime paint I will wear it
19:47:34 <pingou> skvidal: I'll let you know when I pack, we can discuss things
19:47:39 <skvidal> pingou: yes :)
19:47:44 <smooge> sorry for disrailing things
19:47:47 <pingou> smooge: really ?
19:47:47 <marcdeop> nice choice to host  :)
19:47:57 <skvidal> smooge: I think miming is a criminal art in france, too :)
19:48:06 <nirik> anyhow... shall we call it a meeting?
19:48:21 <smooge> [ o>< ] me in a box
19:48:49 <smooge> call it
19:49:46 <relrod_school> ><> fishie. :P
19:49:48 <herlo> lol
19:49:48 <nirik> ok, thanks for coming everyone. :) Do chime up in #fedora-admin, #fedora-noc or #fedora-apps with any questions.
19:49:48 * relrod_school is done ;)
19:49:48 <nirik> #endmeeting