20:00:00 <nirik> #startmeeting Infrastructure (2012-03-08)
20:00:00 <zodbot> Meeting started Thu Mar  8 20:00:00 2012 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:00:01 <nirik> #meetingname infrastructure
20:00:01 <nirik> #topic Robot Roll Call
20:00:01 <nirik> #chair smooge skvidal Codeblock ricky nirik abadger1999 lmacken dgilmore mdomsch
20:00:01 <zodbot> The meeting name has been set to 'infrastructure'
20:00:01 <zodbot> Current chairs: Codeblock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge
20:00:28 * CodeBlock is here
20:00:32 * skvidal is here
20:00:45 * pingou is here
20:00:46 <smooge> is here
20:00:53 <nirik> who all is around for a lovely, exciting, thrilling, action packed, adventure infrastructure meeting?
20:01:09 <CodeBlock> Rainy, tired-ey, last-day-of-school-before-spring-break kind of day.
20:01:46 <pingou> the last one sounds cool :)
20:02:10 <CodeBlock> :)
20:02:11 <nirik> :) ok, lets go ahead and get started then...
20:02:14 <nirik> #topic New folks introductions and Apprentice tasks.
20:02:20 <nirik> If any new folks want to give a quick one line bio or any apprentices
20:02:20 <nirik> would like to ask general questions, they can do so here.
20:03:01 <nirik> anyone? ;)
20:03:14 <nirik> we did have a number of new folks on the list... but the meeting time may not be great for them.
20:03:24 * wsterling here
20:03:54 <nirik> #info new folks, hang out in #fedora-admin and say hi to get started.
20:03:58 <nirik> #topic two factor auth status
20:04:05 <nirik> skvidal: any news on this in the last bit?
20:04:19 <skvidal> yah
20:04:28 <skvidal> so the cgi is posted at github
20:04:36 <skvidal> and I've sent pam_url off to some folks to audit
20:04:42 <skvidal> ricky got back to me and was not happy about pam_url
20:04:48 <skvidal> for a couple of good reasons
20:04:55 <skvidal> one of which looks pretty easy to fix
20:05:00 <skvidal> the others are going to be annoying
20:05:06 <nirik> bummer. ;(
20:05:15 <skvidal> so I'm waiting to hear back from bressers
20:05:16 <nirik> would one of the other implementations be better to go with then?
20:05:25 <skvidal> they are all going to require some hacking now
20:05:40 <nirik> yeah, I wonder if we couldn't ask someone who does security code to pick one and hack on it for us. ;)
20:05:41 <dgilmore> buenos
20:05:42 <skvidal> to get them to work like we want
20:05:51 <skvidal> nirik: I doubt it...
20:06:14 <nirik> possibly not, but we could ask around if we need to.
20:06:20 <skvidal> nirik: but I would love it. C is not my native language (so to speak) and it would be easier to fix pam_url if I was more capable at C
20:06:25 <skvidal> nirik: got any ideas?
20:06:28 <skvidal> on whom to ask?
20:06:40 <smooge> what was that think about pam_python someone mentioned earlier...
20:06:44 <nirik> not sure, I could ask bressers who to ask. ;)
20:07:01 <nirik> but let me ask around, see if I can find anyone.
20:07:02 <skvidal> smooge: 1. it's not in any distribution of anything we have
20:07:16 <smooge> oh got it
20:07:16 <skvidal> 2. pam_python makes a big point on its site of saying how much slower it is to exec python
20:07:29 <skvidal> smooge: believe me, I looked
20:07:36 <smooge> pam_pypy?
20:07:39 <nirik> #action nirik to look around for C coders to work on pam_url for us.
20:07:53 <smooge> I think that would have made a great GSOC
20:07:58 <nirik> so, once we have pam_url working we can setup a test instance and then hopefully deploy?
20:08:01 <skvidal> smooge: umm I doubt it
20:08:13 <skvidal> nirik: I think so - we can definitely get it running and tested
20:08:17 <smooge> if we had people who understand C etc etc.
20:08:45 <skvidal> nirik: I'm even happy with someone who commonly codes in C who can answer some questions for me
20:08:47 <skvidal> w/o mocking me for it
20:08:55 * skvidal is not joking about that last part
20:09:20 <nirik> ok, I'll see what/who I can find.
20:09:25 <nirik> anything else on this?
20:09:47 <skvidal> nothing so far
20:09:51 <nirik> #topic Staging re-work status
20:10:05 <nirik> ok, I meant to work on this some this week... but kept getting sidetracked. ;)
20:10:15 <nirik> I'll try again soon...
20:10:27 <nirik> #topic Applications status / discussion
20:10:43 <nirik> abadger1999 and threebean and lmacken are all out at fudcon...
20:10:44 <skvidal> I think everyone working on apps is at pycon
20:10:47 <skvidal> pycon
20:10:50 <skvidal> not fudcon
20:10:54 <nirik> yeah, sorry, pycon.
20:11:05 <nirik> anyhow, anyone else have any news on any apps?
20:11:10 * pingou updates fedora-gather-easyfix's spec file
20:11:14 <nirik> CodeBlock: how's the search crawler going?
20:11:16 * abadger1999 wishes those were synonymous... 200+ people at fudcon would be awesome
20:11:21 <abadger1999> 2000+
20:11:48 <CodeBlock> well I made some tweaks to the crawler config... some helped, some didn't... still need to tweak more...also need to try to make the web UI faster
20:11:55 <nirik> pingou: is the thought to deploy that to bapp01 and rsync to proxies like we do for websites? or run it on apps? or ?
20:11:55 <CodeBlock> it gets slow with a lot of results sometimes
20:12:28 <nirik> CodeBlock: daMaestro said he could help out, he's run dpsearch before, so ping him if you like.
20:12:41 <CodeBlock> oh cool, will do
20:12:45 <pingou> nirik: the version packaged at the moment is a simple html, rsync is easy, but with the integration with the bz, there is css and js file in addition
20:13:02 <pingou> (but it would still run as a cron)
20:13:28 <pingou> we should see what's the best way
20:13:43 <nirik> ok.
20:14:04 <nirik> any other applications news?
20:14:05 <abadger1999> pingou: would you like to ask for package review on the packages you have that need to get into epel for us to deploy?  (me is busy at pycon)  could be an easyfix if people are already packagers
20:14:16 <nirik> oh, I have one note:
20:14:25 <pingou> abadger1999: done and done :)
20:14:33 <abadger1999> Cool.
20:14:39 <nirik> we now have a way to send httpd error_logs to log02... which could help us when looking for application problems.
20:14:49 <pingou> abadger1999: I meant the two reviews , fedora-gather-easyfix and python-tgcaptcha2
20:14:53 <nirik> however. Some apps are so noisy that it's useless. we need to fix that first
20:15:47 <skvidal> nirik: you're talking about the noise from fas that looked like info-logs?
20:15:58 <nirik> I tried looking at how fas logs for example, but got confused. ;) So, we may want to look at having a VFAD on application logging someday...
20:16:03 <nirik> yes.
20:16:03 <skvidal> ah, yues
20:16:03 <skvidal> :)
20:16:05 <skvidal> sorry
20:16:34 <nirik> if we can get apps to log just tracebacks to error_log (or actual, you know, errors), we can send them to log02 and it will be much easier to see whats happening.
20:16:55 <smooge> skvidal, I can ask around with my former group.. they did various pam stuff for Science!
20:17:00 <skvidal> nirik: I have a dumb question on some of the error logs
20:17:04 <skvidal> I can fix this
20:17:05 <skvidal> File does not exist: /srv/web/favicon.ico
20:17:08 <skvidal> obviously
20:17:21 <skvidal> does anyone care if I do?
20:17:29 <abadger1999> +1
20:17:36 <nirik> yes, please fix.
20:17:42 <skvidal> ok
20:17:44 <nirik> thats one thing I think logs are good helping us do.
20:18:07 <abadger1999> nirik: Fixing logging (to the point you're talking) may be impossible.... I know we've tried for years.
20:18:16 <abadger1999> nirik: But we might be able to cut it down some.
20:18:18 <nirik> abadger1999: ;( thats sad.
20:18:19 <skvidal> done
20:18:32 <smooge> well should we just put one in there?
20:18:34 <skvidal> abadger1999: what is going on where it is outputting so much to the logs?
20:18:35 <abadger1999> I know skvidal was successful on cutting some down for other apps
20:18:46 <skvidal> abadger1999: that's b/c I cut them down with a butcher knife :)
20:18:49 <nirik> if we can't fix it at that level, we may be able to filter them in rsyslog I guess, but man that seems a kludge. ;)
20:18:59 <skvidal> it's not as graceful as a solution as I'd expect you to come up with ;)
20:19:18 <skvidal> abadger1999: what's creating the logs?
20:19:34 <abadger1999> We don't know what's going on -- something about logging is mising tracebacks when we mess with logging in certain ways.
20:19:38 <abadger1999> skvidal: pyhton's logging module
20:19:57 <nirik> it's not just fas right? other things have the same issue?
20:19:59 <skvidal> oh dear god
20:20:05 * skvidal goes to hide his head
20:20:05 <abadger1999> I'm happy for you to cut through the logs with a butcher knife.
20:20:18 <skvidal> logging makes me want to cry and hurt myself.
20:20:22 <abadger1999> yeah.
20:20:33 <nirik> #info we should really try and fix logging of applications so we can send them easily to a log host.
20:20:44 <abadger1999> I was hoping that python-3 was going to get a different logging module but that's another story.
20:21:04 <nirik> yeah, I doubt it would be at all easy, but perhaps if we got everyone together and hacked on it we could figure something out.
20:21:31 <nirik> anyhow, just wanted to mention it.
20:21:39 <nirik> any other application news?
20:21:57 <CodeBlock> do app *servers* count?
20:22:01 <CodeBlock> Most are built as el6 now
20:22:11 <abadger1999> not this week.  Hopefully we'll have some tangential news next week.
20:22:14 <nirik> sure! thanks very much for working on that CodeBlock.
20:22:19 <CodeBlock> np
20:22:25 <skvidal> abadger1999: is there any way to cut down the volume?
20:22:29 <skvidal> abadger1999: or do we lose everything?
20:22:42 <skvidal> you know what... nevermind
20:22:48 <skvidal> I think nirik is right
20:22:48 <nirik> we have just 3 rhel5 boxes left I think now... app07 (which is only serving old community), bapp01 and xen04 (the xen host bapp01 is on)
20:22:50 <skvidal> for post pycon
20:22:55 <skvidal> let's etup a VFAD
20:23:02 <skvidal> so we can spend a day or two just unfucking this
20:23:03 <abadger1999> skvidal: we can cut down the volume... but we tend to lose some tracebacks -- and we aren't sure how to configure logging so that doesn't happen.
20:23:07 <skvidal> either via rsyslog fitlers
20:23:15 <abadger1999> <nod>
20:23:15 <skvidal> or by fixing the app
20:23:20 <skvidal> whatever is more possible
20:23:24 <abadger1999> Cool.
20:23:29 * abadger1999 heads out of room to lunch
20:23:35 <nirik> have fun abadger1999
20:23:41 <nirik> #topic Upcoming Tasks/Items
20:23:46 <nirik> #info 2012-03-07 to 2012-03-14 - Pycon
20:23:47 <nirik> #info 2012-03-10 - drop inactive fi-apprentices
20:23:47 <nirik> #info 2012-03-20 to 2012-04-03 - F17 Beta Freeze
20:23:47 <nirik> #info 2012-03-27 - drop inactive maintainers from packages.
20:23:47 <nirik> #info 2012-04-01 - nag fi-apprentices.
20:23:47 <nirik> #info 2012-04-03 - F17Beta release day
20:23:49 <nirik> #info 2011-04-03 - gitweb-cache removal day.
20:23:51 <nirik> #info 2012-04-10 - drop inactive fi-apprentices
20:23:53 <nirik> #info 2012-04-24 to 2012-05-08 - F17 Final Freeze.
20:23:55 <nirik> #info 2012-05-01 - nag fi-apprentices.
20:23:57 <nirik> #info 2012-05-08 - F17 release
20:23:59 <nirik> That's what I have upcoming until f17 release day. ;)
20:24:14 <nirik> I'd like to get bapp01 replaced before beta freeze if we can.
20:24:29 <nirik> anyone have any other items they want to schedule or note?
20:24:58 <smooge> mailman password cleanup
20:24:58 * nirik listens to the crickets.
20:25:11 <nirik> smooge: ah yeah, are you gonna get that finished today? tomorrow?
20:25:27 <smooge> tomorrow.
20:25:33 <smooge> it should be a short set of things:
20:25:44 <skvidal> smooge: if oyu need help with spamming users yell at me
20:25:47 <smooge> 1) Email a list of people using seths spam-o-matic script
20:25:50 <skvidal> I have that spammer script which makes hatemail easy
20:25:52 <skvidal> oh okay :)
20:25:54 <skvidal> you know :)
20:26:09 <smooge> 2) Force mailman to do my will.
20:26:52 <nirik> cool. Sounds good.
20:27:13 <nirik> ok, anything else for upcoming?
20:27:22 <smooge> not from me.
20:27:44 <nirik> ok, on to this fun topic:
20:27:48 <nirik> #topic Ssh keypair changes still needed.
20:27:49 <CodeBlock> nirik: what special magic needs to happen to rebuild bapp?
20:27:54 <CodeBlock> bah
20:27:55 <nirik> #undo
20:27:55 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x1f057550>
20:28:10 <nirik> CodeBlock: well, bapp01 has a bunch of one-off scripts that only run there.
20:28:27 <nirik> so, we could do a migration instead of a replacement, but I think we can probibly replace it ok.
20:28:47 <nirik> I'd like to wait until toshio is back from pycon to help putting out fires on it.
20:28:59 <nirik> then we just need to schedule a day and replace it and fix anything broken
20:29:07 <CodeBlock> sounds good
20:29:10 * mdomsch votes for "not next week"
20:29:45 <nirik> mdomsch: ok, when are you gonna be around? would be good to have you around too in case. ;)
20:29:53 <nirik> the beta freeze is the 20th. ;(
20:30:27 <nirik> I guess we could do the 19th...
20:30:37 <nirik> and back it out if it became a pain
20:30:54 <smooge> could we do a bapp02 built with el6 and see about putting it in replacement?
20:31:05 <mdomsch> nirik: I'm offline tomorrow 3pm, back Mon 3/19
20:31:34 <nirik> smooge: well we could, but it's going to be a lot more work... having to change what things point to and sync and such. I guess it might be worth it.
20:31:41 <dgilmore> not that it matters im taking tomorrow off
20:31:53 <mdomsch> nothing of MM points at bapp01
20:32:03 <nirik> mdomsch: ok
20:32:08 <nirik> dgilmore: cool. :)
20:32:21 <mdomsch> bapp01 is the source, it pushes to app* from there
20:32:27 <smooge> well I would say it would be a good way to see "what files are on bappX and what ones showed up on bappY and why are they different."
20:32:27 <skvidal> dgilmore: enjoy
20:32:38 <mdomsch> so MM would be fine with a bapp02, get it working, then flip over
20:32:43 <nirik> mdomsch: cool.
20:32:46 <skvidal> dgilmore: before you go - I was meaning to ask -  do you have any thoughts on the buildvm boxes?
20:32:55 <skvidal> dgilmore: are they good enough for us to press forward with more of them?
20:33:09 <nirik> ok, I can look again at how much tweaking a bapp02 would be.
20:33:15 <nirik> #action nirik to look into possibly doing a bapp02 (rhel6) for migration.
20:33:27 <CodeBlock> nirik: more than happy to help with that too
20:34:00 <nirik> one other issue is that we would need to make sure to shut off things on bapp01 as we move them... 2 mirrormanagers both pushing to apps might be very bad.
20:34:22 <mdomsch> nirik: right
20:34:36 <dgilmore> skvidal: quick glance they seem ok. but i want to lookat setting up a heavybuilder channel first, and see how that plays into things, especially with the kernel teasm request yesterday
20:34:57 <skvidal> dgilmore: what do they want?
20:35:41 <dgilmore> skvidal: they are signing all modules at build time. need to look at running rngd or see how else we can make sure there is plenty of entropy for them to use
20:36:05 <dgilmore> skvidal: part of that is making a new gpg key at build time
20:36:17 <nirik> rngd is very easy to add. ;) I wrote up the puppet stuff for it, just didn't commit it.
20:37:15 <nirik> ok, we can continue investigating the bapp01 thing...
20:37:40 <nirik> skvidal: what was the next steps on those? we may want to hold off until we know about what hardware we have in the bladecenter.
20:37:58 <skvidal> not really
20:38:12 <skvidal> the x86-## <higher numbers> are not going anywhere
20:38:17 <skvidal> so what I wanted to do was
20:38:18 <nirik> true enough
20:38:25 <skvidal> lemme find the plan
20:38:26 <skvidal> one sec
20:39:21 <skvidal> take 2 more of the blades
20:39:26 <skvidal> turn them into 4 builders per blade
20:39:37 <nirik> ok, the bc02 ones?
20:39:40 <skvidal> yes
20:39:42 <skvidal> on  bc02
20:39:47 <skvidal> and then 2 more blades
20:39:50 <skvidal> 1 of them one big VM
20:39:53 <skvidal> and the other 2 VM
20:40:00 <nirik> cool. well, whenever dgilmore is ok with you moving ahead...
20:40:12 <skvidal> ideally
20:40:19 <nirik> I've seen no issues with the current buildvms...
20:40:21 <skvidal> if we can move from the builders we have now over to VMs
20:40:31 <skvidal> then we can dismantle the x86## builders
20:40:46 <skvidal> and grow from there... depending on what we want to do.
20:40:53 <nirik> yep.
20:41:03 <nirik> lets revisit next week then? dgilmore ?
20:41:09 <skvidal> sounds round
20:41:32 <nirik> #info will revisit next week on more migrations of builders to vm's
20:41:36 <nirik> #topic Ssh keypair changes still needed.
20:41:41 <nirik> so, some background:
20:42:01 <nirik> we asked in october of last year for everyone to change their password and generate a new ssh keypair.
20:42:15 <nirik> we marked inactive those people who didn't do that in january.
20:42:30 <nirik> then we checked and a number of them had re-uploaded the same ssh key.
20:42:40 <nirik> so, we marked them inactive again and asked them to please change.
20:43:14 <nirik> as of sunday we had several who reactivated their account and didn't still change their key.
20:43:25 <nirik> I don't want to keep playing wack-a-mole here.
20:43:52 <nirik> I'd like to propose we mark these people admin_locked and ask them to come to us to unlock their account...
20:43:53 <dgilmore> nirik: sure
20:44:11 <nirik> can anyone think of a better way to deal with them? or counterproposals?
20:44:34 <skvidal> nirik: +1 lock them
20:44:47 <CodeBlock> ^
20:44:56 <skvidal> nirik: so dumb question
20:44:56 <adrianhannah> Agreed
20:44:58 <skvidal> who is it left?
20:45:17 <nirik> there were 9 as of sunday.
20:45:23 <skvidal> nirik: might be worth getting their contact info out of fas and chasing them down on the phone, if possible
20:45:33 <nirik> people who re-activated since the last time we inactvated them, but didn't bother to change their key
20:46:20 <smooge> well lets just clear their key
20:46:23 <nirik> I suppose, but I'm not sure I want to have a long phone conversation about 'you must do this'... but...
20:46:39 <nirik> smooge: then they just re-upload it again. ;(
20:46:40 <skvidal> nirik: I enjoy being intimidating to folks on the phone :)
20:46:42 <skvidal> nirik: kidding
20:46:47 <smooge> ok we put in my key
20:46:52 <nirik> skvidal: if you want to do it, thats just fine with me. ;)
20:47:02 <skvidal> nirik: gimme the list
20:47:02 <pingou> we should look for time-zone, making sure there is someone they can talk to if they choose irc
20:47:22 <nirik> skvidal: should be in email to sysadmin-main from sunday.
20:47:44 <nirik> or we can run it again... it's in cron on lockbox01
20:47:48 <skvidal> nod
20:47:51 <skvidal> I'll just run it
20:47:55 <nirik> cool.
20:48:14 <nirik> #action skvidal to bug people in this state. Any we can't reach will get another email, then be admin_locked.
20:48:28 <nirik> ok, anything else on this?
20:48:44 <nirik> #topic Tickets from Ages past
20:49:03 <nirik> ok, I thought I would have some fun and try and clean up old tickets...
20:49:21 <nirik> and just drop 1 or 2 a meeting so we can see if we can close them or do something with them
20:49:39 <skvidal> 'fun'
20:49:40 <nirik> we currently have 171 tickets, some of which are...very old.
20:49:49 <skvidal> got a link?
20:50:16 <nirik> https://fedorahosted.org/fedora-infrastructure/report/1?sort=created&asc=1
20:50:28 <nirik> this is the list sorted in created order.
20:50:38 <nirik> so, oldest ticket is:
20:50:44 <nirik> .ticket 116
20:50:46 <zodbot> nirik: #116 (Fedora Poll) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/116
20:51:10 <nirik> This was orig going to be a stand along poll for the front page, then a wordpress plugin, then... I don't know what.
20:51:49 <adrianhannah> There are several wp poll plugins
20:51:52 <nirik> we have a similar ticket for a survey app (althought thats slightly different)
20:52:00 <mdomsch> my ancient ones get a little better with the next MM push, if/when i ever get around to it
20:52:06 <nirik> adrianhannah: yeah. Sadly, we are no longer running wordpress. ;)
20:52:26 <mdomsch> mirror push waits on the FI message bus
20:52:28 <adrianhannah> Aha, that makes sense :p
20:52:35 <skvidal> hmm
20:52:39 <nirik> so, do we still want this? close it? rescope to something we could actually do?
20:52:50 <skvidal> this looks like threebean https://fedorahosted.org/fedora-infrastructure/ticket/151
20:53:06 <skvidal> https://fedorahosted.org/fedora-infrastructure/ticket/506 <- just close this
20:53:47 <nirik> skvidal: I kinda thought 506 was nice to do... not that anyone has any time to work on that kind of thing. ;(
20:53:51 <skvidal> right
20:53:56 <smooge> I have a calendering one...
20:53:58 <skvidal> and it will almost immediately be out of date
20:54:31 <skvidal> nirik: so you'll be writing up a doc that will instantly bitrot, yay
20:54:39 <skvidal> s/you/someone/
20:54:51 <pingou> what about a wiki page ?
20:54:51 <nirik> skvidal: perhaps it could be autogenerated somehow.
20:55:01 <skvidal> nirik: from what? the pkgs we have installed?
20:55:18 <skvidal> nirik: I can dump out a list of pkg names on every system b/c.... that will be unhelpful
20:55:53 <nirik> yeah, not sure. ;(
20:57:09 * nirik reassignes 151 to threebean and updates it.
20:57:32 <pingou> #605 should be duable
20:57:40 <nirik> side note on the poll... I was pointed at http://www.primelife.eu/results/opensource/63-dudle
20:57:53 <pingou> check if the user is in more than one group, if so show the options
20:58:05 <nirik> pingou: cool. Would that be an easyfix type thing? or ?
20:58:31 <pingou> nirik: slightly above imo, it includes a db scheme change
20:58:48 <pingou> nirik: I had a web poll python based some time ago
20:58:52 <nirik> pingou: could you update that ticket with info?
20:59:14 <nirik> ok, we are running low on time...
20:59:47 <nirik> feel free to frob the older tickets as you all see fit... if you can get something moving or it looks like an easyfix, update it.
20:59:50 <nirik> #topic Open Floor
20:59:54 <nirik> anything for open floor?
21:00:01 <skvidal> I am going to bring this up on list
21:00:05 <skvidal> but I thought I'd float it out here
21:00:22 <skvidal> I'd like input on what people use/need/want from our config mgmt system
21:00:33 <skvidal> I've been playing with and adding some patches to another tool recently
21:00:38 <skvidal> and it has been making me think
21:00:48 <skvidal> about what we DO use versus what we need to use
21:01:02 <skvidal> and how complicated those can be to disambiguate
21:01:16 <skvidal> so I'd love to hear back from folks about what it is they want from config mgmt
21:01:24 <skvidal> what features are necessary
21:01:30 <skvidal> this is not about any specific tool
21:01:44 <nirik> cool.
21:01:54 <skvidal> I don't care about cfengine vs puppet vs bcfg2 vs salt - I care about what we're USING from those tools
21:02:25 <nirik> it might be worth mining our git puppet repo and seeing how many of each kind of statement there are? not sure thats easily greppable tho...
21:02:50 <skvidal> nirik: well I think it is reasonable to generate a list of requirements of basic requirements
21:02:56 <nirik> worth thinking about. yep.
21:03:01 <skvidal> my short list is this
21:03:06 <skvidal> pkg install/remove
21:03:11 <skvidal> file replacement
21:03:18 <skvidal> command execution
21:03:37 <skvidal> service 'chkconfig'
21:03:47 <nirik> groups of things? (ie, do all this on 'appservers') ?
21:03:47 <smooge> passwd setting/confirmation
21:03:48 <skvidal> process status checks
21:03:50 <pingou> stg vs prod ?
21:04:05 <skvidal> pingou: I consider stg vs prod to be an ANTI-feature, personally.
21:04:25 <skvidal> nirik: so is grouping something that's in the config mgmt?
21:04:31 <nirik> conditionals? if rhel5 {foo}, elseif AIX {doom}
21:04:39 <skvidal> nirik: or is grouping something that's in the tool that pushes the config mgmt?
21:04:57 <skvidal> nirik: so cases/conditions - what about templating?
21:05:25 <nirik> not sure. We seem to not like templating all that much, but I guess it's a feature. ;)
21:05:56 <skvidal> nirik: conditionalized actions?
21:06:06 <skvidal> ie: if file is changed then run this command?
21:06:10 <nirik> yeah, and related... variables?
21:06:23 <nirik> if bacula5 is true, do this
21:06:38 <skvidal> variables inherited from the system AND applied from the mgmt side
21:07:07 <skvidal> does config mgmt implicitly need to include some sort of inventory system or does it need to simply have access to one?
21:07:23 <skvidal> for example - if we moved to a different tool for the config mgmt but kept using 'facter' - just as an example
21:07:33 <nirik> I don't know that it matters... either way.
21:07:34 <skvidal> facter is the inventory system (to a limited extent)
21:07:37 <skvidal> ok
21:08:05 <nirik> one approach to this might be to look at our more complex machines... and ignore puppet and say 'what all needs to be done here to make this from basemachine'
21:08:13 <skvidal> nirik: +1
21:08:18 <skvidal> I like that idea very much
21:08:30 <skvidal> this came up as part of a couple of things I was working on this week
21:08:32 <nirik> then work back from that to see what would need to be known...
21:08:51 <skvidal> specifically trying to figure out all the places we would need to modify to make all of our http errorlogs go to log02
21:08:56 <skvidal> and to put it unkindly
21:09:00 <skvidal> it's a fracking mess
21:09:11 <nirik> yeah
21:09:21 <skvidal> so I started looking at the yaml catalogs in puppet for each node
21:10:03 <skvidal> and it's kinda ugly - but they do make it easier to see what _should_ be happening
21:10:25 <nirik> yeah.
21:10:30 <skvidal> the thing I've been playing with is called ansible (github.com/ansible/
21:10:36 <skvidal> and let's ignore what it's doing
21:10:40 <skvidal> b/c that's immaterial
21:10:44 <skvidal> but there is one bit in there I like
21:10:47 <skvidal> the playbooks.
21:10:59 <nirik> the nice thing about ansible (or something like it) is that you could write scripts in any lang right? whatever you like best?
21:11:00 <skvidal> essentially a description of what to do on the machine to make it 'right'.
21:11:12 <skvidal> nirik: yah - you write modules in whatever language
21:11:17 <skvidal> as long as they can execute on the remote node
21:11:20 <skvidal> that's all that matters
21:11:31 <skvidal> it pushes them all over to the remote system (using ssh/sftp) and runs them
21:11:46 <skvidal> and reports back results to the person calling them
21:11:51 * nirik nods.
21:11:55 <skvidal> but that's all the 'how they do it' bits
21:12:02 <skvidal> I was comparing a puppet recipe
21:12:04 <skvidal> to a playbook
21:12:15 <smooge> I have to head out to go to a school meeting.
21:12:20 <skvidal> puppet is describing the state
21:12:26 <mdomsch> !
21:12:27 <skvidal> whereas a playbook is describing what to do
21:12:34 <skvidal> mdomsch: ?
21:12:46 <mdomsch> typo
21:12:49 <skvidal> heh
21:12:59 <nirik> I guess there's kinda two parts here... 'is this machine the way it should be now?' if not 'how do I make it be that way'
21:13:13 <nirik> unless you want to just blindly replace files every run or the like.
21:13:21 <skvidal> well you don't have to replace all the files
21:13:23 <skvidal> like puppet
21:13:28 <skvidal> you can xfer a copy
21:13:33 <skvidal> diff the two
21:13:40 <skvidal> if they don't match, then the local one is wrong
21:13:41 <skvidal> replace it
21:13:43 <nirik> sure.
21:13:46 <skvidal> and take action accordingly
21:13:53 <skvidal> but, essentially, the assertion is this
21:13:58 <skvidal> the server is always right
21:13:58 <nirik> I'm just saying it seems like there's a 'collect info on the machine now' and 'act on that'
21:14:03 <skvidal> the node is potentially always wrong
21:14:08 <nirik> right
21:14:23 <mdomsch> skvidal: that sure sounds like puppet or chef to me...
21:14:36 <skvidal> mdomsch: I didn't say it wasn't
21:15:12 <skvidal> mdomsch: like I said - I don't want to hear about the specific tools - I'm trying to figure out what we USE and need to use
21:15:19 <mdomsch> yeah, ok
21:15:26 <nirik> anyhow, we are over time... so do post to list and we can continue discussions. ;)
21:15:31 <skvidal> ok
21:16:09 <mdomsch> new topic - S3 mirror
21:16:11 <nirik> #info skvidal to post to list and gather info on what we actually use in a Config mgmt system.
21:16:20 <nirik> mdomsch: oh yeah... go ahead
21:16:21 <mdomsch> I'll have to spin a copy of s3cmd private to FI
21:16:28 <mdomsch> until upstream takes my patches
21:16:37 <nirik> ick. Ok.
21:16:51 <mdomsch> I've got the mirror stuff working fairly well, a little heavyweight (md5 checks read all the RPMs in the tree)
21:16:57 <mdomsch> but functional
21:17:18 <mdomsch> I plan to deploy on bapp* when that's migrated
21:17:21 <nirik> cool.
21:17:48 <mdomsch> we had discussed here a few weeks ago running on releng* or secondary*, but a) bapp01 has the right mounts already; b) they're r/o
21:18:02 <nirik> yeah, I think thats best.
21:18:03 <mdomsch> that's all
21:18:13 <nirik> cool.
21:18:31 <nirik> #info mdomsch will be uploading a infra version of s3cmd until patches are upstreamed.
21:18:44 <nirik> #info will be deploying mirror sync on bapp01 once it's re-installed.
21:18:55 <nirik> cool.
21:18:56 <nirik> anything else?
21:19:07 <nirik> or shall we call it a meeting?
21:19:28 * skvidal has nothing
21:19:41 <rbergeron> "it's a meeting"
21:19:57 <skvidal> rbergeron: zing!
21:20:04 <nirik> rimshot
21:20:11 <nirik> Thanks for coming everyone!
21:20:14 <rbergeron> :)
21:20:14 <nirik> #endmeeting