21:00:49 <rbergeron> #startmeeting Cloud SIG
21:00:49 <zodbot> Meeting started Thu Feb 10 21:00:49 2011 UTC.  The chair is rbergeron. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:49 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
21:00:50 <nirik> abadger1999: +1
21:00:55 <rbergeron> #meetingname Cloud SIG
21:00:55 <zodbot> The meeting name has been set to 'cloud_sig'
21:01:12 <rbergeron> #topic Roll Call!
21:01:16 <rbergeron> Who's about?
21:01:21 * jforbes is about somewhere
21:01:28 * clalance is here.
21:01:44 <brianlamere> yo
21:01:46 * jsmith is lurking
21:02:05 <rbergeron> clalance :)
21:02:13 <rbergeron> jforbes, clalance, jdarcy. :)
21:02:16 * rackerhacker chirps
21:02:21 <rbergeron> oh, clalance gets 2 smiles.
21:02:30 * rbergeron meant that smiley for brianlamere. and rackerhacker.
21:02:38 <clalance> Haha.
21:02:39 * rbergeron kicks her computer for being odd and laggy
21:02:40 * mmorsi says hi
21:02:41 <clalance> I'm special ;).
21:02:50 * skvidal is here
21:03:19 * rbergeron waves at skvidal
21:03:24 * ke4qqq is here
21:03:30 <rbergeron> who? :)
21:03:38 <skvidal> rbergeron: when it is just one finger it is not a "wave"
21:03:55 * jforbes kicks himself for being odd and laggy...
21:03:56 <rbergeron> skvidal: one finger means I care for you deeply. really.
21:04:02 <jforbes> jgreguske!
21:04:05 <jgreguske> hi!
21:04:11 <jgreguske> forgive my tardiness
21:04:13 <rbergeron> it's my special way of being excellent to you :)
21:04:15 <rbergeron> hi jgreguske :)
21:04:18 <rbergeron> #topic EC2
21:04:23 <rbergeron> jforbes: take it away
21:04:33 * rbergeron will note that she filed a ticket for a test day :)
21:04:35 <jforbes> jgreguske: should I be able to build images in koji again?
21:04:44 <brianlamere> doesn't it mean you think he's number one?
21:04:47 <jforbes> rbergeron: indeed you did, thanks :)
21:05:02 <rbergeron> jforbes: I love filing tickets.
21:05:07 <jgreguske> jforbes: you *should* be able to, however not just yet because of tune2fs breaking
21:05:30 <jforbes> jgreguske: any ETA on the fix?
21:05:51 <jgreguske> jforbes: I have a dirty workaround but dgilmore said he was going to do a little testing himself
21:06:00 <jforbes> okay
21:06:00 <rbergeron> brianlamere: everyone already knows that about skvidal :)
21:06:30 <jgreguske> dgilmore: I know you're busy with the f15 branch... when do you think you'll get to testing appliance-tools as well?
21:07:05 <jgreguske> the dirty workaround is to mount all of /dev in the chroot rather than just /dev/{mapper,shm,pts}
21:07:19 <jgreguske> we could hotfix a koji builder to do that in an image build
21:07:40 <jgreguske> but it's dgilmore's call I think
21:07:59 <dgilmore> jgreguske: hopefully before the weekend
21:08:05 <jforbes> okay, cool
21:08:13 <jforbes> when is alpha release?
21:08:20 <rbergeron> 3-1
21:08:28 <rbergeron> i think
21:08:28 <dgilmore> jf	march 1
21:08:33 <jforbes> Right, so we have a little bit of time
21:08:35 <dgilmore> jforbes: march 1
21:08:36 <rbergeron> yes, i knew it!
21:08:37 <jgreguske> we've got time, but I totally understand jforbes' desire to check a box off :)
21:08:42 <dgilmore> freeze is in 2 weekd
21:09:07 <dgilmore> so we really only have 2 weeks to fix it
21:09:31 <rbergeron> dgilmore: which freeze?
21:09:49 <dgilmore> rbergeron: code/infra etc
21:09:59 <jforbes> Well, I still have to test changes to the publsh script... I am not 100% confident that euca2ools works as documented there
21:10:06 <rbergeron> infra is next week on the 15th
21:10:12 <rbergeron> as is string freeze, etc
21:10:21 <clalance> jforbes: FYI, we've been using it in aeolus, and it does seem to work for us.
21:10:27 <clalance> (though our use-case is kind of constrained)
21:10:34 <dgilmore> maybe ive been looking at the wrong schedule
21:10:39 <clalance> jforbes: There are some slight differences in behavior from ec2-ami-tools.
21:11:24 <rbergeron> dgilmore: everything is on the Schedules wiki page - or here - http://rbergero.fedorapeople.org/schedules/f-15/
21:11:26 <jforbes> clalance: yeah, I have used it from the command line some... Just need to make sure the scripts do what I expect.
21:12:49 * rbergeron wonders if there is anything that she should be actioning people on here or if she should just sit back in her lounge chair and watch
21:13:31 <jforbes> I think we are good on actions, the koji bit has to be fixed for more than just ec2.  I believe it is also used for spins
21:13:45 * rbergeron nods
21:13:56 <rbergeron> Okay. Anything else in ec2-land going on
21:13:57 <rbergeron> ?
21:14:04 <dgilmore> jforbes: its only effecting appliance creation
21:14:15 <jforbes> nopr
21:14:16 <dgilmore> i used koji last night to make the tc1 livecds
21:14:21 <jgreguske> livecd generation goes down a different code path in imgcreate
21:14:26 <jforbes> Hmm, okay
21:14:39 <jgreguske> jforbes: granted f14 is broken for another reason...
21:14:45 <jgreguske> but f15 works as far as I know
21:15:06 <jgreguske> I think dgilmore had some success with it recently
21:15:36 <jforbes> jgreguske: yeah, dont think F14 will be fixed in koji
21:15:57 <jgreguske> jforbes: it just requires a fix in f15 be backported to f14 I think
21:16:06 <jforbes> right
21:16:09 <jgreguske> at the livecd-tools level
21:16:26 <jgreguske> jforbes: is f14 a priority? I figured f15 was the Big Deal
21:17:14 <jforbes> jgreguske: f15 is the big deal
21:17:18 <dgilmore> jforbes: if we really want updated f14 images we need to backport a patch
21:17:39 <jforbes> Well, we can do f14 images another way, though it would be nice to use koji for them
21:18:39 <jgreguske> jforbes: we can fix f14 too, we'll handle it :)
21:18:51 <jforbes> nice :) thanks
21:18:54 <jgreguske> dgilmore: may ping you tomorrow on that, I can help
21:19:13 <brianlamere> ok, I'm going to say it - Fed14 will likely be more stable than 15.  Heck, lots of conversations about that at FudCon - how some releases are more conservative than others
21:19:22 <dgilmore> jforbes: its doable just needs doing
21:19:22 <brianlamere> so a 14 fix would be nice ;)
21:20:39 <jforbes> brianlamere: I would be less concerned about f15 stability on the appliance level... gnome3 is the big shakey piece
21:21:16 <jgreguske> heh
21:21:17 <jdarcy> It would be weird if every release turned out to be exactly as stable as the others.
21:21:23 <rackerhacker> brb
21:21:40 <brianlamere> jforbes: hmm...ok, yeah, true
21:21:42 <clalance> jforbes: Well, the other bit is systemd.
21:22:03 <brianlamere> clalance: and less upstart as a result?  sign me up :P
21:22:18 <clalance> Heh, I think it is a good move, but also a potentially destabilizing one.
21:22:20 <clalance> (just saying)
21:22:25 <jforbes> clalance: sure, but I expect it will be less problematic in this context b release time.
21:22:34 <clalance> Yeah, probably.
21:22:48 <rbergeron> Shall I move on? :)
21:23:13 <jforbes> rbergeron: sure
21:23:38 <rbergeron> sweet.
21:23:41 <rbergeron> #topic CloudFS
21:23:49 <jdarcy> This will be a short section.
21:23:55 <rbergeron> :)
21:24:16 <jdarcy> I requested that CloudFS *not* be included in F15, as I felt I had essentially missed the deadline to come up with something sufficiently useful+usable.
21:24:31 <jdarcy> FESCo approved the request, so it'll be retargeted for F16.
21:24:39 <brianlamere> :(
21:24:48 <rbergeron> jdarcy: are you leaving the code in for people to test/play with?
21:24:52 <jdarcy> Work continues though.  New dev starting next week.  I'm still excited about it.
21:25:16 <rbergeron> hey obino :)
21:25:18 <jdarcy> The project goes on.  It just won't be released in time for F15.
21:25:29 <brianlamere> that's unfortunate; it's a great use of the ephemeral storage which I can't otherwise touch because it's...ephemeral.  heh
21:25:37 <obino> hi rbergeon, I'm late as usual :(
21:25:46 <jdarcy> brianlamere: You can still use what's there, it's just kinda rough.
21:25:49 <rbergeron> obino: no worries
21:26:14 <jdarcy> I think the F16(ish) release will rock.
21:26:27 <rbergeron> :)
21:26:44 <rbergeron> anything else?
21:26:51 <jsmith> jdarcy: I'm a bit confused... are we still going to have packages as a "tech preview"?
21:26:53 <jdarcy> Nope.  Just thanks to everyone.
21:26:53 * rbergeron shall look forward to seeing you at the meetings in the meantime of course :)
21:27:01 <jsmith> jdarcy: Or does it just need more time to cook?
21:27:24 <jdarcy> jsmith: I think it just needs more time to cook.  The feedback I've gotten has been great, I see a clear path to making things *way* better.
21:27:31 <jsmith> Cool.
21:27:35 <jsmith> Thanks for the clarification :-)
21:27:42 <rbergeron> jsmith: I think the definition around "tech preview" really needs more definition for people to feel comfortable doing something like that... at least IMO :)
21:28:24 <rbergeron> #topic Aeolus / Deltacloud
21:28:36 <rbergeron> clalance, mmorsi: Are you bringing me something awesome for F16? :)
21:28:39 <jsmith> rbergeron: Traditionally, "tech preview" means "It's in there, it's not installed by default, but people can play with the packages as they exist."
21:28:48 <jsmith> rbergeron: (We can take futher discussion offline)
21:28:52 <jsmith> s/futher/further/
21:29:02 <clalance> rbergeron: That is the plan.
21:29:11 <clalance> rbergeron: We got closer this week with mmorsi pushing rails 3 into F-15.
21:29:24 <jforbes> jsmith: by that definition, all of virt is tech preview :)
21:29:24 * rbergeron saw. it's in!
21:29:31 <rbergeron> jforbes: :)
21:29:53 <clalance> rbergeron: Let's see, other than that...
21:30:06 <clalance> rbergeron: You saw I released the first version of Oz late last week.
21:30:14 <rbergeron> indeed. thanks for sending that to the list.
21:30:15 <clalance> (since there was interest in that stand-alone)
21:30:26 * gholms walks in
21:30:34 <clalance> Image Factory (which uses Oz underneath) is probably also going to have a release soon.
21:30:57 <clalance> The UI itself *is* stabilizing; the sprint ends next week, and I think we should have packages then.
21:31:06 * rbergeron just wants to make sure that people know that there are always new releases just around the corner - and wiating till f15 is out the door isn't as efficient as just continuing to plow forth on things. :)
21:31:08 <clalance> (still very early alpha stuff, but at least installable)
21:31:16 <clalance> Oh yeah, definitely.
21:31:18 <rbergeron> (I'm pretty sure you are all aware of that, but, you know. :D)
21:31:25 <clalance> We host our own repos for stuff that is not yet in Fedora.
21:31:31 <mmorsi> as always you can install the latest / greatest from our yum repo
21:31:45 <mmorsi> http://repos.fedorapeople.org/repos/aeolus/packages/
21:31:50 * rbergeron nods
21:32:06 <jforbes> clalance: it would be easy to include some of that in the virt-preview repos as you start to stabalize for F16
21:32:06 <mmorsi> yum install deltacloud-configure from there and then   'deltacloud-configure' and you should be good to go
21:32:21 <rbergeron> I really like the updates that you guys post to the list. It's really helpful for us to know what's going on, etc. :)
21:32:23 <clalance> jforbes: Good point.  Though I'm not sure whether it belongs there.
21:32:31 <clalance> (it is semantically a layer on top of virt)
21:32:56 <jforbes> clalance: yeah, just happens to be a simple place to track
21:33:05 <clalance> Yeah, true.
21:33:21 <clalance> I just don't want to confuse the current virt-preview users.
21:33:27 <clalance> But when we have packages, we can discuss further.
21:33:48 <rbergeron> Anything else? :)
21:34:00 * rbergeron moves along
21:34:06 <rbergeron> #topic Eucalyptus
21:34:10 * rbergeron looks at obino and gholms
21:34:19 <obino> lol
21:34:21 * rbergeron figured she'd just ask what's in the works for the future
21:34:24 <rbergeron> :D
21:34:36 * gholms looks up
21:34:43 <gholms> Oh, lol
21:34:46 <rbergeron> You guys have any magical F16 plans? :)
21:34:53 <rbergeron> I mean, I love having you here either way.
21:35:00 <rbergeron> But. :)
21:35:11 * rbergeron is wondering if you guys are thinking about it.
21:35:16 <gholms> I got Eucalyptus building against F14 yesterday.
21:35:48 <obino> it's hard for us to do plan: we are in freezing mode for a release
21:36:00 <gholms> It's still littered with bundled java libs, so once the next release gets out the door that'll be on my agenda to fix.
21:36:06 <obino> my plan is to start populating the wiki page for Eucalyptus
21:36:18 <obino> with gholms help of course :)
21:36:24 <gholms> :)
21:36:34 <rbergeron> Cool.
21:36:36 * obino codeword for gholms will do the job and I'll drink coffee
21:36:44 <rbergeron> lol
21:37:00 <brianlamere> a quick google didn't give me a euca2ools roadmap; either of you have such?  curious how it might be expanding in the future
21:37:00 <ke4qqq> gholms: hosting a yum repo for itsomewhere?
21:37:00 <rbergeron> is that like when I throw someone under the bus? :)
21:37:03 <gholms> I'd rather have you police the forums anyway.
21:37:35 * rbergeron thinks it's amazing how ke4qqq pops in right when i start talking about throwing people under the bus
21:37:39 <rbergeron> :)
21:37:50 <gholms> rbergeron: You're reading into it too much.
21:38:09 <rbergeron> Ah, darnit
21:38:14 <gholms> ke4qqq: We will be.  Up until yesterday all the builds were against F12 because no one knew how to fix linkage failures due to F13's linker change.
21:38:24 <ke4qqq> gholms: cool
21:38:36 <obino> brianlamere: I think the plan is to expand compatibility
21:39:01 <obino> anything in particular you were looking for?
21:39:18 <gholms> Long story short, the person next to me is working on revamping the way we do repos and builds right now, so we can hopefully expect F13 packages some time this month.
21:39:45 <gholms> Yeah, you:  the person staring at my screen right now.
21:39:48 <gholms> Stop being so creepy.
21:40:07 * rbergeron raises an eyebrow at the creepy person
21:40:12 <rbergeron> :)
21:40:15 <brianlamere> well, I was curious about the other toolsets too - such as route53, rds, etc.  Easier in how-to guides to tell people how to use a public tool versus mine which I can't generally give out
21:40:26 * skvidal wonders if gholms is talking to his "friend" who the rest of us can't see
21:40:49 <gholms> skvidal: In this case it's the person sitting next to me who is setting up the new repo layout we're going to use.
21:41:04 <skvidal> gholms: can the rest of us see this person if you are not in the room?
21:41:18 <gholms> Yeeesss...
21:41:20 <gholms> ;)
21:41:21 <rbergeron> lol
21:41:37 <gholms> If not then she's getting paid a lot more than she should be.
21:41:40 <rbergeron> OKAY THEN
21:41:44 <rbergeron> :)
21:41:49 <gholms> :P
21:42:00 <rbergeron> oh, brianlamere had a question - i guess i'll wait on that before i move on
21:42:11 <rbergeron> or maybe a question/statement
21:42:48 <gholms> As I pull deps out of Eucalyptus I'm going to make rpms out of them and submit them for review to Fedora in whatever order makes sense.  Once I finally have enough to build on Fedora I'll start worrying about making EPEL branches for everything.
21:43:04 <brianlamere> well yeah - just whether we can expect expansions to things outside of ec2-api-tools :)
21:43:09 <gholms> Fedora will need ~25 new packages; EPEL ~70.
21:43:17 * gholms shuts up
21:43:34 <ke4qqq> gholms: /me is happy to do review swaps with you - I am in a similiar boat
21:43:38 <brianlamere> but that's a different matter than making them happy as rpms in the current iteration
21:43:50 <obino> brianlamere: you can always submit feature requests :)
21:44:55 <gholms> [Everyone stares at brianlamere]
21:45:09 <rbergeron> [everyone stares at gholms' supposed person sitting next to him]
21:45:17 <brianlamere> ha - ok.  next topic!
21:45:17 <gholms> D:
21:45:25 <rbergeron> #topic BoxGrinder
21:45:29 <obino> gholms: she is not creepy
21:45:31 * rbergeron thought she saw mr. goldmann about
21:45:37 <obino> I can guarantee that she is very lovely
21:45:39 <rbergeron> oh, he's away
21:45:46 <obino> and smells better than gholms :D
21:45:53 <gholms> obino: Only when staring at my screen, reading what I type.
21:46:09 <gholms> Any other BG people here?
21:46:12 <brianlamere> sounds creepy to me
21:46:16 <rbergeron> #action one of you needs to ask her out on a date instead of being creepy :)
21:46:22 <rbergeron> lol
21:46:25 <gholms> rbergeron: She's married
21:46:34 <tatica> o0
21:46:41 <rbergeron> gholms: oh. well
21:46:44 <rbergeron> hmmm
21:46:47 * rbergeron has no comment. MOVING ON
21:46:52 <gholms> Good one.
21:46:56 <rbergeron> no marek today, but he's kicking butt and taking names.
21:47:01 <brianlamere> there was a kickstart discussion that started up via emails involving boxgrinder, dwalsh, etc
21:47:04 <rbergeron> And he did eventually get home from FUDCon. :)
21:47:20 <brianlamere> but I need to poke the conversation I think
21:47:27 <clalance> rbergeron: I had some good discussions with him about integrating BG and Oz.
21:47:31 <gholms> AFAIK you can kickstart a EC2 instance the same way you can kickstart a regular box.
21:47:35 <brianlamere> I really hope it isn't waiting for my input, because that can take AGES sometimes.
21:47:45 <skvidal> gholms: ami-creator seems to be a direction for that, right?
21:47:46 <clalance> (haven't written any code yet, but at least we have some ideas)
21:47:50 <gholms> clalance: Do tell.
21:47:51 <jgreguske> brianlamere: I'd prod it
21:48:08 <clalance> gholms: Essentially, BG would call out to Oz as a backend to do installs.
21:48:16 <gholms> skvidal: What he wants to do is have the instance kickstart itself when it starts.
21:48:26 <clalance> (it requires changes to BG, unfortunately, as there is some overlap between Oz and BG)
21:48:39 <clalance> But if we did that, then BG could install non-Fedora OSs.
21:48:44 <clalance> (and non-Linux OSs, for that matter)
21:48:52 <brianlamere> gholms:  yeah, an instance that can boot up just like PXE/etc would, and knows how to run off and kickstart itself.  lovely idea.
21:48:56 <gholms> That's a gib plus.
21:48:59 <gholms> *big
21:49:19 <clalance> brianlamere: That's kind of what Oz does as well.
21:49:29 <clalance> (though it uses a virtual machine to do so)
21:49:40 <jforbes> Yeah, downside is running bg on an EC2 instance
21:49:48 <gholms> brianlamere: Grab the kernel and initrd from images/pxeboot on the mirror, then hand that kernel the location of your kickstart as a parameter.  Done.
21:50:29 <brianlamere> I think you'd boxgrinder the image that can kickstart itself - I don't think you'd make an image that can boxgrinder an image.  I mean, there's only so far that is useful
21:51:05 <brianlamere> gholms:  sure, but there's specialness about ec2 that might confuse anaconda just a tad.  and thus might need special kickstart args
21:51:26 <skvidal> gholms: so the benefit of being able to kickstart ON the machine is that you can do your normal scripts
21:51:39 <gholms> skvidal: Exactly
21:51:46 <skvidal> but if you can generate/upload an ami using a kickstart as the only input
21:51:47 <brianlamere> of course, I say this as a person that hasn't done VMs locally for years, as I've not needed them.  So I may just have too dusty a memory to be useful.
21:51:49 <skvidal> isn't that the same?
21:52:07 <brianlamere> skvidal:  no
21:52:12 <skvidal> brianlamere: why?
21:52:56 <brianlamere> because a kickstart can source a location for minor tweeks.  it can be told to get info from an amqp message to change what it builds.  it can build the instance with the most updated rpms
21:53:24 <brianlamere> making a static AMI means you have to update it, and make thousands of static AMIs for each and every possible need
21:53:42 <skvidal> perhaps - feels like a big cost for startup - but maybe that's not that big
21:53:47 <brianlamere> better to just have a single AMI that can pop itself out as a fully patched, updated, and exact version of what you need :)
21:54:14 <jdarcy> What's the difference in startup time?
21:54:19 <brianlamere> nope, building it as patched is faster than pushing an AMi and then patching that AMI.  Especially when SELinux gets involved
21:54:22 <gholms> It's very un-cloud-like, but it will still work.
21:54:28 <skvidal> jdarcy: well an image is pretty much instantlyavailable
21:54:38 <skvidal> jdarcy: a kickstart is 'format + download pkgs + install pkgs'
21:55:09 * skvidal wonders if you could build an ami ON a host in ec2
21:55:12 <skvidal> so the upload is faster
21:55:12 <brianlamere> I dunno, I see it as more cloud-like - get a queue message to make exactly what you want, versus some static image harkening back to days of building systems from a blessed gold image on a tape
21:55:15 <skvidal> I suspect so
21:55:39 <jdarcy> skvidal: So even though the actual startup is likely to be quicker than a fully-loaded AMI the total "press button to log in" time would probably be longer, right?
21:55:41 <skvidal> brianlamere: I agree about that part
21:55:59 <brianlamere> user-data, read from a url, read from a queue, whatever - read configs for the kickstart and build that
21:56:00 <skvidal> jdarcy: it varies - I think it would depend on how much you do in %post
21:56:06 <gholms> skvidal: Of course you can.  I ran a Gentoo box on EC2 for a while.  :)
21:56:18 * rbergeron announces that she has to strap on her jetpack in approximately 2 minutes. anyone want to take the reins?
21:56:32 <gholms> rbergeron: For your jetpack?  No way!
21:56:36 <brianlamere> but we haven't talked about s3cmd!
21:56:47 <rbergeron> gholms: of the meeting :)
21:56:48 <skvidal> jdarcy: and depending on where you were installing from - you could end up incurring some bandwidth costs.
21:56:51 <gholms> s3cmd can be next!
21:56:56 <jdarcy> I think the Aeolus folks might have some ideas about image creation vs. subsequent "personalization" too.
21:57:20 * clalance also has to run
21:57:29 * skvidal would generally prefer everything be focused on kickstarts
21:57:34 <skvidal> either making images from kickstarts
21:57:34 * rbergeron spins the wheel of fate
21:57:43 <rbergeron> #chair gholms ke4qqq
21:57:43 <zodbot> Current chairs: gholms ke4qqq rbergeron
21:57:44 <skvidal> or making it so the image can run anaconda and therefore kickstart
21:57:44 <gholms> rbergeron: I can run the meeting...
21:57:52 * rbergeron notes that ke4qqq has something to say when this is done so harass him about it plz
21:58:09 * ke4qqq looks up
21:58:19 <brianlamere> yeah, an image that can run anaconda itself to kickstart a new image would be super-keen.
21:58:32 <rbergeron> ke4qqq: the underside of the bus is shiny, no? :)
21:58:40 <skvidal> brianlamere: pxe for ec2
21:58:44 <ke4qqq> no, actually a bit grimy
21:58:49 <rbergeron> oh
21:58:51 <brianlamere> (er...and that instance becoming the new image - not that instance making a new AMI)
21:58:54 * rbergeron sniffs and leaves, bbiab
21:58:56 * skvidal looks at rackerhacker ^^^^ what about that at rackspace?
21:59:03 * skvidal waves to rbergeron
21:59:09 * jdarcy waves.
21:59:35 <brianlamere> does rackerhacker have a pv-grub like kernel?  (displays ignorance)
21:59:37 <rackerhacker> skvidal: i'm all for it ;)
21:59:40 <brianlamere> err...rackspace
21:59:46 <rackerhacker> brianlamere: we have pv-grub, but it's not usable automatically
21:59:52 <skvidal> rackerhacker: so lemme know when next week that'll be available
21:59:54 <skvidal> :)
22:00:35 <gholms> Shall we move on to s3cmd?
22:01:18 <gholms> #topic s3cmd
22:02:10 <brianlamere> s3cmd!
22:02:15 <gholms> :D
22:02:52 <brianlamere> ok, I'll be honest - my main concern with s3cmd is it is maintained at the same place as s3sync, and I've had lots of problems with s3sync ;)  I tried out s3cmd and it seems to work well enough, BUT...
22:03:34 <brianlamere> s3sync seemed to have problems with larger sets of files....20G +....
22:04:22 <brianlamere> I tried putting up a lot of info with s3cmd and it seemed ok, but I'm not sure at what point it should be blessed.  The repos are massive, but it would cost me real money to do a test similar to that amount of stuff
22:05:01 <brianlamere> sure AWS is cheap, but once you start shoving 900G around, stuff starts adding up :P  and I have no business need for it atm, so I can't "test" that way.
22:05:03 <brianlamere> thoughts?
22:05:15 <gholms> IIRC, jforbes has some sort of magical account that might be useful for testing that sort of thing.
22:05:46 <brianlamere> yeah, I can't nominate myself to use his account though ;)  nor am I sure if it has free S3
22:06:01 <gholms> jforbes: ping
22:06:53 <gholms> Do s3cmd and s3sync use similar code?
22:07:18 <jforbes> gholms: magical account?
22:07:55 <gholms> jforbes: I seem to recall someone or other getting permission to use S3 for testing Fedora-related stuff.
22:08:08 <jforbes> gholms: Umm, not really
22:08:09 * rbergeron is back
22:08:20 <jforbes> gholms: We have 1 account which is for hosting official images only
22:08:33 <gholms> There goes that idea.
22:08:35 <jforbes> gholms: and another account which we actually pay for, used for testing fedora images
22:09:08 <gholms> Would a similar account for putting repo mirrors on S3 be possible?
22:09:27 <jforbes> gholms: not sure what the arrangement is there
22:09:28 <brianlamere> jforbes:  so my concern is that s3sync broke down with large sets of data.  So to really know if s3cmd would work, it should be tested with data sets similar to what it would actually be used for
22:09:40 <gholms> It sounds like brianlamere's small-scale testing was successful, so I think that's the next logical step.
22:10:01 <brianlamere> jforbes:  would it be worth it for me to try to re-ping the AWS folks and see if they'd lend me a temporary account to test with?
22:10:47 <jforbes> brianlamere: more than welcome to, but knowing the amount of overhead required for them to do anything, it might be a massive waste of time
22:10:50 <brianlamere> I don't necessarily like contacting an org and present myself as a representative of another org, without really being blessed to do it for that action - heh
22:11:25 <brianlamere> I'm happy to waste my time, so long as someone can just say it's cool to ask for it on the Fedora Community's behalf
22:11:31 <jforbes> brianlamere: heh, dont know that I would contact them as a representative of fedora myself to do that
22:11:33 <gholms> Does it help at all when the requests come from rh.com addresses?
22:12:06 <ke4qqq> brianlamere: consider yourself blessed if the org you are representing fedora cloud-sig :) default to you have permission :)
22:12:15 <jforbes> gholms: I don't know the extent of the RH relationship with them, but I dont think they are even offering us free space for repos mirrors, so no clue
22:12:24 <mgoldmann|away> baaack!
22:12:26 <gholms> :-\
22:12:31 <rbergeron> hey marek :)
22:12:42 <gholms> rbergeron: Jetpack broke down?
22:12:42 <mgoldmann|away> hey rbergeron
22:12:49 <brianlamere> they were suggesting that fedora repo space would be offered, but man...that's been months.  I'll dust off those conversations ;)
22:12:54 <gholms> Unfortunately the best I can do is test stuff against Eucalyptus, and I'm not sure we have that much storage space.
22:13:09 <rbergeron> gholms: I successfully retrieved younglings from educational stuff
22:13:24 <gholms> (And I'm not sure if I have the cycles to test it right now)
22:13:24 <jforbes> brianlamere: last I heard, there were discussions on that, and it was decided that they would not, but those discussions were with gregdek, so not sure
22:13:35 <rbergeron> brianlamere: it doesn't hurt to try :)
22:13:35 <brianlamere> gholms:  the problem is that it is s3 that sometime burps, and a tool needs to handle that well enough...so if your local stuff handles it, that's not really a test
22:13:41 <brianlamere> s3 disappears sometimes
22:13:53 <gholms> brianlamere: Yep
22:14:14 <gholms> Then again it's easy to make walrus hiccup as well, since you can bounce it easily enough.
22:14:29 * rbergeron makes a note to try that sometime
22:14:48 <gholms> Interpret that however you want.
22:15:13 <brianlamere> walrus hiccups are rather smelly, fyi
22:15:19 * rbergeron notes that as well
22:15:46 * rbergeron looks for more comments on s3cmd ... ?
22:16:04 <gholms> #info Small-scale testing with s3cmd was successful.  We now need a way to do larger-scale testing with a full-size Fedora repo.
22:16:15 <ke4qqq> so wait - brianlamere wants to do testing - he either needs funds or a free account
22:16:37 <ke4qqq> who can get him funds for account or free account?
22:16:53 <rbergeron> i have no idea how to get money out of anyone
22:16:56 <brianlamere> ke4qqq:  well I've also been assuming I'd have a large set of files of my own to upload at some point, it just hasn't happened.  there's still a chance that will happen Any Day Now
22:17:01 <rbergeron> oh, wait: that's not true
22:17:05 <gholms> It sounds like Fedora repos on S3 are something red hat will eventually need to pay for.
22:17:14 <gholms> rbergeron: Challenge them to bowling games?
22:17:17 <brianlamere> gholms:  maybe, I've not lost hope ;)
22:17:20 <rbergeron> gholms: i lost to spevack twice
22:17:22 * ke4qqq knows how to get money
22:17:35 <mgoldmann|away> I'm sorry, but I need to go now:( will read the logs and attend next week's meeting, latesr and have a nice evening!
22:17:43 <rbergeron> mgoldmann|away: have a good night :)
22:17:53 <mgoldmann|away> thanks, bye!
22:17:55 <rbergeron> brianlamere: how much money are we talking about here?
22:18:04 <rbergeron> would we guesstimate
22:18:05 <gholms> *Someone* will have to pay for it.  If RH can't even get free hosting for Fedora images there's little chance we'll get them to do it for repos.
22:18:07 <brianlamere> remember - if fedora already has an account, they can create a bucket and give my account temporary ability to upload to just that bucket, and nothing else.  IAm to the rescue
22:18:29 <brianlamere> sec, lemme relook at the prices and I can tell yu
22:18:31 <brianlamere> you
22:18:31 <rbergeron> jforbes: the account tha tyou have that spevack is hooked up with - is that all worked out?
22:19:01 <rbergeron> or is that still in the air with the moving around of accounts
22:19:16 <jforbes> Yeah, that's all worked out
22:19:16 <gholms> 647G * 0.15 USD per GB per month == 97.05 USD per month.
22:19:22 <jforbes> That one comes out of his budget though
22:19:28 <ke4qqq> thats ok
22:19:31 <ke4qqq> it's under 500 bucks
22:19:33 <gholms> That's for one mirror, so we would need four of those.
22:19:34 <ke4qqq> just do it
22:19:47 <ke4qqq> you don't even need to ask him for permission
22:19:48 <gholms> There are also charges to transfer the stuff in.
22:19:50 <brianlamere> $0.10/G for data transfer in.  $0.14/G for storage.  Now...how many gigs is in a repo?  ;)
22:19:53 <rbergeron> would we need this continuously every month?
22:20:04 <gholms> brianlamere: Right now, 647G.
22:20:05 <rbergeron> or just 1x for testing purposes and then moving it over to engineering stuff
22:20:39 <rbergeron> jforbes: can we just allocate him some space in the way he just listed?
22:20:40 <brianlamere> would probably want to transfer the block up, then mess with it a lot - remove random sections of it, change things, etc - you know, testing the syncing
22:20:53 <brianlamere> so it would probably push a TB by the end of it all
22:21:03 <jforbes> rbergeron: I am just a user of the account, you would have to ask spevack about that
22:21:14 <rbergeron> spevack would have to do the allocating?
22:21:29 <jforbes> or tell someone to do it
22:21:54 <jforbes> The account is really there for Fedora infrastructure stuff as I recall.  We just use it for image testing to simplify billing
22:22:10 <ke4qqq> hmmm perhaps more people need privs to unblock that.
22:22:18 <ke4qqq> perhaps smooge or skvidal
22:22:19 <brianlamere> and I can shoot a simple set of instructions on how to lock down my ability to do *only* that activity
22:22:32 <skvidal> jforbes: that's neat
22:22:36 <skvidal> b/c I've never seen it :)
22:22:56 <ke4qqq> lols
22:23:09 <jforbes> not really a priv thing, more of a budget thing.
22:23:31 <ke4qqq> still - this strikes me as less than 500 bucks, we ought to just make it happen.
22:23:32 <rbergeron> jforbes: and there is a separate engineering account somewhere too?
22:23:55 <rbergeron> or am i insane
22:24:04 <gholms> ke4qqq: When running this in production we would need four copies, though.
22:24:09 <brianlamere> well I can also just help show what would be a sufficient test, if someone else wants to just run it themselves :)  if infrastructure already has ~674G of files sitting around on an instance, they're halfway there so far as the cost is concerned
22:24:13 <jforbes> rbergeron: I am sure there is a RHEL account for engineering, I dont know anything about it though
22:24:13 <skvidal> rbergeron, jforbes: ping spot about this
22:24:25 <skvidal> rbergeron, jforbes: b/c we were told to start collecting accounts
22:24:27 <brianlamere> yeah, lots less than $500
22:24:28 <rbergeron> skvidal: about wha tthe purposes is?
22:24:37 <ke4qqq> gholms: yes, but still less than 500 even with 4 copies
22:24:44 <gholms> Plus a *lot* of transfer charges
22:24:45 <rbergeron> ke4qqq: on a monthly basis
22:24:47 <skvidal> rbergeron: apparently wre're not the only ones at rh who have aws accounts :)
22:25:08 <rbergeron> skvidal: I think we just need to sort out whose accounts are for what.
22:25:20 <skvidal> yes
22:25:21 <rbergeron> If brianlamere wants/needs to do stuff on spevack's ec2 setup, that's fine by me.
22:25:32 <rbergeron> If that's almost meant for ingfrastructure stuff, i want to make sure you guys have access.
22:25:46 <rbergeron> If there is another infrastrucutre account that spot has, we should figure that out as well.
22:25:55 * rbergeron feels herself writing herself a lot of tasks
22:26:06 <rbergeron> #action talk to spevack about adding brianlamere to community ec2 account
22:26:11 <rbergeron> #undo
22:26:11 <zodbot> Removing item from minutes: <MeetBot.items.Action object at 0x2b025f120dd0>
22:26:12 * brianlamere cheers for rbergeron
22:26:16 <gholms> Who will do that?
22:26:18 <rbergeron> #action rbergeron talk to spevack about adding brianlamere to community ec2 account
22:26:24 <gholms> Thanks!
22:27:05 <rbergeron> #action rbergeron to send a mail to spot, skvidal, smooge, jforbes, spevack, and anyone else suggested about "what are all the accounts, and what stuff should be in those accounts" and copy cloud folks
22:28:16 <rbergeron> brianlamere: we'll get you fixed up. we're not going to block on small purchases, though if you can describe them a bit so that spevack understands what the random increase in cost is so he's not wondering if something crazy is going on, that would be appreciated.
22:28:34 <rbergeron> Anything else?
22:28:38 <brianlamere> rbergeron:  will do
22:28:55 * gholms raises hand
22:29:18 <rbergeron> gholms: I meant on this topic of s3cmd.
22:29:22 <gholms> Yeah
22:29:23 <rbergeron> if that's the case
22:29:30 <rbergeron> you don't have to raise your hand :)
22:29:40 <gholms> Just making sure I have time to type
22:29:50 <rbergeron> oic
22:30:12 <gholms> Fair warning:  uploading Fedora is going to cost about 65 bucks.  That doesn't include storage costs.
22:30:42 <gholms> It's $0.10 per GB in, last I checked
22:31:22 <brianlamere> gholms:  yeah, upload it to an ec2 instance, and then it's free to upload to s3 from the same region.  that way you can remove large blocks and only pay for the transfer once
22:31:22 <gholms> Hopefully that gives you an idea of how much to expect this to cost.
22:31:38 <gholms> brianlamere: I like that plan.  :)
22:32:08 <rbergeron> gholms: would you mind condensing your budget related estimates in an email as well? I think it's helpful to have on-list as it makes it easier for me to haras^H^H^H^H^H explain things to the folks with the credit cards.
22:32:09 <brianlamere> nuke a large amount of stuff, resync.  change a lot of files, resync.  then you're just paying for storage and the initial transfer
22:32:49 <gholms> All you really need to know are the magic number "647GB" and the pricing data on http://aws.amazon.com/s3/#pricing
22:32:54 <gholms> rbergeron: ^
22:33:20 <brianlamere> and I /think/ S3 is an averaged storage amount?  so 300G for 10 days would cost the price of 100G for a month?  maybe.  I think.  they don't actually itemize the storage charges for S3 on the bills, it's a mystery ;)
22:33:35 <gholms> brianlamere: Now that I think about it this might be a good use for RRS.
22:33:43 * gholms is out of things to talk about
22:33:45 <brianlamere> gholms: true
22:33:49 <rbergeron> kay.
22:34:02 <rbergeron> #topic Announcements
22:34:06 * rbergeron throws a bus at ke4qqq
22:34:20 <ke4qqq> ohhh so there's a build and open source cloud day at SCaLE
22:34:22 <gholms> D:
22:34:28 * gholms slowly backs away from rbergeron
22:34:45 <ke4qqq> currnetly cloudstack, and openstack are coming as well as opscode with chef
22:34:59 <ke4qqq> dell supposedly might be showing up
22:35:07 * rbergeron provides a link
22:35:09 <rbergeron> http://www.socallinuxexpo.org/scale9x/special-events/build-open-source-cloud-day
22:35:11 <gholms> obino: Are we sending anyone to SCALE?
22:35:12 <ke4qqq> so if you are inthe LA area - join us
22:35:50 <ke4qqq> eof
22:35:54 * gholms needs to find out if eucalyptus is going to go
22:36:00 <brianlamere> hmm...maybe I'll run up to that...
22:36:11 <ke4qqq> gholms: tell them to send you
22:36:29 * rbergeron will be there, and as an added bonus i'll be talking (NOT at cloud stuff) too, if you want to snicker.
22:36:32 <rbergeron> :)
22:36:33 <brianlamere> it is just a bit over an hour away on my bike, after all...and isn't on a day I go to school...
22:36:35 <gholms> I don't have the cycles to go...
22:37:02 <rbergeron> gholms: saturday? :)
22:37:02 <gholms> rbergeron: You don't need to talk if you want to evoke snickers.  Just make a funny face or something.
22:37:20 <gholms> Oh, right.  Saturday.
22:37:31 * gholms should find transportation
22:37:33 <rbergeron> gholms: buildacloud thing is friday - but scale is all weekend
22:37:45 <rbergeron> you can come up and like, hang out, n stuff.
22:37:49 <rbergeron> same to brianlamere
22:37:50 <rbergeron> and obino
22:38:21 <rbergeron> gholms: just keep it in mind :)
22:38:31 <rbergeron> #topic Any other business??
22:39:25 <rbergeron> [waits for gholms to make a snarky remark]
22:39:32 <rbergeron> :D
22:39:36 * gholms snarks at rbergeron's snarks
22:39:40 <rbergeron> rawr!
22:39:43 * rbergeron counts backwards from 87
22:39:51 <rbergeron> 35
22:39:51 <rbergeron> 22
22:39:52 <rbergeron> 17
22:39:53 <gholms> BY ONE!
22:40:02 <rbergeron> 1
22:40:04 <rbergeron> :)
22:40:06 <rbergeron> #endmeeting