19:01:32 <rbergeron> #startmeeting Cloud SIG 19:01:32 <zodbot> Meeting started Fri Mar 2 19:01:32 2012 UTC. The chair is rbergeron. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:32 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:01:35 <rbergeron> #meetingname Cloud SIG 19:01:35 <zodbot> The meeting name has been set to 'cloud_sig' 19:01:38 <jforbes> damn, beat me to it :) 19:01:41 * dgilmore is hungry 19:01:43 <gholms> Hehe 19:01:46 <rbergeron> #chair gholms jforbes 19:01:46 <zodbot> Current chairs: gholms jforbes rbergeron 19:01:58 <rbergeron> #topic Who's here? 19:02:04 * dgilmore 19:02:24 * tdawson_ raises his hand. "Here" 19:02:32 <mull> here 19:02:34 * mdomsch 19:02:43 * gholms hums a few bars 19:02:44 * kkeithley is here 19:02:50 * jforbes is hereish 19:02:58 <rbergeron> ;) 19:03:08 * nteon 19:03:13 * obino lurking in the background 19:03:40 <nteon> (I'm mostly lurking, but I'd be interted in helping with ec2 image related things) 19:03:48 <rbergeron> ;) 19:03:50 <rbergeron> obino! hi :) 19:03:55 * rkukura is here 19:04:00 <obino> hello there :) 19:04:12 <rbergeron> okay, well, full house it would seem 19:04:51 <rbergeron> i suppose we'll get started :) 19:05:04 <rbergeron> #topic Check Your Packages, yo 19:05:38 <rbergeron> #info just a sweet and friendly reminder that features should be at 100% by 3/20. 19:05:46 <rbergeron> Is anyone freaking out about that happening? 19:05:50 <rbergeron> hey russellb 19:05:54 * russellb waves 19:06:10 <gholms> Freaking out? Not yet. 19:06:11 <ayoung> \O/ 19:06:13 * nilsson is late to the meeting 19:06:14 <rbergeron> LOL 19:06:28 <rbergeron> hey nilsson! welcome :) 19:06:34 <nilsson> hello 19:06:42 <rbergeron> gholms: okay. well, let me know, so i can get my camera... err, well, you know. :) 19:07:11 <gholms> #help Reviewer needed for Apache Rampart/C: https://bugzilla.redhat.com/show_bug.cgi?id=796462 19:07:14 <rbergeron> okay, that's all the flogging there i'll do today, unless anyone has any specific packaging drama they want to talk about 19:07:17 <gholms> plzkthx 19:07:45 <rbergeron> gholms: word 19:08:18 * rbergeron notes that if anyone is trying to get sponsored that might be a good package to review in conjunction with your kind sponsor 19:08:34 <rbergeron> alrighty, moving onwards 19:08:57 <rbergeron> #topic Different delivery formats 19:09:00 <nilsson> sponsored for what? 19:09:00 <rbergeron> So: 19:09:21 <rbergeron> https://lists.fedoraproject.org/pipermail/cloud/2012-February/001229.html 19:09:47 <rbergeron> nilsson: if you want to become a packager, you need a packaging sponsor, who will normally make you review other folks' packages as part of the "becoming sponsored" process. :) 19:10:01 <nilsson> ok, thanks 19:10:33 <rbergeron> soo - dgilmore sent out the note above a few weeks ago - I'm not sure we got anywhere with it. 19:11:07 <rbergeron> So I thought perhaps we could gather some thoughts and maybe pick some direction and apply help towards dennis' way. 19:11:14 <rbergeron> Don't you all go speaking at once now. 19:11:17 * rbergeron looks at russellb 19:11:23 <dgilmore> rbergeron: im not sure what the exact value is 19:11:30 <jforbes> I think the xz compressed raw is good personally 19:11:52 <nilsson> is there any demand for vmdk images? 19:11:56 <ayoung> agreed. smaller images == faster downloads 19:11:59 * mull thinks we should at least pay attention to what Ubuntu does: http://cloud-images.ubuntu.com/releases/11.10/release/ 19:12:00 <dgilmore> nilsson: no 19:12:07 <ayoung> and from raw you can convert to most other formats 19:12:10 <mull> we don't have to follow them, but maybe some good ideas there 19:12:18 <ayoung> dgilmore, yes there is, but we can let VMWare do that.... 19:12:29 <dgilmore> the biggest issue i see is how do you get onto the system 19:12:55 <dgilmore> ayoung: well ive not had a single request for vmdk images 19:13:17 <dgilmore> the only thing ive been actually requested was qcow2 19:13:38 <dgilmore> the images would work just fine in say eucalyptus 19:14:01 <dgilmore> but how do you log into the image if you bring it up on your desktop system? 19:14:04 <mull> right, we (euca) do conversion for vmware ourselves 19:14:28 <dgilmore> since the images are setup to use cloud-init 19:15:03 <dgilmore> while ive had requests for them. im not sure how useful they actually are 19:15:11 <dgilmore> or if we need to say make 2 images 19:15:26 <dgilmore> one with cloud-init and one that runs firstboot 19:16:03 <ayoung> dgilmore pointer to issues with cloud-init? 19:16:10 <nilsson> would it be feasible to ship locked-down images, and provide documentation for how a user could use libguestfs to inject their own ssh keys into the image? 19:16:12 <rbergeron> mull: what on that page do you see that is immediately useful and/or with what specific audiences? 19:16:19 <dgilmore> personally i think its easy enough to just use a kickstart and do a install 19:16:43 <dgilmore> ayoung: well the issue is cloud-init goes off and fetches your ssh key so that you can ssh in as the ec2-user 19:17:24 <ayoung> dgilmore, I'm googling as we speak. where does it get them from? Perhaps we can make it possible to work for the desktop case. 19:17:29 <mull> rbergeron, well, first off, the idea of delivering info about how the image was produced is good 19:17:33 <dgilmore> so putting the image into your private euca cloud they should work just fine 19:17:45 <mull> also, they provide an ovf... not sure who uses it, but maybe there's a good reason 19:17:50 <dgilmore> ayoung: an amazon/euca webservice 19:18:57 <dgilmore> ayoung: there is a specific url that it talks to 19:19:11 <dgilmore> we could provide something to mimic it 19:19:27 <dgilmore> but its an extra something someone would need to run 19:20:12 <mull> dgilmore, for the desktop use case, does root console login with no password work? or is that explicitly disabled ? 19:20:37 <dgilmore> mull: i believe thats explictly disabled 19:20:57 <dgilmore> it would be a bad idea to have enabled 19:21:18 <dgilmore> if you happened to break out of the hypervisor you could access any other guest on the host 19:21:20 <mull> well, in the ec2 case it doesn't matter because there's no console on which to log in 19:21:33 <mull> I understand your point 19:21:52 <mull> but I've seen images which allow root login if you can get to a console, for debugging purposes 19:22:00 <ayoung> dgilmore, is it possible to put something into the .xml config file for the virtual machine that would indicate "run first-boot instead of cloud-init" ? 19:22:35 <jforbes> If you happen to break out of the hypervisor, you have serious issues 19:23:05 <gholms> There might be a way to make cloud-init suppress what it usually runs and do something else. 19:23:08 <dgilmore> ayoung: i dont know of anything 19:23:29 <dgilmore> jforbes: i dont disagree 19:23:37 <dgilmore> i may be too paranid 19:23:38 * mull wonders if you could just have cloud init take a metadata service url as a boot parameter... and that url could be file:/// to do something baked into the image 19:24:09 <mull> I may be talking crazy 19:24:12 <obino> mull: I think it can, doens't it? 19:24:19 <ayoung> I think just cloud-init to start is OK 19:24:35 <mull> obino, maybe it does... I have not used it much 19:24:37 * obino looking for gholms to say something about it 19:25:07 <ayoung> but we can post an article saying : to set up a private cloud-init for your desktop, run apache with the following .... 19:25:11 <gholms> That rings a bell. I can't recall if that is true, but... 19:25:32 <dgilmore> ayoung: we can. 19:25:56 <dgilmore> i guess it largely depends on how we expect the images to be consumed 19:26:08 <ayoung> what happens if you boot the VM and it can't reach the public internet, or the cloud-init request fails? Is the VM still usable? 19:26:39 <ayoung> like, reboot init 1 and edit /etc/firstboot 19:27:19 * rbergeron wonders if we are aiming for decision-making in this meeting (not trying to hurry anyone up, just trying to ensure we get something constructive or actionable here) 19:27:56 <mull> ayoung, reaching the public internet is not required... the metadata service is typically on a zeroconf IP 19:28:00 <dgilmore> ayoung: curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/aws-key 19:28:06 <ayoung> rbergeron, question is do we need one image or two. I'm leaning toward one, but don't have the necessary info to propose. Personally, I'd rather just have first-boot, but I see the benefit of cloud-init 19:28:16 <dgilmore> thats what we used to run to get the key before cloud-init 19:28:42 <dgilmore> ayoung: first-boot cant be used in ec2 19:28:55 <dgilmore> ayoung: you dont have a interactive console 19:29:00 <mull> I think we find a way to only need one. I'm _guessing_ that ubuntu has solved this problem, since they appear to provide an ovf for use with virtualbox 19:29:06 <dgilmore> ayoung: you have to use cloud-init in ec2 19:29:08 <ayoung> dgilmore, so long as there is a clear way to cloud-init in the desktop case, I say we go with one image, and document it on the download page 19:30:03 <dgilmore> ayoung: listen on 169.254.169.254 and provide a ssh key at /latest/meta-data/public-keys/0/openssh-key 19:30:34 <ayoung> dgilmore, by listen I assume you mean HTTP? 19:30:43 <dgilmore> ayoung: yes http 19:31:12 <dgilmore> ayoung: as long as you do that i think cloud-init should work and you could then log in as ec2-user with ssh 19:31:14 <nteon> dgilmore: could cloud-init start first-boot if it can't connect to 169.254.169.254? 19:31:27 <dgilmore> nteon: maybe gholms ^ 19:32:04 <dgilmore> nteon: we dont actually install firstboot 19:32:09 <ayoung> nteon, or perhaps we modify first-boot to run cloud-init, and fall back to its default processing upon failure? 19:32:12 <nteon> hmm 19:32:13 <gholms> We can't rely on that because the metadata service will not always be up beforehand. 19:32:14 <mull> http://cloud-images.ubuntu.com/releases/11.10/release/ubuntu-11.10-server-cloudimg-i386.ovf <--- please read this 19:32:22 * ke4qqq shows up late 19:32:53 <rbergeron> howdy :) 19:33:07 <mull> it might give people ideas and avoid reinventing some wheels, just sayin' 19:34:20 <ayoung> mull, so you suggest we wrap the VM in an ovf and tell people that they can edit the xml to vary how to deploy? 19:35:07 <dgilmore> mull: we dont install the firstboot bits 19:35:15 <dgilmore> i guess we could and disable it 19:35:26 <dgilmore> the people could do something to tweak it 19:35:34 <rbergeron> tweakers! 19:35:44 <mull> ayoung, possibly. right now I'm reading this: https://help.ubuntu.com/community/UEC/Images 19:35:54 <mull> they have a section on "Ubuntu Cloud Guest images on Local Hypervisor" 19:36:02 <mull> which seems to match what we're talking about 19:36:07 <dgilmore> so seems we are sidetracked 19:36:20 <dgilmore> we could find ways to make things useable i guess 19:36:43 <dgilmore> the question then is do we just compress the raw image and say here you go 19:36:55 <nteon> gholms: what do you mean the metadata service won't be up beforehand? 19:36:59 <dgilmore> or do we also convert it to a compressed qcow2 image 19:37:15 <nteon> gholms: is there any other way to reliably detect that we're running on an ec2-like cloud? 19:37:33 <ayoung> should we create a # action for figuring out how to chose between cloud-init and first boot and leave it at that> 19:37:35 <ayoung> ? 19:37:42 <gholms> nteon: 169.254.169.254 is not always immediately accessible when cloud-init runs. 19:37:55 <mull> ayoung, seems reasonable 19:38:15 <mull> although the decision may be to do something simpler than firstboot 19:38:31 <gholms> That sounds reasonable. I'm pretty sure cloud-init has some way of setting stuff up via boot args or something. 19:38:38 <ayoung> #action figure out how to chose between cloud-init, firstboot or different init process for vm images 19:39:00 <rbergeron> #chair ayoung 19:39:00 <zodbot> Current chairs: ayoung gholms jforbes rbergeron 19:39:04 <rbergeron> ayoung: press up and do it again 19:39:05 <ayoung> #action figure out how to chose between cloud-init, firstboot or different init process for vm images 19:39:08 <rbergeron> you rock 19:39:21 <gholms> rbergeron: You don't need to be a chair to use #action. :) 19:39:22 <ayoung> not according to zodbot I don't 19:39:42 <nteon> hrmph 19:40:31 <rbergeron> okay, so. have we ... baked that cookie well enough for now? 19:40:41 <ayoung> #action ayoung, dgilmore: figure out how to chose between cloud-init, firstboot or different init process for vm images 19:40:45 <ayoung> meh 19:41:09 <rbergeron> ayoung: zodbot doesn't give a response to actions, if that's what you're looking for 19:41:26 <rbergeron> oh, you were adding names ;) 19:41:28 <gholms> I should be around to answer stuff about cloud-init. If all else fails, ask smoser in #ubuntu-cloud. 19:41:28 <dgilmore> the question remains 19:41:46 <dgilmore> do we ship just compressed raw, or also qcow2 19:41:48 <rbergeron> #chair dgilmore 19:41:48 <zodbot> Current chairs: ayoung dgilmore gholms jforbes rbergeron 19:42:25 <ayoung> dgilmore, was there a significant size difference? 19:42:30 * ke4qqq would argue for raw, qcow2 and some other formats as well 19:42:51 <ayoung> I'd say one (the smallest) format, with explanations how to convert 19:43:08 <dgilmore> ayoung: compressed raw is about 114mb 19:43:20 <dgilmore> the compressed qcow2 is about 220mb 19:43:46 <dgilmore> i would put the on the mirrors http://dl.fedoraproject.org/pub/fedora/linux/releases/test/17-Alpha/Cloud/ or http://dl.fedoraproject.org/pub/fedora/linux/releases/test/17-Alpha/Virt/ 19:43:52 <mull> quite the paradox. :) 19:43:54 <dgilmore> so it would all get mirrored out 19:43:57 <ayoung> converting raw to qcow is just a matter of running qcom-img, correct? 19:44:03 <dgilmore> ayoung: yep 19:44:10 <gholms> Everything can use raw anyway, right? 19:44:12 <ayoung> lets go raw. 19:44:27 <mull> +1 19:44:34 <russellb> is size a problem? if not, it's nice to just download what you need and go ... 19:45:31 <dgilmore> russellb: well it would be a compressed 10gb raw image 19:45:46 <dgilmore> russellb: xz compressed it came in at 114mb 19:45:53 <russellb> i mean, space on the dl site 19:46:02 <dgilmore> russellb: mirrors like us to use less 19:46:06 <russellb> since i saw mention of just having one format 19:46:18 <mdomsch> 114mb is no problem 19:46:25 <nteon> dgilmore: uncompressed, is the image sparse, or does it take a full 10 GiB on disk? 19:46:29 <mdomsch> GBs would be a problem 19:46:39 <dgilmore> nteon: its not sparse 19:47:03 <mull> hmm, sparse would be nice 19:47:14 <ayoung> mull, +1 19:47:16 <nteon> dgilmore: I know tar can do sparse, any reason we don't? 19:47:28 <dgilmore> mdomsch: right. if we added 4 formats for 2 arches we would be looking at probably a couple of gb 19:47:36 <mull> ayoung, that's really a +1 to nteon 19:47:36 <mull> :) 19:47:55 <dgilmore> nteon: we dont get sparse images out of koji 19:48:09 <nteon> dgilmore: aah! 19:48:12 <nteon> gotcha 19:48:27 <dgilmore> we would need to do something to make them sparse 19:48:38 <dgilmore> down the road we should look at it 19:48:48 <dgilmore> but today its what we have 19:49:02 <nteon> sure, makes sense 19:49:40 <mdomsch> +1 for pushing sooner w/o sparse, add sparse when we can 19:49:40 <dgilmore> ok so to start we will just do a compressed raw image 19:49:56 <mdomsch> 2GB is fine 19:50:06 <dgilmore> if there is demand for more formats we can look at it 19:50:39 <ke4qqq> how would you measure demand? 19:50:55 <dgilmore> #action dgilmore will make compressed raw images and put under /Cloud or /Virt next to /Live and /Fedora for Beta 19:51:00 <ayoung> ke4qqq, if we all come back in a month and say "give us qcow" 19:51:05 <dgilmore> ke4qqq: people coming and asking 19:51:19 <mdomsch> we can always put images under /pub/alt also 19:51:25 <ke4qqq> dgilmore: consider me asking for vmdk, vhd and qcow2 :) 19:51:25 <dgilmore> mdomsch: sure 19:51:26 <mdomsch> fewer mirrors, but fewer requests 19:51:43 <mdomsch> mirrors should be used for content we know will get used regularly 19:51:50 <dgilmore> mdomsch: we only put spins there for final 19:52:09 <dgilmore> mdomsch: so at least for Beta ill go with twhat i said above 19:52:10 <ke4qqq> mdomsch: makes sense - elevate to mirrors if volume increases 19:52:32 <dgilmore> for Final we could put it next to /Spins and /Mega on alt 19:52:36 * mdomsch is protective of the mirrors 19:53:04 <dgilmore> mdomsch: im very considerate of them also 19:53:12 <mdomsch> dgilmore: indeed 19:54:21 <ayoung> Anything else? 19:54:25 <dgilmore> nope 19:54:30 <dgilmore> can i go find food now? 19:54:35 <rbergeron> yes :) 19:54:38 <rbergeron> thanks, dennis :) 19:54:52 <gholms> Heh 19:54:57 <rbergeron> #topic Other Things 19:55:06 <gholms> Other things, eh? 19:55:13 <mdomsch> s3cmd sync - I've got a bunch of patches 19:55:31 <gholms> Any idea what upstream thinks of them? 19:55:33 <rbergeron> #info ke4qqq is on the record asking for vmdk, vhd, and qcow2, with a smiley face at the end 19:55:33 <mdomsch> last one (mtime compare) I don't like - way too slow and expensive 19:55:37 <rbergeron> #chair mdomsch 19:55:37 <zodbot> Current chairs: ayoung dgilmore gholms jforbes mdomsch rbergeron 19:55:39 <mdomsch> upstream has been a little quiet 19:55:51 <mdomsch> he thought he had written the mtime comparison code already 19:56:13 <mdomsch> anyhow, what we have is functional, albeit suboptimal 19:56:42 <mdomsch> I need to package up my changes into a version deployed into FI only for now 19:57:01 <mdomsch> dgilmore, is there a way to get notified when you do a push? 19:57:10 <mdomsch> e.g. that whole magic message bus thing we talked about years ago? 19:57:25 <rbergeron> MAGIC BUS 19:57:26 <mdomsch> so I don't have to put it on a cronjob, but instead catch a notification? 19:57:39 <dgilmore> mdomsch: not today 19:57:44 <mdomsch> ok 19:57:50 <dgilmore> mdomsch: i want to work on a composedb for fedora 19:57:50 <rbergeron> mdomsch: is this related to the magic bus spot talked about at the engineering meeting perhaps :) 19:58:08 <mdomsch> rbergeron, I thought that was for fucdon travel! 19:58:10 * rbergeron apologizees to spot for using his name in vain, er, in a meeting 19:58:19 <dgilmore> mdomsch: the composedb could tell you when the last pushes were done 19:58:35 <mdomsch> hmmm 19:58:40 <mdomsch> easy to query, from, say, bapp1? 19:58:47 <dgilmore> but you would still need to ask it 19:58:53 <dgilmore> mdomsch: it would be 19:59:00 <mdomsch> hmmmmm 19:59:04 <dgilmore> mdomsch: but the composedb is in very early planning stages 19:59:18 <mdomsch> ah 19:59:30 <mdomsch> well, plan on throwing a message somewhere (db trigger?) 19:59:40 <mdomsch> until then, I'll just cronjob a few times a day 20:00:01 <mdomsch> and wait for upstream to take or reject my patches 20:00:09 <nirik> yes, a message bus could solve this issue. 20:00:15 <dgilmore> mdomsch: :) the first parts of it i hope to have in place for f18 20:00:27 <nirik> "I am doing a compose" "I have finished a compose" for anything that listens to it. 20:00:50 <mdomsch> nirik, I need "I have written to the netapp in the /foo directory" 20:00:59 <nirik> sure. 20:01:35 <rbergeron> i hope it's called the magic bus project. and when people say, "Who is workign on it," we nod and say, "Yes. Exactly." 20:01:44 <dgilmore> nirik: hopefully ill cater for consumers 20:02:05 <nirik> yeah, I think the early thought is 0mq... 20:02:09 <dgilmore> rbergeron: we all point at you 20:02:35 * rbergeron will assume nobody got her musical joke and pouts 20:02:39 <rbergeron> nirik: yeah 20:03:04 <nirik> that build compose script sure plays a mean pinball! 20:03:06 * rbergeron notes that magic buses is one of russellb's favorite topics too, lol 20:03:10 <spot> why does it sound like CSI in here? 20:03:10 <rbergeron> nirik: OH GOD 20:03:17 <russellb> \o/ 20:03:23 * gholms gets dragged off to another meeting 20:03:26 <rbergeron> i thought only spot played a mean pinball 20:03:36 <rbergeron> gholms: gregdek is cruel! 20:03:45 <spot> rbergeron: i've got nothing on that deaf, dumb, and blind compose script. 20:03:56 <gholms> This one is $bossman's, not gregdek's. :/ 20:04:08 <rbergeron> okay. we are descending into bad jokes and i'm not sure if mdomsch is done :) 20:04:25 <rbergeron> gholms: oh 20:04:33 * rbergeron doesn't want to interrupt good train of thought 20:06:06 <rbergeron> .... okay, i think i just killed us somehow 20:06:16 <rbergeron> does anyone else have anything? 20:06:59 <nilsson> rbergeron, yeah, I'm looking for stuff to do 20:07:10 <rbergeron> #topic introducing nilsson 20:07:20 <rbergeron> nilsson: welcome. i think a number of folks saw your intro mail :) 20:07:56 <rbergeron> of course you come to an interesting group looking for things to do - because I am pretty sure there are many things to do. and there are probably people heere who could throw them at you. 20:08:07 * rbergeron also notes we started a google summer of code page for fedora, hoping to get that rolling 20:08:19 <rbergeron> ...so if anyone has any ideas, that might be a good place to look (or put ideas) as well 20:08:22 * rbergeron puls out said link 20:08:22 <nilsson> I see 20:08:40 <rbergeron> http://fedoraproject.org/wiki/Summer_coding_ideas_for_2012 20:09:09 <rbergeron> nilsson: that might be an interesting place for you to look. I think everyone just vanished for eating, but there are several folks doing packaging for projects that could use help, etc. 20:09:33 <nilsson> I noticed there is a Draft cloud doc, who is the point of contact for that? 20:09:40 <nilsson> rbergeron, any packages in particular? 20:09:58 <rbergeron> ah. the folks in #fedora-docs, mostly. i would talk to sparks. 20:10:04 * rbergeron pokes at sparks 20:10:16 <nilsson> ok, thanks 20:10:24 <rbergeron> nilsson: i know that the eucalyptus and cloudstack folks are hard at work, as are the opennebula folks. 20:10:51 <rbergeron> but it may depend on your experience, your interests, etc. :) 20:11:01 <nilsson> true 20:11:16 <nilsson> ok, that's really all the questions I had 20:11:40 <rbergeron> okay. I'll drop you a mail on list with a few suggestions :) 20:11:46 <rbergeron> thanks for coming! 20:11:48 <rbergeron> :) 20:11:53 <rbergeron> #topic Closing down in 10 seconds 20:12:04 <rbergeron> Bye, folks. :) see y'all next week. 20:12:10 <rbergeron> #endmeeting