20:02:10 <poelcat> #startmeeting
20:02:11 <zodbot> Meeting started Thu Feb  4 20:02:10 2010 UTC.  The chair is poelcat. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:02:12 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:02:19 <gregdek> I defer my clock to the more accurate collective clock.  :)
20:02:25 <poelcat> #chair gregdek
20:02:25 <zodbot> Current chairs: gregdek poelcat
20:02:31 <poelcat> who is in the house?
20:02:36 * gregdek is here
20:02:44 * jforbes is here
20:02:48 * gholms takes a seat in the bleachers
20:02:49 * jboggs here
20:02:49 * Oxf13 
20:02:56 <gholms> I've asked smoser from Canonical to stop in and help answer any EC2-related questions we might have.
20:02:57 * jgreguske lurks
20:03:02 * huff 
20:03:08 * ewan is mostly lurking/watching
20:03:43 <gregdek> A good crowd, hm?
20:03:45 <gregdek> Shall we begin?
20:03:47 * mikeburns listening
20:03:51 <poelcat> welcome everyone
20:04:02 <poelcat> #topic creating current fedora EC2 images
20:04:31 * poelcat thought the bot was supposed to do something with that but oh well
20:04:45 <jforbes> Some less than ideal news on that front
20:04:47 <poelcat> who can give a run down on where we are at today?
20:04:49 <gholms> (Maybe that doesn't work here...)
20:04:49 <gregdek> LOL.  Shall we start with the state of the current Fedora kernel?  jforbes?
20:04:51 * lutter here
20:05:10 <jforbes> We are waiting on 2.6.32 to be released as an update for F12, it fixes a couple of issues
20:05:32 <jforbes> Unfortunately the 2.6.32.7 in updates-testing is not going to make it, and we are waiting on 2.6.32.8
20:05:34 <Oxf13> unfortunately the first offering broke a number of things too
20:05:46 <lutter> what's the story for rawhide ?
20:05:57 * mmcgrath is here but in another meeting as well
20:06:18 <jforbes> lutter: the rawhide kernel is daily snapshots from the linus tree still, I expect we can really look at it once 2.6.33 releases
20:06:22 * mattdm is here but fixing a security problem in another window. awesome.
20:06:46 <poelcat> jforbes: how far out is that?
20:07:36 <lutter> jforbes: it would be great if we can have F13 images in EC2 as soon after F13 release as possible
20:07:44 <jforbes> poelcat: which? 2.6.32.8 for F12 will probably hit updates-testing in the next day or so.  2.6.33 proper, sometime this month
20:07:54 <gregdek> F13 is likely to be based on 2.6.33, then?
20:07:57 <jforbes> lutter: I would liek to see F13 images in EC2 concurrent to real release
20:08:05 <jforbes> gregdek: Yes
20:08:36 <poelcat> jforbes: ulimately i think we're looking for a clearer idea of when F12 might be EC2 ready?
20:08:40 <lutter> jforbes: would be awesome
20:08:53 <gregdek> How much extra work is it to get the 2.6.32 stuff working in F12?  Are we talking small incremental work, or big pain in the ass work?
20:08:57 <jforbes> Remember, we have to make kernels that we put there public for people to use, so I dont think we want to upload every rawhide kernel
20:08:58 <poelcat> F13 is topic #2
20:09:45 <gholms> We also need to do the paperwork to get an AWS account with ACLs that let us upload Fedora kernels.
20:09:51 <gregdek> Is it that we essentially believe that the Xen issues are fixed in 2.6.32 generally, and we're just waiting for a stable enough version to test F12 stuff around?
20:10:02 <poelcat> jforbes: is there a list of steps other people could be working on to help get the F12 piece ready?
20:10:03 <gregdek> gholms: We've got that covered, actually.  I'm working that in parallel with folks at Amaozn.
20:10:04 <jforbes> gregdek: Getting F12 kernels working is not too difficult with 2.6.32, but we need 2.6.32 to actually be an update.  More a technicality, as the issues they are seeing with the current kernel are all about video and kms, not ec2
20:10:11 <gholms> Oh, nice.
20:10:24 <jforbes> gholms: that's done, and we are ready to upload as soon as we have a good kernel to upload
20:10:46 <poelcat> jforbes: when do we think we'll have a good kernel?
20:10:47 <jforbes> poelcat: We still need to define the packageset for F12 (and F13) on what will be the official Fedora
20:11:13 <poelcat> who is working on defining the packageset?
20:11:15 <gregdek> Sounds to me like kernel stuff is just hurry up and wait, meanwhile we should be focusing on AMIs.
20:11:19 <Oxf13> I'm still fuzzy on the tools involved
20:11:22 <gregdek> poelcat, I don't think anyone right now.
20:11:28 <Oxf13> package set/
20:11:29 <Oxf13> eh?
20:11:35 <jforbes> poelcat: Hopefully by next week. Depends on what is the status for the video issues on desktops
20:11:36 <gregdek> Well...
20:11:36 <Oxf13> that seems like an easy one right?
20:11:39 <gholms> I'd start with a minimal image; maybe core and part of base.  I've built some F12 images privately in the past.
20:11:39 <gregdek> ...yes.
20:11:43 <Oxf13> enough to get booted, logged in, and add more packages
20:11:49 <gregdek> Oxf13, yes, I think that's easy.
20:12:02 <gregdek> Oxf13, we just need to identify tools and process and how it actually gets done.
20:12:10 <Oxf13> I'd imagine a kickstart file
20:12:12 <lutter> does anybody have a ks that they could post to the mailing list ?
20:12:13 <jforbes> gregdek: did anyone look at the rpath ami tools for setting up the ssh keys, etc?
20:12:17 <Oxf13> since that's how every other image we create is managed
20:12:25 <gregdek> jforbes, no.
20:12:29 <gregdek> Here's my take:
20:12:35 <huff> lutter: aos ks: has ssh, yum, dhcp, selinux, etc. these were all requirements back in the day
20:12:39 <gregdek> A good AMI is more than just a spin.
20:12:43 <huff> when we started
20:12:52 <lutter> yeah, I'd imagine that's a good start
20:12:52 <gregdek> A spin process is a necessary precondition, but not sufficient.
20:13:01 <mattdm> +1 to that, greg.
20:13:18 <gregdek> So we should probably:
20:13:27 <gregdek> 1. Identify a good spin that we think would be a nice starting point;
20:13:51 <gregdek> (And maybe that's creating some kind of new default "web server" spin or something, some default that will be most beneficial to the most EC2 users);
20:14:13 <gregdek> 2. Figure out what goes on top of that spin to make it a good out-of-the-box AMI.
20:14:14 <Oxf13> I can see the debate now
20:14:34 <gregdek> Oxf13, I'm comfortable making massively wrong choices just to get us moving.  :)
20:14:50 <jforbes> Sure, this can be a moving target.  We just need to define a starting point
20:14:52 <gregdek> "Dear so-and-so, yes, you're right, these packages are wrong.  Don't care.  Moving on."
20:15:06 <Oxf13> yep
20:15:26 <jforbes> gregdek: but we still need something to talk to amazon and set things up... We can't ship the binary ami tools
20:15:35 <gregdek> Is this the time to stand up some dumbed down "web server" spin, or not?
20:15:38 <gholms> That's where euca2ools come in.
20:15:39 <gregdek> jforbes: Good point.
20:15:49 <mattdm> is euca2ools sufficient?
20:16:09 <jforbes> gholms: Yeah, we had also looked at the rpath tools, they are a plugin system as well
20:16:09 <gregdek> You are still required to install the Amazon ec2tools locally, aren't you?
20:16:23 <Oxf13> gregdek: well, we don't have any such spin in existence right now, so we'd be creating it more than "propping it up"
20:16:31 <gholms> gregdek: Required to?
20:16:36 <gregdek> Oxf13, you tell me.  Good idea?  Bad idea?
20:16:41 <huff> gregdek: only if your goign to rebundle
20:16:43 <huff> afask
20:16:43 <jforbes> gregdek: with the rpath tools, there is no requirement for the amazon ec2tools
20:17:06 <gholms> What part of the ec2 tools needs to go on the VMs?
20:17:09 <Oxf13> gregdek: it's no worse than any other idea.  It's a bike shed.  I'm fine with saying the default EC2 image is designed to start as a simple webserver.  You can adjust from there as necessary
20:17:25 <gregdek> gholms, to do anything with EC2, you still need the EC2 client tools locally -- start an AMI, set up your keypair, etc.  I don't think we can ever ship those tools, can we?
20:17:28 * Oxf13 thinks we're talking about two different topics at once.
20:17:32 <gregdek> Yep.
20:17:44 <gregdek> Oxf13 is right.
20:17:50 <gregdek> Can we talk package set for now?
20:17:53 <Oxf13> #proposal EC2 image is a simple webserver at it's base, plus whatever we need for deployment in amazon.
20:17:59 <gregdek> Like it.
20:18:03 <gregdek> Can we put a name next to that?
20:18:08 * poelcat just created a document on gobby cloud-2010-02-04 ... lets add lists, requirements, etc. there
20:18:17 <mattdm> "Fedora Simple Web Server AMI".
20:18:20 <huff> I can provide a base ks file
20:18:26 <huff> based off aos
20:18:32 <gregdek> huff, I like.
20:18:34 <mattdm> Relevant to more fancy steps: anyone who is interested in making awesome Fedora AMIs but hasn't looked at turnkeylinux.org really should.
20:18:42 <gregdek> +1000.
20:18:48 <gregdek> They are doing it right, from what I hear.
20:18:48 <bobmcw> fwiw, we have an httpd (+mod_cluster for jboss) AMI we build
20:19:21 <gregdek> OK, so we've got some proposed package sets.
20:19:27 <gregdek> How do we actually build an AMI?  :)
20:19:38 <bobmcw> thincrust, boxgrinder
20:19:45 <gholms> thincrust is a good start.
20:19:59 <Oxf13> those playing at home may not know what these terms are
20:20:03 <gholms> Although thincrust hardcodes a couple big no-nos.
20:20:07 <bobmcw> thincrust.net
20:20:18 <huff> gholms: i i can fix that
20:20:18 <Oxf13> from the fedora point of view, we have two current tools at our disposal.  livecd-creator and appliance-creator
20:20:19 * jforbes has only built them by hand, and with rBuilder (which won't work for us)
20:20:23 <gregdek> thincrust looks lovely to me.  gholms, what are the nonos?
20:20:24 <bobmcw> http://jboss.org/stormgrind
20:20:37 <Oxf13> both use the same backend to basically make a chroot, install packages into the chroot, and do something with that chroot after the fact
20:20:45 <Oxf13> either isofs, or loopback fs or whatever.
20:20:46 <gholms> gregdek: Non-Fedora kernel and ec2-api-tools on the AMIs.
20:20:52 <huff> im currently adding some functalitu to AC now and should do a release soon if you have any extras send them my way
20:20:58 <gregdek> gholms, ah, right.
20:21:11 <gregdek> ec2-api-tools is a sticky question.
20:21:12 <Oxf13> so what is thincrust?
20:21:22 <bobmcw> don't really *need* ec2-api-tools on the image
20:21:25 <gregdek> thincrust = turn a kickstart into a VM with one command.
20:21:34 * gholms agrees with bobmcw
20:21:48 <Oxf13> is thincrust in Fedora?
20:21:51 <gregdek> And thincrust also contains a "make this VM into an AMI" tool -- jboggs, is that basically right?
20:21:55 <gregdek> Oxf13, yes it is.
20:21:56 <gholms> Oxf13: appliance-tools
20:21:57 <jboggs> yeah
20:21:59 <huff> appliance-tools is in fedora
20:21:59 <bobmcw> ec2-converter, yeah
20:22:09 <bobmcw> boxgrinder builds upon all of that, replacing some bits
20:22:09 <Oxf13> thincrust == appliance-tools ?
20:22:16 <gregdek> Oxf13, basically.
20:22:32 <huff> appliance-tools is on part
20:22:36 <huff> creates teh image
20:22:37 <huff> the
20:22:56 <gregdek> So there's "generic vm" and there's "AMI".
20:23:06 <bobmcw> raw and ami, yessir
20:23:22 <bobmcw> most things can run raw, or be converted to an appropriate format from raw
20:23:24 <gregdek> One tool creates "generic vm" from kickstart, and another tool "transmogrifies raw into ami".  True for both thincrust and boxgrinder?
20:23:27 <huff> there is an image and an ami is that image with meta-data
20:23:44 <gregdek> bobmcw, is boxgrinder in Fedora?
20:23:48 <bobmcw> no, not yet
20:23:58 <gregdek> Can we add that as a TODO that we track?
20:24:04 <gholms> An ami also has a custom fstab and init script.
20:24:17 <gregdek> bobmcw, what's the diff between boxgrinder and thincrust?
20:24:18 <bobmcw> gregdek: boxgrinder is currently tied to jboss tech
20:24:23 <bobmcw> so we need jboss in fedora also :)
20:24:36 <gregdek> bobmcw, hee hee!  That's another meeting.  :)
20:24:41 <bobmcw> mgoldmann's the boxgrinder lead, I mostly observe these days, but...
20:24:42 <huff> bobmcw: what does boxgrider give us ontop of the thincrust tools, im just not fam with the project
20:25:13 <bobmcw> the same general principles as thincrust, we do some things differently in our conversions (libguestfs to crack images) and to expedite building multiple types of VMs
20:25:30 <bobmcw> ie, we build a base, then crack and add Ec2-specific bits, then do the same from the base for vmware-etc
20:25:44 <bobmcw> since we assume we need to produce N types of images for any input
20:25:51 <bobmcw> raw, ec2, vmware, etc
20:26:03 <gregdek> bobmcw, is boxgrinder java/x-platform?
20:26:04 <bobmcw> handling things like disk format and other weirdness
20:26:15 * mgoldmann is back
20:26:16 <bobmcw> boxgrinder is primarily ruby, and until recently was purely c-ruby
20:26:19 <gholms> Ooh, that would make providing downloadable VMware images easy, too.
20:26:29 <bobmcw> on top of the functionality, we also have a 3-tier arch on it
20:26:31 <Oxf13> so eventually boxgrinder might be useful
20:26:37 <bobmcw> boxgrinder-build as the main library of logic
20:26:41 <gregdek> Sounds like boxgrinder is getting more cycles... but has more deps to figure out.
20:26:43 <mgoldmann> http://www.jboss.org/stormgrind/projects/boxgrinder.html
20:26:43 <bobmcw> -rest as a REST API server, as a service
20:26:51 <Oxf13> but in the context of a stated goal of producing ec2 images, it seems that thincrust/appliance-creator is enough to get going
20:26:55 <bobmcw> and -studio, purely in my head at this point, to help in the construction of images
20:26:58 <bobmcw> akin to SuSE Studio
20:27:01 <mgoldmann> http://community.jboss.org/wiki/StormGrindBoxGrinderDocumentation
20:27:03 <gregdek> Oxf13, for now I think that's right.
20:27:14 <huff> bobmcw: your still using appliance-creator to build the base image
20:27:16 <bobmcw> the servery bits are targetting TorqueBox at the moment
20:27:22 <mgoldmann> huff: yes
20:27:30 <bobmcw> torquebox.org, the ruby appserver built on jboss AS
20:27:33 <gregdek> bobmcw, if boxgrinder is all ruby, where is jboss required?
20:27:39 <gregdek> Ah, ok.
20:27:42 <mgoldmann> that's the one thing we're using from thincrust tooling
20:27:54 <mgoldmann> gregdek: it's not required
20:27:54 <bobmcw> and starting to use stuff like the jboss MQ tech to drive build farms of appliance building nodes
20:27:58 <gregdek> Hm!
20:28:11 <gregdek> Then we should really make it a priority to get boxgrinder into fedora.
20:28:16 <bobmcw> but we could move to some other tech.  just torquebox is my baby, so I push it where I can :)
20:28:18 * stickster comes in late and apologizes, just got off an impromptu meeting with mgr.
20:28:19 <gregdek> In parallel.
20:28:26 <mgoldmann> we just wanted a tool we could build images for cloud for JBoss projects, that's all
20:29:18 <lutter> is there any way to resolve the weird thincrust/boxgrinder split ? It's very confusing to the outside world
20:29:49 <bobmcw> lutter: thincrust seems to be winding down as a project
20:29:51 <gregdek> 1. start with thinrust;
20:29:59 <gregdek> 2. commit to moving to boxgrinder when we are able.
20:30:00 <gregdek> Make sense?
20:30:06 <gholms> gregdek: +1
20:30:07 <mgoldmann> sure!
20:30:09 <bobmcw> lutter: but it could also just be considered the foundation upon which we build, sorta
20:30:13 <lutter> bobmcw: it's anybody's project for the taking
20:30:15 <Oxf13> gregdek: that's my thought too
20:30:26 <lutter> (well, huff or bkearney might have a different opinion)
20:30:27 <gregdek> I like thincrust, but it's soon to be deprecated, it seems. Still, we wanna start now.
20:30:44 <huff> no sounds good to me
20:30:45 <bobmcw> gregdek: one thing we ship is a meta-appliance, so you can easily prop up a boxgrinder server on ec2 to spew out more
20:30:47 <bobmcw> easy bootstrap
20:30:52 <gregdek> Clever.  ;)
20:30:59 <gregdek> All right.
20:31:08 <bobmcw> plus, pushing your AMIs to S3 is free/fast if you're already on EC2
20:31:08 <gregdek> So now that we know we're starting with thincrust for now...
20:31:14 <gregdek> ...what do we do, exactly?
20:31:30 <mattdm> e-mailed thincrust list the other day, asking if it was still active. Bryan Kearney replies "Yes, it is an ongoing concern."
20:31:37 <Oxf13> if we've decided on a toolchain
20:31:41 <gregdek> Put together a script that transmogrifies spin into vm into ami automagically and see if it works?
20:31:50 <Oxf13> then what we need to do is take the previous decision on the package set, and get it into a format appropriate for the toolchain
20:31:53 <Oxf13> which should be a .ks file
20:32:07 * gholms wants a very minimal baseline
20:32:20 <Oxf13> gholms: we've already decided that it's going to be a simple web server
20:32:27 <gholms> Yeah...
20:32:33 <mattdm> there's a project goals question here....
20:32:42 <mattdm> initially, simple web server is good...
20:32:43 <Oxf13> gholms: @core, perhaps some of @base, and the httpd stack
20:32:45 <mgoldmann> bear in mind, that current thincrust ec2-converter doesn't work and it was building only 32 bit images earlier...
20:33:04 <mattdm> but is the intention of this project to eventually produce application-level appliances that just magically work out of the box?
20:33:21 <mattdm> or are we planning to work entirely below that level?
20:33:32 <Oxf13> mattdm: I'm not sure about eventual.  I am only looking at the first goal of getting usable useful recent Fedora offerings into Amazon EC2
20:33:50 <Oxf13> that's an obtainable goal, with clear benefits
20:33:52 <gholms> I would rather just provide "Fedora in the Cloud" and let users customize it.
20:33:57 <mattdm> just want to be explicit about that.
20:34:04 <jforbes> mattdm: once the base image is public, and the kernel is publically available, people can build there own images easily
20:34:11 <gregdek> mattdm, I think one step at a time.  Initial goal is "usable default AMIs that don't suck and users can expand upon."
20:34:19 <gregdek> Everything must start from there anyway.
20:34:21 * mattdm nods
20:34:27 * gholms nods
20:34:35 <jforbes> s/there/their
20:34:48 <Oxf13> ok, I think we've reached two conclusions, maybe 3
20:35:00 <poelcat> can we document this as we go in Gobby or using meetbot?
20:35:10 <Oxf13> is meet bot in here?
20:35:12 <gregdek> poelcat, can you restate in meetbotspeak?
20:35:13 * poelcat does not have time to create notes or do recap today
20:35:17 <gregdek> Ah, ok.
20:35:18 <gholms> Oxf13: Yep
20:35:22 <gregdek> Who knows meetbotspeak well?
20:35:25 <Oxf13> is this a live meeting?
20:35:29 * gholms raises hand
20:35:30 <gregdek> Oxf13, yep.
20:35:33 <gholms> Oxf13: Yep
20:35:34 <gregdek> gholms?
20:35:40 <poelcat> gregdek: if i understood what to capture... we've covered 10 different topics at once :)
20:35:44 <gholms> I can give it a shot.
20:35:45 <gregdek> LOL
20:35:57 <Oxf13> #info Initial goal of project is to provide Useful usable recent Fedora images in Amazon EC2
20:36:08 <gregdek> Oxf13, yep.
20:36:23 * gregdek waits for the full Oxf13 recap.  :)
20:36:26 <Oxf13> #info The initial image will consist of a simple webserver and the tools to add/remove packages from there, plus whatever is necessary for EC2
20:36:49 <mgoldmann> A JEOS-like appliance would be sufficient IMHO, at least for start. http://github.com/stormgrind/boxgrinder-build/blob/master/appliances/jeos.appl
20:36:57 <Oxf13> #info The initial toolset to accomplish this will be thincrust/appliance-creator, however we will be looking at stormgrind in the future
20:37:02 <gregdek> The big open question is WHO and WHEN?
20:37:12 <Oxf13> Ok, that's the end of my recapping
20:37:16 <gregdek> I guess that's two questions.  :)
20:37:36 <Oxf13> so action items!
20:37:40 <gholms> At the very least we want something in time for F13.
20:37:49 <Oxf13> I do believe huff volunteered to author the .ks file for the appliance
20:37:53 <Oxf13> huff: is that correct?
20:37:55 <huff> I can delever a fist pass base ks with min packageset extra stuff in post for amazon spicific stuff,
20:37:58 <huff> yea
20:38:04 <gregdek> Maybe I'm naive but seems to me that if we get a decent ks, we can start making broken-but-instructive images almost immediately.
20:38:14 <Oxf13> #action huff to deliver a first pass base ks with min packageset extra stuff in post for amazon specific stuff.
20:38:22 * jforbes has the kernel
20:38:23 <mgoldmann> FYI: we are in #stormgrind if you have additional questions about BoxGrinder
20:38:32 <Oxf13> #action jforbes continues to wrangle a kernel for us to use
20:38:40 <Oxf13> the big question I have in my mind right now
20:39:06 <Oxf13> who is working on figuring out what tools we can or need to use in order to get the image to run in EC2, and to get the image uploaded to EC2
20:39:22 <gregdek> 1. Uploading.
20:39:26 <gregdek> We have two accounts:
20:39:27 <mgoldmann> boxgrinder has this out of the box
20:39:33 <gholms> euca2ools can bundle and upload.
20:39:40 <Oxf13> mgoldmann: but is it using software that is acceptable for Fedora?
20:39:45 <gregdek> One "official" account that will be the home of all "official" spins.
20:39:47 <mgoldmann> Ruby?
20:39:49 <Oxf13> same question for gholms
20:39:57 <bobmcw> mgoldmann: do we rely on the ec2-tools though?
20:39:59 <gregdek> And one "scratch" account that we can use for all unofficial stuff.
20:40:01 <mattdm> is there anything we need that euca2ools can't do?
20:40:08 <mgoldmann> bobmcw: no
20:40:10 <bobmcw> or just the right_aws gems?
20:40:10 <Oxf13> lets take a step back here
20:40:10 <gholms> Oxf13: euca2ools has passed approval
20:40:15 <gregdek> Is euca2ools in Fedora?
20:40:21 <mgoldmann> bobmcw: ah, yes, we are
20:40:32 <Oxf13> in previous discussions it seemed that amazon required  use of some non-free stuff to either create the image, or upload the image
20:40:34 <jforbes> can euca2ools set up the ssh keys, etc to a running instance?
20:40:37 <mgoldmann> bobmcw: bundling is done via ec2-ami-tool IIRC
20:40:42 <gholms> jforbes: Yes
20:40:46 <jforbes> excellent!
20:40:48 <Oxf13> is that not the case?
20:40:54 <gregdek> OK, I want to understand clearly.
20:40:55 <bobmcw> jforbes: "set ssh keys" just means "pass a string to launch"
20:41:04 <bobmcw> ec2 itself creates and holds your pubkey
20:41:25 <mattdm> euca2ools is a reimplementation implementation of the non-free amazon tools.
20:41:30 <Oxf13> #topic image upload
20:41:34 <jforbes> bobmcw: Yeah, familiar with it, just making sure.  We had a different tool at rPath, but it has a plugin system and a bunch of crap that is useless to fedora
20:41:45 <bobmcw> mattdm: name-compatible?  ec2-describe-instances and such?
20:41:46 <Oxf13> #info We have two accounts with amazon
20:41:46 * mattdm is typing faster than he is thinking
20:41:50 <gregdek> Here's what I *think* I understand: Amazon has API hooks, and then they create a set of APIs, licensed in a funky way, that use those hooks.  Is euca2ools a clean rewrite of those tools, basically?
20:41:53 <bobmcw> iff so, then the boxgrinder stuff could use it
20:41:58 <gholms> bobmcw: s/ec2/euca/
20:42:04 <bobmcw> 'k, we could adjust
20:42:10 <Oxf13> #info one "official" account for "official" spins, and one scratch account for unofficial stuff
20:42:22 <bobmcw> gregdek: amazon publishes a SOAP WSDL API document
20:42:29 <gholms> bobmcw: I can't provide symlinks without breaking ec2-ami-tools from rpmfusion.
20:42:31 <bobmcw> along with a client capable of speaking to it
20:42:35 <mattdm> no not same names. s/ec2/euca/.
20:42:48 <bobmcw> gholms: we can accomodate that sort change
20:42:52 <gregdek> bobmcw, ok, and euca2ools is written to that spec?
20:42:59 * gregdek gets it.  I think.
20:43:01 <bobmcw> gregdek: no idea, but I gotta assume so
20:43:08 <Oxf13> just so we're clear...
20:43:11 <gregdek> bobmcw, lol.  Can someone provide clarity there?
20:43:16 <bobmcw> WSDL is like CORBA IDL
20:43:24 <mattdm> gholms: maybe symlinks in a subpackage?
20:43:44 <bobmcw> $TOOL_PREFIX="euca" in our scripts will suffice if needed
20:44:06 <Oxf13> euca2ools is A) all we will need to upload images, and B) non-reliant on anything non-free or non-redistributable, and C) not going to cause the wrath of amazon to fall upon us?
20:44:08 <gholms> mattdm: It's still a file conflict; fesco would shoot me.
20:44:09 <bobmcw> gregdek: I'm pretty ignorant of eucalyptus in general
20:44:23 <gregdek> Oxf13, +1.
20:44:30 <gregdek> We need a clear answer to that question.
20:44:32 <Oxf13> gholms: other option is use of Alternatives for amazon tools
20:44:38 <huff> also, AFAIK, you need special tools to bundle and upload aki and ari's and to assioacite them
20:44:44 <jforbes> The only thing I know about eucalyptus is that tybstar is working there now :)
20:44:49 <huff> this may have changed
20:44:51 <jforbes> huff: Yes, and those tools are NDA
20:44:56 <Oxf13> huff: now we're talking jibberish again :)
20:44:59 <gholms> Oxf13: That's a possibility.
20:45:00 <gregdek> huff, that's ok, we can separate that out.
20:45:01 <bobmcw> kernels and ramdisks
20:45:01 <huff> always
20:45:19 <Oxf13> gholms: can we get an answer to my above question?
20:45:37 <gregdek> I believe that Oxf13 is the 100% critical path question.
20:45:45 * gholms scrolls up
20:45:48 <huff> Oxf13: what ?
20:45:50 <gregdek> That we must have a clear answer to before we can proceed at al.
20:45:58 <gregdek> euca2ools is A) all we will need to upload images, and B) non-reliant on anything non-free or non-redistributable, and C) not going to cause the wrath of amazon to fall upon us?
20:46:01 <gregdek> (Restating)
20:46:14 <bobmcw> I think (c) is probably the only question
20:46:34 <gholms> A:  Yes for AMIs, I can't say for the rest
20:46:35 <bobmcw> sounds like eucatools does (a) and (b)
20:46:37 <jforbes> I don't think C is a question, Amazon provides the API for a reason
20:46:45 <gregdek> If (a) and (b) are true, I can't imagine that (c) will be an issue.
20:46:45 <gholms> B: completely FOSS
20:46:47 <mattdm> B:  is true.
20:46:52 <gregdek> All right.
20:46:58 <gholms> C: unlikely
20:47:03 <bobmcw> and normal mortals don't get to upload kernels normally, do we?
20:47:07 <gregdek> Then we need to take an action to expedite getting euca2ools into Fedora.
20:47:10 <Oxf13> perhaps we need a recap of terms
20:47:11 <bobmcw> gotta do a dance with Bezos to get a new kernel up there
20:47:16 <Oxf13> right now we were talking about an OS image
20:47:22 <Oxf13> but there are also kernel/initrd images yes?
20:47:28 <bobmcw> yes
20:47:29 <jforbes> Oxf13: yes
20:47:32 <Oxf13> alright
20:47:36 <jforbes> Oxf13: AMI is the os image.
20:47:37 <Oxf13> so correct me if I'm wrong
20:47:40 <gregdek> Yep.  And that is a separate discussion.  Only we will have the right to upload AKIs.
20:47:46 <gholms> gregdek: The review is done; I'm working on fixing a problem with adding it to pkgdb.
20:47:49 <Oxf13> but eucatools is only for uploading the OS image (sans kernel)?
20:47:52 <gregdek> Which is a tricky thing, tbh.
20:47:54 <bobmcw> Oxf13: yes
20:47:55 <gregdek> Oxf13, correct.
20:47:57 <mattdm> Will it be possible to upload new kernels for _every kernel update_ from Fedora?
20:48:04 <jforbes> mattdm: kind of
20:48:05 <bobmcw> mattdm: if we have friends in Amazon
20:48:06 <Oxf13> #info eucatools is only for uploading the OS image.  The kernel is handled differently
20:48:06 <gregdek> mattdm, that's our call.
20:48:12 <Oxf13> Ok, so.
20:48:12 <gregdek> bobmcw, we do have friends at amazon.
20:48:17 <bobmcw> 'k
20:48:17 <Oxf13> to recap
20:48:32 <jforbes> mattdm: the expectation is that security fixes will get updated to ec2, but not necessarily every kernel with bugfixes that don't apply
20:48:44 * gregdek notes that a former FPL works at amazon.  :)
20:48:47 <Oxf13> #info simple webserver ks -> appliance-creator -> eucatools == OS image in Amazon EC2
20:48:56 <gregdek> Oxf13, yep.
20:48:58 <mattdm> good enough for me.
20:49:04 <Oxf13> gregdek: he works about 40 minutes from my house
20:49:13 <jforbes> gregdek: msw is there now too
20:49:18 <gregdek> Oxf13, buy that dirty ol' bastard a drink for me sometime.
20:49:21 <Oxf13> Ok, with the above recap, I think we can close out the OS image portion of this discussion.
20:49:26 <gregdek> +1
20:49:27 <poelcat> smoser: how do you guys do this process?
20:49:35 <Oxf13> which brings us to the kernel side of things
20:49:38 <Oxf13> #topic kernel upload
20:49:50 <smoser> reading
20:50:26 <smoser> euca2ools is not capable of uploading a kernel.
20:50:33 <gregdek> Nor should it be.
20:50:35 <Oxf13> so before we can even boot an OS image, regardless of the OS image, we have to have a kernel available.
20:50:45 <Oxf13> how do we get a kernel available?
20:50:46 <gholms> Sure it should.  That's what account ACLs are for.
20:50:49 <smoser> you have to use the blessed ec2-api-tools and ec2-ami-tools to do that.
20:50:51 <gregdek> Oxf13, correct.  And right now, the only kernel up there is FC8.
20:51:01 <jforbes> kernel upload is fairly restrictive, though a simple process (made even easier with dracut)
20:51:03 <huff> so my thoughts are to generate the aki and ari in post of ks
20:51:04 <smoser> we publish all kernels that are used in a build.
20:51:10 <gregdek> gholms, ok, maybe.  But it our case, not.  :)
20:51:16 <gholms> gregdek: :)
20:51:23 <smoser> we use buckets and names to differenciate: https://wiki.ubuntu.com/UEC/Images/NamingConvention
20:51:51 <Oxf13> lets back up a second here.
20:51:52 <huff> not sure if that will work or not
20:51:59 <jforbes> Oxf13: I have the kernel tools, and will be writing something up to share with kylem and davej as well, to make sure kernels can be uploaded, and I am not the only one capable
20:52:24 <Oxf13> to upload the kernel, you need a vmlinuz and initramfs right?  Anything else?
20:52:34 <gholms> You upload them separately.
20:52:42 <gregdek> Funky.
20:52:44 <jforbes> Oxf13: actually we specifically need a vmlinux, vmlinuz is not allowed
20:52:51 <Oxf13> oh
20:52:52 <gregdek> Super funky.  :)
20:53:01 <jforbes> debuginfo to the rescue
20:53:03 <Oxf13> so then, we cannot use the existing Fedora RPMS
20:53:11 <gregdek> Not for kernel, i guess.
20:53:15 <Oxf13> at least without some modification
20:53:20 <huff> alsothe initrd has to be differnt for lagre (64 bit) and small
20:53:22 <gholms> Do the images even need the kernels themselves installed?
20:53:24 <huff> 32bit
20:53:30 <gholms> Modules, sure.
20:53:37 <huff> gholms: yes for the modules
20:53:39 <gregdek> We should prob bang out a clear list of all packages that need to have ec2 versions.
20:53:54 <Oxf13> #info vmlinux (not vmlinuz) and initramfs files need to be uploaded (separately)
20:53:55 <huff> ec2 versions?
20:53:57 <jforbes> Oxf13: well, we can use the vmlinux from the debuginfo, but we need an dracut image
20:54:06 <Oxf13> jforbes: right
20:54:20 <Oxf13> my spidersense is pinging awfully hard right now.
20:54:25 <gholms> Does it need to differ from the stock one?
20:54:30 <gregdek> Oxf13, yep.  That's why you're here.  :)
20:54:31 <Oxf13> gholms: there is no stock one
20:54:47 <Oxf13> gholms: initramfs is created upon kernel install to your filesystem
20:54:47 <gholms> Oh, right...
20:54:50 <gregdek> At some point, this probably requires some funky compromises.  Part of why we're here is to figure out what those are ASAP.
20:55:03 <Oxf13> we'll get to my spidysense in a moment.
20:55:06 <huff> initrd can but build in post of ks
20:55:30 <Oxf13> before we dive into the mechanics of generating the vmlinux and initramfs, lets talk more about the upload process
20:55:36 <jforbes> huff: the kernel image should be there before you are even building your ks
20:55:49 <Oxf13> What are the requirements to upload, provided we have the necessary vmlinux and initramfs in hand?
20:56:22 <jforbes> Oxf13: we use their nda'd tools to create and upload the image, then we set the permissions to make them publicly visible
20:56:37 * gregdek wanders afk for a sec...
20:56:52 * gholms wonders how the eucalyptus people add kernels
20:56:55 <huff> ec2-bundle needs to be run on 1.raw disk image, 2.initrd image and, 3.vmlinux
20:57:09 <huff> then ec2-upload need to be runon all three
20:57:16 <Oxf13> jforbes: it's necessary for the /creation/ phase as well?
20:57:24 <Oxf13> jforbes: can you walk us through that briefly?
20:58:07 <huff> then you have to link them togeather
20:58:26 <jforbes> Oxf13: creation? Essentially we call the bundle tool with a path to the vmlinux file to create the kernel image, then call the upload, then set the acls.. do the same for the ramdisk
20:58:41 <huff> yep, build, bundle, upload,link
20:58:50 <jforbes> huff: kind of, the bundle tool is different for aki and aris, and nothing binds them together othe than the ami definition
20:58:57 <huff> right
20:59:15 * gregdek back
20:59:38 <Oxf13> jforbes: do we know what it does to that vmlinux file?  Is it bundling something into it that wasn't already there?
20:59:53 <huff> just splits it up and adds meta data
20:59:57 <Oxf13> is it doing anything more than just taking what's there and re-arranging it slightly for the upload?
21:00:06 <jforbes> Oxf13: nope, not doing anything to it of concern
21:00:50 <huff> amazon just restricts who can upload kernels, thats why the bundle tool is different for aki and aris
21:00:51 <Oxf13> ok.
21:00:53 <jforbes> Oh, we do have to worry about regions, though that is a different discussion
21:01:17 <huff> jforbes: there is a migrate tool to move toEU oncein US
21:01:28 <jforbes> huff: there are 2 US now
21:01:32 <huff> o
21:01:39 <Oxf13> ok, so the tool adds nothing material to the image, nothing that is used beyond their infrastructure
21:01:46 <Oxf13> #info the tool adds nothing material to the image, nothing that is used beyond their infrastructure
21:01:47 <jforbes> new, as f last month I believe
21:01:52 <jforbes> Oxf13: correct
21:02:06 <Oxf13> and I assume the tool authenticates you somehow?
21:02:16 <gholms> Oxf13: It uses x.509.
21:02:21 <jforbes> Oxf13: with a pem, a key, and UID
21:02:23 <Oxf13> ok.
21:02:43 * poelcat has to leave for another meeting
21:02:45 <Oxf13> does the fact that you uploaded it using their tool count as the "blessing" or is there anything else?
21:02:54 <Oxf13> poelcat: thanks, I'll keep meetbotting the notes
21:03:06 <jforbes> Oxf13: that is the blessing, and one of the reasons it is restricted
21:03:09 <gregdek> We have a pem for the Fedora "scratch account" for anyone who wants it, btw.
21:03:16 <gregdek> poelcat, thanks.  :)
21:03:36 <gholms> FYI:  euca2ools doesn't yet implement image migration
21:03:39 <Oxf13> #info uploading by the tools counts as the blessing
21:04:23 <Oxf13> jforbes: do the best of your knowledge, is the vmlinux/initramfs bundle/upload step the only part of this process which requires non-free tools?
21:04:39 <jforbes> Oxf13: correct
21:04:48 <huff> Oxf13: also chaning attrubutes
21:04:55 <jforbes> Oxf13: for everything else, if a free tool isn't written, the API exists
21:05:06 <Oxf13> #info vmlinux/initramfs bundle/upload step is the only part of the process which requires non-free tooling.
21:05:09 <huff> ie: assiocate our kernel to our images
21:05:17 <jforbes> huff: p
21:05:17 <Oxf13> huff: that can't be done via the API?
21:05:22 <jforbes> huff: err, no
21:05:29 <gholms> That can be done with euca2ools.
21:05:44 <jforbes> huff: once the kernels/ramdisks are uploaded and public, anyone can link them to any image via the APU
21:05:47 <jforbes> err API
21:06:08 <ewan> My understanding is that each AMI machine image essentially says which kernel it wants to use - yes?
21:06:17 <gholms> ewan: Yes.
21:06:20 <huff> are you sure you can change the images kernel and ramdisk attrbute with euca2ools.
21:06:26 <jforbes> ewan: yes, and by not stating it implies the default
21:06:52 <huff> how do we set the default
21:06:59 <gholms> huff: euca-bundle-image takes --kernel and --ramdisk parameters.
21:07:01 <huff> ie our image should default to our images
21:07:10 <huff> gholms: ok
21:07:28 <Oxf13> alright, lets break out some action items here
21:07:53 <jforbes> huff: we set the default when we create our image
21:08:07 <Oxf13> jforbes: can I assign you the task of documenting the kernel/initramfs upload process, and sharing credentials with the other fedora kernel folks?
21:08:10 <jforbes> huff: outside of that, there is only one default, the amazon kernels
21:08:16 <jforbes> Oxf13: absolutely
21:08:29 <Oxf13> #action jforbes will document kernel/initramfs upload process
21:08:31 <gholms> I will push euca2ools as soon as infrastructure lets me.
21:08:39 <Oxf13> #action jforbes will share upload credentials with Fedora kernel developers
21:08:57 <Oxf13> We've talked this subject up a buch, but there is one concern I'm going to lay down
21:09:04 <Oxf13> and it's going to be ... uncomfortable.
21:09:08 <mattdm> yay
21:09:15 <Oxf13> GPL compliance.
21:09:15 * gholms braces himself
21:09:20 <Oxf13> #topic GPL Compliance
21:09:32 <gholms> Dun dun duuuun!
21:09:33 <Oxf13> We actually stopped pre-producing initramfs images in our kernel builds due to this
21:09:54 <Oxf13> the initramfs includes bits and pieces of many other software projects, only in binary form
21:10:14 * gholms notes that #topic didn't work again
21:10:15 <Oxf13> were we to distribute that initramfs, we'd be on the hook to provide the source to those projects as well
21:10:29 <jforbes> Oxf13: But if they are only built using the released repository bits, we are already providing them right?
21:10:33 <gregdek> Hm... zodbot has op.  Weird.
21:10:46 <Oxf13> jforbes: thats if, and only if, we only provide the /release/ kernel, and not any update kernel
21:11:02 <jforbes> Oxf13: we ship source with update kernels too
21:11:11 <Oxf13> that also assumes that the Amazon kernel images are not available longer than the time which our Fedora source is available.
21:11:42 <Oxf13> jforbes: we don't guarantee that the version of tools you use during creation of an initramfs from updates will always be available
21:11:46 * gregdek wonders if we should consider moving fedora source to amazon at some point...
21:11:50 <Oxf13> jforbes: we expire updates rather rapidly
21:12:36 <Oxf13> so if you build an image today with updates that includes say plymouth, and in 3 weeks a new plymouth comes out, the version  you used will be removed from our servers, and thus the source to the bits in your image are no longer available in the assumed location
21:13:08 <Oxf13> there are obviously things we can do to track and store sources somewhere, however in the interest of getting F12 out the door we decided to just not pre-generate the images, and side step the issue
21:13:09 <gholms> How much can be rebuilt from cvs?
21:13:14 <mattdm> (and practically speaking i've run into that as a real problem. not with kernels but with other updates.)
21:13:26 <Oxf13> gholms: it could be rebuild, provided nobody moved a tag.
21:13:32 <Oxf13> gholms: it could also be rescued from Koji
21:13:44 <Oxf13> but the point is that the content would not be in the typical place.
21:13:49 <gregdek> Is our relationship with Amazon a useful way to solve this problem by throwing massive amounts of source into S3?
21:14:06 <Oxf13> and since the binary bits are sitting in amazon, without source next to it, it breaks how we comply with GPL requirements
21:14:19 <Oxf13> with Fedora, we publish source alongside binaries, and we keep them in lock step
21:14:30 * mattdm idly considers putting the source into the initramfs....
21:14:45 <Oxf13> I don't imagine there is anywhere within the Amazon EC2 infrastructure that we could drop all the srpms involved with the kernel
21:14:57 <Oxf13> mattdm: this is a problem for more than just the kernel
21:15:06 <Oxf13> it's worse for the kernel due to the initramfs
21:15:08 <gholms> Not even a EBS snapshot?
21:15:14 <Oxf13> but we also have to provide srpms for the OS image
21:15:25 <Oxf13> or provide some sort of written notice as to where the srpms can be found
21:15:30 <gregdek> Oxf13, why on earth not?  s3 is one of the most massive datastores on the planet.
21:15:35 <Oxf13> we've never done the written notice.
21:15:40 <jforbes> Oxf13: for one thing, the expectation on images going forward is that the kernel is the only thing we will update
21:15:44 <mattdm> written-notice route is a pain because of its requirements.
21:15:50 <Oxf13> gregdek: I do believe there is specific language about distributing source with binary
21:15:58 <mattdm> better to provide the source.
21:16:01 <Oxf13> gregdek: saying that "they're both in amazon, go find them" isn't sufficient
21:16:03 <jforbes> Oxf13: so we build new ramdisks on release with no updates outside of kernel
21:16:32 <Oxf13> jforbes: or, we make sure that we build kernel images only with bits found in the OS image we've uploaded, and make sure to track the sources used in the OS image
21:16:41 <Oxf13> basically we're going to need some accounting here
21:16:50 <Oxf13> what binaries go into the OS image, and what binaries go into the kernel images
21:17:08 <Oxf13> and a place to stash the srpms and a way to connect the two for GPL sake
21:17:33 <huff> I can generate a list of rpms inthe os image
21:17:45 <huff> we do this for ovirt node
21:17:52 <Oxf13> relevant GPL(v2) language:
21:17:53 <Oxf13> 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following:
21:17:57 <Oxf13> a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,
21:18:32 <Oxf13> On the Fedora download pages we provide source links next to all the binary links
21:18:44 <Oxf13> in the case of the live images, we bundle a README.sources file
21:19:39 <jforbes> And this would be very much like a live image
21:19:43 <Oxf13> Sources for all packages used in this live image can be found under:
21:19:43 <Oxf13> http://download.fedoraproject.org/pub/fedora/linux/releases/12/Everything/source/
21:19:49 <Oxf13> that's the content of the readme file
21:20:14 <Oxf13> things get fuzzy when we do OS images or kernel images from updates, again because we don't guarantee the life of the srpm
21:20:20 <gholms> That would mean no kernel/busybox/etc updates.
21:20:20 <Oxf13> unlike with the full releases
21:20:46 <Oxf13> and since Amazon separates the kernel/initramfs from the os image, we'd have to have a notice bundled with the kernel
21:20:53 <Oxf13> as well as a notice bundled with the OS image
21:21:03 <jforbes> Oxf13: users cannot see the kernel
21:21:12 <Oxf13> which is a problem
21:21:14 <gholms> But they load it.
21:21:20 <jforbes> Oxf13: xen booting with the kernel outside of the image, the kernel is never visible
21:21:26 <Oxf13> how do we inform the users where they can get the source code to the binary that's being provided to them?
21:21:42 <jforbes> Oxf13: the same kernel should be installed into the image
21:22:25 <Oxf13> so we have to ensure that we always have at least one official image that matches one official kernel, maintaining at least a 1:1 relationship
21:22:27 <jforbes> Though realistically once a kernel is public, anyone can link it to any image
21:22:36 <Oxf13> and deliver the "find the sources here" type message in the OS
21:22:44 <Oxf13> jforbes: yeah, that's a concern.
21:22:48 <ewan> The AMIs will surely /need/ to have the same kernel installed as they boot with if they're to be able to load modules.
21:23:13 <jforbes> Oxf13: no more of a concern than once we ship any binary rpm anywhere, anyone can put it whever the hell they choose
21:23:16 <Oxf13> by uploading the kernel to amazon and making it "public" we're essentially distributing those binaries
21:23:19 <gholms> They let users specify alternate kernels at their own peril.
21:23:34 <Oxf13> jforbes: it it different
21:23:45 <Oxf13> jforbes: when we distribute a binary rpm, we distribute the srpm along side it
21:23:48 <gholms> Would it be sufficient to put the sources in the same bucket as the vmlinux?
21:23:57 <Oxf13> jforbes: when we stop distributing the binary rpm, we take down the srpm
21:24:04 <gregdek> If you can tell what kernel you're using at runtime, surely there can be a script that generates a link to the source of the kernel you're currently running?
21:24:06 <jforbes> Oxf13: We do, but people redistributing it might not, which is what we are saying here as well
21:24:18 <Oxf13> jforbes: except /we/ are the ones "distributing" it
21:24:27 <gholms> gregdek: Then we'd have to hold onto the srpms for three years.
21:24:28 <mattdm> or, amazon is.
21:24:31 <Oxf13> we put it on amazon, we marked it as public.  that constitutes distribution
21:24:54 <Oxf13> at least in my mind it does
21:24:56 <gregdek> gholms, for the kernels, yes.  But amazon makes that *way* easier than it is now.
21:24:56 <jforbes> Oxf13: Oxf13 no, we distribute it once to amazon, just like when we upload a kernel rpm to an ftp server or anywhere else
21:25:20 <gholms> smoser: How do you folks deal with GPL compliance for kernel and initramfs updates?
21:25:58 <jforbes> Oxf13: we include a source link in our image as well.  Once that is done, people have the ability to do whatever they want with it... Our distribution of it is clean, other people might choose to redistribute, and we cannot control what they do with it
21:26:21 <Oxf13> jforbes: we're still missing a step though
21:26:32 <Oxf13> jforbes: when we put a binary up on our master server, we put the srpm along with it
21:26:34 <jforbes> Oxf13: of course Amazon has packaged our kernels before this as well
21:26:44 <Oxf13> jforbes: so that when folks choose to consume it, they have the opportunity to consume the srpm as well
21:26:54 <jforbes> Oxf13: and when we put our images on EC2, we will have a link to the sources as well
21:26:56 <Oxf13> jforbes: if we put our kernel up on amazon, where do we put the matching source?
21:27:04 <Oxf13> ok, where does that link go?
21:27:06 <gholms> Oxf13: Would it be sufficient to put the sources in the same bucket as the kernels?
21:27:16 <Oxf13> gholms: define "bucket"
21:27:20 <huff> in the same bucket
21:27:25 <gholms> An S3 bucket
21:27:35 <gregdek> My guess is that Amazon, or more likely EC2 users, are likely breaking these rules out of ignorance currently.
21:27:54 <gregdek> Who, for instance, uploaded the current F8 kernels?
21:28:01 <jforbes> gregdek: Amazon did
21:28:14 <gholms> The plot thickens!
21:28:15 <gregdek> Do they have the srpms available?
21:28:18 <gregdek> I'll wager not.  ;)
21:28:27 <Oxf13> my bet is on simple ignorance.  I think about these things because I'm paid to :/
21:28:34 <jforbes> gregdek: And they probably counted on the fact that the source was available from us when they put it there.  It ins't anymore unless you look at cvs
21:28:46 <Oxf13> jforbes: it is on the archive server
21:28:50 <gholms> There's a chance it's in the archives.
21:28:52 <gregdek> My point is not to slam Amazon, of course.  My point is that it's not Amazon's business to solve this problem; it's ours.
21:29:00 <Oxf13> gregdek: +1
21:29:22 <Oxf13> and really, we just have to make a reasonable effort to provide people who get the binaries with the option to get the sources
21:29:22 <gregdek> And we have much less wiggle room, because it's our name on the line.  That's part of what this whole exercise is about.
21:29:34 <mattdm> +1
21:29:38 <gregdek> OK, good discussion...
21:29:39 <Oxf13> we certainly don't have to shove the source down their throat
21:29:43 <gregdek> ...where are we?
21:29:44 <jboggs> the F8 kernel amazon has is some random koji build that never quite lined up with any update
21:29:50 <Oxf13> I think we've made some decisions
21:29:51 <gregdek> jboggs, lulz
21:29:56 <Oxf13> jboggs: awesome.
21:29:59 <gregdek> EXACTLY!  :)
21:30:02 <jforbes> gregdek: so maybe we bring this up with amazon, and see about making a sources server available.  the sources go into an S3 bucket that is visible from the EC2 web server
21:30:14 <jboggs> and koji dropped it so gone for now, i think bobmcw has a copy
21:30:20 <Oxf13> lets recap a bit
21:30:36 <gregdek> Yep.  I can bang out a note to nthomas.  It may take a while to resolve, but we can get started.
21:30:37 <Oxf13> I think we decided to deliver a README-SOURCES file with the OS version, that points to the release tree from whence the image came
21:30:51 <Oxf13> is everybody in agreement with that?
21:30:54 <jforbes> gregdek: maybe get amazon to pick up the tab for a machine instance to run that web server, and we inlcude only the sources from the kernels we upload to EC2 in that server
21:30:55 <gregdek> +1
21:31:01 <gregdek> jforbes, definitely.
21:31:02 <gholms> +1
21:31:03 <mattdm> so far good.
21:31:05 <Oxf13> ok
21:31:12 <Oxf13> #info deliver a README-SOURCES file with the OS version, that points to the release tree from whence the image came
21:31:31 * gregdek hopes zodbot is getting all this... :/
21:31:39 <huff> Oxf13: is that on the image?
21:31:40 <Oxf13> also, to simplify, for now OS images will only come from GOLD releases, they will not include updates
21:31:40 <gholms> gregdek: http://meetbot.fedoraproject.org/fedora-cloud/2010-02-04/fedora-cloud.2010-02-04-20.02.log.txt
21:31:55 <Oxf13> huff: with the live torrents, we've placed teh file in the torrent, along side the iso
21:32:00 <gholms> We can let people update them after launch if need be.
21:32:08 <Oxf13> huff: I'm not exactly sure where we'd place it on EC2, implementation detail
21:32:13 <Oxf13> gholms: yep!
21:32:30 <mattdm> can we make them update automatically on first boot?
21:32:33 <Oxf13> #info to simplify the initial offering, images will only come from GOLD releases, they will not include updates.
21:32:34 <jforbes> That's the problem with kernel, people cannot update it after launch, so we need to handle security
21:32:48 * gholms nods
21:32:52 <ewan> IIRC S3 buckets can be directly web-accessible, so you don't necessarily need a web server on EC2 to export the sources; just publisj the S3 URLs.
21:32:55 <gregdek> fwiw, gafton asked the pointed question "how often will you update?"  So they're looking ahead to some of this stuff.
21:33:01 <Oxf13> mattdm: we can highly recommend it, but if we don't do that for Fedora, that would mean alteration of the image, and that gets us into "Remix" land as opposed to "Fedora" land
21:33:05 <gregdek> ewan, I agree.
21:33:24 <Oxf13> and as for the kernel/initramfs
21:33:31 <mattdm> eh, we can make a package that does it and get it in fedora.
21:33:47 <jforbes> ewan: if that's the case, we can solve this ourselves, just drop the sources into a bucket for kernel updates, and include that URL in the sources file
21:33:50 <Oxf13> I think what we have is the idea of providing a README-SOURCES with the kernel/initramfs that directs people to the same place the OS image sources are found
21:34:26 <Oxf13> and we have agreed to keep the kernel and OS image in lock step
21:34:31 <Oxf13> does that sound right?
21:34:34 <gregdek> +1
21:34:35 <gholms> Ubuntu's EC2 init script downloads the instance's user-data and executes it if it begins with #!.  Something like that would make it really easy for users to auto-update if they wish.
21:34:55 <Oxf13> ok.
21:35:19 <Oxf13> #info We have the idea of providing a README-SOURCES with the kernel/initramfs that directs people to the same place the OS image sources are found
21:35:30 <jforbes> Oxf13: kind of, if we do kernel updates, we would be pointing to a different place for sources, so just make the README-SOURCES point to the gold release and the kernel s3 bucket
21:35:34 <Oxf13> #info We have agreed to keep the kernel/initramfs and OS image in lock step as to share the same sources
21:36:00 <Oxf13> #info we will have to do something different if we provide updated kernel images
21:36:49 <gregdek> Which, at some point, we will.
21:36:52 <Oxf13> alright
21:36:55 <gregdek> But not today.  :)
21:37:07 <jforbes> Why not just do it from the beginning?
21:37:10 <Oxf13> dangit I had the tickle of another point, but I've forgotten it
21:37:26 <gregdek> Hm.
21:37:27 <Oxf13> jforbes: we lack the tools to proprely discover what binaries in the initramfs came from which srpms
21:37:27 <jforbes> When we put the first kernel there, include sources in the kernel S3 bucket?
21:37:33 <gregdek> Ah, right.
21:37:38 <Oxf13> the kernel is easy, the initramfs is much harder
21:37:51 <gregdek> Oxf13, can we document the need for that tool?
21:37:59 <jforbes> Oxf13: no, OS image comes from gold, all except for kernel.  We create the initrd on a gold image
21:38:29 <Oxf13> #info in the future, to provide updated kernel/initramfs images with matching kernels, we'll need a tool to discover which initramfs binaries came from which sources
21:38:38 <Oxf13> jforbes: that's not going to work if we have to fix a bug in something that's in the initramfs
21:38:39 <mattdm> I think self-updating-by-default is an extremely important goal, even if we can't do it from the start. It's very different from the typical fedora desktop install.
21:38:51 <mattdm> But I'm okay with whatever decision we need to make now to get going.
21:38:57 <jforbes> Oxf13: those should be release blockers, and not make gold in the future
21:39:08 <Oxf13> jforbes: and I'd like a fully functioning crystal ball (:
21:39:24 <Oxf13> riding on a pony
21:39:33 <Oxf13> farting a rainbow
21:39:44 <gholms> ...
21:39:48 <jforbes> mattdm: self updating by default is difficult, because users have to pay for bandwidth used to update unless we host an updates server there (Ubuntu does)
21:40:11 <gregdek> We can negotiate that, I think.
21:40:14 <gregdek> And should.
21:40:20 <mattdm> we should definitely host an updates server, then. that's important.
21:40:23 <gregdek> In fact, I think it's already come up.
21:40:27 <gholms> How hard would it be to host three mirrors there?
21:40:36 <jforbes> gregdek: I agree.  But until we can do that, we cannot enforce self updating AMIs
21:40:44 <gregdek> Yeah.  One step at a time.
21:40:46 <mattdm> fair enough.
21:40:52 <gregdek> But it's an intention we should make clear asap.
21:40:58 <Oxf13> ok, I think I'm done with the GPL compliance issue
21:41:02 <Oxf13> and I'm ready to move on
21:41:16 <Oxf13> unless anybody else has something to say about it
21:41:16 <gholms> Do we need to talk about init scripts and the like?
21:41:16 <gregdek> How do I denote an action item?
21:41:22 <Oxf13> gregdek: #action
21:41:22 <gholms> gregdek: #action
21:41:36 <Oxf13> gholms: yeah, I think the next topic would be "running the image"
21:41:48 <gregdek> #action ask amazon to host srpms on s3 for fedora akis
21:42:02 <Oxf13> gregdek: gotta assign it to somebody
21:42:03 <gregdek> #action ask amazon to host update servers for fedora
21:42:10 <gregdek> Oh, they're both me
21:42:12 <Oxf13> well, you don't, but.
21:42:18 <gregdek> I'll know it from the notes.  :)
21:42:33 <huff> i have a feeling there not gettign recorded anyhow
21:42:39 <gregdek> huff, me too.
21:42:42 <Oxf13> they are
21:42:50 <gregdek> I may be trolling the log -- at least I know the log file is writing.
21:42:54 <Oxf13> we've confirmed multiple times that zodbot is catching this
21:42:58 <gregdek> And this will all go into a wiki anyway.
21:43:06 <Oxf13> ok, #topic Running the Image
21:43:08 <Oxf13> er
21:43:11 <gholms> #topic Running the Image
21:43:12 <Oxf13> #topic Running the Image
21:43:17 <gholms> (sorry)
21:43:26 <huff> #topic Running the Image
21:43:27 <Oxf13> Lets assume we've got the stuff uploaded, sorted out source issues, and a user is ready to go
21:43:30 <gregdek> #topic Running the image
21:43:34 <gregdek> Ah HAH!
21:43:34 <gholms> :D
21:43:42 <gregdek> You have to be /op to tell zodbot what to do, I think.
21:43:45 <gregdek> Grr.
21:43:51 <Oxf13> gregdek: you must be the meeting chairperson
21:43:51 <gregdek> Which means it's lost most of this stuff.
21:43:56 <Oxf13> only the chair can update the topic
21:44:00 <gregdek> Ah, ok.
21:44:03 <Oxf13> but I think it'll catch #info from other people
21:44:05 <gholms> That reminds me, no one set the meeting title.
21:44:10 <gregdek> Well, sorry for discovering that late.  :)
21:44:21 <Oxf13> gregdek: can you do #chair Oxf13  ?
21:44:50 <gregdek> #chair Oxf13
21:44:50 <zodbot> Current chairs: Oxf13 gregdek poelcat
21:44:50 <Oxf13> well anyway
21:44:59 <gregdek> Kinda late now.  ;)
21:45:05 <Oxf13> yep, but oh well
21:45:25 <Oxf13> What happens when a user decides they want to run Fedora in EC2?
21:45:43 <jforbes> Oxf13: it runs?
21:45:52 <huff> Oxf13: you do ec2-run #ami number and pass it an ssh key
21:46:02 <jforbes> Oxf13: There are a couple of interfaces, users can select a public image, and just run it
21:46:04 <Oxf13> how do they pick it, what is displayed to them, what happens when it boots for the first time?
21:46:10 <gholms> The user could also pull up the AWS Console and look in the list of Fedora AMIs.
21:46:25 <gregdek> Oxf13, huge lists, visible by api search tool or web search on ec2 web ui.
21:46:45 <jforbes> Oxf13: every time it boots is the first time unless they are using persistent storage, which is a new option, and I am unfamiliar with
21:46:49 <gholms> After booting the user gets an IP address that he or she can SSH to with an SSH key he or she created beforehand.
21:47:02 * mattdm has to go. thanks everyone.
21:47:10 <huff> the image has to grab that ssh key
21:47:13 <gholms> (We need to write an init script to fetch the public keys)
21:47:18 <gregdek> Oxf13, it may be useful for you to just go and do it.  I can send you a great jumpstart document and our x509 cert (which i'll send gpg armored).
21:47:19 <huff> in the past i did this in rc.local
21:47:29 <huff> but an init script would be bettter
21:47:42 <gholms> Maybe a ec2-config package with an init script?
21:48:05 <huff> we also wrote a cheap first boot utility
21:48:06 <Oxf13> ok, so we somehow need to get the user's ssh key stuffed into the system, into root's authorized_keys file?
21:48:21 <jforbes> That is one of the things that amiconfig did
21:48:22 <Oxf13> does the root account have a password by default, or is that something we do in the image creation?
21:48:24 <gholms> Oxf13: Yep
21:48:31 <Oxf13> jforbes: but we're not using amiconfig are we?
21:48:35 <jforbes> Oxf13: root account has an ssh key
21:48:38 <huff> root has no pw
21:48:40 <jforbes> Oxf13: we can, it is open source
21:48:48 <gregdek> Oxf13, that's the user's business.  They set it up using the apis or the ec2 web ui.
21:48:49 <gholms> Typically one completely disables SSH password auth.
21:48:51 <jforbes> Oxf13: that is the rPath tool
21:48:52 <Oxf13> ok..  lets try to gather some of this.
21:49:07 <Oxf13> #info Every boot is the "first boot" without use of persistent storage
21:49:12 <gregdek> Yep.
21:49:19 <gregdek> Which is where EBS comes in.
21:49:21 <Oxf13> #info User's SSH key needs to get into the system's root authorized_keys file
21:49:25 <gregdek> EBS = Elastic Block Storage
21:49:48 <gholms> EBS lets a user create persistent disks.  You can boot from these.
21:49:55 <Oxf13> is getting the key implanted the job of a outside of the image tool, or something that has to go inside the image?
21:50:00 <gregdek> Oxf13, we're providing a raw image.  The user configures that image thru tools themselves.
21:50:07 * gregdek goes to find a good walkthru...
21:50:14 <gregdek> It's better if you just read.
21:50:15 <gholms> Oxf13: That has to go in the image.
21:50:17 <huff> Oxf13: in the image
21:50:36 <smoser> gholms, way above, the kernels and ramdisks are available just like any other ubuntu build
21:50:44 <smoser> and as such, source is available that way
21:51:06 <gholms> smoser: How long do the sources stick around?
21:51:13 <gholms> A user could modify the non-persistent image we provide and give his or her own copy persistence using EBS.
21:51:25 <smoser> i'm not sure what the normal ubuntu policy is, but "quite a while"
21:51:35 <jforbes> That is an interesting sticking point here
21:51:36 <smoser> definitely for any supported release you can get it
21:51:50 <jforbes> Images are tied to a kernel/ramdisk by their config
21:51:53 <gholms> smoser: Then you just reference it in a readme file somewhere?
21:52:15 <smoser> i really dont' know. its just no difference
21:52:16 <Oxf13> #info the tool to stuff user's ssh key goes /inside/ the image
21:52:17 <jforbes> Once our kernels are public, we won't know what images will reference them.  We need a policy on how long a kernel will stick around
21:52:19 <smoser> from everything eles
21:52:41 <gholms> jforbes: How long do we support gold releases?
21:52:43 <gregdek> http://paulstamatiou.com/how-to-getting-started-with-amazon-ec2
21:52:56 <jforbes> gholms: just over a year typically
21:53:04 <gregdek> This is a great overview for n00bs on how a user starts their own image.  Mustread, I think.
21:53:15 <gholms> Mmm... mustard...
21:53:22 <smooge> sorry for the meeting
21:53:24 <gholms> jforbes: I mean, do we make people update before offering them help?
21:53:28 <smooge> missing th emeeing
21:53:32 <gregdek> smooge, np.
21:54:01 <jforbes> gholms: AFAIK, if we delete a kernel image, a users AMI which references it will no longer boot...
21:54:40 <Oxf13> that's going to suck
21:54:42 <gholms> If we support the kernel that ships with releases for the lifetime of the release then we could stick with the two-releases-plus-a-month rule.
21:54:54 <gholms> We could also just leave old kernels up there.
21:54:55 <Oxf13> right, that's a 13 month life span
21:55:18 <Oxf13> after 13 months, we don't guarantee that the URLS used to access the source remain valid
21:55:26 <jforbes> gholms: right, both are options, though I prefer to not leave ancient kernels there
21:55:32 <gholms> Heck, Amazon still hosts FC4 kernels.
21:55:36 * jgreguske bails for another meeting, and ceases active reading
21:55:51 <Oxf13> that's darn near criminal
21:55:55 * gholms nods
21:56:26 <gholms> I think the question here is whether or not we want to break everyone's EBS images after a period of time, and if so, how long.
21:56:35 <Oxf13> ok, it seems to me that we'll need to create and package whatever mechanism we want to use to handle the ssh key
21:56:40 <Oxf13> and make sure we include that in the image
21:56:52 <gregdek> Um... that's part of euca2ools, right?
21:56:58 <Oxf13> do we have something like that written somewhere?
21:57:08 <gholms> Oxf13: appliance-tools has a short script that does that.
21:57:22 <huff> we do it in rclocal
21:57:36 <huff> i think
21:57:43 * gregdek waitwaitwait.
21:57:50 <gregdek> In order to ssh to an ec2 box...
21:57:52 <Oxf13> if it's a interpreted script that'sfine
21:57:59 <jforbes> rclocal isn't of value here
21:57:59 <gregdek> ...you must create a keypair.
21:57:59 <Oxf13> since that is source itself.
21:58:42 <huff> gregdek: yep an when you launch the image you pass it the the name
21:58:43 <gholms> Oxf13: Here's a script that will do it:  http://fpaste.org/LOyT/
21:58:51 <gregdek> OK.  Got it.
21:58:58 <gholms> By "it" I mean download public keys.
21:59:14 <gregdek> Wait, public keys?
21:59:25 <Oxf13> is that a magic IP address there?
21:59:29 <gholms> The instance has to download your public keys so you can log in.
21:59:35 <huff> Oxf13: yea
21:59:36 <gholms> Oxf13: It is.
21:59:51 <gholms> Oxf13: People on Eucalyptus usually have to change that.
21:59:58 <gregdek> We *are* assuming creation of a new keypair at time of "user decides to pick image foo", yes?
22:00:18 <gholms> gregdek: We shouldn't.
22:00:19 <huff> gregdek: you have to have one assiocated with your aws account
22:00:41 <gholms> That would mean I would have to create a new key for every instance type I want to launch.
22:00:53 <huff> no you can use the same one for multiple
22:00:54 <gregdek> Ah, right.  User creates keypair as part of account for all machines, ok.
22:00:59 <gregdek> Nevermind.  :)
22:01:16 <Oxf13> ok
22:01:20 <gregdek> You just specify which of *your* keypairs you use when you instantiate an image.
22:01:30 <gholms> We can leave key creation and specification up to the web UI and euca2ools.  All the image has to care about is downloading them.
22:01:33 <Oxf13> Then they're up and running and everything is great
22:01:52 <Oxf13> which leads us to....
22:01:55 <Oxf13> #topic Updates
22:02:07 <Oxf13> this goes back to gregdek's action plan of getting a Fedora mirror in S3
22:02:21 <Oxf13> if you get that, we need to make sure S3 becomes a mirrormanager site default mirror
22:02:27 <gregdek> Yep.
22:02:35 <gregdek> This has two pieces: tech and admin.
22:02:37 <Oxf13> gregdek: would you want to take that on as well?  It's really simple.
22:02:43 <gregdek> yeah, I can do that.
22:02:50 <Oxf13> (and by take that on I mean file the ticket for mdomsch to do it)
22:02:59 <gregdek> LOLyupyup
22:02:59 <Oxf13> #action gregdek will handle getting S3 setup in MirrorManager
22:03:03 <gholms> It's simple as long as we know what all of EC2's IPs are.
22:03:15 <Oxf13> gholms: right, "simple" (:
22:03:33 <gholms> There's also the matter of differing internal and external IP addresses.
22:03:33 <gregdek> I suspect there will be "complications", so it may take a while to work through Amazon's process underbelly.  But I'll hase it.
22:03:35 <Oxf13> #info some would like images to autoupdate, perhaps by force
22:03:36 <gregdek> chase it.
22:03:52 <gregdek> I dunno about forcing updates on users.
22:03:55 <Oxf13> I had read earlier about loading user data in and executing any scripts found
22:03:58 <gregdek> I think that's kinda not what cloud is about.
22:04:12 <Oxf13> Can somebody expand upon that?  Is there a way for us to provide reference scripts to do the updating, without forcing it?
22:04:21 <Oxf13> gregdek: I'm with you on that, I just wanted it noted for the record
22:04:25 <gregdek> ok :)
22:04:30 <gholms> A user can specify a "user data" file/string.  Ubuntu's init script executes that on bootup if it begins with #!.
22:04:43 * jforbes doesn't like autoupdate either, these are servers
22:04:45 <Oxf13> gholms: ok, so that's specific to Ubuntu
22:05:00 <gholms> So if a user hands euca-run-instance a file with "#!/bin/sh \n yum -y upgrade" that will accomplish it.
22:05:15 <gregdek> Oooh, cool.
22:05:17 <Oxf13> provided we had something in the image that knows to execute the incoming files?
22:05:17 <jforbes> gholms: that works
22:05:23 <gholms> But we would have to write that into the init script.
22:05:29 <Oxf13> what is "the init script" ?
22:05:49 <gholms> An init script that I presume we would have to write alongside the ssh key script.
22:05:58 <gregdek> So I'm guessing that some of these things will start to become obvious as we start doin' stuff.
22:06:09 <gregdek> And we're at the 2 hour mark.
22:06:12 <Oxf13> yep
22:06:16 <gregdek> Can we get to the "next meeting" topic?
22:06:20 <Oxf13> I think we made a bunch of progress today
22:06:31 <gregdek> Yepyep!
22:06:34 <Oxf13> and I think gregdek is right, there is only so far we can go without having images in hand to walk through it
22:06:47 <Oxf13> #topic Next Meeting?
22:07:01 <Oxf13> other than the infrastructure meeting conflict, this day worked for me
22:07:10 <Oxf13> should we try one hour later next week?
22:07:14 <gregdek> Is 4pm eastern better for folks?  I know that infrastructure would prefer it, and having those dudes available will be really helpful.
22:07:31 <Oxf13> it's fine by me
22:07:33 <gregdek> I know that's 10pm for CET...
22:07:39 <jforbes> Works for me
22:07:46 * gholms has to leave at 6pm EST on Thursdays
22:08:00 <gregdek> All the more reason to keep the meetings to an hour.  :)
22:08:01 <Oxf13> hopefully we won't be doing 2 hour meetings every week
22:08:02 <gholms> Otherwise that works for me.
22:08:03 <gregdek> Is weekly good for now?
22:08:11 <Oxf13> i think so, as things are moving fast
22:08:14 <gholms> Sure.
22:08:16 <gregdek> okeydoke.
22:08:31 <jforbes> Yeah, I think weekly is a good idea until things are a bit more developed
22:08:35 <gholms> Should I post that init script somewhere and #link to it before we're through here?
22:08:40 <gregdek> yespls
22:08:51 <gholms> (Of course now it'll be under the wrong topic)
22:09:04 <gregdek> meh, that's ok.
22:09:10 <gregdek> It's all fodder for a wiki page anyway.  :)
22:09:22 <jforbes> gregdek: one more bit, I am going to do the virt status update, and I thought I would point people to the cloud-sig since there is a lot of cross interest there.  Do we have a wiki page yet, or where should I point people?
22:10:01 <gregdek> jforbes, one sec...
22:10:32 <gholms> #link http://www.physics.umn.edu/~holms/download-sshkeys
22:10:36 <gregdek> https://fedoraproject.org/wiki/Cloud_SIG
22:10:38 * gholms hopes that worked
22:10:41 <gregdek> I will create this page now.  :)
22:10:50 <jforbes> gregdek: thanks :)
22:11:02 <Oxf13> zodbot will pick up links without #link
22:11:11 <gholms> Good to know...
22:11:28 <gregdek> OK, send them there.  It will have content soon.
22:11:34 <gholms> Anything else?
22:11:39 <Oxf13> Everybody with action items should be prepared to report on them next week
22:11:46 <gregdek> #action gregdek will update wiki page with recap, mailing list info, irc info, etc.
22:11:54 <gholms> I should have euca2ools pushed by then.
22:12:27 <gregdek> So we're done?
22:12:43 <gholms> There's also a public Eucalyptus instance people can use for testing if anyone's interested.
22:12:48 <gregdek> gholms, link?
22:13:05 <huff> he posted it
22:13:06 <gholms> https://mayhem9.cs.ucsb.edu:8443/
22:13:08 <gregdek> ah ok
22:13:10 <gregdek> :)
22:13:19 <gregdek> How do we call this meeting over?  :)
22:13:25 <gholms> #endmeeting
22:13:40 <gregdek> Anyone have anything else?
22:13:44 <gregdek> Meeting over in 5...
22:13:47 <gregdek> 4...
22:13:49 <gregdek> 3...
22:13:52 <gregdek> 2...
22:13:55 <gregdek> 1...
22:13:56 <gholms> 0.5!
22:13:59 <gregdek> 1/2...
22:14:01 <gregdek> :)
22:14:03 <gregdek> #endmeeting