20:00:27 <pwhalen> #startmeeting --> starts the meeting and zdbot
20:00:27 <zodbot> Meeting started Wed Jul 18 20:00:27 2012 UTC.  The chair is pwhalen. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:27 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:00:27 <pwhalen> #chair pwhalen jonmasters bconoboy ctyler pbrobinson dgilmore
20:00:27 <zodbot> Current chairs: bconoboy ctyler dgilmore jonmasters pbrobinson pwhalen
20:00:39 <ctyler> .fas ctyler
20:00:39 <zodbot> ctyler: ctyler2621 'Christopher Tyler' <chris@mowisp.com> - ctyler 'Chris Tyler' <chris@tylers.info>
20:00:42 <pwhalen> .fas pwhalen
20:00:42 <zodbot> pwhalen: pwhalen 'Paul Whalen' <pwhalen@redhat.com>
20:00:48 <fossjon> .fas fossjon
20:00:48 <zodbot> fossjon: fossjon 'Jon' <jonc_mailbox@yahoo.ca>
20:00:53 <dmarlin> .fas dmarlin
20:00:54 <zodbot> dmarlin: dmarlin 'David A. Marlin' <dmarlin@redhat.com>
20:00:59 <amaxm> .fas maxam@
20:00:59 <zodbot> amaxm: 'maxam@' Not Found!
20:01:04 <amaxm> .fas maxam
20:01:04 <zodbot> amaxm: maxam '' <masihul.abed@senecacollege.ca> - maxamaxim '' <maxamaxim@me.com> - maxamillion 'Adam Miller' <admiller@redhat.com>
20:01:06 <bconoboy> .fas blc@
20:01:07 <scientes> .fas scientes
20:01:08 <zodbot> bconoboy: blc '' <blc@redhat.com>
20:01:10 <zodbot> scientes: 'scientes' Not Found!
20:01:41 <agreene> .fas agreene
20:01:41 <zodbot> agreene: agreene 'Andrew Greene' <agreene@learn.senecac.on.ca> - tag4fedora 'Tim Greene' <tagreene@flowserve.com>
20:02:04 <pwhalen> #topic 1) Builders on F17 - status update
20:02:17 <Frojoe> Alright
20:02:35 <jonmasters> .fas jonmasters
20:02:35 <zodbot> jonmasters: jcm 'Jon Masters' <jonathan@jonmasters.org>
20:02:47 <Frojoe> So thanks to pwhalen's help yesterday, we have two test f17 builders up and running in the farm
20:02:57 <bconoboy> farm?
20:03:06 <ctyler> buildsystem
20:03:10 <Frojoe> Sorry, buildsystem
20:03:14 <bconoboy> whew, ok
20:03:15 <jonmasters> mooo
20:03:19 * bconoboy has a farm
20:03:24 <jonmasters> eieio
20:03:28 <Frojoe> v___v
20:03:29 <ctyler> farm=[f]edora [arm]
20:03:31 <fossjon> haha
20:03:37 <dmarlin> build farm
20:03:42 <Frojoe> Yes, build farm
20:03:43 <jonmasters> (Enforce In-Order Execution of IO - my favorite IBM POWER instruction)
20:04:01 <fossjon> wow
20:04:13 <Frojoe> Anyways, the performance seems reasonably in line with the f15 builders, and there hasn't been any stability issues
20:04:14 <jonmasters> anyway, yay builders!
20:04:23 <ctyler> Frojoe: which hostnames?
20:04:23 * djdelorie is not here ;-)
20:04:28 <bconoboy> frojoe: are those pandas?
20:04:36 * jonmasters suggests leaving the remaining builders on F15 until after the mass rebuild
20:04:42 <Frojoe> cdot-panda-7-2 and cdot-panda-12-3 are currently f17
20:04:46 <ctyler> jonmasters: we will be
20:04:50 <jonmasters> #idea suggests leaving the remaining builders on F15 until after the mass rebuild
20:05:00 <jonmasters> ctyler: just saying it for the record, so it's noted down
20:05:03 <Frojoe> We were discussing the possibility of moving some of the gurus to f17 as well, but apparently there are some kernel issues
20:05:11 <Frojoe> I haven't had a chance to look into that yet
20:05:20 <ctyler> trimslices?
20:05:55 <Frojoe> Also, I've done some additional work on our BCFG2 schema so that we can just drop basic f17 rootfses into the farm and have them fully configured in about 3-4 minutes
20:05:57 <bconoboy> I think it's okay to take 14-1 off nightly building duty and have it do builds.
20:06:03 <amaxm> ctyler: we have two ... one is the arm-nightly compose
20:06:20 <bconoboy> is 13-1 being used for anything? if not please put it back into koji
20:06:21 <amaxm> the other one will be back up shortly
20:06:29 <Frojoe> If the trimslice rootfs is fine, then we can throw those into the farm as well
20:06:31 <amaxm> bconoboy: it will be
20:06:45 <bconoboy> amaxm: great.  do the same with 14-1?
20:07:25 <amaxm> bconoboy: awesome!
20:07:29 <dgilmore> there is no reason not to have the trimslices running f17
20:08:02 <bconoboy> dgilmore: it's 2 commands to get 14-1 back as a f15 builder, no need to reinstall
20:08:14 <Frojoe> bconoboy, like I said, the pandas seem to be stable
20:08:16 <ctyler> bconoboy: so then no nightlies?
20:08:19 <bconoboy> I wouldn't mind 13-1 going to f17 though, it has the usb reset bug in its kernel
20:08:42 <bconoboy> ctyler: We can call F17 nightlies cooked at this point.  The churn is pretty small now.  We can turn them abck on after the mass rebuild if there's demand.
20:09:02 <ctyler> rawhide nightlies would be useful
20:09:07 <jonmasters> #agreed nightly F17 images to be discontinued in line with PA
20:09:12 <dgilmore> bconoboy: yeah, but a resinstall will make sure we have the usb patch
20:09:39 <dgilmore> ctyler: we dont make rawhide images for primary
20:09:54 * jonmasters suggests whatever is the path of least resistance to get as many builders up as possible - re-installing should be trivial, but 2 commands are even faster and less risky
20:09:55 <dgilmore> we have all the patches to anaconda slowly working their way upstream
20:09:55 <bconoboy> differnt subject, let's hold off on that
20:10:13 <dgilmore> which means we can switch to livemedia-creator
20:10:26 <dgilmore> im ok either way
20:10:38 <dgilmore> lets get all the build hardware we can up and running
20:10:54 <bconoboy> f17 builders subject: let's keep things where they are today unless we're installing or need to reinstall.  In that case we will use f17. Yes?
20:11:13 <bconoboy> We're bringing a few more heavy builders online in hsv- they'll all have f17 on them
20:11:20 <bconoboy> well, nearly all
20:11:37 <bconoboy> It'd be good to have a copy of the panda image cdot is using.  If we need to reinstall hsv pandas we can use it.
20:11:56 <dgilmore> bconoboy: sure
20:12:24 <ctyler> Frojoe: please share that image ^
20:12:27 <jonmasters> good, agreement is great
20:12:45 <bconoboy> pwhalen: I think we're all in agreement here.  DO we need a summary or shall we move on?
20:13:09 <Frojoe> ctyler, can do
20:13:11 <pwhalen> #topic 2) F18 Mass rebuild
20:13:33 <bconoboy> pbrobinson/dgilmore: What's the plan?
20:13:42 <bconoboy> (where is pbrobinson?)
20:14:08 <misc> (in board meeting)
20:14:14 <bconoboy> ah!
20:14:20 <bconoboy> dgilmore: what's the plan? :-)
20:14:33 <misc> (discussing on SB, so expect to wait a bit :) )
20:14:52 <fossjon> secret business
20:15:19 <fossjon> secure boot
20:15:33 <scientes> why does that require a mass rebuild?
20:15:38 <bconoboy> it doesn't
20:15:43 <bconoboy> f18 is doing a mass rebuild, we must follow
20:15:46 <ctyler> for a rebuild, do we need to shadow it, or just run through the list? I don't see build dependencies being an issue this time.
20:15:59 <ctyler> a straight run through could be faster
20:16:08 <bconoboy> Okay, absent the koji-shadow masters, let's talk a bit more about build hosts:
20:16:19 <ctyler> build sequencing, I meant
20:16:42 <jonmasters> ctyler: suggest deferring to pbrobinson when he is around later on koji-shadow vs. manual builds, since he'll be doing it
20:16:43 <jonmasters> :)
20:16:49 <bconoboy> How are the cdot builders doing? Is everything online that can be?
20:17:19 <pwhalen> a couple hosts need reboots
20:17:38 <ctyler> bconoboy: pretty much. we can bring some additional smarttops up, but not sure of the value of that. Our heaviest builders are tasked with other stuff at the moment because we anticipated the rebuild starting on the 30th.
20:17:39 <dgilmore> the plan is to follow along the f18-rebuild tag on arm
20:17:53 <dgilmore> we are well into submitting the over 12k builds
20:18:27 <dgilmore> ctyler: we could modify build-previous  to build--current
20:18:30 <bconoboy> ctyler: are any heavybuilders left in cdot?  I think 13-1 and 14-1 would be good choices.  pandas not so much.
20:18:32 <dgilmore> and send them that way
20:18:50 <ctyler> 13-1 and 14-1 are heavybuilders as are oscar and felix
20:18:59 <bconoboy> oscar and felix are busy or available?
20:19:03 <dgilmore> /var/lib/mock performance on the cdot pandas makes them not useful as heavybuilders
20:19:10 <ctyler> busy at the moment
20:19:20 <bconoboy> dgilmore: I *think* the cdot pandas are no longer listed as heavy builders
20:19:23 <bconoboy> can somebody confirm?
20:19:28 <dgilmore> bconoboy: :)
20:19:37 <ctyler> agreene changed the panda status this afternoon
20:19:47 <bconoboy> \o/
20:19:48 <ctyler> we might move some back since we have some local storage for some of them
20:20:08 <bconoboy> like local usb disks?
20:20:10 <ctyler> yes
20:20:14 <bconoboy> sure- those would be fine
20:20:23 <bconoboy> more or less the same as the trimslices
20:20:35 <ctyler> or more accurately, the same as the hsv pandas
20:20:47 <bconoboy> well, yeah
20:20:55 <bconoboy> which are more or less the same as trimslices disk performance wise
20:21:09 * ctyler hasn't compared them
20:21:10 <dgilmore> should be
20:21:16 <dgilmore> since panda storage is on usb also
20:21:46 <pwhalen> shall we revisit the rebuild near the end then, move on?
20:21:51 <bconoboy> please
20:21:55 <pwhalen> #topic 3) Raspberry Pi Remix update
20:22:51 <ctyler> Progress on two fronts
20:23:51 <ctyler> Ironing out the remaining compose issues. I had hoped to knock that off the list today, still hopeful to finish it.
20:24:16 <ctyler> fossjon is doing an armv6hl test rebuild to compare performance.
20:24:29 <fossjon> assuming im building these right sure
20:24:42 <amaxm> Go Linus!
20:24:43 <fossjon> almost 200 pkgs done out of our 650 requested
20:24:45 * jonmasters reports receiving his Raspi finally today. I'm sending one of the two I got to dgilmore
20:24:57 <ctyler> cool :-)
20:25:09 * djdelorie read that you can order more than one at a time now...
20:25:23 <ctyler> djdelorie: I just ordered 10
20:25:25 <jonmasters> yep
20:25:47 <jonmasters> ctyler: that means you'll have 9 new best friends soon
20:25:51 <bconoboy> ctyler: what's blocking a booting image at this point?
20:26:00 <amaxm> jonmasters: make that 5
20:26:02 <jonmasters> :)
20:26:46 <ctyler> bconoboy: The current image boots but not to a login prompt
20:27:04 <fossjon> i think it hangs on some systemd service
20:27:04 <ctyler> I just compared the package lists with an older working compose and they're nearly identical
20:27:15 <fossjon> i had to disable a bunch of them to skip past it
20:27:17 <ctyler> except that the new one has updates, so I'm now rebuilding without updates
20:28:14 <bconoboy> ctyler: would you like help?
20:28:25 <ctyler> Once we get a good compose, we need to test the firstboot and resize stuff, and then I'd like to see how it shapes up against the armv6hl build
20:28:48 <fossjon> :/ still a bit to do
20:28:51 <bconoboy> I'd like to see a remix come out before f18 :-(
20:29:04 <ctyler> bconoboy: this piece doesn't parallelize well, but as soon as we get a good compose, yes, lots of help wanted
20:29:12 * pbrobinson is sorry he's late
20:29:12 <bconoboy> ctyler: pis are no longer unobtanium. perhaps a vfad to solve issues would be appropriate.
20:30:03 <bconoboy> ctyler: I think you're missing out on a lot of community support at this stage
20:31:01 <fossjon> this is true
20:31:05 <fossjon> composing images takes a bit
20:31:08 <ctyler> bconoboy: we're really, really close to having a pick-up-and-play piece, which makes this widely accessible
20:31:24 <ctyler> and then I hope quite a few will jump in
20:31:42 <pwhalen> shall we go back to the rebuild?
20:31:43 <bconoboy> ctyler: okay. I'm just suggesting you schedule a vfad to take images for a test drive real soon.
20:32:09 <pwhalen> #topic 2) F18 Mass rebuild
20:32:36 <pwhalen> welcome pbrobinson! whats the plan?
20:32:54 <dgilmore> follow allong f18-rebuild
20:33:18 * dgilmore expects that the builds will all be submitted tonight sometime on primary
20:33:39 <dgilmore> if we just grab the build and send it to koji it will speed things up a bit
20:33:43 <dgilmore> much less newRepos
20:33:55 <dgilmore> but there will be a risk of out of order building
20:34:05 <pbrobinson> so the plan is as dgilmore says to follow f18-rebuild
20:34:24 <ctyler> dgilmore: out of order building isn't really an issue, though, is it? we don't have major soname bumps that I have heard about
20:34:28 <pbrobinson> from a core f18 build up  to now i believe we're in the best state of build we've been in for a long time
20:34:37 <dgilmore> ctyler: we are close enough it should be ok
20:34:55 <dgilmore> there is 42 x86 builders on primary
20:34:56 <pbrobinson> and I've been pouncing on core issues as quickly as possible
20:36:53 <ctyler> pbrobinson: what is the URL for viewing koji-shadow status?
20:36:54 <bconoboy> pbrobinson: Are you aware of any issues that need to be solved today?
20:37:07 <pbrobinson> so I'm hoping for a relatively straight forward mass rebuild
20:37:09 <bconoboy> (IE, before rebuilding the world)
20:37:17 <pbrobinson> without too many issues
20:37:46 <pbrobinson> bconoboy: nope, none, we're as close to mainline fedora as we've ever been
20:38:00 <bconoboy> awesome.
20:38:06 <dgilmore> how many builders do we have?
20:38:39 <bconoboy> from hsv we'll be throwing about 20 heavies in.
20:38:53 <dgilmore> bconoboy: and half sfp and half hfp?
20:38:58 <bconoboy> pretty much
20:39:22 <dgilmore> the x86 boxes are all quad core 4gb ram
20:39:36 <dgilmore> i expect that the builds will take 3-4 days
20:39:54 <dgilmore> for arm we are probbaly more likely to tak 7-10?
20:40:04 <dgilmore> though thats a random stab in the dark
20:40:04 <ctyler> that's my expectation
20:40:07 * bconoboy thinks it will be 14+
20:40:33 * pbrobinson thinks somewhere between 7 and 14
20:41:02 * bconoboy assumes hardware failure happens, plus a critical blocker comes up in gcc or something
20:41:14 * dmarlin thinks we should start a pool
20:41:19 <bconoboy> +1
20:42:01 <ctyler> but we all have some level of influence over the results, hmm...
20:42:09 <fossjon> rm -fr .....
20:42:13 <fossjon> oops
20:42:15 <fossjon> wrong channel
20:42:23 <pbrobinson> I assume we'll have some build issues on core dependencies
20:42:46 <pwhalen> #topic 4) Your topic here
20:42:51 <pbrobinson> fossjon: who's server were you about to destroy :-P
20:43:06 <fossjon> :)
20:43:11 <bconoboy> beyond mass rebuild...
20:43:17 <bconoboy> ... anything we need to get in before f18 alpha?
20:43:26 <fossjon> stale nfs locks?
20:43:27 <bconoboy> I'm thinking kickstart, kernel testing, etc
20:43:37 <fossjon> nm
20:44:00 <dgilmore> bconoboy: yeah we need to get a test tree up with the anaconda patches in and get kickstart and interactive install testing done
20:44:04 <pbrobinson> bconoboy: what's the status of the anaconda bits for custom kickstart builds?
20:44:13 <dgilmore> we need to test the 3.5 kernels
20:44:20 <pbrobinson> I think that would be a good bit to land for alpha
20:44:41 <bconoboy> dgilmore: is that something you're doing on your own or do you want help?
20:45:16 <dgilmore> bconoboy: everyone can help
20:45:23 <pbrobinson> bconoboy: dgilmore will need help as he's not taking all his ARM hardware to australia on the weekend and probably doesn't have all of it
20:45:26 <pbrobinson> 4tttt
20:45:33 * dmarlin thinks we need to "test" the 3.4 kernels
20:45:43 <dgilmore> i will only be taking 3 or so systems
20:45:49 <dgilmore> dmarlin: no
20:45:49 <pbrobinson> dmarlin: what's that got to do with f18 alpha?
20:45:56 <dgilmore> dmarlin: rawhide is 3.5 now
20:46:08 <pbrobinson> dmarlin: and it will soon be moving on to 3.6
20:46:09 <dgilmore> and f17 will be 3.5 when 3.5.1 lands upstream
20:46:22 <pbrobinson> I think we need to start looking at what is needed to support dtb
20:46:27 <dmarlin> just that we've never tested any of them before, and I think we should ... stability
20:46:38 <dgilmore> we do need to work out how to support dtb
20:46:49 <dgilmore> dmarlin: never in what way?
20:46:51 <pbrobinson> dmarlin: I've tested lots of 3.4 kernels? What have you been using?
20:47:05 <dmarlin> booting is not the same as "testing"
20:47:32 <dgilmore> dmarlin: 3.4 will soon be irrelevant
20:47:34 <dmarlin> what test suites have you run
20:47:34 <bconoboy> if 3.5 is going to hit f17 we shoudl definitely start using it right away
20:47:37 <pbrobinson> dmarlin: I test all sorts of bits other than just booting, I have no idea what your test plan is ;-)
20:47:52 <dmarlin> right, no idea... we need one
20:48:13 <dgilmore> f17 will likely also get 3.6 down the road
20:48:28 <pbrobinson> dmarlin: ultimately testing kernels depends on the HW and how people use it, what usb devices they plug in, if they use GPIO etc.
20:48:51 <pbrobinson> dgilmore: lets not get ahead of ourselves too much :)
20:48:56 <dmarlin> pbrobinson: seems random and haphazard... not a real plan
20:49:31 <dmarlin> but I fully endorse "testing" kernels
20:49:37 <dgilmore> dmarlin: http://fedoraproject.org/wiki/Kernel
20:50:03 <pbrobinson> dmarlin: yes, but so is a lot of mainline kernel testing as well
20:50:19 <dmarlin> pbrobinson: that's discouraging
20:50:44 <pbrobinson> because it's impossible to test all HW combinations so you aim for best effort and fix when people report issues with their special combination
20:51:14 <pbrobinson> it's likely easier on ARM because the SoC/RAM and most peripherals are fixed
20:51:25 <ctyler> pbrobinson: but there's a lot more to the kernel than HW. There's crypto, and selinux, and network stack, ... which can be tested regardless of your HW.
20:51:37 <dmarlin> ctyler: +1
20:51:42 <dgilmore> we should be doing testing
20:51:52 <dgilmore> but we should focus on testing whats coming
20:51:59 <bconoboy> indeed, we need more testing if we're going to make it to PA
20:52:06 <dgilmore> because we move so fast what we have today will be updated soon
20:52:10 <pbrobinson> ctyler: agreed but ultimately network and other components generally depend on the behaviour of the HW
20:52:11 <bconoboy> anaconda kickstarting ties into that
20:52:24 <ctyler> dgilmore: if we automate it, we can run the tests against whatever comes down the pipe
20:52:34 <ctyler> or even if we write good manual test plans
20:54:11 <dgilmore> ctyler: i dont disagree
20:54:45 <bconoboy> perhaps next week when we're twiddling our thumbs as the mass rebuild is going we can discuss PA strategy, pi strategy, etc?
20:55:06 <ctyler> Question on kernels from a meeting yesterday: would we be open to supporting an OLPC target in our kernel RPM? # This can wait until thumbtwiddle time too
20:55:16 * dmarlin does not anticipate much twiddling
20:55:24 * ctyler neither
20:55:42 <pwhalen> autotest has some good kernel tests, I'll look into those as an option for us
20:55:47 * bconoboy is being facetious
20:55:49 <pbrobinson> what was the state of anaconda and the ability to use media-creator to build images?
20:55:54 <dmarlin> pwhalen: +1
20:56:15 <dmarlin> pbrobinson: upstream, local testing?
20:56:51 <dmarlin> pbrobinson: I am using livemedia-creator for trimslice images on a regular basis... using what is in xpfa
20:56:57 <pbrobinson> dmarlin: to produce nightly builds and for the average joe user that wants to produce their own hadoop build
20:57:03 <dmarlin> pbrobinson: just building things to try panda
20:57:22 <dmarlin> pbrobinson: have tested highbank as well
20:57:55 <dmarlin> pbrobinson: have also performed highbank kickstart installs
20:58:06 <dmarlin> wiki pages available
20:58:11 * pbrobinson would like to see the move of nightly image builds to anaconda / media-creator in time for alpha so we start to dogfood it
20:58:19 <dmarlin> pbrobinson: we are actively working to push these upsteam
20:58:38 <pwhalen> I'm going to test the panda ks dgilmore added to git with the newly built packages from dmarlin
20:58:39 <pbrobinson> is there a git branch?
20:58:48 <dmarlin> pbrobinson: agreed, but some platforms are more challenging than others
20:59:02 <dgilmore> pbrobinson: there is for kickstarts
20:59:08 <pbrobinson> I don't doubt it!
20:59:19 <dgilmore> pbrobinson: probably should make one on fedorapeople.org for anaconda patches
20:59:23 <dmarlin> pbrobinson: only locally (internal git tree)
20:59:46 <ctyler> open ftw!
21:00:20 <dmarlin> pbrobinson: all the patches have been submitted to the anaconda list (and in the archives)
21:00:41 <pwhalen> we're coming up on the hour, any last thoughts or important items?
21:00:41 <dmarlin> pbrobinson: and everything that has been tested (known good) are in xpfa, including sources
21:01:01 <pbrobinson> ultimately i think it would be a good idea to have some public git for the few people that are interested in testing, you might even get patches to fix other random devices
21:01:12 <pbrobinson> what is xpfa?
21:01:46 <dmarlin> pb the lookaside repo for fedora arm packges that are not yet in fedora... same one I've been posting for a year or so
21:01:47 <dgilmore> dmarlins repo for testing arm packages
21:02:11 <pwhalen> pbrobinson, http://fedoraproject.org/wiki/Architectures/ARM/Installer
21:02:24 <pwhalen> shall we wrap?
21:02:41 <dmarlin> pbrobinson: also see: http://fedoraproject.org/wiki/Architectures/ARM/Anaconda
21:02:42 <pwhalen> #endmeeting