15:59:59 <adamw> #startmeeting 2009-09-21 Fedora QA meeting
15:59:59 <zodbot> Meeting started Mon Sep 21 15:59:59 2009 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:59:59 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:05 <adamw> #topic gathering people
16:00:09 <adamw> QA meeting time, folks: who's here?
16:01:00 * kparal is here
16:01:03 <adamw> wwoods: dpravec: kparal: oxf13: ping
16:01:11 <adamw> poelcat: ping
16:01:14 <dpravec> pong
16:01:55 <adamw> hmm, must be a case of the mondays :)
16:02:15 <adamw> well, let's get going and see what happens
16:02:31 <adamw> #topic last meeting follow up
16:02:51 <adamw> the only real follow-up topic that I see is zsync
16:03:34 <adamw> we tried to get the packaging problem kickstarted, but it doesn't appear to really be going anywhere yet. the -devel-list discussion thread is here, for reference:
16:03:36 <adamw> #link https://www.redhat.com/archives/fedora-devel-list/2009-September/msg00525.html
16:03:55 <adamw> kparal, nirik, anything else to add?
16:04:34 <kparal> unfortunatelly i have skipped this conversation somehow, so i have to first read it
16:04:45 <kparal> which means nothing to add atm
16:04:55 <adamw> alright
16:05:05 <adamw> anyone want to follow up on anything else from last week's meeting?
16:05:57 * nirik doesn't have anything to add there. Someone needs to start talking to various upstreams...
16:07:17 <Oxf13> adamw: pong, but I'm at LinuxCon
16:07:30 <adamw> Oxf13: ah, ok.
16:08:02 * jeff_hann here
16:08:08 <adamw> hi, jeff.
16:08:25 <adamw> well, next topic on the agenda is:
16:08:28 <adamw> #topic autoqa update
16:08:31 <adamw> wwoods: around yet?
16:09:31 <adamw> hmm, i guess we can come back to it
16:09:50 <adamw> so, onto:
16:09:53 <dpravec> heh i think we need james to be back soon
16:10:15 <adamw> dpravec: meanie! :)
16:10:26 <dpravec> adamw:  :)
16:10:33 <adamw> then, onto:
16:10:37 <adamw> #topic upcoming events
16:10:49 <adamw> so we're coming up on beta release reasonably soon now
16:11:02 <adamw> we have the test compose process starting up this week and a beta blocker review on fridauy
16:11:22 <adamw> obviously we need to get as much testing done on the test composes, particularly installation, as we can
16:11:43 <adamw> there's quite a few beta blocker bugs in MODIFIED status that need fix confirmation, so if people can help out with that it'd be great.
16:12:06 <jeff_hann> adamw: link please
16:12:11 <adamw> Jeff_S: sec
16:12:13 <adamw> grr
16:12:16 <adamw> jeff_hann: a sec
16:12:23 <jeff_hann> :) ok, no prob
16:12:36 <kparal> adamw: should i tag bugs recently reported against anaconda with the f12blocker keyword?
16:12:42 <dpravec> adamw:  not drectly related to this, but i was talking to anaconda guys... and they told me that they need to have stable api 14 days before each freeze/release.
16:12:45 <adamw> #link https://bugzilla.redhat.com/showdependencytree.cgi?id=507678&hide_resolved=1
16:12:52 <kparal> meaning my own bugs
16:13:02 <adamw> kparal: tag bugs as blockers that you think are blockers :)
16:13:09 <adamw> f12blocker is the *final release* blocker
16:13:10 <dpravec> do you think we can do something for them to make the life easier for anaconda devels?
16:13:18 <adamw> if you think it should block the beta release, use f12beta
16:13:31 <adamw> dpravec: what do you mean by 'have stable api'? they're the ones defining the API...
16:13:40 <dpravec> they are using other tools
16:13:49 <adamw> ah, i see.
16:14:28 <adamw> that's more of a devel discussion, i think, freezing everything that anaconda depends on two weeks before every pre-release would be a fairly major change
16:14:39 <adamw> but of course for the final release there's already a general freeze in place at that point
16:14:44 <dpravec> not freezing, just api freeze
16:15:03 <adamw> still...but yeah, that could be practical. um.
16:15:22 <adamw> Oxf13: what do you think of that?
16:15:45 <dpravec> so they could use comandline tools for managing for example lvm
16:16:07 <Oxf13> adamw: that really falls into the critical path stuff
16:16:08 <Oxf13> and autoqa
16:16:20 <Oxf13> freezing without testing doesn't help nearly as much as just testing
16:17:17 <kparal> that's true
16:17:25 <adamw> i think we need jlaska back =)
16:17:38 <adamw> denise: are you around? any input on this from the anaconda team POV?
16:17:59 <denise> reading back ...
16:18:12 <adamw> denise: key quote: "<dpravec> adamw:  not drectly related to this, but i was talking to anaconda guys... and they told me that they need to have stable api 14 days before each freeze/release."
16:18:28 <dpravec> maybe before release
16:19:08 <denise> even if people just stopped moving libraries around!
16:19:20 <dpravec> they explained to me that when other packages will upload changes in last day before freeze, anaconda will have big problem
16:19:24 <adamw> jeff_hann: did you get the link? as you can see, there's currently four bugs marked as f12beta and modified, they're all installer issues
16:19:41 <dpravec> i mean those system packages anaconda cooperates with
16:20:00 <adamw> denise: this sounds like maybe more a communication issue than anything else. are the maintainers of all components that are critical to anaconda _aware_ that they're critical to anaconda? are you in regular touch with them?
16:20:04 <jeff_hann> adamw: link pleaseas,s jut looking at 522675
16:20:04 <denise> dpravec, or even if they just TOLD the anaconda team before adding changes likely to affect insta;;
16:20:07 <Oxf13> denise: seriously.  wtf nss?!
16:20:10 <denise> ;-)
16:20:14 <jeff_hann> adamw: link pleaseas,s jut looking at 522675s
16:20:15 <adamw> jeff_hann: https://bugzilla.redhat.com/showdependencytree.cgi?id=507678&hide_resolved=1
16:20:21 <jeff_hann> omg
16:20:29 <jeff_hann> sorry
16:20:42 <jeff_hann> something remained in konversation's buffer
16:20:57 <jeff_hann> I meant I was just looking at bug nr. 522675
16:21:24 <jeff_hann> couldn't it be a HAL problem?
16:21:32 <dpravec> i think automated testing of anaconda could be a big help in this and thats probably what i would like to do
16:21:36 <adamw> specific discussion would be best in the bug thread or friday's meeting
16:21:50 <denise> dpravec +100
16:21:51 <jeff_hann> adamw: ok, will do
16:22:03 <adamw> dpravec: well, autoqa already does that to some extent.
16:22:16 <adamw> denise: are you guys in the loop on the autoqa stuff? you know where to go for the test results?
16:22:18 <dpravec> adamw: yes i know.. but not enough
16:22:21 <adamw> right
16:22:56 <denise> adamw, i'm using https://fedorahosted.org/pipermail/autoqa-results/2009-September/date.html#end   - correct?
16:23:13 <adamw> denise: yup. cool
16:23:22 <adamw> so, as i said, to me this kinda feels like a communication issue
16:23:28 <denise> adamw: shared w/lwang too so kernel team knows
16:23:40 <adamw> for e.g. i wouldn't have guessed off the top of my head that nss would be critical to anaconda (as it is if i'm understanding the above discussion right)
16:23:50 <denise> adamw, agreed but some way to autodetect would be big help
16:24:12 <denise> nss is onlythe most recent of many
16:24:15 <adamw> is this...uh...Elio who's making all these exciting breakages^H^H^H^H^H^Hchanges to nss aware of it? :)
16:25:14 <adamw> denise: autodetect what?
16:25:15 * Oxf13 fears that there is an elephant in the room
16:25:21 <Oxf13> when it comes to late breaking changes in things like NSS
16:25:52 <denise> adamw, dpravec, forwarding dcantrell's api checking idea - jlaska has seen this
16:26:05 <adamw> thanks
16:26:39 <adamw> Oxf13: what's the elephant?
16:26:46 <Oxf13> RHEL6
16:26:56 <adamw> rhel? what is this rhel of which you speak? =)
16:27:04 <Oxf13> There seems to be a number of maintainers who are told THEY HAVE TO GET THIS IN NOW for RHEL6
16:27:31 <adamw> yeah. it does feel like sometimes the walls between fedora and rhel processes are rather...chinese.
16:27:32 <Oxf13> and so people who aren't typically plugged into the Fedora thing are tasked with work and they make it happen, stumbling into problems with our Fedora process
16:28:02 <adamw> i'm really not sure how we should go about addressing that, but it's definitely a valid concern...
16:28:54 <Oxf13> adamw: i think it's going ot take more impressing upon RHT managemment that Fedora policies and procedures cannot just be tossed aside under the banner of RHEL
16:29:07 <Oxf13> which isn't something those in the Fedora community can do easily
16:29:27 <adamw> Oxf13: right, we redhat-fedora people should probably take the ball on that one
16:29:29 <denise> i'm trying to take some of the pressure off by starting the official list of things RHEL6 will rebase from F13
16:29:37 <adamw> sounds like something we should just escalate through the traditional management set up
16:29:56 <Oxf13> adamw: we do have a bi-weekly meeting for just such things
16:30:07 <Oxf13> adamw: and we're identifying more and more folks who should be represented in such meetings
16:30:11 <Oxf13> so there is progress on that front
16:30:15 <adamw> Oxf13: great
16:30:22 <Oxf13> and things are far better than the RHEL4/FC6 era
16:30:42 <adamw> Oxf13: will you be able to pass on the concerns from this meeting? i think we can definitely say we all agree with your characterization
16:30:59 <jeff_hann> aye
16:32:17 <adamw> aside from that, denise / dpravec, i'm sure wwoods would be happy to take any suggestions for more detailed automated installer testing
16:32:54 <dpravec> adamw: i would love to get  dogtail tests of anaconda going
16:33:10 <adamw> pity he doesn't seem to be here for the meeting :/
16:33:11 <dpravec> and testing it everyday for everything possible
16:33:34 <denise> dpravec, and more testcases to automate - mgracik is available to help
16:34:26 <dpravec> denise: yes, we started talking about it, thats why i hear they (as a team) want some stable api 2 weeks  before freeze...
16:35:01 <denise> dpravec, and a pink pony ;-)
16:35:10 <dpravec> heh :)
16:35:13 <dpravec> i love ponies
16:35:39 <adamw> right :) my instinct is it'd be hard to get people to agree to a real freeze, but we can certainly improve on the 'not breaking stuff just for the hell of it with no regard to fedora processes' front
16:36:08 <poelcat> adamw: pong
16:36:17 <adamw> poelcat: was just pinging for attendance :) hi!
16:36:39 <poelcat> oh sorry... here at LinuxCon w/ Oxf13 and others
16:37:11 <Oxf13> denise: adamw: moving the feature freeze back a week from Alpha freeze was an attempt at getting things to stop moving around and time for Anaconda to sync up before entering the freeze
16:37:41 <denise> I'd settle for a heads-up on what's changing ...
16:38:09 <denise> at least it saves the debug WTF changed now time
16:38:49 <denise> the yum guys handled this well - gave anaconda a test image early
16:39:23 <Oxf13> denise: and to be fair, shit ain't supposed to be changing at this point
16:39:33 <Oxf13> people who are changing things now are doing it wrong.
16:40:04 <denise> although sometimes there are justifications - harald wants to rebase udev to pick up many buffer overflow fixes
16:40:38 <denise> and security is generally a good reason
16:40:59 <Oxf13> there are smart ways of doing that
16:41:06 <denise> and he looked for dependencies and gave us a heads-up
16:41:17 <Oxf13> building it quietly one night and waiting for the shit to fall over to see what needs fixing... is not how to do that.
16:41:25 <denise> agreed!!!
16:41:38 <adamw> uh...it isn't? /me grabs notepad
16:41:40 <adamw> =)
16:41:40 <Oxf13> coordinating with you, rel-eng, etc.. that's the way to do it.  Get a test build, get a test compose, see what happens /before/ doing the rawhide build.
16:41:56 <Oxf13> I can't wait until we have a 'rawhide-testing' like setup
16:42:25 <kanarip> maybe a rawhide-next like with linux-next
16:42:49 <kanarip> everything that goes in there should just work perfectly fine and has been tested to some extent
16:42:54 <adamw> ok, so, looks like we have avenues to explore here...
16:43:00 * kanarip back to lurking
16:43:02 <dpravec> yes thats my dream too
16:43:10 <dpravec> having some kind of working rawhide
16:43:16 <adamw> does anyone else have anything to raise in regards to the upcoming stuff for f12 beta? oxf13, anything you want to highlight?
16:43:29 <adamw> dpravec: rawhide works fine if you're careful enough. it's the only thing i have installed on this system =)
16:43:30 <skvidal> I have an item
16:43:34 <Oxf13> well, there is lots of risk out there
16:43:34 <adamw> skvidal: go ahead!
16:43:48 <Oxf13> broken live images, broken input on x86_64 stage1/live
16:43:55 <Oxf13> I'm quite worried about the upcoming freeze
16:44:03 <skvidal> so I have two pkgs of yum for F12 - one is 3.2.24-  which is more or less what's in rawhide right now
16:44:15 <adamw> Oxf13: what's broken on the live images? i've had people using recent ones for bug report testing, didn't get any reports they weren't working
16:44:27 <adamw> erf, sorry, we're on two conversations: let's go with skvidal for now
16:44:30 <skvidal> the other option is 3.2.25 (soon to be released) which will include the yum history work geppetto did
16:44:39 <skvidal> it's been tested by clumens in anaconda
16:44:44 <Oxf13> adamw: I fixed up the broken deps on friday and composed live images from it at my house, both failed to boot pretty badly
16:44:44 <skvidal> and it works and doesn't break things
16:44:54 <skvidal> but it is NOT required by any wild stretch for f12 GA
16:45:25 <skvidal> so long story short - should I wait on 3.2.25 until GA or not?
16:45:45 <adamw> it's kind of a grey area, i suppose
16:45:47 <skvidal> given the discussion above I thought this was relevant
16:46:01 <adamw> i mean, if we were considering it a Fedora 12 Feature - which we certainly could have done - the answer now would be no, it's too late
16:46:11 <skvidal> but it's not a feature
16:46:15 <adamw> but since it's _not_ a feature, i don't think there's anything technically preventing it going in
16:46:15 <denise> but you definitely did the right thing for anaconda
16:46:23 <adamw> (which is a sorta loophole)
16:46:29 <adamw> Oxf13: wdyt?
16:46:37 <Oxf13> I've already talked about it with them.
16:46:49 <Oxf13> I'd /prefer/ to wait, but I won't throw a fit if it goes in
16:47:04 <Oxf13> I reserve the right to throw a fit if it goes in and something breaks and we lose rawhide for a day or more
16:47:18 <adamw> that's kinda my instinct too. and i can see a downside to waiting - we wind up splitting development effort, i guess
16:47:27 <skvidal> adamw: not really
16:48:13 <skvidal> okay, I'll do this
16:48:25 <skvidal> I'll issue a 3.2.24+HEAD patch for rawhide today that has history in it
16:48:43 <skvidal> if things go sideways for any reason I'll patch it out and bump for rawhide
16:49:02 <adamw> i guess that works
16:49:13 <skvidal> there's no issue reverting a patch at all
16:49:31 <adamw> is there a realistic chance it could break something important subtly in a way we don't notice for a bit?
16:49:37 <adamw> (i know that's a hard question, but...)
16:49:41 <skvidal> okay
16:49:45 <skvidal> so here is what history does
16:49:51 <skvidal> it records extra info during each transaction
16:50:05 <skvidal> it doesn't change anything else on the system - just in /var/lib/yum/history/
16:50:17 <skvidal> it writes out a sqlite db
16:51:10 <skvidal> so I'll do the HEAD into rawhide and see if anyone has any issues with it
16:51:24 <skvidal> and I'll be immediately happy to roll it back out if we run into something
16:51:25 <adamw> yeah, i think that sounds ok.
16:51:33 <adamw> thanks for running it by us, though.
16:51:36 <skvidal> nod
16:51:49 <adamw> and working with the anaconda guys, that was awesome.
16:52:06 <denise> much appreciated
16:52:09 <skvidal> thank geppetto, too - he passed the bits over to clumens
16:52:25 <adamw> praise all around!
16:52:29 <skvidal> hah
16:52:52 <adamw> Oxf13: ok, so, your beta concerns...
16:52:58 <adamw> erf, just a sec
16:53:24 <adamw> eek gotta step out for a minute, got a parcel to collect downstairs :) brb
16:53:30 <Oxf13> k
16:54:52 <poelcat> Oxf13: wondering if we have a good wiki page to point people to on this topic besides the freeze pages
16:55:35 <poelcat> or is it just more common sense?
16:55:38 <Oxf13> poelcat: at one point, the Overview page within the releng name space tried to provide a high level view of how releases were done, and why we freeze at certain points, etc..
16:55:46 <Oxf13> I haven't done a good job maintaining that page over time
16:56:07 <Oxf13> I wanted it to be the page that new maintainers could read through to grasp our development and release process
16:57:12 <adamw> Oxf13: so, at least a couple of people i have checking out things for bug reports had no problem booting with the 20090917 auto-generated live images
16:57:34 <Oxf13> adamw: right, 17th was OK (that's what snap3 was made from)
16:57:38 <adamw> ah
16:57:44 <adamw> not sure if i've seen anyone test since then...
16:57:46 <Oxf13> adamw: something changed on the 18th that made me unable to get a live image booted
16:57:55 <adamw> erk.
16:57:59 <Oxf13> there were no nightly live images on the 18th due to dep issues.
16:58:12 <adamw> we have 20090920 images up now
16:59:07 <Oxf13> worth trying them out
16:59:27 <adamw> k
16:59:27 <poelcat> Oxf13: let's talk more I want to help make these pages better
16:59:43 <adamw> what was the failure you saw with 20090918?
17:00:03 <Oxf13> adamw: couldn't get the live root fs mounted.  It died halfway through bootup
17:00:22 <Oxf13> adamw: it was only fleeting, I was working on some other things, and expecting family to show up so I didn't get a lot of time to file it or debug it
17:00:31 <Oxf13> I just saw it with both i686 and x86_64
17:00:48 <adamw> interesting, sounds like the failure we were having with live images transferred to USB sticks which was recently fixed...
17:01:00 <adamw> well, if everyone can try out the 20090920 images and see if they work that'd be good
17:01:08 <adamw> the other thing, x86-64 input problems
17:01:09 <Oxf13> yeah, i wondered if the fix broke the non-usb case
17:01:24 <adamw> would that be https://bugzilla.redhat.com/show_bug.cgi?id=522675 ?
17:01:25 <buggbot> Bug 522675: high, low, ---, kernel-maint, NEW, mouse,keyboard don't work when boot from LiveCD
17:01:36 <Oxf13> I can't confirm that it is the same problem on x86_64 live images
17:01:38 <dpravec> Oxf13: can you give me url of the page you are talking about? (overview)
17:01:42 <Oxf13> that's why I was building live images to test with
17:01:56 <adamw> Oxf13: i see
17:02:05 <Oxf13> dpravec: http://fedoraproject.org/wiki/ReleaseEngineering/Overview
17:02:11 <dpravec> ah ty
17:02:20 <Oxf13> adamw: I spent most of friday comparing a i386 initrd to an x86_64 initrd looking for missing files, and not finding anything
17:02:24 <adamw> well, i'm grabbing the 20090920 image now, i'll try it out here and see how it goes
17:02:34 <Oxf13> last time we had something like this, the x86_64 buildinstall was dying part way through and not all files were making it to the initrd
17:02:44 <Oxf13> that's not the case this time
17:02:53 <adamw> k
17:02:53 <Oxf13> (or it's a cleverly hidden file somewhere)
17:03:47 <adamw> so yeah, our big takeaway here is that we really need people to be banging on testing at present: the test compose when it comes out, direct rawhide install, the nightly live builds, whatever you like
17:04:00 <adamw> just grab something and test it :) with an eye to the issues oxf13's raised, and the bugs on the f12beta list
17:05:07 <adamw> and again, remember the blocker bug meeting on Friday, 15:00 UTC, #fedora-bugzappers
17:05:12 <adamw> please come out if you can
17:05:23 <adamw> poelcat will be sending an official announcement soon.
17:05:54 <adamw> ok, we're over time as usual =) moving on...
17:05:58 <adamw> #topic open floor
17:06:04 <adamw> open floor time! anyone have anything to raise?
17:07:55 <adamw> no? then closing meeting in 3...
17:08:09 <adamw> 2...
17:08:15 <adamw> 1...
17:08:21 <adamw> thanks for coming, everyone!
17:08:23 <dpravec> adamw: thanks !
17:08:23 <adamw> #endmeeting