15:00:16 <jlaska> #startmeeting Fedora QA Meeting
15:00:16 <zodbot> Meeting started Mon Oct  4 15:00:16 2010 UTC.  The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:16 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:18 <jlaska> #meetingname fedora-qa
15:00:18 <zodbot> The meeting name has been set to 'fedora-qa'
15:00:30 <jlaska> #topic Waiting for attendees
15:00:41 * kparal me me me!
15:00:46 * saccia waves to jlaska
15:00:48 <jlaska> kparal for the win
15:00:53 <jlaska> saccia: hello there
15:01:09 <adamw> yo
15:01:14 <jlaska> adamw: hey hey
15:01:24 <jlaska> kparal: can I appoint you #chair for the tail end of the meeting today?
15:01:27 * jskladan here :)
15:01:31 <jlaska> jskladan: greetings
15:01:40 <kparal> jlaska: sure
15:01:47 <jlaska> kparal: thank you
15:01:50 <jlaska> #chair kparal
15:01:50 <zodbot> Current chairs: jlaska kparal
15:02:13 <jlaska> note ... wwoods will be out today
15:02:24 * jsmith lurks
15:02:26 <tk009> so the rest of us have to stand? no its ok I see how it is
15:02:27 <jlaska> lurker!
15:02:38 <jlaska> tk009: hah
15:02:41 <jsmith> jlaska: I know... 'nother meeting in 30 minutes :-/
15:02:51 <jsmith> jlaska: C'est la vie on Mondays for me
15:03:12 <jlaska> alright, we'll get started in 30 seconds
15:03:26 <jlaska> kparal: is Martin available today?
15:03:46 <kparal> jlaska: nope, he will come only tomorrow
15:04:46 <jlaska> #topic Previous meeting follow-up
15:05:07 <jlaska> Shame on me, I never sent wiki versions of the minutes from last week
15:05:18 <jlaska> However, I had no follow-up items listed
15:05:33 <jlaska> So unless there is anything specific folks would like to call out ... we'll continue
15:06:06 <jlaska> alrighty ...
15:06:16 <jlaska> #topic F-14 Testing
15:06:30 <jlaska> Okay all, prepare to increase the pantscon level
15:06:49 <jlaska> I've listed the upcoming F-14 testing milestones on the meeting wiki page (I'll list here as well)
15:07:10 <jlaska> #info Oct 07 - Fedora 14 acceptance test run planned (see rel-eng ticket#4140)
15:07:21 <jlaska> this friday ...
15:07:26 <jlaska> #info Oct 08 - F14Blocker review meeting #2 planned
15:07:37 <jlaska> followed by next week ...
15:07:47 <jlaska> #info Oct 14 - F14Blocker TC1 test run planned (no rel-eng ticket available yet)
15:07:58 <jlaska> #info Oct 21 - F14Blocker RC1 test run planned (no rel-eng ticket available yet)
15:08:12 <jlaska> and lastly ...
15:08:20 <jlaska> #info Oct 26 - F-14 Go/No Go meeting (see https://fedoraproject.org/wiki/Go_No_Go_Meeting)
15:08:52 <jlaska> so this means we have less than 3 weeks to have all F14 blocker bugs closed
15:09:21 <adamw> yay fun
15:09:24 <jsmith> w00t!
15:09:29 <jlaska> We hosted the first of the F-14-Final blocker review meetings last Friday (http://meetbot.fedoraproject.org/fedora-bugzappers/2010-10-01/f-14-blocker-review.2010-10-01-16.00.log.html)
15:09:41 <jlaska> I suspect adamw is still in pain from that one
15:09:45 <jlaska> it was a *long* meeting
15:09:57 <jlaska> meetings that require an intermission ... ouch!
15:10:18 <jlaska> one question I have regarding this weeks planned acceptance test run ...
15:10:42 <jlaska> previously, we use these checkpoints to run the rawhide acceptance test plan (https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan)
15:11:03 <jlaska> this was mainly intended to identify any big gotchas with doing a basic install, boot and login to the desktop
15:11:23 <jlaska> do we want to continue with this plan ... or should we consider additional installer or desktop testing?
15:11:30 <jlaska> or perhaps, just bug verification?
15:11:35 <jlaska> any suggestions / ideas ?
15:11:49 <adamw> we can't easily change this, can we?
15:11:55 <adamw> since it requires Code
15:12:20 <jlaska> adamw: sorry, change what?
15:12:32 <jlaska> the automation used to run the acceptance plan?
15:12:37 <adamw> what the parameters for automated testing are
15:12:40 <saccia> I say we make sure we add something in that relates to dual hard drive setups... that's been nipping at us recently
15:13:13 <jlaska> adamw: yeah, I don't imagine we'll be able to adjust the automated testing very quickly for this week
15:14:15 <jlaska> saccia: let's continue to test the scenario you, Southern_Gentlem and I were discussing in #fedora-qa earlier.  That's a tough challenging test scenario in that the 'expected results' aren't very explicit in most cases.
15:14:21 <adamw> i mean, i thought it was generally understood the long-term goal was simply to automate as many of the validation tests as we can, and what we have implemented so far is simply what we've managed
15:14:32 <jlaska> adamw: yes certainly
15:14:54 <jlaska> adamw: for the milestone this week, my question wasn't so much ... how do we want to change the automation we run this week
15:15:10 <jlaska> more ... do we want to include _other_ tests/activities during this event
15:15:26 <jlaska> to help identify issues earlier, or verify MODIFIED/ON_QA issues etc...
15:15:43 <adamw> oh. like, manual testing?
15:15:56 <jlaska> yeah, that could be one thing
15:16:14 <kparal> "manual", that sounds awful :P
15:16:21 <jlaska> it sure does!
15:16:39 <jlaska> awfully fun you meant, right? :P
15:17:11 <adamw> i dunno, i'm not sure it's sensible to load other stuff into the rawhide acceptance testing system
15:17:20 <adamw> reviewing blocker issues isn't really part of that is it?
15:17:25 <jlaska> adamw: I wouldn't suggest that either
15:17:28 <adamw> what did you have in mind?
15:17:38 <jlaska> I'm asking for ideas/suggestions in _addition_ to the planned acceptance run
15:17:48 <jlaska> sorry if that wasn't clear
15:18:09 <jlaska> I'm just worried that we only have about 5 days to find and fix bugs against TC1
15:18:15 <jlaska> in order for them to land in RC1
15:18:25 <jlaska> for Beta ... TC1 landed late
15:18:28 <jlaska> leaving less time
15:18:46 <kparal> we can do standard installation matrix even for that acceptance compose
15:19:24 <jlaska> adamw: do we have a lot of ON_QA or MODIFIED bugs on the blocker list?
15:19:43 <jlaska> kparal: I was wondering about that too, whether we had cycles to support additional installer tests
15:20:07 <adamw> 7
15:20:29 <adamw> kparal's idea isn't a bad one, though that effectively turns the acceptance compose into TC1
15:20:38 <jlaska> adamw: true, is that bad?
15:21:15 <adamw> not necessarily, just it would mean that we're not happy with the current process and we should probably change it formally rather than re-purposing things
15:21:46 <jlaska> actually, repurposing is just what I had in mind
15:21:54 <jlaska> I think that's what kparal's suggesting
15:22:15 <jlaska> okay, I'll take this offline ... I want to run kparal's idea by Hurry and Newgle too
15:22:30 <jlaska> either way, we're going to continue with the planned acceptance run this week
15:22:53 <jlaska> any ideas to help front-load any of the TC1 or RC1 firestorm ... please don't hesitate
15:24:04 <jlaska> #idea potentially add some of the validation matrix testing to the planned acceptance run?
15:24:30 <jlaska> #action jlaska to check-in with rhe for thoughts on doing a limited test run of validation matrix this week
15:24:43 <jlaska> #topic F14 Test Days
15:24:50 <jlaska> alright, so this is another FYI topic
15:25:10 <jlaska> Unless you were on another planet last week ... Adam and company hosted the always popular X test week
15:25:15 <jlaska> #info Sep 28-30 - Graphics test week
15:25:28 <adamw> i haven't done a summary yet, planning that for today
15:25:36 <jlaska> From what I can tell, the event rocked?
15:25:46 <jlaska> adamw: okay, sweet
15:26:19 <jlaska> adamw: any issues or highlights to note here?
15:26:48 <kparal> and the 'most unsatisfied tester prize' wins... kparal. 5 bugs! :o)
15:26:58 <adamw> all went ahead fine
15:27:01 <kparal> (meaning Intel test day)
15:27:09 <jlaska> haha, I like that prize :)
15:27:10 <adamw> it wasn't as busy as the last few, my fault for not promoting more ahead of time; it kind snuck up on me
15:27:13 <adamw> thanks a lot kparal
15:27:47 <jlaska> adamw: cool, feel free to log that or any other thoughts to https://fedoraproject.org/wiki/Fedora_14_QA_Retrospective
15:28:01 <jlaska> I've got a bullet there for the virt test day as well
15:28:06 <jlaska> #info Oct 07 - Open slot
15:28:11 <jlaska> so no test day planned this week
15:28:40 <jlaska> which is partly why I was thinking about taking our typicaly test day time, and devoting it in some way to working on F-14-Final tests/bugs/$something
15:28:49 <jlaska> #info Oct 14 - OpenLDAP and nss
15:29:07 <jlaska> The wiki appears to be well underway - https://fedoraproject.org/wiki/Test_Day:2010-10-14_OpenLDAP/NSS
15:29:10 <kparal> I help with that
15:29:17 <kparal> those guys are pretty active
15:29:24 <kparal> I think all should be prepared well in time
15:29:30 <jlaska> kparal: awesome!  Any next steps you want to note?
15:29:49 <jlaska> Are they planning to test-announce and planet.fedoraproject.org the event?
15:29:54 <kparal> I just need to ensure LiveCD can be built and sent out announcements
15:29:55 <jlaska> or should one of us handle that
15:30:06 <jlaska> kparal: oh, it needs a custom live image?
15:30:15 <kparal> may be
15:30:27 * kparal needs to check out :)
15:30:28 <jlaska> I actually can build live images now, unlike previously when adamw asked for live image creation help
15:30:38 <adamw> heh
15:30:49 * adamw can build 32-bit ones now too, whatever the bug was with that, it's fixed
15:30:57 <jlaska> alright, I think that covers test days
15:31:12 <jlaska> anything else before moving on? (moving in 30 seconds)
15:31:46 <jlaska> alrighty ...
15:32:02 <jlaska> two more topics before open-discussion (autoqa and NTH update)
15:32:09 <jlaska> #topic AutoQA Package Acceptance Update
15:32:25 <jlaska> kparal: I have 2 follow-up items on the list from last week ... do you want to take it away?
15:32:37 <kparal> sure, what are they?
15:33:03 <jlaska> wwoods will also add an rpm test transaction to depcheck to catch file conflicts
15:33:07 <jlaska> jskladan and wwoods investigating mash integration
15:33:43 <kparal> wwoods is out right now, I don't think I have any news about that
15:34:10 <kparal> let's keep it to next time
15:34:16 <jlaska> sounds good
15:34:25 * jlaska will leave those on the wiki for next week
15:34:44 <kparal> apart from that, the last week was mainly involved around upgradepath
15:34:53 <kparal> so, we have received some new patches
15:35:12 <kparal> we stopped checking -testing repos, because we're not sure how to do that properly
15:35:22 <kparal> we need to consult with FESCo about it
15:35:47 <kparal> also there were some small fixes and cleanups in upgradepath code
15:36:23 <kparal> I have started a thread about timing-sensitive tests, that will be a main topic soon when we plan to actually use it in production
15:36:37 <kparal> upgradepath is one of them, the other is depcheck
15:37:09 <kparal> as for other new, jlaska posted a few fixes and we received a huge resultsdb patch from jskladan
15:37:42 <jskladan> eh, it was not a patch :-D more of recapitulation of the current state :)
15:37:54 <kparal> #info we stopped checking -testing repos in upgradepath test, because we're not sure how to do that properly - we need to consult FESCo
15:38:20 <kparal> #info we started a thread about timing-sensitive tests - upgradepath and depcheck - how and when to correctly execute them
15:38:30 <jskladan> but sure - currently resultsdb patches for autoqa are waiting to be pushed to master, so we can start storing some test results (together with sending emails to the -results list)
15:38:59 <jskladan> so once there is some reasonable dataset, we can hop into creating frontends
15:39:42 <jskladan> the aforementioned email <https://fedorahosted.org/pipermail/autoqa-devel/2010-September/001187.html>
15:40:05 <jskladan> also describes a 'metadata' proposal
15:40:09 <kparal> #info resultsdb patches for autoqa are waiting to be pushed to master
15:40:48 <jskladan> which can be used to describe testplans in a unanimous way, so we can create a "generic testplan frontend"
15:41:16 <jskladan> which will visualize testplans using the structure defined in the provided metadata
15:41:33 <jskladan> (but that is kind of 'happy future' for now :) )
15:41:47 <kparal> #info jskladan prepares a way for us to be able to create "generic testplan frontend"
15:41:56 * jskladan ends the boring monologue
15:42:03 <kparal> as a side note, in autoqa-devel there is also an ongoing discussion with atodorov how to handle whitelisting of particular lines in rpmlint test case. feel free to join in.
15:42:19 <kparal> that should be all of autoqa I think
15:42:46 <kparal> #topic F-14-NTH (Nice to have) proposal status
15:42:58 <kparal> adamw: you want to say a few words about this?
15:43:25 <adamw> sure
15:43:35 <adamw> not much to say beyond the proposal on the ml, which we've been acting on provisionally
15:43:53 <adamw> if i don't get any more feedback to the mail i'll just go ahead and push the policy, so be warned =)
15:43:59 <kparal> #link http://lists.fedoraproject.org/pipermail/test/2010-September/093833.html
15:44:07 <adamw> it seems to be working out fine so far, we reviewed and accepted several NTH bugs at the blocker review meetin
15:44:40 <kparal> #info adamw threatens to push the policy, hurry with comments :)
15:45:37 <kparal> ok, thanks adamw
15:45:47 <kparal> #topic Open discussion <your topic here>
15:45:57 <kparal> any proposals for discussion?
15:47:11 <adamw> nothing from me
15:47:25 * kparal declares this monday as the Sleepy Monday
15:47:45 <kparal> closing in 1 minute if no other topics
15:48:37 <jlaska> kparal: thanks for driving :)
15:48:45 <kparal> jlaska: you're welcome
15:48:57 <kparal> have a nice day everyone
15:49:02 <kparal> #endmeeting