16:00:07 <jlaska> #startmeeting
16:00:17 <jlaska> #meetingtopic Fedora QA meeting
16:00:25 <jlaska> #topic gathering
16:00:59 <jlaska> Hey gang, show of hands folks attending the Fedora Quality Assurance meeting
16:01:09 <jlaska> I know adamw and f13 are unable to attend today
16:01:28 <Viking-Ice> hellú..
16:01:32 * wwoods waves
16:01:42 <jlaska> Viking-Ice: wwoods: Hey guys!
16:01:44 * mclasen pays a visit
16:01:53 <jlaska> mclasen: welcome :)
16:02:51 <dpravec> hello all, i am new here, i have joined fedora qa in Brno today
16:03:14 <jlaska> dpravec: glad to have you, thanks for joining in from home!
16:04:10 <jlaska> okay, I think that captures most folks ... let's dive into the brief follow-up items from last week
16:04:15 * skvidal is lurking
16:04:15 <jlaska> #topic Previous meeting follow-up
16:04:26 <jlaska> skvidal: hey there
16:05:01 <jlaska> Okay, I've got the still outstanding item of updating our goals page to be more current
16:05:04 <jlaska> *  [jlaska] - update QA/Goals wiki document
01:45:43 * [jlaska] - update QA/Goals wiki document
16:05:38 <jlaska> I don't have big updates here, other than I finally dusted off my fingers and am getting started.  I should have draft content soon.
16:05:58 <jlaska> One of the pre-reqs for me was pulling together the test day feedback ... so having that done helps me now
16:06:22 <jlaska> as always, input welcome on the test day survey or recommendations ... https://www.redhat.com/archives/fedora-test-list/2009-July/msg00009.html
16:06:27 <jlaska> s/welcome/encouraged/
16:06:39 <jlaska> #  [viking_ice] - check-in with abrt
16:06:55 <jlaska> Viking-Ice: I've got you down next for a update on some discovery efforts around abrt?
16:09:00 <jlaska> uh oh, we may have lost Viking-Ice
16:10:05 <jlaska> okay, we can come back to this.
16:10:30 <jlaska> I spoke to Viking-Ice earlier today on the abrt subject, so I believe he's been poking around in that space and offering some RFE's for the abrt team.
16:10:50 <jlaska> * [wwoods] - add post-install network test to basic functionality in rawhide acceptance plan
01:45:43 * [wwoods] - add post-install network test to basic functionality in rawhide acceptance plan
16:11:24 <jlaska> I believe you had this done before last weeks meeting concluded?
16:13:14 <jlaska> wwoods: did you go where Viking-Ice went?
16:13:26 <wwoods> yeah, I added two cases to the plan
16:13:32 <wwoods> post-install network and post-install input, I think
16:14:26 <jlaska> Nice, what's the linky?  just https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan#Basic_Functionality
16:15:19 <jlaska> okay ... going with that for now
16:15:40 <jlaska> #topic F12 - Raising the bar
16:15:59 <jlaska> we're missing a few folks today, so much of the agenda is just touching in on several fronts
16:16:28 <jlaska> The first is an effort mclasen announced to the list ... see https://www.redhat.com/archives/fedora-test-list/2009-June/msg00741.html
16:16:38 <mclasen> not sure if I need to say something about it ?
16:16:58 <mclasen> we want to do a series of user-task focussed test days
16:17:03 <mclasen> rather than feature-focussed
16:17:20 <mclasen> I hope some of the fine qa folks come by...
16:17:34 <mclasen> We want to do the first one next Tuesday
16:18:02 <jlaska> there's been some interesting feedback on the f-d-list for this, but overall this is a really cool initiative.  I'm looking forward to contributing!
16:18:10 <Southern_Gentlem> ?
16:18:15 <jlaska> mclasen: is the schedule of events - http://fedoraproject.org/wiki/Fit_and_Finish ?
16:18:41 <mclasen> yes, as far as we have it yet
16:19:02 <mclasen> I expect this effort to start a bit slow, and then pick up speed
16:19:11 <jlaska> mclasen: are there any additional preparations you guys are looking to make for the event?
16:19:16 <mclasen> as more of my team finish their f12 feature work and can join in
16:19:31 <mclasen> jlaska: we are trying to get live cds building and working
16:19:38 <mclasen> ajax was looking into that earlier today
16:19:53 <mclasen> so far we have the first (ie have them building)
16:20:05 <mclasen> but not the second (boots into runlevel 3, but no X)
16:20:16 <mclasen> also, they're a bit oversize
16:20:25 <jlaska> mclasen: you guys are closer to the issues affecting building them on rawhide right now, but if we can help ... just shout
16:20:41 * jlaska hasn't tried building one this month ... kind of a nice break from that :)
16:20:49 <mclasen> I expect ajax will maybe post an update on that sometime soon
16:20:52 <mclasen> I'll ask him
16:20:59 <mclasen> even better, he's here...
16:21:37 <mclasen> other than that, for preparations, I will spend some more time filling up the test day page with information
16:21:49 <mclasen> and I have to figure out how we are going to handle the issues that are found
16:22:00 <jlaska> mclasen: you've got some nice focus areas listed http://fedoraproject.org/wiki/Fit_and_Finish/Display_Configuration
16:22:05 <mclasen> bugs with a tracker bug, I guess...
16:22:12 <jlaska> that's right
16:22:45 <Southern_Gentlem> mclasen, so you are using rawhide or f11 for your test days?
16:23:22 <jlaska> so for the QA folks, I encourage folks to join in.  Our teams are working together so that we can provide a consistent test day experience ... while also helping to identify use case gaps + bugs
16:24:09 <mclasen> Southern_Gentleman: the plan is to use rawhide, but I don't think it is essential for user-task testing that everyone runs the same
16:24:24 <mclasen> you may well find issues in F11 that have a high chance to still be relevant in rawhide
16:24:55 <jlaska> as we move further into F12 development cycle, I suspect the gap will become wider?
16:25:59 <mclasen> hopefully we'll fix some of the issues that are found, so yes...
16:26:23 <jlaska> cool
16:27:02 * Viking-Ice back
16:27:09 <jlaska> Thanks for kicking things off on the Test Day front.  Having you and your crew involved should really help fast track issues that come out of this
16:27:45 <jlaska> I've updated the upcoming events section to point to #fedora-fit-and-finish next Tuesday
16:27:46 <Viking-Ice> btw mclasen reporters are missing what to report on you Fit and finish
16:28:01 <Viking-Ice> which files to include etc ..
16:28:37 <mclasen> I will try to clarify some of that when I get to responding in that thread
16:28:52 <jlaska> mclasen: Viking-Ice raises a good point.  Something that surfaced from last weeks meeting was the need for improved details on writing good bug reports
16:29:18 <jlaska> mclasen: if it makes sense at all to link to upstream docs or add some content into https://fedoraproject.org/wiki/Category:Debugging ... that's an area we can assist in
16:29:29 <Viking-Ice> mclasen: all X components are covered
16:29:53 <Viking-Ice> https://fedoraproject.org/wiki/Xorg/Debugging#All_bug_reports
16:30:29 <jlaska> okay ... going to move on to next topic (there is a meeting room conflict in 30 minutes)
16:31:34 <jlaska> #topic AutoQA - update from wwoods
16:31:44 <jlaska> wwoods: what's the word on the autoqa front?
16:31:59 <wwoods> So the plan (as mentioned earlier) is complete: https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
16:32:07 <wwoods> 7/12 cases have been written
16:32:42 <wwoods> looking over the roadmap: https://fedorahosted.org/autoqa/milestone/israwhidebroken.com
16:33:07 <wwoods> the stuff there isn't really in order but all the steps are there.
16:33:24 <wwoods> so the next step after finishing writing test cases is to attempt to automate one
16:33:46 <wwoods> which requires an autotest instance, which requires packaging autotest
16:33:53 <wwoods> that's https://fedorahosted.org/autoqa/ticket/9
16:34:15 <wwoods> jlaska and f13 are handling that and it seems to be going well
16:34:21 <jlaska> yeah ... good progress on that front
16:34:36 <jlaska> I'm happy with the autotest packaging, and have started futzing with the autoqa package to tie into it
16:34:48 <wwoods> hopefully by next week we'll have the test cases all finished and at least one automated
16:34:53 <wwoods> once we know how to write tests for autotest
16:35:00 <jlaska> I have to discuss with f13 how we'll manage some of the python dependencies to get this into shape for fedora-infrastructure
16:35:05 <wwoods> we can start trying to figure out how to organize and package those tests in the autoqa repo
16:35:44 <jlaska> wwoods: was there a test you had in mind that seemed the most ready for automating?
16:36:20 <jlaska> something where we can use the off-the-shelf approach?
16:36:26 <wwoods> the repodata/comps validity stuff is really easy to automate
16:36:37 <wwoods> as is installer image existence
16:36:53 <wwoods> there are existing scripts / tools / example code to mess with that stuff
16:37:10 <jlaska> cool, that should help!
16:37:30 <jlaska> I've got some bits of that .treeinfo validation if it's useful
16:37:43 <wwoods> the other thing to figure out is.. how to allow manual test reporting
16:37:58 <wwoods> since there's always going to be things that we can't automate (or haven't figured out how to automate yet)
16:38:34 <jlaska> wwoods: is this a general test result reporting solution?
16:38:45 <wwoods> that's a whole big can of worms, because then you need identification/authentication to know who is allowed to report the results
16:38:58 <wwoods> I really don't want to rewrite something like testopia
16:39:32 <wwoods> and we've been promised some magical Test Case Management System for months and months and it never seems to happen
16:39:39 <wwoods> because it's a huuuge problem to solve
16:39:46 <wwoods> so no - that would get way, way out of hand
16:39:52 <jlaska> agreed!
16:39:53 <wwoods> I'm trying to keep this as simple as possible
16:40:37 <jlaska> I don't know how robust it is, but I think autotest does support remote communication
16:40:45 <jlaska> so that might help solve the immediate need, who knows
16:42:46 <jlaska> wwoods: thanks for the update, we'll stay tuned to fedora-test-list for any updates or if test review is needed
16:43:29 <jlaska> wwoods: any other updates on the autoqa front?
16:44:08 <wwoods> nothing I can think of at this time
16:44:17 <jlaska> okay, changing topic ...
16:44:22 <jlaska> #topic Install Testing SOP draft - update from lili
16:45:00 <jlaska> hopefully Liam is sleeping right now (being 12:44am for him) ... but wanted to give a quick update on an SOP he posted for review
16:45:31 <jeremy> jlaska: this is fedora, we don't allow sleeping :-P
16:45:42 <jlaska> Liam is trying to document the tasks involved when planning+preparing and executing a collaborative installation test event (see mail at - https://www.redhat.com/archives/fedora-test-list/2009-June/msg00734.html)
16:45:50 <jlaska> jeremy: shhh!  aren't you supposed to be sleeping!
16:45:52 <jlaska> :)
16:46:36 <jlaska> A few folks have chymed in already and helped clean up the wiki, so thanks to arxs and beland
16:47:33 <jlaska> while this largely affects the folks who have been testing RC's, feedback from a wider audience is encouraged ... as that's the audience we're hoping benefits from added transparency
16:47:38 <jlaska> https://fedoraproject.org/wiki/User:Liam/Draft_Install_Test_SOP
16:48:06 <jlaska> unless any questions/comments ... changing topics ...
16:48:28 <jlaska> #topic Improve Bug Reporting proposal - update from viking_ice
16:48:51 <jlaska> Viking-Ice: wanted to give you an opportunity for any updates on the proposal you drafted
16:49:01 <jlaska> looks like some good stuff come from that discussion last week
16:49:13 <Viking-Ice> Yup things are emerging..
16:49:24 <Viking-Ice> Took a look at abrt
16:50:06 <Viking-Ice> Filed an rfe for gui improvements https://fedorahosted.org/abrt/attachment/ticket/53/Automatic%20Bug%20Reporting%20Tool%20Mockup1.png and https://fedorahosted.org/abrt/attachment/ticket/53/ABRT_simple_bug_filing_form.png
16:51:13 <jlaska> Viking-Ice: I know the abrt guys follow their trac instance, but you may want to check in with them directly.  I know they are keenly interested in improving the look'n'feel (and ease of use) of abrt
16:51:35 * mclasen recommends iterating over that design a bit  :-)
16:52:31 <jlaska> mclasen: you're much closer than me to what is helpful when it comes to designing GUI apps
16:52:32 <Viking-Ice> Bringing the gui closer to the simpler bug reporting from that's coming with the upcoming release of bugzilla The Idea here is to reduce the learning curve for reporters so if users have gotten used to work in the upcoming bugzilla simple reporting form they have no problem getting used to use abrt and also if we need to get reporters for some reason to bugzilla they would be familiar with the reporting form there...
16:53:44 <Viking-Ice> mclasen: over my proposed designed or the current?
16:53:51 <jlaska> I guess it's a question for the abrt team whether abrt is intended as a graphical desktop client for filing bugs, or a generic problem reporting tool ... where one of the reporting "plugins" is bugzilla?
16:54:13 <mclasen> Viking-Ice: both
16:54:34 <mclasen> the abrt guys need to get together with some of our designers, really
16:54:46 <jlaska> definitely!
16:55:17 <jlaska> Look at what huge improvements they've suggested for setroubleshoot ... https://fedoraproject.org/wiki/Design/SETroubleshootUsabilityImprovements
16:55:20 <Viking-Ice> Sorry to say but I think they should try to have this as close to the bugzilla form as possible..
16:55:45 * mclasen doesn't even think bugzilla should figure at all in this...
16:55:55 <mclasen> anyway, probably offtopic here
16:56:06 <Viking-Ice> true
16:56:27 <jlaska> Viking-Ice: the Debugging pages were a nice offshoot from your topic last week, I know Adam is raising these during BugZapper meetings
16:56:36 <jlaska> did you have any news or recommendations on that front?
16:57:15 <Viking-Ice> debugging visualization seems complete https://fedoraproject.org/wiki/Reporting_virtualization_bugs
16:57:46 <Viking-Ice> mean virtualization
16:58:06 <jlaska> yeah, markmc and the virt team have done a great job outlining that page
16:58:17 <jlaska> it came in handy during the virt test day ... and hopefully will for the F12 virt testday
16:58:35 <Viking-Ice> oh yeah I should mentioned selinux
16:58:49 <Viking-Ice> https://fedoraproject.org/wiki/Design/SETroubleshootUsabilityImprovements
16:59:24 <Viking-Ice> Notice the report Bug <--
17:00:11 <jlaska> are there things here you think the QA team should be considering?
17:00:44 <wwoods> yeah the "report bug" dialog there is closer to what you'd want to see from a bug-reporting client
17:00:46 <Viking-Ice> The only thing flawed in this design is calling the button Report Bug instead of Report Alert
17:01:36 <Viking-Ice> This is a security application and thus user might think all alerts are just bugs and start to ignore it
17:02:43 <Viking-Ice> Just thought to mention what's happening on selinux front for users and the great job they are doing...
17:03:23 <Viking-Ice> Anyway i'm done
17:03:39 <jlaska> thanks for the update ... anything you want to check-in on next week on this subject
17:03:53 <jlaska> or things you want the larger f-t-list to review/consider before next meeting?
17:04:09 <fhornain> ping ?
17:04:36 <jlaska> #topic Open Discussion
17:05:00 <jlaska> We've run over, but I just wanted to toss a note out there for any feedback/input on the Fedora 11 test day survey results
17:05:03 <jlaska> https://www.redhat.com/archives/fedora-test-list/2009-July/msg00009.html
17:05:06 <Viking-Ice> nope low on time these day's.. we have to finish something next week before everybody start bailing for vacations..
17:05:16 <Viking-Ice> here @ work..
17:05:27 <jlaska> I've gathered the feedback to the wiki and starting to toss recommendations on the list for how we can improve F12 based on the feedback
17:05:34 <jlaska> Viking-Ice: no worries, understood.
17:05:45 <jlaska> That's all I had for today ... any other thoughts / concerns?
17:06:13 <fhornain> Can we use the room ?
17:06:20 <jlaska> fhornain: sure can, we are wrapping up
17:06:33 <fhornain> Thx
17:06:51 <jlaska> Thanks for your time today, let's keep things going on f-t-list and #fedora-qa
17:06:57 <jlaska> #endmeeting