15:01:45 <adamw> #startmeeting Fedora QA meeting
15:01:45 <zodbot> Meeting started Mon Sep 26 15:01:45 2016 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:45 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:45 <zodbot> The meeting name has been set to 'fedora_qa_meeting'
15:01:48 <adamw> #meetingname fedora-qa
15:01:48 <zodbot> The meeting name has been set to 'fedora-qa'
15:01:51 <garretraziel> afternoon afternoon
15:01:52 <adamw> #topic Roll call
15:02:00 <adamw> ahoyhoy folks
15:02:03 <adamw> who's around?
15:02:06 <adamw> hi garret
15:02:08 <garretraziel> \me is here
15:02:11 * kparal is here
15:02:21 * tenk is here
15:02:26 <garretraziel> ha, just like in Windows
15:02:32 * pschindl is here
15:02:32 * coremodule is here!
15:02:32 <adamw> garretraziel: review my patches damn you :P
15:02:35 <garretraziel> c:\me
15:02:49 * tflink is here
15:03:02 <garretraziel> adamw: do you know how long it takes to run whole test suite on my laptop :-D?
15:03:33 <adamw> .fire garretraziel longer than it takes me to fire you!
15:03:33 <zodbot> adamw fires garretraziel longer than it takes me to fire you!
15:03:51 * cmurf thinks he's here
15:04:23 <adamw> cmurf: sorry, but no.
15:04:23 <adamw> :P
15:04:28 <adamw> alrighty, let's get rolling!
15:04:40 <adamw> #topic Previous meeting follow-up
15:04:42 <cmurf> adamw: you need to wait longer to see if that sinks in and what effect it has
15:05:03 <adamw> next week on Doctor Who, be baffled and bored as the Doctor goes back in time to fix some embarrassing bugs in Fedora releases before they go out
15:05:13 <adamw> (hi Pharaoh)
15:05:18 <cmurf> oh really?
15:05:25 * satellit listening
15:05:40 <adamw> so i guess sumantro didn't make it - he let me know he'd be on the road today
15:05:49 <adamw> i'll try and cover his topics
15:06:04 <adamw> #info "sumantro to check testdays webapp is working ahead of test days starting up" - pretty sure that got done, since we've run a Test Day using the app since then :)
15:06:26 <adamw> #info "adamw to do something about a wayland test day" - not done yet, still on my todo list (i'll do that first thing this week)
15:06:31 <adamw> #action adamw to arrange a Wayland test day
15:07:13 <adamw> hmm, next one is "sumantro to organize a test day for https://fedoraproject.org/wiki/Changes/BetterSwitchableGraphicsSupport"
15:07:23 <adamw> i don't see a ticket for that yet, or anything; anyone know the status?
15:08:18 <cmurf> no
15:08:41 <cmurf> I could ping one of the stakeholders if they're listed in the change, but I thought sumantro had done that already
15:09:07 <adamw> yeah, let's wait till he's around'
15:09:13 <adamw> i'll put the item back on the list for next time
15:09:16 * handsome_pirate shows up for a meeting for once
15:09:19 <cmurf> it might be best to leave it to sumantro rather than add confusion
15:09:35 <adamw> #info "sumantro to organize a test day for https://fedoraproject.org/wiki/Changes/BetterSwitchableGraphicsSupport" - we're not sure on the status of this and sumantro is not here today, will re-list
15:09:42 <adamw> #action sumantro to organize a test day for https://fedoraproject.org/wiki/Changes/BetterSwitchableGraphicsSupport
15:10:37 <adamw> #info "sumantro and a2batic to work on planning migration from trac" - looks like a2batic sent an email on that topic this morning, so this is moving along: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/WYNY2LISKV7S3QVEGQGTD6IGSLKOAO5H/
15:11:08 <adamw> do we want to dive into that now, or shall we follow it up on the lisT?
15:11:47 <cmurf> pagure transition?
15:11:53 <adamw> yeah
15:12:21 <cmurf> i'm fine with followup on the list, i'm not sure what to chat about it yet anyway
15:12:59 <adamw> alrighty
15:13:09 <adamw> then i think that's all the follow-up
15:13:15 <cmurf> infra moved to pagure over the weekend, so we'll let them shake things out :D
15:13:26 <adamw> ;)
15:13:31 <adamw> #topic Fedora 25 Beta status
15:14:07 <adamw> so, freeze is 09-27 00:00 UTC, which - if my math is correct - is like 16 hours away
15:14:14 <adamw> so we're getting down to the business end of Beta
15:14:31 <cmurf> I'm slightly surprised to not see many bugzilla emails early Monday morning
15:14:44 <adamw> we've got 4 beta blockers ATM
15:15:01 <adamw> i got about a dozen, but yeah, it's a light monday...
15:15:24 <coremodule> Whoo!
15:15:34 <cmurf> I guess I'm not cc'd on enough bugs haha
15:16:37 <adamw> cmurf: you should make like robatino
15:16:42 <adamw> he CCs himself on every blocker bug
15:16:53 <adamw> every time i fail a blocker the next mail I get is 'robatino CCed himself', heh
15:16:57 <cmurf> so freeze causes things in u-t to stay in u-t, that's the gist?
15:17:03 <adamw> yes
15:17:31 <cmurf> so if at freeze time something doesn't have enough karma to go stable, it stays in u-t
15:17:53 <adamw> after freeze nothing goes from u-t to 'stable' (the 'fedora' repo used for the composes) unless it fixes a freeze exception or blocker bug
15:18:18 <adamw> but more importantly it means we have two weeks until go/no-go :)
15:18:26 * adamw looks at https://www.happyassassin.net/testcase_stats/25
15:18:40 <cmurf> holy k rap
15:18:48 <adamw> we're missing the RAID tests, so I guess I'll try and get them done this week
15:18:54 <adamw> #action adamw to run firmware/hardware RAID tests
15:19:51 <cmurf> so there's a raid bug that's a rather odd duck, in that it's affecting CentOS7, Fedora 25, 24, 23, and maybe 22?
15:20:17 <cmurf> I'm like - that stuff got tested so has something else changed? Anyway, can talk about it on #fedora-qa but it's sort of a WTF bug
15:20:27 <adamw> RAID bugs are always wacky
15:20:34 <adamw> and quite frequently depend on the previous state of the disk...
15:20:45 <adamw> so yeah, test coverage doesn't look bad but there's a few tests that haven't been run since Alpha
15:20:54 <cmurf> right that's what I'm thinking is something new touched the drive and then something old is doing the WTF
15:21:04 <adamw> #info https://www.happyassassin.net/testcase_stats/25 looks pretty good but there's a few tests that haven't been run since Alpha: please help out with covering those
15:21:34 <adamw> anyone have any specific bugs/issues/situations to cover for Beta, or are we just kinda rolling along?
15:22:04 <cmurf> roll along more on the agenda
15:22:08 <adamw> tflink: coremodule: which of us is supposed to be looking after Cloud, again?
15:22:25 <adamw> the Cloud tests don't seem to have been run since Alpha, it'd be good to avoid another last-minute 'oh crap, the cloud images don't work' situation
15:22:28 <cmurf> oh yeah good point
15:22:33 <coremodule> I can do it. I'll do it today!
15:22:39 <adamw> that's the spirit
15:22:41 * adamw sends gin
15:22:48 <coremodule> Delicious!
15:22:48 <adamw> #action coremodule to run Cloud tests
15:22:58 <cmurf> I keep trying to test the atomic ISO but until there's a new anaconda in a compose it blows up
15:23:05 <adamw> coremodule: i'm not sure if nightlies go to EC2, best ask in releng if you want to try 'em there
15:23:28 <adamw> cmurf: yeah, openQA knows that one is broken, but it's not the blocking image
15:23:42 <cmurf> Cloud folks want to make atomic iso their official release, i.e. release blocking image
15:23:44 <adamw> i really need to add a space to the matrix for that image in fact...
15:23:55 <cmurf> I'm kinda like, let's maybe do that for F26...
15:23:56 <adamw> cmurf: yep, but i think still the installer ISO wouldn't be blocking, and it's not till 26 at least.
15:23:59 <cmurf> kinda short notice
15:24:08 <coremodule> adamw, Okay, I will.
15:24:21 <cmurf> The impression I got is they want it release blocking for F25...
15:24:40 <cmurf> There are other ducks to get in a row before that though IMO
15:24:42 <adamw> #info freeze is in 16 hours from this meeting, go/no-go is in two weeks (2016-10-07)
15:24:56 <adamw> cmurf: wow, that sounds...radical...do you have references? is there a ticket or something?
15:25:07 <adamw> i thought the blocking list was supposed to be set in stone some time before Alpha
15:25:22 <cmurf> right me too, just on the face of it
15:25:37 <cmurf> it's on the cloud list and in meeting logs
15:25:48 <cmurf> I don't know if there's a ticket, it's a discussion
15:26:06 <cmurf> And maybe I threw cold water on it when I said, OK all the stuff in custom partitioning needs to work for you to do that
15:26:18 <adamw> heh...well, only if they wanted the ISO.
15:26:19 <cmurf> pretty much nothing in custom partitioning works on atomic ISO
15:26:25 <adamw> but thanks for the heads up, let us know if it goes any further...
15:26:37 <cmurf> yep
15:27:01 <adamw> any other Beta topics? or shall we move on?
15:27:16 <cmurf> move on more interesting stuff to cover
15:27:28 <adamw> #topic Test Day status
15:27:34 <adamw> well, this one will be a bit tricky with sumantro not around
15:27:41 <adamw> but let's just throw him lots of action items while he's away ;)
15:27:46 * cmurf snickers
15:27:58 <satellit> MediaWriter for mac and windows? work for me but slow downloading .isos    FedoraMediaWriter-mac-3.97.1   https://bugzilla.redhat.com/show_bug.cgi?id=1377825#c6
15:28:14 <tenk> +1 on Windows
15:28:22 <tenk> very slow download and very very slow check
15:28:24 <cmurf> satellit: weird you get it to even launch, the Mac version stopped working for me entirely
15:28:43 <satellit> nice on windows also even did wks f25 in other
15:28:50 <adamw> #info the Fedora Media Writer test day went ahead and we had great attendance and test coverage, thanks everyone: http://testdays.fedorainfracloud.org/events/9
15:29:04 <adamw> looks like a lot of bugs were found and there's already been a big fmw update
15:30:03 <tenk> Is it normal that there is a new version of the tested product during the test day?
15:30:28 <satellit> there were several new ones..
15:30:42 <adamw> tenk: not super common, but fmw is a fairly new and quite important thing
15:31:18 <adamw> i'd say the most common pattern is that the maintainers of whatever we're testing try to get all the latest bits for testing lined up a few days before the test day, we test them during the event, and then they fix the bugs we find in the next week or two
15:31:30 <satellit> --best in linux is 3.97.0
15:31:42 <adamw> it has happened before though, if there turns out to be some really obvious bug in the thing-being-tested or whatever and we need to fix it rapidly.
15:32:03 * satellit no 3.97.1
15:32:33 <adamw> #info the i18n Test Day will be on Wednesday 2016-09-28, please do come along if you can help test non-English input etc. - https://fedoraproject.org/wiki/Test_Day:2016-09-28_I18N_Test_Day
15:32:55 <adamw> #action adamw and sumantro to blog, mail, social media etc. i18n Test Day
15:33:02 <adamw> oh yeah, another one for sumantro...
15:33:09 <adamw> #action sumantro to send post-event report for FMW Test Day
15:33:19 * adamw loves giving people work =0
15:33:46 <adamw> #info Wayland test day does not yet exist because adamw is bad and should feel bad
15:34:42 <adamw> any other test day notes before we move along? thoughts on the FMW event, especially from newer members - did you find it easy/fun to take part?
15:35:11 <tenk> That could be good to have  official UTC time of the start of the test
15:35:19 <tenk> end the end
15:36:13 <adamw> we usually try not to put a specific time frame on it so as not to discourage people from testing (as really, we can take test results from as soon as we have the test materials available up until a few days later)
15:36:31 <adamw> but maybe we can find a middle ground :)
15:36:42 <tenk> yes the second point is to know what version we should test
15:36:51 <adamw> (in the past we put specific times on, and people who wanted to test but happened not to be available at those times would just be discouraged)
15:37:13 <satellit> It would be nice to alert testers to do download just before test...as it did change versions
15:37:28 <adamw> yeah, sounds like that caused some problems
15:37:35 <tenk> yep
15:37:53 <adamw> i think i saw kparal or someone asking the maintainer to put versions in the filenames...
15:38:04 <kparal> they are there now
15:38:22 <adamw> #info update of FMW during test period caused confusion, we should try to avoid changes to test materials during test days as much as possible, and message clearly when they're needed
15:38:29 <tenk> It's just that the result page we don t know on which version the test have be made
15:38:29 <adamw> cool
15:38:38 <satellit> I noted version in comments on result page
15:38:52 <adamw> tenk: yeah, i think when sumantro set up the page we didn't expect there would be multiple versions
15:39:15 <adamw> thanks for the feedback, it's helpful :)
15:39:20 <tenk> and no possibility to edit result also
15:39:47 <satellit> I did add 2nd result by clicking top header though
15:40:52 <cmurf> 20m remaining
15:41:06 <adamw> tenk: yeah, the app is a fairly quick job that could do with some improvements
15:41:11 <adamw> thanks for the heads up cmurf
15:41:31 <adamw> #info people are still missing the obvious possible improvements for the testdays webapp (e.g. result editing)
15:41:48 <adamw> yipes, two more topics...let's get along :)
15:41:53 <adamw> #topic Workstation Atomic / Flatpak testing
15:42:13 <adamw> so this is kind of a heads-up, as i realized workstation WG have been talking to me about this but i didn't really pass it along
15:42:28 <adamw> they're planning to make a Workstation Atomic image, and they're interested in getting testing for it
15:43:02 <adamw> it would be non-release-blocking at first, of course, so testing would be on the same level as the Xfce/LXDE/Cinnamon images etc
15:43:26 <cmurf> it is a different world that is for sure
15:44:01 <adamw> so far i've basically been suggesting we could add it to the Desktop validation page and run mostly the same tests on it as we run on the regular Workstation image, with the update and upgrade tests adjusted for Atomic-ness, and maybe add a Flatpak install test which we could run on both images
15:44:06 <adamw> does that sound sensible?
15:44:10 <cmurf> user facing behavior should be similar or identical to now, but for QA people needing to see things under the hood, it's radically different
15:44:42 <cmurf> adamw: yes but I also think this starts to relate to your Project Ambition
15:45:06 <cmurf> because the old way of testing just doesn't scale as you're pointing out
15:45:07 <adamw> haha
15:45:30 <adamw> well, this would only grow an extra column on the 'non-blocking desktops' table really so it's not super horrible
15:45:34 <adamw> but yeah
15:45:56 <adamw> we can only stuff *so* many columns into wiki tables
15:46:01 <cmurf> right I get that, but it's also fitting a square peg into a round slot so...
15:46:28 <cmurf> better than nothing so yea sure do it
15:46:56 <adamw> this would be whenever they actually manage to get the image built as part of official composes by releng, that's kinda the point i was looking at.
15:47:16 <cmurf> yes i just saw releng's meeting notes
15:47:48 <cmurf> "ostree repo management needs redesigned and unified in pungi and bodhi"
15:48:21 <adamw> #info Workstation team are planning a Workstation Atomic image, expecting Flatpak to be the primary app delivery mechanism, and would like testing for it; we plan to add it to the Desktop matrix as a column in the non-blocking desktops table
15:48:22 <cmurf> releng changes, installer changes, file system changes...
15:48:31 <adamw> CHANGE ALL THE THINGS
15:48:37 <cmurf> change it now!
15:48:40 <adamw> i don't know if that's specifically related to workstation
15:48:43 <adamw> aaaaanyho
15:48:58 <cmurf> it's related to all atomic stuff (rpm-ostree really)
15:49:00 <adamw> did that get everyone up to date? any questions/objections before we move on to Project Ambition? :P
15:49:02 <adamw> right
15:49:11 <cmurf> Project Ambition
15:50:11 <adamw> alrighty, project ambition it is
15:50:18 <adamw> so we only have ten minutes, but quickly...
15:50:31 <adamw> #topic Release validation NG
15:50:57 <adamw> so in case anyone didn't see, i sent a mail with my initial thoughts on this: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/XO5WAZ6CVXUPD2W5DACXOFA3A257DJBV/
15:51:19 <cmurf> what's NG?
15:51:20 <adamw> the executive summary is: let's build a release validation testing submission/display webapp that's better than doing it in mediawiki
15:51:25 <adamw> Next/New Generation
15:51:28 <cmurf> gotcha
15:51:30 <adamw> just a silly name
15:51:32 <kparal> pity that jskladan is not here. but I think one of the major issues with using resultsdb frontend (e.g. similar to current testday app) is lack of auth support
15:51:42 <tflink> not just frontend
15:51:48 <kparal> for testdays, we simply disallowed editing and removal of results
15:51:52 <adamw> kparal: well, if we're using a webapp as an intermediary is that really a problem?
15:52:10 <tflink> FWIW, this is a usecase we've been talking about for years
15:52:18 <tflink> not that any progress has been made on it :-/
15:52:19 <adamw> we can do the auth there and just grant the webapp rights to do whatever it likes to resultsdb...
15:52:30 <adamw> tflink: yeah, this is indeed Episode ~8 in an ongoing story
15:52:39 <tflink> adamw: how do you track who submitted what, then?
15:53:14 <tflink> i suppose we could have a convention for a key/value pair attached to human-submitted resultsd
15:53:15 <adamw> tflink: fair point, but it would be easier to just add a field to resultsdb that it doesn't have to actually authenticate (it just trusts the webapp told it the right thing), wouldn't it?
15:53:15 <kparal> I'm a bit afraid we will need to implement quite a lot of stuff - what about edit history, do we care?
15:53:46 <tflink> I'd rather not allow result changing in resultsdb
15:53:50 <adamw> kparal: while i've been thinking about it i've been trying *not* to plan too much stuff, there's a lot we *could* do, but i think if we just focus first on making 'something better than the wiki' it wouldn't be too big
15:54:08 <adamw> there's lots to think about, i guess
15:54:36 <kparal> I'm definitely for the idea of using a more appropriate tool. otoh with wiki we have a lot of features for free
15:54:44 <adamw> but i think if we want something a) really tuned to our workflow and b) using resultsdb it's kinda gonna have to be 'build it yourself'...i think moztrap was our best shot at 'use something else' and it still didn't quite fly
15:54:57 <kparal> yeah
15:55:04 <cmurf> I think it's reasonable to have an ideal goal in mind that's also realistic, and then break that up into v1, v2, v3 milestones
15:55:06 <tflink> I'm about to use the s-word but it shouldn't be too bad to throw something together as a PoC assuming someone has the time to do it
15:55:10 <adamw> kparal: we *could* build a webapp which acts as an intermediary to *the wiki*, but that's getting dangerously close to insanity...:P
15:55:24 <tflink> adamw: you're on your own for that one :-P
15:55:28 <kparal> we need more layers
15:55:34 * satellit need to keep it simple for nubies to use   still not up to speed in editing in git and pagure...:/
15:55:35 <adamw> tflink: i was basically volunteering to at least *try* and do that. you know, with my extensive webapp development background
15:55:46 <adamw> satellit: oh, it'd be much easier than that. hopefully.
15:55:48 <cmurf> satellit: that's the idea
15:55:54 <satellit> +1
15:55:56 <cmurf> satellit: make it easier for people to contribute
15:56:09 <adamw> satellit: i'd hope it'd be pretty much like the wiki only easier: see a gap, click it, click pass/fail/warn, type your comment, profit
15:56:10 <tflink> adamw: that's the hardest I've laughed for a while :-D
15:56:10 <kparal> adamw: there has to be some useful related code somewhere on stackoverflow, no worries :P
15:56:21 <adamw> kparal: right? how hard can it be
15:56:30 <adamw> i googled flask yesterday, i've totally got this
15:56:33 <tflink> bah, I can't read
15:56:40 <cmurf> well, you need to figure out where to steal the underpants from first i guess
15:57:13 * tflink laughed because he thought adamw was volunteering someone else to do the webapp work
15:57:22 <adamw> tflink: oh, well, i can do that too :P
15:57:55 <adamw> so i'm guessing it'd be too much to hope there's a webapp development ninja idling here who just thought 'hey, this would be a great project for my huge amount of spare time'?
15:57:58 <adamw> ...
15:58:07 <adamw> ....
15:58:15 <adamw> thought so!
15:58:17 <cmurf> hence, where to steal underpants from
15:59:05 <garretraziel> he wouldn't be ninja if you could see him, would he
15:59:15 <adamw> so i'm gonna try and find some cycles to at least start fiddling with this and try to come up with some kind of mock-up (and teach myself webapp development at the same time!), but if anyone wants to help out / do it instead because they'd be better at it, please say so
15:59:16 <cmurf> maybe put the idea on devel?
15:59:34 <adamw> cmurf: good point, might be people on other lists interested in helping out; i'll send out a mail shot
15:59:50 <cmurf> yeah if it has a design writeup, that might help someone better understand what scope and action they'd be getting into
15:59:59 <adamw> #action adamw to troll for interest in release validation app development project outside of test@ list
16:00:11 <cmurf> step 1. steal underpants
16:00:15 <adamw> maybe i can talk to mizmo about it and come up with a design...
16:00:24 * tenk find the edit of the wiki for result quite simple
16:00:37 <adamw> tenk: yikes, really? i think you're the first person who ever said that :P
16:00:53 <tenk> adamw: yes really
16:00:54 <satellit> easy for me also...
16:01:02 <tflink> adamw: it'd be worth looking at the fancy js frameworks (eg angular.js) for the app instead of flask
16:01:17 <adamw> oh christ please don't make me code things in js
16:01:23 <adamw> anyhoo, we're over time
16:01:26 <tflink> :)
16:01:26 <cmurf> i think i busted the wiki the first three times i edited it, adamw has just blocked from memory having to fix it
16:01:36 <adamw> cmurf: haha, maybe a lil bit
16:02:02 * satellit can always roll back
16:02:04 <adamw> tenk: satellit: still that's interesting feedback, maybe it's not as big of a problem as i thought...maybe i'll talk to mo about doing some kind of user survey for it
16:02:15 <adamw> satellit: it gets tricky when someone else edited since the break, though
16:02:21 <adamw> alrighty, we should be over in #fedora-blocker-review right now
16:02:29 <cmurf> food and coffee time
16:02:30 <adamw> so let's wind things up here, and i'll start that meeting up...
16:02:45 <tenk> adamw: I just talk about to add test result on the wiki, not to add a new test or something else  (not try yet)
16:02:54 <adamw> #action adamw to do some very preliminary work on validation app planning, maybe with design WG
16:03:14 <adamw> tenk: please do poke me if you're interested in that stuff, it'd be great to have someone else who knows how the bits work
16:03:30 <adamw> tenk: be warned, though, if you start getting into how the pages are made, it gets into some crazy wikicode :P
16:03:38 * adamw doesn't even remember how it all works any more...
16:03:49 <adamw> alrighty, thanks for coming along everyone
16:04:01 <tflink> adamw: thanks for running the meeting
16:04:04 <adamw> release blocker review meeting starting in #fedora-blocker-review right now
16:04:14 <adamw> sorry for missing open floor - please send any urgent topics to the list
16:04:31 <adamw> (or bring them up in blocker review open floor, but usually only two people are alive by that point)
16:05:15 <adamw> #endmeeting