fedora-qa
LOGS
16:00:17 <adamw> #startmeeting Fedora QA meeting
16:00:17 <zodbot> Meeting started Mon Nov  2 16:00:17 2015 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:17 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:21 <adamw> #meetingname fedora-qa
16:00:21 <zodbot> The meeting name has been set to 'fedora-qa'
16:00:29 <adamw> #topic Roll call
16:00:39 <adamw> ahoyhoy folks, who successfully navigated the clock change?
16:00:51 <adamw> and didn't drink themselves to death testing F23?
16:01:07 * roshi is here! mostly adjusted to time and not drunk to death, yet
16:01:22 * pschindl is here
16:01:50 * kparal is here
16:03:03 <adamw> .fire roshi clearly not trying hard enough
16:03:03 <zodbot> adamw fires roshi clearly not trying hard enough
16:03:21 * tflink is here
16:03:24 <roshi> lol
16:05:04 <adamw> alrighty
16:05:08 <adamw> #topic Previous meeting follow-up
16:05:35 <adamw> oh hey looks like there isn't any
16:05:40 <adamw> #info no action items from previous meeting
16:05:52 <adamw> so we can go right on to...
16:06:06 <adamw> #topic Fedora 23 status and final tasks
16:06:28 <adamw> #info RC10 was signed off as Fedora 23 at the Go/No-Go meeting on Friday
16:06:39 <adamw> #info thanks to all for the huge amounts of testing
16:06:43 * kparal is working on documenting some of the remaining commonbugs
16:06:55 <adamw> yeah, that's one of the things we need to do now
16:07:11 * roshi put a couple of the cloud ones on there last week
16:07:16 <adamw> #action kparal and adamw to update Common Bugs
16:07:26 <adamw> kparal: we need to switch it over to the 'release mode' too, i can do that if you like
16:07:38 <kparal> huh?
16:07:49 <adamw> kparal: the boilerplate text changes a bit at release time
16:07:53 <adamw> there's a template for it, i can do it
16:07:58 <kparal> I see, thanks
16:08:26 <adamw> we also need to take care of updates
16:08:52 <adamw> the dnf-plugin-system-upgrade packages for F21 and F22 ideally need to get into stable, but i found a dep issue with lorax last week
16:09:07 <adamw> #action adamw to check in with bcl and wwoods on dnf-plugin-system-upgrade updates status and lorax
16:09:41 <adamw> other than that, i need to check in on the kf5 updates and make sure we don't regress stuff there...
16:09:55 <adamw> and there's the freeipa upgrade mess to document
16:10:10 <roshi> that's fun
16:10:48 <adamw> anyone think of any other final F23 tasks?
16:11:16 * roshi will being doing the heroes of fedora post this week
16:11:22 <adamw> cool
16:11:29 <adamw> you'll want to use relval git master, or wait for me to cut a release
16:11:49 <adamw> otherwise relval will trip up somewhat on the double-digit compose issue. not sure how bad it'd affect hte stats commands, but just in case
16:12:05 <roshi> sure, I can hold off
16:14:16 <adamw> alrighty
16:14:42 <adamw> there's some other housekeeping crap i'll do, mostly creating f25 blocker bugs, if anyone's interested see https://fedoraproject.org/wiki/BugZappers/HouseKeeping and https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers
16:15:16 <adamw> #topic Fedora 23 retrospective
16:15:40 <adamw> so, we didn't put the retrospective wiki pages up for f22, i can do that for f23 if there's interest
16:15:46 <adamw> but i figured we could kick it around here too
16:15:58 <adamw> the idea being, thoughts on how the f23 cycle went - what was good, what was bad, what can we improve next time
16:16:51 <kparal> openqa was awesome
16:17:20 <kparal> more coverage next time! :)
16:17:34 <Southern_Gentlem> it would be better if everyone would get their stuff done and QA didnt have to go into hero mode every release
16:18:10 <Southern_Gentlem> aka features
16:19:57 <adamw> Southern_Gentlem: yeah, this is something that comes up frequently...
16:20:04 <adamw> not sure what we can do about it in concrete terms though
16:20:35 <roshi> yell louder so people know that they, too, have to test?
16:20:42 <adamw> kparal: for f24 cycle we should have a lot more workers for openQA and hopefully we can also ditch i386 testing, which would be nice
16:21:11 <tflink> the only thing I can think of (and I'm not fully suggesting this) would be to drop features (even major ones) if they're not ready
16:21:15 <adamw> roshi: yeah, to pivot a bit, i'm worried in general that the project as a whole seems to be more or less assuming the release blocker / validation process is the Magic Thing That Tests Everything
16:21:41 <adamw> i keep having conversations which are implicitly predicated on the idea that all important bugs *must* be release blockers or FEs and if a bug isn't one of those things it's just not going to get fixed
16:21:44 <adamw> which isn't actually how it works
16:22:07 <adamw> tflink: that's supposed to be a part of the Change process, i think so far we've dropped maybe one or two Changes
16:22:36 * danofsatx has arrived. the meeting can start now.
16:22:47 <adamw> uh, okay, we'll, uh, start half way through the f23 retrospective
16:22:57 <tflink> the only thing I can think of is to start talking more about testing in general in the hopes that some non-qa folks would read it and start understanding more
16:23:03 <adamw> #info kparal says openQA was great, more coverage next time
16:23:25 <adamw> #info southern_gentlem notes that as usual, we had significant changes landing late
16:23:50 <roshi> yeah, that's true adamw
16:24:06 <adamw> i've got one: we badly need to come up with a better way of handling serious bugs that the 'release blocker' process doesn't really cover
16:24:22 <adamw> so i'm gonna write one! but if anyone else has great ideas for one, please let me know so i don't have to
16:24:25 <tflink> adamw: we could start tracking "serious/significant" buts
16:24:32 <jsmith> adamw: Can/should we publicly shame the features that landed late?
16:24:34 <tflink> bugs
16:24:44 <kparal> I think 'release blockers' covers those issue, we just consider it to be a 'compose blocker', which is a different thing
16:24:46 <adamw> tflink: that was more or less my idea, yeah, i'm a deeply unimaginative person: more trackers and more process! everyone loves process.
16:25:10 <adamw> kparal: yeah, to some degree it's semantics, but the process as currently constructed is really about the compose process
16:25:12 <tflink> adamw: it'd be nice to make it a more hands-off "process" that doesn't feel so much like a process
16:25:37 * danofsatx has a couple of ideas
16:25:40 <adamw> fire awawy
16:25:47 <tflink> something along the lines of "submit bug as a significant problem here", can be voted on by anyone, vetoed by a subset of folks
16:25:59 <danofsatx> I'll let 'em percolate for a bit, though, and send to list. They're not ready for public consumption yet.
16:26:40 <roshi> danofsatx, getting our hopes up...
16:26:50 <adamw> tflink: that's more or less what we had years ago when there was no formal review of the blocker / 'Target' trackers; anyone could just throw a bug on there. what tended to happen was no-one really cared
16:26:54 <danofsatx> it's called "antici...........pation'
16:26:57 <adamw> tflink: especially about the 'Target' tracker
16:27:07 <adamw> ooh, you tease, dan
16:27:12 <tflink> i think that we'll hit that no matter what, honestly
16:27:23 <tflink> it'll be an issue, rather
16:28:06 <tflink> "yet another tracker and system that devs dislike"
16:28:08 <adamw> tflink: to a degree, yeah, but i think - being very machiavellian - there's an effect where the more officious process nonsense is going on, the harder it is to ignore stuff
16:28:09 <roshi> yeah, I think so too
16:28:34 <adamw> partly the current blocker process works because we actually enforce it by slipping releases, but it also works just because we make a lot of noise about it and bug people and send them emails
16:28:50 <tflink> yeah, I don't think it's an insurmountable thing - just that it'll take more work than just "hey, here's this list of bugs that <people> think are significant"
16:28:58 <adamw> if we have a system where we more or less just make a list of bugs but don't have a big gong-banging din-making process around it, it's easier to ignore
16:29:36 <tflink> the other possible trap is giving submitters too much hope if it seems like a "vote here for whether or not bug X should be fixed"
16:29:40 <adamw> software engineering as public theatre :P
16:29:46 <adamw> tflink: yep, indeed
16:29:53 * danofsatx wonders if we can set an "Acceptable"  number of known bugs
16:30:08 <tflink> kinda like the terminology issue we had with NTH bugs
16:30:08 <adamw> danofsatx: i think that'd get extremely susceptible to fuzzing
16:30:15 <danofsatx> not just blocker-worthy, but total bugs
16:30:21 <roshi> yeah
16:30:36 <tflink> that strikes me as a terrible idea
16:30:51 <danofsatx> it would make people take notice
16:31:02 <tflink> when you start putting artificial limits on the process, the gaming starts and things get much worse
16:31:05 <danofsatx> I didn't say it was a good idea, it was just a thought
16:32:42 <roshi> perhaps if you want a feature in the next release, you have to commit to so many testcase runs per compose
16:32:57 <tflink> danofsatx: sorry if I seem like I'm jumping down your throat - that's just an example case I've seen  used to illistrate the destructive power of bad management
16:32:58 <roshi> which we can track with relval - hold features ransom for testing
16:33:26 <danofsatx> tflink: no, I understand - I was just thinking out loud. I expected to get shot down ;)
16:33:26 <tflink> what's a testcase run?
16:33:34 <adamw> yeah, i was wondering that
16:33:39 <roshi> result on the matrices
16:33:44 <tflink> it'd get gamed
16:33:47 <roshi> is the simplest thing I could think of
16:33:52 <adamw> roshi: what if the feature isn't really anything to do with release validation?
16:33:56 <tflink> and it would dilute the usefullness of the results as a whole
16:34:16 <roshi> I don't care about that adamw - they agree to doing release validation in addition to testing their feature :p
16:34:16 <danofsatx> if it is a marketed new feature, it should be validated
16:34:28 <tflink> sure, not arguing with that part
16:34:40 <adamw> so anyway, yeah, for the 'special blockers' stuff - which is USB writing app issues, upgrade issues, anything that would get fixed somewhere outside the release media - I will draft something up (taking account of all these thoughts), unless anyone else wants to take it
16:34:44 <tflink> metrics should be used to ask questions, not answer them
16:34:46 <roshi> yeah, I'm not really sure there is anything *to* be done
16:34:53 <adamw> roshi: haha. buy one feature, get one bunch of testing free
16:34:56 * linuxmodder late
16:35:12 <adamw> does anyone else want it?
16:35:15 <adamw> ahoy linuxmodder
16:35:36 <tflink> adamw: i can help but I don't have the spare cycles to take it on right now
16:35:41 <adamw> rgr
16:35:56 <linuxmodder> adamw,  howdy  saw your ipa post  that is  nasty
16:36:41 <linuxmodder> I picked up a clusterf of a webmaster gig or I'd be up for cycles
16:36:47 <adamw> #action adamw to draft up proposal for better handling of 'special blockers'
16:37:13 <adamw> so another thing i think bears mentioning, though i don't have any fixes: Test Days are kinda falling off lately
16:37:37 <adamw> again it's something we've noticed for a while and even been trying to counter, but we seem to getting sort of sucked into release validation all day every day
16:37:41 * danofsatx noticed the distinct lack of Test Days for 23
16:38:22 <linuxmodder> adamw,  next one  ping me  a few days ahead I'll add to my blog / G+  etc
16:38:23 <roshi> between release validation and working on tooling, that's where all our cycles have been going recently
16:38:35 <roshi> I think it'll get better as tools get closer to production
16:38:54 <roshi> openQA coverage, taskotron, etc
16:38:57 <adamw> well, we can hope ;)
16:39:11 <adamw> someone insert xkcd graph here
16:39:13 <roshi> yup
16:39:14 <danofsatx> well, there are a few of us that don't have the skillset for the tools, so why don't we get off our arses and run the Test Days?
16:39:32 <adamw> it would be great if more folks helped out with running test days, absolutely
16:39:33 <adamw> and it '
16:39:43 <roshi> for sure
16:39:45 <adamw> and it's something you basically need no special knowledge or permissions or anything to do
16:39:54 <danofsatx> I admit, I'm guilty of it myself.
16:40:07 <adamw> https://fedoraproject.org/wiki/QA/SOP_Test_Day_management is the guide to running a test day, for anyone who hasn't seen it
16:40:16 <danofsatx> only excuse I have is "Hey, life happens."
16:40:30 <adamw> #info general agreement that Test Days have been falling off lately and it'd be good to get them more alive again
16:41:00 <adamw> i'd note that it *does* take a decent chunk of time to run a test day really well, though, and it's great if you can commit to it - i used to spend a decent chunk of my working hours on them
16:43:01 <linuxmodder> adamw,  come  new year my schedule for such **Should** clear up fall /winter si always  shit for me
16:43:08 <adamw> that's good
16:43:41 <adamw> i guess the other interesting topic in the f23 cycle was: i386 keeps freaking breaking
16:43:52 <adamw> there are general project-wide moves afoot to stop caring so much about i386
16:43:58 <roshi> one thing to note, for F24 - if Atomic is going to be a blocking deliverable, we should have a couple people getting familiar with it and docker so it can be tested
16:44:14 <roshi> dropping i386 will make testing easier
16:44:15 <adamw> should we throw our weight behind them? if nothing else it reduces our combinatorial explosion nightmares a bit
16:44:31 <Southern_Gentlem> adamw,  well the problem is third party software that requires i686
16:44:33 <roshi> as I pretty much *only* did 32 bit testing
16:44:57 <Southern_Gentlem> skype steam etc
16:45:13 <adamw> Southern_Gentlem: you don't need an i686 install for those.
16:45:16 <adamw> only the relevant i686 packages.
16:45:20 <linuxmodder> roshi,   I'm attempting to wrap around  dockeer
16:45:34 <adamw> we don't have to stop building i686 packages at all, just stop blocking releases on the 32-bit media.
16:45:38 <danofsatx> 3rd party...do we care? it's not Fedora.
16:45:52 <linuxmodder> ^^ lol
16:45:52 <adamw> and yeah, someone has to make the first jump and fedora has a proud history of being it
16:46:02 <roshi> that's true
16:46:09 <tflink> i suspect that if we stop blocking on 32 bit, it'll stop working well
16:46:15 <adamw> we aren't that important in ourselves, but when we do something it creates a precedent for other distros to do the same and it comes with a heaping side order of implication that RHEL will do it at some point
16:46:32 <Southern_Gentlem> adamw,  i agree i dont see the need for 32bit media at all
16:46:45 <adamw> tflink: people who really care about 32-bit third party apps can always do the necessary work to make sure their dependencies work alright
16:46:51 <danofsatx> CentOS 7 didn't have 32 bit until just a couple months ago (and I'm not sure its "officially supported")
16:47:07 <tflink> adamw: what if the installer doesn't work on 32 bit because it hasn't been tested?
16:47:12 <adamw> danofsatx: yeah, i may be behind the times in terms of RHEL actually, i have no idea what RHEL ships / blocks on.
16:47:19 <kparal> tflink: I'm quite sure steam's deps will be kept in good shape :)
16:47:42 <adamw> tflink: well, i mean, that's kinda the question.
16:47:55 <brunowolff> Some of us still use i686 machines. (Though mine seem to be dying off this year.)
16:48:10 <adamw> tflink: do we get behind the various proposals to stop blocking on 32-bit media or shipping them at all
16:48:25 <roshi> I say we back them on it
16:48:34 <roshi> +1 to dropping it to a secondary arch
16:48:54 <adamw> roshi: it's not quite the same thing as secondary arch
16:48:55 <tflink> when the cloud WG changed their "main" deliverable, there were assurances that the base cloud images would still work even if they weren't blocking. i don't think that the same could be said about non-blocking 32 bit install images
16:49:18 <tflink> and if there are no working 32 bit install images, why build 32 bit packages at all?
16:49:39 <roshi> tflink: the use case I could see is, install from older media, and upgrade
16:49:41 <adamw> tflink: see above.
16:49:42 <brunowolff> Not blocking seems to be pretty reasonable. I think there should still be some way to easily install Fedora on an i686 machine.
16:49:56 <adamw> brunowolff: the thing is, if we don't block on it working, it's very likely not to.
16:50:06 <roshi> or, we could just do i386, since it will run on x86 hardware
16:50:10 <roshi> and then it'd be done
16:50:10 <tflink> adamw: for which concern?
16:50:12 * roshi ducks
16:50:19 <adamw> tflink: running third-party stuff that's 32-bit only
16:50:28 <adamw> i.e. Steam.
16:50:29 <tflink> oh, good point
16:50:45 <tflink> not sure how i forgot about that
16:50:56 * tflink blames monday morning with only 1 cup of coffee
16:51:05 <adamw> fwiw, i'd vote entirely in favour of any move to drop i686 media. yes, there's a cost to doing everything, but we definitely need to reduce the scope of releases somehow
16:51:16 <tflink> roshi: so we'd eventually be saying "install f22 and good luck upgrading to f26"
16:51:18 <linuxmodder> make libs available but not  the os tree maybe?
16:51:40 <brunowolff> I still have two i686 machines I use that typically run branched. So I am likely to notice busted kernels or live images, install specific issues not so much.
16:51:53 <adamw> the Fedora 23 release has *61* images
16:51:55 <roshi> tflink: it was an answer to "if we don't have 32bit installer why have 32 bit packages" - but that's where it'll likely end up, yes
16:51:55 <tflink> yeah, I think that not blocking will effectively be the same thing as not building it
16:51:56 <adamw> that's ridiculous
16:52:09 <tflink> roshi: adamw's answer was better :)
16:52:17 * tflink just wasn't thinking correctly
16:52:24 <adamw> if we ditched i686 that would cut out 21 of them, so we'd have 40, which is still ridiculous, but somewhat less so. :)
16:52:31 <roshi> sure - but that's also what I'd have expected :p
16:52:45 <brunowolff> We may want to have just one supported media for installing i686.
16:53:05 <adamw> in practice that's probably the path we'll wind up going down
16:53:06 <tflink> would that work? only have server netinstall for 32 bit?
16:53:17 <adamw> what's actually happening is that various WGs are proposing to drop their 32-bit media
16:53:23 <adamw> tflink: i think Server may well be dropping 32-bit for F24.
16:53:49 <adamw> so it sounds like we have a few caveats but we're generally in favour of proposals to drop 32-bit media/stop blocking on them?
16:53:56 <tflink> or some netinstall to reduce the number of images
16:54:16 <tflink> I'm pretty -1 on not blocking on 32 bit media
16:54:25 <tflink> either block on them or don't build them as part of the release process
16:54:59 <Southern_Gentlem> i really think this is a FESCO question
16:55:08 <adamw> tflink: yeah, that's a reasonable point
16:55:17 <roshi> Southern_Gentlem: this is just if we, as QA, want to support the idea
16:55:22 <roshi> not make the call
16:55:29 <tflink> the "build and don't block on" thing is going to cause confusion and wasted disk space
16:55:35 <adamw> Southern_Gentlem: in some cases it's up to WGs, in others up to FESCo, i just wanted to see if we had a common PoV on it as a group
16:55:36 <tflink> s/is going/would
16:55:38 <roshi> yeah, that's true
16:55:47 <adamw> Southern_Gentlem: so we can then agitate for that PoV in WG/FESCo/whatever discussions
16:55:48 <brunowolff> Wouldn't this be more a council thing than FESCO?
16:56:03 <adamw> brunowolff: doesn't really matter, point is it's not us :)
16:56:14 <adamw> i just think it's handy if we can say 'QA thinks X' and have this discussion to document it
16:56:26 <Southern_Gentlem> adamw,  how are you seeing 61 images
16:56:40 <adamw> Southern_Gentlem: count them in the table at the top of https://fedoraproject.org/wiki/Test_Results:Fedora_23_Final_RC10_Installation
16:57:08 <adamw> ok, a few are boot.iso dupes, so the real count is probably 54
16:57:23 <brunowolff> Right, but we should tell a more encompassing group that supporting QA for i686 is a lot of QA work, for probably little benefit.
16:58:04 <adamw> brunowolff: i was figuring wherever the discussions come up, we can provide our input
16:58:08 <adamw> ok, so we're nearly at time
16:59:00 <adamw> #agreed there's a general consensus among QA that 32-bit install media produce a lot of testing/bug fixing work for little benefit and we're generally in favour of dropping most or all 32-bit install media
16:59:05 <adamw> that sound alright?
16:59:38 <roshi> ack
16:59:59 <tflink> generall in favour/wouldn't fight - depends on desired strength
17:00:02 <brunowolff> ack
17:00:25 <adamw> tflink: i prefer 'generally in favour' if you guys will wear it ;) i really really want to kill them with fire.
17:00:38 * tflink doesn't have strong feelings about it
17:00:56 <tflink> it -> that particular phrasing
17:00:58 <tflink> ack
17:01:22 <kparal> ack
17:01:54 <tflink> do we want to add "not in favor of having non-blocking 32 bit release media built as part of the release process"
17:02:17 <roshi> I'd be ok with that
17:02:29 <Southern_Gentlem> ack
17:02:42 <tflink> block or drop :)
17:02:49 <adamw> if it's a choice between that and no change at all i'd prefer that, but i'd prefer dropping them entirely over having non-blocking ones, yeah
17:03:10 <danofsatx> wait, is it favor or favour?
17:03:18 <adamw> how about: #agreed QA also doesn't think that having non-blocking 32-bit install media is a great compromise, if it's proposed, and would prefer to drop them entirely
17:03:30 <adamw> danofsatx: former is US, latter is UK
17:03:31 <Southern_Gentlem> yes
17:03:34 <tflink> i'd make it a bit stronger
17:03:47 <adamw> okay...
17:03:58 <danofsatx> adamw: I know, just trying to cause trouble. I'll shut up now ;)
17:04:06 <adamw> then: #agreed QA thinks it would be a bad idea to have non-blocking 32-bit install media, and dropping them entirely would be preferred
17:04:13 <roshi> ack
17:04:31 * roshi is still kinda sad to see 32bit go, since there's still so much 32bit hardware out there...
17:04:33 <tflink> like: #agreed QA also thinks that having non-blocking 32-bit install media would be a bad move, leading to stale bits in the release. If non-blocking install media is proposed, we would prefer to drop them entirely
17:04:36 <danofsatx> ack
17:04:38 <brunowolff> ack
17:04:41 <Southern_Gentlem> ack
17:04:44 <tflink> that works, too
17:04:49 <tflink> and is shorter :)
17:04:50 <tflink> ack
17:05:08 <adamw> #agreed QA thinks it would be a bad idea to have non-blocking 32-bit install media, and dropping them entirely would be preferred
17:05:10 <adamw> alrighty, we're over time
17:05:14 <adamw> so very quickly
17:05:16 <adamw> #topic Open floor
17:05:23 <adamw> anyone have anything burning that can't hold for next week?
17:05:39 * roshi has nothing
17:05:45 <danofsatx> see ya next week
17:06:11 <adamw> thanks dan!
17:07:02 <brunowolff> I saw some wierd size changes in the games live live image composes, For F24 I hope to have time to figure out what is going on if I still see them.
17:07:17 <adamw> dgilmore probably has some idea, and  bcl
17:07:50 <brunowolff> They didn't seem likely to be caused by package changes, but rather something else in the composes.
17:07:52 <adamw> alrighty, thanks for coming everyone!
17:08:04 <adamw> brunowolff: yes, i mean, they're the folks who know the compose process and the tools
17:08:29 <adamw> see you on the lists
17:08:31 <adamw> #endmeeting