fedora-qa
LOGS
16:00:15 <adamw> #startmeeting Fedora QA meeting
16:00:16 <zodbot> Meeting started Mon Feb 16 16:00:15 2015 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:16 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:22 <adamw> #meetingname fedora-qa
16:00:22 <zodbot> The meeting name has been set to 'fedora-qa'
16:00:25 <adamw> #topic Roll call
16:00:29 <adamw> ahoyhoy folks, who's around?
16:00:34 * roshi is here
16:00:48 * kparal just joined
16:00:48 * mkrizek is here
16:01:02 * satellit_e here for 3/4 hr
16:01:03 * nirik is lurking in the back by the bacon
16:01:15 * pschindl is here
16:01:31 * tflink is alwo here
16:02:02 * jskladan lurks
16:02:39 <adamw> #chair stellit_e jskladan
16:02:39 <zodbot> Current chairs: adamw jskladan stellit_e
16:02:47 <adamw> and now let's try that gain with real people
16:02:51 <adamw> #chair satellit_e
16:02:51 <zodbot> Current chairs: adamw jskladan satellit_e stellit_e
16:02:57 * dextone listening
16:03:15 * jreznik is around
16:03:31 <adamw> thanks for coming out, everyone
16:03:39 <adamw> all having a good monday?
16:03:52 <roshi> oh yeah, got a bunch of snow last night :)
16:04:03 <adamw> damn yooooouuu
16:04:15 <adamw> http://www.snow-forecast.com/resorts/Whistler-Blackcomb/6day/mid
16:04:23 <roshi> rugby world series over the weekend, what's not to like? :)
16:04:36 <dextone> almost tuesday here
16:04:50 <adamw> so you can make a very accurate assessment!
16:04:53 <adamw> where's here?
16:05:04 <dextone> Indonesia :)
16:05:05 * adamw counts...er...india?
16:05:09 <adamw> ah, not too far.
16:05:12 <dextone> haha
16:05:18 <adamw> (time zone speaking.)
16:05:26 <jreznik> it's nice winter-spring weather here :)
16:05:44 <adamw> alrighty, gonna skip previous meeting follow-up if  no-one minds as the only action item was for the test day and we have a test day topic anyway
16:05:57 <adamw> sooo moving right along to:
16:06:02 <adamw> #topic Fedora 22 Alpha status
16:06:21 <adamw> well, obviously, about a dozen manual testers, anaconda CI, and coconut all tell us...it's broke.
16:06:51 <adamw> in case anyone didn't see it yet, current f22 installer showstopper is https://bugzilla.redhat.com/show_bug.cgi?id=1192702 , will likely be resolved in the next day or two
16:07:20 * satellit_e installed with calamares  but not in anaconda 22.19-1
16:07:30 <adamw> aside from that, though, wanted to get a broader overview of where we are - anyone have any thoughts or notes?
16:08:43 <Corey84> .fas corey84
16:08:44 <zodbot> Corey84: corey84 'Corey84' <sheldon.corey@gmail.com>
16:09:20 <roshi> not really, though I think we got some good coverage of anaconda from the testday
16:09:29 <Corey84> the net install still broke ?   havent had chance to check
16:09:41 <satellit_e> yes no boot.iso today
16:09:46 <satellit_e> that works
16:09:50 <adamw> right
16:10:10 <adamw> yeah, i think we're mostly in decent shape; there are various improvements we could make to the test case set but we can still do that as we go along i guess
16:10:18 <adamw> release criteria revisions mostly got done i think
16:10:50 <adamw> dgilmore: nirik: either of you around? any estimate on the difficulty of doing tc1 once we have a non-showstopped package set?
16:13:21 * Kinokoio is here
16:14:16 <adamw> welp, guess not.
16:14:42 <adamw> moooving right along
16:14:52 * Corey84 hopes to get  this  MBP i7 equipped to make this  testing much easier
16:15:07 <adamw> #info Fedora 22 status is that we're waiting for showstoppers to clear and 22 Alpha TC1 to land, we have reasonable coverage so far through nightlies and anaconda/DNF test day
16:15:16 <adamw> #topic Project Coconut (OpenQA)
16:15:44 <nirik> adamw: I would think it would be pretty fine to make one once we get the goahead. I dont know of any stoppers.
16:15:46 <adamw> so i hope the folks working on this don't mind me blowing their cover :) but it seemed like some smart cookies had it figured already
16:15:51 <adamw> nirik: rgr, thanks
16:16:10 <adamw> nirik: guess i'm just a bit worried about the product-y netinst / generic netinst stuff but we'll burn that bridge when we come to it
16:16:55 <nirik> yeah, not sure where that is
16:17:28 <Corey84> adamw, netinstall stuff ?
16:18:06 <adamw> ok, switch topics back again...
16:18:06 <idraulico> quit
16:18:13 <adamw> #topic Fedora 22 status part 2
16:18:18 <adamw> just to keep things organized
16:18:31 <adamw> Corey84: there's a planned change to the media for f22
16:18:52 <adamw> server and workstation (at least, not sure about cloud) will have dedicated network install images that are actually different
16:19:10 <Corey84> ah will have to relook  as was not terribly happy with  the netinst  script fails on 21  tbh
16:19:37 <roshi> cloud will only have raw/qcow2 and AMIs - no need for a netinst, doesn't make any sense aiui
16:19:38 <adamw> and there will also be a 'generic' network install iamge
16:20:02 <adamw> this is the plan, anyhow, and releng apparently has an idea how to do it, but as with the f21 plan, we'll see what happens when we actually have to buiild it :)
16:20:53 <adamw> Corey84: script fails?
16:20:57 <Corey84> so 2  purpose builts then one that is akin to the current one that is no usb/cd   user friendly?
16:21:17 <Corey84> yeah,  one sec
16:22:10 <adamw> Corey84: basically. i suspect the differences between the three will end up just being the default package set and logos and maybe one or two other things, but that's the design.
16:23:16 <adamw> allrightly, moving back along...
16:23:21 * nirik still thinks it's overkill, but hey, whatever folks want to do
16:23:46 <adamw> nirik - giving the people what they want
16:23:59 <Kinokoio> One ISO to install them all is not better?
16:24:23 <Kinokoio> in case of netinstall, of course
16:24:27 <Corey84> its in the pyanaconda scripts that fail adamw
16:24:29 <roshi> depends on what angle you're viewing the term "better" from
16:24:29 <adamw> Kinokoio: well, if the products want to be significantly different from each other, it becomes difficult to do that with the way the installer's built, aiui
16:24:48 <adamw> there's kind of a whole nexus there which is being worked out sort of...fuzzily...
16:25:30 * satellit_e DVD with generic/workstation/server first screen like multi DVD we used to do?
16:25:53 <adamw> heh, rebirth of multi-image media
16:25:59 <adamw> that's actually not such a bad idea
16:26:11 <adamw> wonder if anyone had considered it yet
16:26:19 <satellit_e> : )
16:27:07 <Corey84> that would be a huge dvd tho would it not
16:27:25 <Corey84> like cOS7 large
16:27:35 <adamw> not just for netinsts, no
16:28:42 <adamw> ok, moving back along...
16:28:50 <adamw> #topic Project Coconut
16:29:31 <adamw> so folks had been noticing big piles of validation results from 'coconut' in the matrices
16:30:33 <adamw> coconut is a sort of quick hack automated QA system a few of the rh folks have been working on, using openQA
16:30:48 <adamw> https://os-autoinst.github.io/openQA/
16:31:39 <adamw> the openQA folks have been bugging us about it for years and got a demo running against fedora at christmas, so a couple of folks finally decided why not try and get something rough running until we have install testing in taskotron or whatever, and it came up rather quickly
16:32:10 <kparal> big kudos to jskladan and garretraziel
16:32:31 <adamw> because it's a quick hack project the instances that are up and running atm are behind the rh firewall, but all the code and scripts and tests and stuff is open and you could set up an instance elsewhere pretty easily i believe
16:32:36 <nirik> has it found any breakage yet?
16:32:59 <adamw> though we haven't bothered making openqa itself run on fedora yet, the coconut boxes run opensuse :)
16:33:00 <kparal> I think lbrabec reported one bug today
16:33:11 <adamw> nirik: it caught the same showstopper everyone else did
16:33:27 <nirik> cool.
16:33:29 <kparal> for the moment, openqa reports just passes into the wiki, and we manually inspect the failures
16:33:31 <adamw> but so far it's actually been designed mostly to do box-ticking for passes
16:33:38 <nirik> excellent work
16:33:39 <kparal> but jskladan can probably tell you more
16:33:41 <adamw> it's configured to only report passes to the public wiki
16:33:54 <adamw> i'd quite like to get it to run through the bug submission process when it hits a crash and report a failure, but hey
16:34:16 <jskladan> nirik: in practice, the Coconut will not report failures/bugs on itself, though. We only want to have it reporting passes, so the ever-repeating work on testing all the "simple" testcases is a bit less boring...
16:34:27 <adamw> so, i thought it'd be good to explain what's going on there and maybe see if folks had any questions or were interested in helping (if we can work that out) and maybe we could talk a bit about how much more we want to do with it in f22 timeframe
16:35:16 * satellit_e I assume sugar (soas) testing will remain manual?
16:35:28 <adamw> satellit: coconut doesn't take anything over
16:35:37 <adamw> all it ultimately 'does' is file wiki reports, like any other tester
16:35:57 <adamw> everyone's free to run the same tests manually, of course, as well
16:36:03 <satellit_e> k
16:36:05 <adamw> right now it does 7 or 8 of the core Installation tests
16:36:51 <satellit_e> another test we do not do is installation of 2nd DE and compatability
16:37:11 <jskladan> adamw: https://bitbucket.org/rajcze/openqa_fedora_tools/src/ee062fac7b729c1600af1664010241f62e76988d/PhaseSeparation.md?at=master these are the covered testcases ATM
16:38:08 <adamw> jskladan: thanks
16:38:09 <roshi> slick :)
16:38:14 <satellit_e> nice
16:38:19 <adamw> so yeah, if anyone wants to play with the bits, they're all there:
16:38:24 <adamw> https://bitbucket.org/rajcze/openqa_fedora_tools/
16:38:25 <jskladan> adamw: ad plans - the consensus (at the moment) is that garretraziel will provide tests/checks for the rest of the un-covered Beta testcases, and then we'll "let it live by itself", checking up on the failures, and (if caused by a bug outside of the openqa) reproducing the errors manually. This is something at least pschindl, garretraziel and lbrabec will be participating on (IIRC)
16:38:57 <nirik> so eventually this might live in taskotron or beaker? (once there are disposable clients?)
16:39:00 <adamw> there's even an 'installation guide' tim wrote when he set up a new box, which should more or less cover what you'd need to do to set up your own coconut - but obviously we don't want too many automation bots running around reporting results wildly, so use with care :)
16:39:44 <adamw> oh, btw, any reports from 'colada' are from the instance of coconut i have to play around with.
16:40:32 <kparal> nirik: there are some big deployment challenges atm. and it does not support distributed test clients
16:40:34 <tflink> nirik: depends on what you mean by "this"
16:40:43 <adamw> jskladan: cool. i had some thoughts/plans along the lines of integration with compose releases, but we're just coming to the end of nominated nightlies for f22 so it might be best to wait for f23 there
16:41:10 <adamw> nirik: openqa is a pretty 'vertically integrated' system with its own unique test format and stuff
16:41:11 <nirik> sure, lots to be figured. ;)
16:41:25 <adamw> i suppose you could have mini-coconuts *within* taskotron, that'd be nice and insane!
16:41:30 <nirik> can/will coconut go on with rawhide?
16:41:35 <adamw> nirik: it can for sure.
16:41:36 <kparal> everything runs on a single box, that's the only supported approach by openqa
16:41:43 <jskladan> nirik: this (as in OpenQA) is not really in any way tied to disposable clients, and we do not plan on running it in beaker/taskotron ATM. The thing is, that you need to have an OpenSuSe box to run it, and we are not really going to put the time into porting it all on  Fedora, as it would immediately loose the bang-for-the-buck advantage it has now
16:41:58 <nirik> ah, I see. ok
16:42:25 <adamw> nirik: it can run on just about anything, at least with my patches to use fedfind
16:42:32 <adamw> which jskladan is going to review real soon now right? :P
16:42:55 <adamw> i'd like to hack it up a magic mode where it runs on everything it can find every day - rawhide, branched, tc/rc.
16:42:59 * jskladan just applied the patches in his git ;)
16:43:35 <kparal> poor jskladan, it's really hard to keep up with adamw's rate of development :P
16:45:20 <adamw> so my brief sketch plan of an interesting thing to do with it was invent a new type of result page that was for coconut's results only
16:45:42 <adamw> Test Results:Automatic (compose version) or something
16:46:09 <adamw> coconut could report results for all builds it could find every day, and then we could do stuff with that data
16:46:16 <adamw> like not nominate nightlies that coconut found were DOA
16:46:38 <adamw> oh, the other thing i'd like to play with is getting it to emit results via fedmsg
16:47:04 <kparal> that's going to be tricky, because fedmsg is not packaged for opensuse, I believe
16:47:08 <adamw> kparal: it's in pypi
16:47:22 <adamw> i have it on the qa-01 box atm
16:47:29 <kparal> ok
16:47:32 <adamw> looks like it needs some config file or something but shouldn't be too hard to fix
16:48:53 <adamw> those will probably be fun time projects though now we're into alpha/beta phase, as they don't achieve anything we can't do relatively easily semi-manually (it's easy enough just to fire off a coconut run manually on a tc/rc before announcing it)
16:49:19 <adamw> soo, yeah, thanks a lot to the folks who got that up and running! hopefully it'll save us monkey work on some of the boring test cases for the 22 cycle
16:49:35 <adamw> if anyone has q's outside the meeting, qa-devel@ list is probably the place
16:51:38 <adamw> any other thoughts/notes on it before we move on?
16:51:53 * roshi is going to have to play with it :)
16:52:20 * satellit_e afk
16:52:24 <adamw> roshi: it's pretty neat to be honest, i'm having fun
16:52:49 <kparal> as long as you don't need to get your hands dirty with perl :)
16:52:58 <roshi> it looked like it when I glanced at it before
16:53:01 <roshi> lol
16:53:13 <adamw> roshi: openqa_fedora_tools just wraps the whole thing in a bunch of python :?
16:53:45 <adamw> kparal: jskladan: oh, i talked to richard brown about coconut a bit and he said some of the stuff you're doing in fedora_tools looks like stuff openqa can do in core, i'll pass his notes along in a bit...
16:54:04 <kparal> adamw: great
16:54:22 <adamw> alrighty, just quickly:
16:54:25 <adamw> #topic Test Days
16:54:29 <adamw> oh damn
16:54:33 <adamw> #topic Project Coconut
16:54:51 <threebean> adamw: real quick on coconut.  for the results page, you could just copy the source for the releng-dash
16:54:58 <threebean> and modify it to query datagrepper for coconut messages.
16:54:59 <adamw> #info Coconut is a quick-and-dirty deployment of openSUSE's OpenQA running a limited set of tests on Fedora composes and reporting results to the wiki.
16:55:13 <adamw> #info see: https://bitbucket.org/rajcze/openqa_fedora_tools and https://os-autoinst.github.io/openQA/
16:55:56 <adamw> threebean: oh, i have a whole thing that does results pages, i'm the crazy-haired mad genius there. ;)
16:56:31 <adamw> #topic Test Days
16:56:45 <adamw> so the Anaconda/DNF Test Day went off pretty well i thought? did everyone have a good time?
16:57:06 <roshi> yeah, I thought it went well
16:57:16 <roshi> and we have a few more proposed/scheduled as well
16:58:17 <danofsatx> I'm late
16:58:24 <adamw> i'll try and get results from the anaconda/DNF day transferred into the wiki and a follow-up email sent out soon
16:58:36 <adamw> did anyone have any issues with the event or thoughts on how we could do it better or anything?
16:58:55 <roshi> the only thing I thought would have been nice
16:59:16 <roshi> since kickstarts were a focus, is a good noob guide to writing kickstarts
16:59:25 * roshi can write it
17:00:28 <pschindl> or there is a tool - system-config-kickstart
17:00:45 <pschindl> for making simple kickstarts
17:00:54 <roshi> ah, didn't know about that
17:00:59 <pschindl> it is gui-tool btw
17:01:17 <pschindl> Me neither until I saw him one hour ago :)
17:01:34 <adamw> it's pretty old, i'm never sure how up to date it is :/
17:02:34 <adamw> but yeah, that would've been a good idea
17:02:45 <adamw> i wasn't quite sure where to pitch the whole thing between beginners and experienced testers
17:02:57 <Kinokoio> When will be scheluded the next tests for anaconda?
17:02:58 <adamw> #info Anaconda/DNF Test Day went off well with good attendance and multiple bugs found
17:03:10 <adamw> Kinokoio: we'll just have the regular validation testing from this point on
17:03:15 <adamw> we'll have lots of other test days, though
17:03:29 <adamw> roshi: what's coming up next?
17:04:08 * roshi looks
17:04:36 <roshi> Cockpit Test day in March
17:04:46 <roshi> 2014-03-24
17:04:46 <adamw> okey dokey
17:04:51 <adamw> mark your calendars, folks
17:04:59 <adamw> #info currently next planned Test Day is Cockpit Test Day on 2015-03-24
17:05:01 <adamw> (FTFY)
17:05:30 <adamw> really quickly:
17:05:33 <adamw> #topic Open floor
17:05:41 <adamw> any other thoughts / notes / philosophical ramblings?
17:06:56 <roshi> Blocker meeting to commence after this meeting and a 5 minute coffee break sound good?
17:08:28 <adamw> sure
17:08:58 <kparal> ok
17:09:24 <adamw> alrighty, setting the TinyFuse
17:11:18 <adamw> thanks folks!
17:11:20 <adamw> 3.2.1
17:11:22 <adamw> #endmeeting