16:54:05 <mattdm> #startmeeting Council (2015-06-08)
16:54:05 <zodbot> Meeting started Mon Jun  8 16:54:05 2015 UTC.  The chair is mattdm. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:54:05 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:54:08 <mattdm> #meetingname council
16:54:08 <zodbot> The meeting name has been set to 'council'
16:54:10 <mattdm> #chair mattdm jreznik jwb cwickert rdieter langdon sgallagh decause
16:54:10 <zodbot> Current chairs: cwickert decause jreznik jwb langdon mattdm rdieter sgallagh
16:54:12 <mattdm> #topic Introductions, Welcomes
16:54:21 <mattdm> #topic Fedora QA subproject report
16:54:46 <mattdm> We're getting the hangout going....
16:54:52 <rdieter> hola
16:55:08 * cwickert is here
16:55:20 <mattdm> http://youtu.be/7arJcuUNHGg
16:56:19 * decause cracks knuckles and prepares for live transcription
16:57:35 <mattdm> we're waiting for langdon to get to the beach.
16:57:56 <decause> langdon++
16:58:56 <decause> here we go
16:58:59 <decause> hi everybody
16:59:03 <decause> this is mattdm, of hte fedora project
16:59:09 <decause> this is the 3rd time we've done this
16:59:29 <decause> we wanted to get into the routine of checking in with diff parts of the projects, with semi-formal presentations
16:59:38 <decause> we have roshi, reporting back on the QA Team
16:59:51 <decause> they work to make sure that when the bits go to users, they are not broken
16:59:57 <decause> can everyone see slides?
16:59:58 <decause> good
17:00:17 <decause> I'm mike, I live CO
17:00:24 <decause> we're going to go over the state of th eteam
17:00:26 <decause> and future plans
17:00:36 <decause> and then finally a list of stuff we'd like that would make our lives easier
17:00:49 <decause> roshisdog++
17:01:13 <jkurik> hi there
17:01:29 <decause> #topic Current State of QA
17:01:38 <decause> adam just sent out some emails
17:01:41 <Oneiroi> gives a whole new meaning to "the dog ate my homework"
17:01:56 <decause> we have a wellknown/understood process for release candidates, where to get them, and how to report on them
17:02:05 <decause> this helps us do the minimum amount of testing
17:02:18 <decause> getting base validation of the images we put out is a pretty well oiled machine
17:02:33 <decause> our updated testing, we have a pretty well doc'd way of testing packages as they go into bodhi
17:02:40 <decause> not sure if everyone is famliar with easykarma
17:02:51 <decause> but it will show all updates that need karma, or are pending
17:02:54 * rdieter hugs easykarma
17:02:57 <decause> then allows you to provide karma from cli
17:02:59 <decause> #link plz
17:03:12 <decause> there is fedora gui karma, which is the same thing, but with a GUI
17:03:34 <decause> we don't have releases that are about to be EoL
17:03:43 <decause> there is a somewhat lack of folks doing the up/down voting there
17:03:47 <decause> but the process is solid
17:03:52 <decause> we need a good way to do that
17:03:53 <rdieter> https://apps.fedoraproject.org/packages/fedora-easy-karma
17:04:13 <decause> #action decause work with QA on a "EoL" series of karma badges
17:04:14 <rdieter> https://fedoraproject.org/wiki/Fedora_Easy_Karma
17:04:25 <decause> taskotron has replaced autoQA
17:04:35 <decause> it is mostly under the hood though, so most don't notice th difference
17:04:43 <decause> once it is done, it will help automate many of our tests
17:04:54 <decause> tflink and martin and chamille(sp) have worked hard
17:05:02 <decause> and they ahve a well defined roadmap
17:05:11 <tflink> chamille - kamil
17:05:16 <decause> tim will continue writing blogposts to help show what things people are working on
17:05:21 <decause> tflink++
17:05:22 <zodbot> decause: Karma for tflink changed to 3:  https://badges.fedoraproject.org/tags/cookie/any
17:05:23 <decause> thanks
17:05:28 <tflink> np
17:05:33 <decause> through F22, we were testin ga tool from OpenSUSE called OpenQA
17:05:35 <decause> tl;dr
17:05:44 <mattdm> tflink blog posts are at:
17:05:45 <mattdm> #link http://tirfa.com/
17:05:47 <decause> spins up a VM, and then analyzes a screenshot of install as it happens
17:05:54 <mattdm> #info for updates on tasktron http://tirfa.com/
17:05:57 <decause> uses Needles to look for buttons in a certain area, etc
17:06:01 <decause> we hvae tests running in QA
17:06:12 <decause> not a public instance yet, but you can see the results for a user named coconut
17:06:15 <decause> that is openqa
17:06:23 <mattdm> https://fedoraproject.org/wiki/Test_Results:Fedora_22_Final_RC3_Installation
17:06:26 <decause> I know adam is working on getting all the relevant bits into Fedora
17:06:35 <decause> once everything is packaged, we'll migrate and expand it
17:06:42 <decause> we can get more than just base validation from there
17:06:53 <decause> freeing up time from manual testing to do exploratory testing
17:07:05 <mattdm> (this explains the kind of scary robot logo by the coconut user in the test matrixes)
17:07:10 <decause> with changes now, we really only have time to cherrypick what looks like the most obvious disrupters in a change
17:07:16 <mattdm> #user "coconut" in test matrix is actually from openqa
17:07:19 <mattdm> uh
17:07:22 <mattdm> #info user "coconut" in test matrix is actually from openqa
17:07:25 <decause> we can test on that, but we rarely get to thoroughly test every change, due to lack of manpower
17:07:40 <decause> #topic Future Plans
17:07:54 <decause> 3 things in Taskotron
17:07:58 <decause> 1) disposable clients
17:08:18 <decause> it will boot a cloud image, in a VM, and then insert all the required stuff for the taskotron task, and report back to the host where the vm is run
17:08:28 <decause> we're still just out of the Proof of concept stage
17:08:36 <decause> this will enable much more automated desktop tests
17:08:48 <decause> GNOME has a suite we hope in the next coupel years, to have the caffolding to run too
17:08:51 <mattdm> #info taskotron disposable clients will allow better automated desktop tests
17:08:53 <decause> 2) Fedmsg integration
17:09:05 <decause> martin is working on getting taskotron to emit fedmsgs
17:09:09 <decause> it listens already
17:09:16 <decause> but getting to send, with taskotron trigger
17:09:25 <decause> it would run the dep-check tests, based on Fedmsg
17:09:39 <decause> #info fedmsg trigger is 2 way communication and amazing
17:10:06 <decause> we'd then get a repository of tasks, and then we can have folks send a "pull-request" to get tasks into the queue
17:10:13 <decause> that makes writing tasks locally easy
17:10:33 <decause> Actually #2) Expand OpenQA testing
17:10:47 <decause> we have an instance, but we want to mak emore public, for other sto write tests too
17:10:54 <decause> we want our installation testing to get into OpenQA
17:11:11 <decause> I hesitate to name how many Fedora Installs (even just bare metal) I've done in the past year
17:11:17 <decause> that will help with anaconda changes too
17:11:23 <decause> and will free up a lot of time
17:11:30 <decause> #3) Revising the TC/RC process
17:11:49 <decause> adamw: has been talking with Rel-eng about this
17:12:08 <mattdm> whooo CI
17:12:11 <decause> the proces of stable push requests for example, getting that process to the point of like a continuous integration
17:12:23 <decause> roshi: he's been at the Rel-Eng FAD since Thursday
17:12:41 <decause> #4 increased test coverage
17:12:46 <decause> the cloud image, for example
17:12:55 <decause> I took the base test cases, and applied them with small tweaks
17:13:09 <decause> "can I install packages" "Can I update packages" "Is SELinux working?"
17:13:17 <decause> one place I'd like ot have more coverage is cloud-init
17:13:24 <decause> I was workign with dusty and Digital Ocean
17:13:31 <decause> certain directives weren't working
17:13:41 <decause> there is a bug between cloud-init and dnf, arond permissions issues
17:13:50 <decause> this is not just with DO, but in many places
17:14:06 <decause> I'd like to have that with more testing, and any other infrastructure core bits, ahving more solid coverage
17:14:34 <decause> #5 Getting more into OpenQA and taskotron for things that need coverage but hav ebeen done manually
17:14:39 <decause> it is easier to miss if we do it manual
17:14:47 <decause> we want to increase the number of automated tests
17:14:52 <decause> #topic Things QA would like
17:15:01 <decause> #1 Bodhi 2.0
17:15:09 <mattdm> #info bodhi 2.0 -- a new version of the updates system
17:15:24 <decause> leaving comments on updates is a patch to make it work, as opposed to an API to call
17:15:34 <decause> we've got a lot of code that is doing stuff with monky patches
17:15:41 <decause> bodhi 2.0 will allow major streamlining
17:15:48 <mattdm> lmacken: ^ :)
17:15:50 <decause> it isn't breaking anything not having it
17:16:01 <decause> better sync with Rel-Eng
17:16:11 <decause> right now we rely on releng to get us images to test
17:16:19 <decause> having good sync between teams is going to make things a lot easier
17:16:21 <lmacken> mattdm: yep, sitting next to decause listening :)
17:16:29 <mattdm> lmacken++
17:16:29 <zodbot> mattdm: Karma for lmacken changed to 6:  https://badges.fedoraproject.org/tags/cookie/any
17:16:32 <decause> lmacken++
17:16:41 <decause> it would make life easier, big time
17:16:57 <decause> we need more package maintainers to run taskotron tasks
17:17:07 <decause> this will help integrate packages, and provide test cases
17:17:16 <decause> getting some features into a more public facing domain
17:17:21 <decause> this will help us get ready for production
17:17:26 <decause> taskotron could do a lot more
17:17:35 <decause> it is not just a testing framework, it is meant to run tasks
17:17:53 <decause> one task will be testing, but hte framework the QA-devel gusy are workign to make taskotron do a whole lot more
17:18:01 <decause> I expect to see major benefits in other places too
17:18:13 <decause> Even more testing form other WG's
17:18:17 <decause> we need to increase our test cases
17:18:27 <decause> and gettin gth ecore groups working on their edition/flavor
17:18:34 <decause> that will help the QA side, and mkae everyone happier
17:18:42 <decause> this helps brin gerros, and then bring fixes
17:18:51 <decause> #topic QA Q&A
17:19:02 <decause> mattdm: I'm curious about the dist-git items
17:19:15 <decause> dist-git is a series of source control repos for a packge in fedora
17:19:23 <decause> all the maintainers, and the proven packages, can access it
17:19:40 <decause> repos for pacakges, then cant hen have tests, and proven QA peopel to run tests
17:19:53 <decause> roshi: yes, attempt to decrease learning curve, and keep workflow the same
17:20:07 <decause> if we can keep the workflow in taskotron tasks, it will be easier to build more tasks themlsevs
17:20:11 <decause> mattdm: do you konw the roadmap?
17:20:25 <decause> roshi: righ tnow, as I undrestand, I'm working work tflink on the disposable clients bit
17:20:29 <decause> that runs testcloud
17:20:33 <decause> which is a tool to boot images locally
17:20:43 <decause> for now, it is time to suss out all the features
17:20:45 <decause> test all the code
17:20:50 <decause> make sure it does what it needs
17:20:52 <decause> we're not blocking
17:20:59 <decause> we're in heads-down dev mode
17:21:13 <decause> I don't hve a time frame, but tflink might be able to give you one on the milestone completion
17:21:26 <decause> mattdm: that makes sense for disposable clients, particularly for crazy tests
17:21:29 <decause> roshi: yes
17:21:43 <decause> roshi: up untli now, we couldn't run disruptive tests, as it is aon a build-bot machine
17:22:02 <decause> we can have tests to do 'rm -rf slash' and it won't destroy anytning really
17:22:05 <decause> :)
17:22:08 <decause> that is it as far as I understand it
17:22:20 <decause> I have a little bit of insight into what taskotron team is doing
17:22:26 <decause> but mostly I'm doing docs and disposable clients
17:22:44 <decause> mattdm: I know there was talk about moving the test matrix out of the wiki. UPdates?
17:22:51 <decause> adamw: he wrote some cli tools
17:23:00 <decause> we understand the not goodness of wiki-munging
17:23:12 <decause> we've done a good job with getting the squre peg into th eround hole
17:23:22 <decause> we've looked at some other toosl, but none really fit our workflow
17:23:40 <decause> we had TCMS (test-case management system) brainstorm
17:24:07 <decause> current, there can be image sanity checks, and whenever say, TC1 gets release RELVal can
17:24:24 <decause> we can spin up a rawhide test page, and then report results before we branch
17:24:31 <decause> up until maybe 6mos ago, that was all created by hand
17:24:44 <decause> instead of creating 10 templates by hand, we have one command for that now
17:24:56 <decause> it is afire we have, but it isn't one anyone has been close enough to put out yet :P
17:25:06 <decause> mattdm: when I do do tests, I worry I will mess up the wikipage
17:25:15 <decause> roshi: that is the good thing abou tht wiki, we can always revert
17:25:23 <decause> mediawiki makes it hard to actually destroy things
17:25:27 <decause> mediawiki++
17:25:51 <decause> roshi: all of us QA guys stil lmkae errors now and then, and we fix them no problem and are always willing to help
17:25:55 * decause catches breath
17:25:58 <decause> mattdm++
17:26:14 <decause> langdon: I do. If you guys looked at behavior oriented test models?
17:26:35 <decause> roshi: if we have, I havne't been in those conversations. I'm happy to talk about it though, and we're always open to new and better test coverage?
17:26:56 <decause> roshi: I've spitballed a tool to make it easier to run through all the test cases
17:27:14 <decause> roshi: I've realized I could run 5-10 unique tests on a machine
17:27:20 <decause> so I wrote a tool to streamline that
17:27:33 <decause> I'd be open to revising that tool with feedback at some point
17:28:09 <decause> langdon: are there more better ways we can encourage more everyday users to contribute more info back?
17:28:19 <decause> langdon: is abrt helping get feedback through?
17:28:30 <decause> langdon: where is the line between enduser/tester support?
17:28:42 <decause> roshi: abrt we rely on it pretty heavily when we get big crashes
17:28:48 <decause> it pulls everyting automagically into one place
17:29:00 <decause> roshi: we could use some more visibility on tools like Fedora Easy Karma
17:29:06 <decause> which interfaces with bugzilla/bodhi
17:29:15 <decause> you can run it on an existing system, and tells you what you can test
17:29:19 <mattdm> #info try easykarma!
17:29:27 <decause> roshi: we totally should get a badge for doing that somehow
17:29:29 <mattdm> #help more people running easykarma would be helpful
17:29:43 <mattdm> #link https://fedoraproject.org/wiki/Fedora_Easy_Karma
17:29:44 <decause> roshi: when folks intereste din QA and don't konw where to start, we point them to updates testing
17:29:58 <decause> roshi: I just dont' think everyone knows it is there
17:30:16 <decause> roshi: if a new usser wants to help, even if they want to go outside of QA, we help them find a way to get invovled
17:30:29 <decause> roshi++
17:30:29 <zodbot> decause: Karma for roshi changed to 5:  https://badges.fedoraproject.org/tags/cookie/any
17:31:05 <decause> #action mattdm include Easy Karma in five things in fedora magazine article
17:31:21 <decause> roshi: peple don't realize how many tools/packages they use even to just go online
17:31:30 <decause> roshi: you don't realize how much stuff runs int he background
17:31:36 <decause> mattdm: I have a diff question
17:31:54 <decause> mattdm: bundled updates. SOmething we've kicked around at FLOCk. Have you folks talked about htat?
17:32:06 <decause> roshi: you mean 20 updates all pushed out as 1 update
17:32:20 <decause> mattdm: maybe pushing them out weekly/monthly as long as not security upsteas
17:32:51 <decause> roshi: off the top, I would think that the biggest thign to watch out for is making sure that manually testing isn't impacted
17:33:10 <decause> I can see arguemnts for pushing daily, and batched
17:33:24 <decause> there is increased procedural overhead, and "fun times" to manage
17:33:35 <decause> mattdm: yeah, it would dpened on automated testing higly
17:33:48 <decause> mattdm: I've heard proposed a daily stream
17:34:01 <decause> roshi: so updates piecemeal and updates firehose then?
17:34:12 <decause> mattdm: yeah, maybe updates-collected?
17:34:25 <decause> roshi: without konwing the overhead, getting to tools in place will be the key factor
17:34:45 <decause> mattdm: I hear it from people that there is too much change between releases, and users would likely appreciate it.
17:34:53 <decause> roshi: we could maybe make that work
17:35:00 <decause> mattdm: it would help with updated cloud images
17:35:17 <decause> mattdm: we've respinned with a batch, and improve with periodic updates
17:35:28 <decause> roshi: this will help with increased test coverage in cloud init
17:35:45 <decause> when we do spin an kupdated verison, we can get through in a timely fashion. all those would work together to make that less painful
17:35:55 <decause> "sweet! new tested update! let's go!"
17:36:05 <decause> mattdm: yeah, we want it to feel like a great fun new thing, yeah
17:36:11 <decause> mattdm: any other questions
17:36:17 <decause> questions from IRC?
17:36:17 <mattdm> any questions from irc?
17:36:39 <decause> mattdm: great!
17:36:43 <decause> thanks for the update
17:36:46 <langdon> mattdm, i would like to make sure someeone (eg. roshi) add links for all the tihngs he mentioned
17:36:50 <decause> it was very informative
17:36:58 <decause> langdon: #agreed
17:37:07 <stickster> I had one thing to add, not a question
17:37:12 <decause> before we end the IRC meeting
17:37:32 <stickster> Just thanks to the QE folks for continued vigilance over release criteria, and keeping the quality bar high for Fedora releases
17:37:32 <decause> mattdm: stickster has something to add
17:37:39 * stickster not sure how far behind "live" he is
17:37:52 <decause> stickster: very close to lieve
17:37:58 <stickster> mattdm: ^^
17:38:00 <decause> transcripts have been live ;)
17:38:11 <decause> mattdm: sticker question
17:38:20 <decause> roshi: thanks!
17:38:23 <kushal> decause, Your transcripts came first, audio/video came way later for me ;)
17:38:41 <decause> kushal: :)
17:38:59 * decause waits for links
17:39:05 <jkurik_mtg> decause: the same for me; the video is approx. 5 minutes delayed comparing to the transcipt
17:39:11 * roshi gathers links
17:39:11 <stickster> roshi: ^^ Oh well, my comment didn't get picked up, but QA++
17:39:36 <ncc1701lj> Impressed with the speed. I got transcript from the future! Time travel transcripts. :)
17:39:37 <roshi> thanks :)
17:39:37 <decause> things that need #info: EasyKarma, Taskotron, GuiKarma, OpenQA, cloud-init
17:39:56 <decause> err, things that need #link, rather
17:40:06 <decause> anybody with links can add them plz
17:40:18 <tflink> #link http://fedoraproject.org/wiki/Taskotron
17:40:19 <decause> I may have be missing others too
17:40:24 <decause> thanks tflink
17:40:36 * tflink is looking for the others
17:40:36 <roshi> #link https://github.com/os-autoinst/openQA
17:40:58 <roshi> #link https://fedoraproject.org/wiki/Fedora_Easy_Karma
17:41:27 <roshi> #link https://fedoraproject.org/wiki/Fedora_Gooey_Karma
17:41:50 <roshi> #link http://cloudinit.readthedocs.org/en/latest/
17:41:58 <roshi> I think that about sums it up
17:42:04 <decause> roshi++
17:42:06 <mattdm> jkurik_mtg: ah, good to know that it's on tape delay :)
17:42:06 <decause> thanks so much
17:42:08 <tflink> roshi: that link to fedora gooey karma is out of date
17:42:14 <decause> roshi: you were really on point in that meeting
17:42:21 <mattdm> roshi++
17:42:37 <decause> roshi: stickster wanted to give you folks a shoutout for your vigilance and efforts too, in case you missed it
17:42:54 <tflink> but I don't have a better one :-/
17:43:07 <decause> roshi: I'd like to hear more about easykarma at some point. I love the concept of abrt, and I was really glad to hear that all these years I've been helping in userland :)
17:43:07 <roshi> I don't have a better one either...
17:43:17 <mattdm> just a reminder that we are running out of cookies https://fedorahosted.org/fedora-badges/ticket/379
17:43:22 <roshi> yeah, thanks stickster :)
17:43:29 <roshi> thanks decause
17:43:35 <decause> roshi: no problem
17:43:38 * langdon has to disconnect..
17:43:43 <langdon> roshi, thanks for the update
17:43:52 <mattdm> yep, anything else to add?
17:44:01 <decause> mattdm: link to youtube?
17:44:12 <mattdm> decause: it's up there at the beginning
17:44:18 <decause> mattdm: kk, great!
17:44:28 * decause doesn't have any other loose ends then
17:44:30 <roshi> glad to be able to give it -feel free to ping iwth more questions if you have them
17:44:32 <mattdm> #link http://youtu.be/7arJcuUNHGg
17:44:36 <tflink> it should be transcoded to something other than h264 after a while
17:44:39 <roshi> if I don't have an answer, I'll help you find one :)
17:44:48 <decause> tflink: nod nod
17:44:56 <decause> roshi++
17:44:59 <tflink> it took an hour or two for the last council meeting
17:45:10 <mattdm> tflink: *nod*
17:45:13 <mattdm> okay here we go
17:45:15 <mattdm> #endmeeting