16:00:29 <adamw> #startmeeting Fedora QA Meeting
16:00:29 <zodbot> Meeting started Mon Nov 25 16:00:29 2019 UTC.
16:00:29 <zodbot> This meeting is logged and archived in a public location.
16:00:29 <zodbot> The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:29 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:29 <zodbot> The meeting name has been set to 'fedora_qa_meeting'
16:00:35 <adamw> #meetingname fedora-qa
16:00:35 <zodbot> The meeting name has been set to 'fedora-qa'
16:00:39 <tflink> morning
16:00:40 <adamw> #topic Roll call
16:00:44 <frantisekz> .hello2
16:00:45 <zodbot> frantisekz: frantisekz 'František Zatloukal' <fzatlouk@redhat.com>
16:00:47 <adamw> morning morning
16:00:49 <adamw> how's everyone doing?
16:00:51 <tablepc> .hello2
16:00:52 <zodbot> tablepc: tablepc 'Pat Kelly' <pmkellly@frontier.com>
16:00:55 * satellit listening
16:01:00 * kparal is here
16:01:17 * coremodule is present!
16:02:36 <lruzicka> .hello2
16:02:37 <zodbot> lruzicka: lruzicka 'Lukáš Růžička' <lruzicka@redhat.com>
16:02:54 <cmurf> .hello chrismurphy
16:02:55 <zodbot> cmurf: chrismurphy 'Chris Murphy' <bugzilla@colorremedies.com>
16:03:55 <adamw> ah, presence, the attribute i value most
16:04:20 <cmurf> be here or be square?
16:04:37 <adamw> be here or be rectangular
16:04:43 <coremodule> adamw, is that presence or presents? because I like both
16:04:47 <tablepc> but only a single instance
16:04:55 <adamw> heh
16:04:56 <lruzicka> I'd rather be 'round
16:05:06 <tablepc> Multiple instances might be handy
16:05:08 <fbo> .hello2
16:05:09 <zodbot> fbo: fbo 'Fabien Boucher' <fboucher@redhat.com>
16:05:21 <adamw> welp, SNL called and said none of us have a future in comedy, so i guess let's start the meeting
16:05:29 <adamw> #topic Previous meeting follow-up
16:05:55 <adamw> #info "adamw to follow up on the printing criteria proposal and ask sgallagh if he thinks we should push it out as-is" - that one got overtaken by events later in the meeting
16:06:08 <adamw> #info "adamw to create F32 release criteria pages" - I did that! I'm helping!
16:06:20 <adamw> "sgallagh to add proposed printing criterion to F32 criteria once they're ready"
16:06:42 <adamw> it does not look like he's done that
16:06:44 <adamw> sgallagh, around?
16:06:53 <sgallagh> On PTO this week.
16:07:18 <sgallagh> I went to do that and I thought I saw that you had done it. Am I misremembering?
16:07:58 <sgallagh> If so, I’ll do it on Monday unless someone gets there first.
16:08:53 <adamw> er
16:09:01 <adamw> possibly?
16:10:18 <adamw> huh
16:10:34 <lruzicka> I would like to ask one question related to this criterion.
16:10:39 <adamw> i think we got stuck in a time warp somewhere, because the references say i put it there in april
16:10:42 <adamw> well, color me confused
16:10:44 <adamw> sure!
16:11:27 <adamw> #info "sgallagh to add proposed printing criterion to F32 criteria once they're ready" - it seems we actually added this criterion to the page back in april, so all is well here
16:11:35 <lruzicka> There is this test case, which I think handles the criteria: https://fedoraproject.org/wiki/QA:Testcase_Printing_New_Printer
16:12:27 <lruzicka> The criteria are, if I remember correctly, that it has to print over a PDF printer and a real IPP printer (at least)
16:12:35 <lruzicka> am I right?
16:13:14 <adamw> i think so
16:13:19 <frantisekz> IDK what IPP stands for, but the TC seems to mention any printer should work
16:13:28 <frantisekz> any/some
16:13:38 <lruzicka> IPP stands for "Internet Printing Protocol"
16:13:38 <cmurf> There is some minimum IPP Everywhere version that includes a requirement to directly consume/print PDF
16:14:10 <lruzicka> so we are pretty safe here with our office printer ... if it works, it means that printing works according to that criterion
16:14:23 <adamw> yeah, i think that was what we decided back in march
16:14:27 <adamw> the criterion link is https://fedoraproject.org/wiki/Fedora_32_Final_Release_Criteria#Printing
16:14:51 <lruzicka> I suggest splitting the test case into two cases, one could be automated, I am trying this now, which would print over CupsPDF.
16:14:59 <lruzicka> The real life printer cannot be automated.
16:15:25 <cmurf> If it's on a network it could be?
16:15:40 <frantisekz> can't we just install camera near our printer in the office to check results :D ?
16:15:57 <lruzicka> cmurf, well, it theoretically could be, but it could not be our office printer :D
16:16:07 <cmurf> frantisekz: haha that's true I forgot about the actual result needs to be LOOKED AT
16:16:11 <frantisekz> :D
16:16:22 <lruzicka> cmurf, exactly :D
16:16:55 <cmurf> but perhaps the discovery and "hey it says it printed it!" could be automated
16:17:01 <adamw> so we print to a printer whose out tray is under a webcam, then have openqa download the latest image from the webcam and match it...:P
16:17:15 <frantisekz> yeah, that's what i've meant adamw
16:17:17 <cmurf> that would be hilarious
16:17:34 <adamw> lruzicka: if anything i'd just have two result columns for the test case, one for the pdf result one for the 'real' printer result
16:17:37 <lruzicka> adamw, the matching part would be the easiest I think
16:17:41 <adamw> splitting the test case itself seems unnecessary
16:17:54 <lruzicka> adamw, yeah, I am ok with that :)
16:18:15 <adamw> lruzicka: actually i'm thinking about how you solve the problem of the printer not printing anything *at all* but the successful output from the previous day still being visible
16:18:33 <kparal> adamw: include timestamp on the papaer
16:18:40 <adamw> clearly what we need is a robot that snatches the paper from the out tray after the webcam has fired 🤔🤔
16:18:46 <cmurf> QR code
16:18:50 <adamw> kparal: that was my first thought, but then how do you generate the needles
16:19:10 <adamw> we *could* write an ungodly piece of perl (to be authentic) that creates the needles, i guess...
16:19:14 <lruzicka> lruzicka, I will check that there is a file really printed?
16:19:32 <adamw> man, i think if someone actually did this it would finally knock wikitcms off its coveted "most ridiculous system in qa" perch
16:20:22 <lruzicka> adamw, well, an automated process to create the needles would be uuuuuuuusefullll
16:20:38 <kparal> back to the lruzicka's proposal/adjustment, I don't think criteria need to reflect what is and what is not automatable. They reflect the state that we want to release in. How we implement it is not that related. We can have an automated test case that doesn't cover the criterion completely, but that is still valuable.
16:21:12 <adamw> kparal: we've done it a couple of times for practical reasons: if a test case is fully covered by an automated test we can submit the result of the test to the wiki
16:21:14 <adamw> if it isn't, we can't
16:21:17 <adamw> so the result is less visible
16:21:17 <cmurf> yes, automation can sanity test some initial basics that really should work and if they don't a human needs to find out why
16:21:23 <adamw> there's value in the results all being visible in the wiki together
16:21:32 <adamw> but in this case i think just split results columns would be fine
16:21:49 <cmurf> if automation can find show stoppers early, that's useful
16:22:21 <adamw> anyhow, we can deal with that as it comes up, i think
16:22:40 <kparal> so are we talking about criteria or a test case?
16:22:57 <adamw> #info "tablepc to make the 'disk dismount' test case proposal into a draft wiki page" - he did that, at https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot , and it'll come up for discussion later
16:23:01 <tablepc> Yes, I am confused on that point
16:23:01 <kparal> doh, I misread
16:23:27 <kparal> I take back my comment, I read criteria and lruzicka said testcase. my fault.
16:23:30 <adamw> heh
16:23:36 <adamw> "coremodule and sumantro to work on a heroes of fedora blog post" - where do we stand?
16:23:45 <frantisekz> .fire kparal for fault
16:23:45 <zodbot> adamw fires kparal for fault
16:23:47 <frantisekz> :)
16:24:06 <coremodule> The posts have been written, I believe sumantro intends to publish them on 2 Dec...
16:25:11 <lruzicka> yeah, I can confirm, I have seen them.
16:25:39 <adamw> eeeeexcellent
16:25:49 <adamw> #info "coremodule and sumantro to work on a heroes of fedora blog post" - this is done, and the post is awaiting Magazine publication
16:26:00 <adamw> any other follow-up items before i move on?
16:26:05 <tablepc> Do they get badges for being Heros?
16:26:20 <coremodule> tablepc, that is something that is in the works currently...
16:26:27 <coremodule> but as of now, no.
16:27:45 <adamw> #topic Proposed disk unmount test case
16:28:13 <adamw> so, tablepc asked for an agenda item to review his proposed test case, https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot
16:28:26 <adamw> the proposal is to add this to the Base matrix, i guess?
16:28:29 <tablepc> I test a note to QA list this morning with a link to all the list discussion
16:28:43 <tablepc> Yes, that's the proposal
16:29:12 <adamw> i'd kinda prefer if it was written in the format most of the other test cases we have are - with the results in the 'expected results' section, not mixed in with the test steps
16:29:44 <tablepc> Okay I can do that.
16:29:59 <adamw> but aside from that it looks pretty fine to me
16:30:02 <adamw> did anyone else have notes?
16:30:04 <cmurf> failure to unclearly umount at reboot/shutdown is definitely a bug of some kind, but whether it's blocker worthy will need to be taken up case by case
16:30:10 <kparal> I assume it's an Optional test case?
16:30:22 <cmurf> off hand I can't think of a simple metric
16:31:13 <cmurf> s/unclearly/uncleanly
16:32:14 <tablepc> This used to be a base test case. Anyone know know the history or how it was handled back then
16:32:25 <adamw> different question
16:32:30 <adamw> it should definitely go on the Base page
16:32:49 <adamw> the question is whether the Milestone column should say Beta, Final or Optional
16:33:05 <cmurf> basic?
16:33:19 <adamw> the shutdown criterion says "It must be possible to trigger a clean system shutdown using standard console commands."
16:33:23 <cmurf> i mean, if it's corrupting stuff badly enough that e.g. fsck can't fix, then that's a big deal
16:33:24 <adamw> with a footnote:
16:33:33 <adamw> "The system must shut down in such a way that storage volumes (e.g. simple partitions, LVs and PVs, RAID arrays) are taken offline safely and the system's BIOS or EFI is correctly requested to power down the system."
16:33:44 <cmurf> and if it's merely journal replay fixing stuff up it may not be blocker no matter the milestone
16:34:08 <adamw> so arguably it can be Basic with reference to that criterion...
16:34:46 <cmurf> that's interesting...
16:35:07 <cmurf> "taken offline safely" does that mean a clean umount or remount ro is mandatory?
16:35:21 <kparal> if this is something more than Optional, I'll probably have a dozen nitpicks, starting with formatting and ending with it being too verbose and explaining non-related stuff, e.g. how to create a user
16:35:49 <adamw> wow, a harsh reviewer has appeared :P
16:35:53 <cmurf> kparal: haha
16:35:55 <frantisekz> :D
16:36:05 <adamw> cmurf: i mean, i don't see how else you read it, really.
16:36:19 <tablepc> I can pull out the nits
16:36:56 <tablepc> They are there because the old case had them.
16:37:15 <kparal> what's the point of using "halt"?
16:37:36 <cmurf> well halt is common on devices that don't have power off, you have to unplug them
16:37:39 <adamw> so you can see the output without the system actually turning itself off, i think.
16:37:48 <cmurf> but you use halt to stop everything and cleanly unmount file system before you do that
16:37:52 <lruzicka> adamw, yes
16:38:03 <cmurf> e.g. raspberry pis
16:38:08 <kparal> but the output might not contain what you're looking for
16:38:11 <adamw> if this is based on an old test case, i think at that time, late shutdown messages weren't written to any log, so you couldn't see them after reboot
16:38:14 <kparal> cmurf: poweroff works just fine on rpi
16:38:19 <adamw> nowadays they probably make it to the journal
16:38:32 <cmurf> i think poweroff falls back to halt on systems that don't support actual poweroff
16:38:42 <adamw> ok, so if kparal has nits to pick, propose we ask tablepc and kparal to work on it together and come back with a revised draft?
16:38:58 <cmurf> adamw: +1
16:39:01 <kparal> sigh. what did I do
16:39:03 <frantisekz> adamw +1
16:39:14 <kparal> my question stands, is it Optional or Alpha/Beta/Final?
16:39:26 <cmurf> i'm thinking Basic
16:39:31 <cmurf> Basic/Alpha
16:39:37 <adamw> kparal: it seems like it should be basic, given the criterion
16:39:39 <kparal> right, no more Alphas
16:40:02 <adamw> not *everything* in the test case is a criteria violation if it fails, but that's not unusual (we have quite a lot of test cases like that)
16:40:24 <kparal> ok, I'll just reply to this list. if this is something we should check regularly, I'd like to be written well
16:40:40 <cmurf> the central part of why it could be a big deal is essentiallly none of the bootloaders know how to do journal replay; so depending on the situation, it's possible the fs is fine per kernel code but the bootloader might not be able to boot
16:40:43 <cmurf> or boot the right entry
16:40:46 <cmurf> or other
16:41:28 <kparal> I found the emails in test list to be too scattered around and hard to follow
16:41:31 <cmurf> anyway feel free to loop me in on fs stuff if you want
16:41:39 <kparal> is there a back story for this test case?
16:41:55 <cmurf> well that's because i'm super verbose by default :P
16:41:56 <frantisekz> and here comes epics and user stories time...
16:41:58 <frantisekz> :D
16:42:36 <kparal> no, just a simple question - why this test case?
16:43:00 <tablepc> From the discussion so far it seems like we need such a test case.
16:43:29 <tablepc> The question seems to be what phase of development and nits
16:43:53 <kparal> ok, so nothing spectacularly broke that we want to prevent in the future or anything. ok, just asking
16:44:05 <adamw> kparal: there's some history in https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/ZYGQL6ETJOXQPGPWZBDH5DX5SO6OO23F/
16:44:29 <cmurf> tablepc did have an example early on, somewhere during prerelease but it's not happening right now
16:44:56 <cmurf> i have an old example I can refresh memories of offline
16:45:08 <adamw> #info we generally agreed this seems like a useful proposal and support adding the test case to the Base matrix, but there were several suggestions for revisions
16:45:18 <kparal> ok, thanks
16:45:30 <adamw> #action kparal to reply to the list with his suggestions, tablepc to do a new draft of the test case after seeing kparal's suggestions
16:45:33 <adamw> humm
16:45:35 <adamw> let's split that
16:45:36 <adamw> #undo
16:45:36 <zodbot> Removing item from minutes: ACTION by adamw at 16:45:30 : kparal to reply to the list with his suggestions, tablepc to do a new draft of the test case after seeing kparal's suggestions
16:45:51 <tablepc> Okay
16:45:53 <adamw> #action kparal to reply to the list with his suggestions for revisions to the proposed shutdown test case
16:46:00 <adamw> #action tablepc to do a new draft of the proposed test case after seeing kparal's suggestions
16:46:04 <adamw> that good for everyone?
16:46:21 <kparal> yes. is this something we'll try to automate?
16:46:46 <frantisekz> I hope, lruzicka, any thoughts?
16:46:49 <adamw> hadn't really got that far yet, but it *should* be
16:47:17 <lruzicka> if there will be clear things to look for in the logs, it might be doable
16:47:57 <kparal> that's what I'm thinking. instead of reading random output on the halt screen, see the mount logs after reboot
16:48:28 <adamw> i'm sure we can figure something out
16:48:55 <cmurf> ext4 and XFS report info when there is journal replay, but it's the PRIOR logs for shutdown that we need to know why it was uncleanly unmounted
16:49:23 <lruzicka> adamw, do you know out of the blue, what is the symbol for the win key in openqa? it does not seem to be win nor meta.
16:49:46 <cmurf> I've asked Btrfs upstream for log tree replay info lines similar to ext4 and XFS. (On Btrfs things are different because even without replay the file system is considered consistent.)
16:50:02 <kparal> lruzicka: probably best to ask that in #fedora-qa so that we don't mix up discussions
16:50:31 <adamw> lruzicka: i believe there actually *isn't* one
16:50:38 <lruzicka> adamw, ok
16:50:53 <adamw> or, wait
16:50:54 <adamw> imbw
16:51:01 <kparal> I'll follow up on the list
16:51:03 <adamw> on right. i am
16:51:05 <adamw> lruzicka: it's super
16:51:10 <adamw> send_key "super";
16:51:14 <lruzicka> thanks
16:51:23 <adamw> ok, so, let's try and get through the rest of the agenda quick
16:51:40 <adamw> #topic Test systems update: Taskotron, autocloud, Zuul, oh my
16:51:55 <cmurf> Zuul?
16:51:59 <adamw> so there happened to be a lot of news about various test systems this last week so i thought i'd throw together a quick agenda item to keep folks up to date
16:52:03 <adamw> going in order:
16:52:33 <cmurf> as in, https://vignette.wikia.nocookie.net/ghostbusters/images/3/37/ZuulTerrorDog1.png/revision/latest?cb=20140515200520
16:52:37 <tablepc> Zull is a bad guy in an old scifi vid
16:52:47 <tablepc> zuul
16:52:49 <adamw> #info there was some discussion of retiring Taskotron very soon due to F29 EOL, but in the end, kparal managed to get it working on F30. so now Taskotron will stick around until F30 EOL
16:52:59 <adamw> that accurate, kparal/tflink?
16:53:05 <tflink> yeah
16:53:10 <kparal> tflink managed to get it working
16:53:19 <kparal> apart from that, yes
16:53:20 <adamw> ok who wants this credit
16:53:21 <adamw> come take it
16:53:24 <adamw> #undo
16:53:24 <zodbot> Removing item from minutes: INFO by adamw at 16:52:49 : there was some discussion of retiring Taskotron very soon due to F29 EOL, but in the end, kparal managed to get it working on F30. so now Taskotron will stick around until F30 EOL
16:53:25 <tflink> hit some issues in stg but I think that it's a stg flag/network issues
16:53:32 <adamw> #info there was some discussion of retiring Taskotron very soon due to F29 EOL, but in the end, tflink managed to get it working on F30. so now Taskotron will stick around until F30 EOL
16:54:47 <adamw> #info autocloud *is* retiring with F29 EOL. we have extended openqa to run the tests autocloud was previously running, on the Base_Cloud qcow2 images only for now. this is the 'cloud_autocloud' test case in openqa, it shows as 'compose.cloud_autocloud' in resultsdb
16:54:58 <adamw> #undo
16:54:58 <zodbot> Removing item from minutes: INFO by adamw at 16:54:47 : autocloud *is* retiring with F29 EOL. we have extended openqa to run the tests autocloud was previously running, on the Base_Cloud qcow2 images only for now. this is the 'cloud_autocloud' test case in openqa, it shows as 'compose.cloud_autocloud' in resultsdb
16:55:07 <adamw> #info autocloud *is* retiring with F29 EOL. we have extended openqa to run the tests autocloud was previously running, on the Cloud_Base qcow2 images only for now. this is the 'cloud_autocloud' test case in openqa, it shows as 'compose.cloud_autocloud' in resultsdb
16:55:08 <adamw> sigh.
16:56:05 <adamw> as for "zuul":
16:56:05 <adamw> Test systems update: Taskotron, autocloud, Zuul, oh my
16:56:08 <adamw> ...grr
16:56:13 <adamw> https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/message/CXDD2VYR6PHTR76JCJ5H5VEBIRG7YL37/
16:57:09 <adamw> there was an announcement of a new system for running checks on pull requests, based on Zuul. there's more info in that post and the subsequent discussion
16:58:22 <adamw> it seems the idea is for this to be complementary with the existing 'fedora ci' stuff, with this focusing on PRs and the pipeline focusing on testing package builds, or something along those lines
16:58:59 <adamw> there's also a wiki page with more info - https://fedoraproject.org/wiki/Zuul-based-ci - and https://src.fedoraproject.org/rpms/nodepool/pull-request/6 was given as an example of a PR tested under this system
16:59:57 <frantisekz> hmm, clicking on zuul flag expecting to see ci results doesn't work... :/
17:00:09 <adamw> #info there is a new Zuul-based CI/CD system for dist-git pull requests in operation, more details at https://fedoraproject.org/wiki/Zuul-based-ci and the other links above
17:00:25 <adamw> whoops, and we're at time
17:00:37 <adamw> any quick questions on the test systems or other urgent business?
17:01:37 <tablepc> Have a Great Day!
17:01:57 <adamw> you too
17:02:24 <frantisekz> i don't think I have anything, thanks for the meeting adamw and others!
17:02:56 <kparal> thanks
17:04:00 <lruzicka> have a great day everyone and do not get eaten by zuuls
17:06:10 <adamw> thanks for coming, everyone
17:06:16 <adamw> #endmeeting