f41-blocker-review
LOGS
<@adamwill:fedora.im>
16:00:48
!startmeeting F41-blocker-review
<@meetbot:fedora.im>
16:00:49
Meeting started at 2024-10-07 16:00:48 UTC
<@meetbot:fedora.im>
16:00:49
The Meeting name is 'F41-blocker-review'
<@adamwill:fedora.im>
16:00:58
!topic Roll Call
<@frantisekz:fedora.im>
16:01:07
!hi
<@nielsenb:fedora.im>
16:01:13
!hi
<@zodbot:fedora.im>
16:01:30
František Zatloukal (frantisekz)
<@zodbot:fedora.im>
16:01:31
Brandon Nielsen (nielsenb)
<@decathorpe:fedora.im>
16:03:47
!hi
<@zodbot:fedora.im>
16:03:48
Fabio Valentini (decathorpe) - he / him / his
<@lruzicka:matrix.org>
16:04:04
Hey, hi, ho, ho
<@adamwill:fedora.im>
16:05:02
it's off to work we go?
<@adamwill:fedora.im>
16:05:07
i suppose it is
<@nhanlon:beeper.com>
16:07:38
!hi
<@zodbot:fedora.im>
16:07:39
Neil Hanlon (neil) - he / him / his
<@davdunc:fedora.im>
16:07:56
.hi
<@davdunc:fedora.im>
16:08:01
!hi
<@zodbot:fedora.im>
16:08:02
David Duncan (davdunc) - he / him / his
<@davdunc:fedora.im>
16:08:06
old habits.
<@adamwill:fedora.im>
16:08:56
hi hi everyone
<@adamwill:fedora.im>
16:09:05
boilerplate time!
<@nhanlon:beeper.com>
16:09:17
yum, boiler
<@adamwill:fedora.im>
16:09:19
!topic Introduction
<@adamwill:fedora.im>
16:09:19
!info The criteria for release blocking bugs can be found at:
<@adamwill:fedora.im>
16:09:19
<@adamwill:fedora.im>
16:09:19
<@adamwill:fedora.im>
16:09:19
<@adamwill:fedora.im>
16:09:19
<@adamwill:fedora.im>
16:09:19
<@adamwill:fedora.im>
16:09:19
!info The bugs up for review today are available at:
<@adamwill:fedora.im>
16:09:19
!info We'll be following the process outlined at:
<@adamwill:fedora.im>
16:09:19
!info Our purpose in this meeting is to review proposed blocker and nice-to-have bugs and decide whether to accept them, and to monitor the progress of fixing existing accepted blocker and nice-to-have bugs.
<@adamwill:fedora.im>
16:09:19
Why are we here?
<@adamwill:fedora.im>
16:09:45
!info 2 Proposed Blockers
<@adamwill:fedora.im>
16:09:45
!info 4 Accepted Blockers
<@adamwill:fedora.im>
16:09:45
!info 1 Proposed Freeze Exceptions
<@adamwill:fedora.im>
16:09:45
!info 2 Accepted Freeze Exceptions
<@adamwill:fedora.im>
16:09:54
who wants to secretarialize?
<@frantisekz:fedora.im>
16:09:58
o/
<@adamwill:fedora.im>
16:10:06
!info František Zatloukal will secretarlalize, thanks!
<@adamwill:fedora.im>
16:10:11
let's get started with:
<@adamwill:fedora.im>
16:10:14
!topic proposed Final blockers
<@adamwill:fedora.im>
16:10:23
!topic (2316066) dbus activated apps including the welcome dialog suffer 30-45 sec delay and/or crash on Workstation Live image randomly
<@adamwill:fedora.im>
16:10:23
!info Proposed Blocker, dbus, NEW
<@adamwill:fedora.im>
16:10:23
<@adamwill:fedora.im>
16:10:23
<@adamwill:fedora.im>
16:10:23
!info Ticket vote: FinalBlocker (+2,2,-0) (+asciiwolf, +jmaybaum, geraldosimiao, nielsenb)
<@adamwill:fedora.im>
16:10:30
i'm not...really convinced about this one
<@frantisekz:fedora.im>
16:10:45
I'd incline to 0 or -1 here, unless we have some signifficant data for it
<@frantisekz:fedora.im>
16:10:49
I'd never seen it
<@adamwill:fedora.im>
16:10:56
the impact doesn't sound that bad, and i couldn't reproduce it myself, though i do kinda think i saw it brought up somewhere else
<@frantisekz:fedora.im>
16:12:22
so, -1, or leaving it up on the tracker with 0? I am fine for both, maybe -1, we can resurrect it if needed
<@adamwill:fedora.im>
16:13:01
any other votes?
<@nielsenb:fedora.im>
16:13:26
I remain pretty unconvinced too, I haven't seen it across quite a few different hardware combinations
<@nielsenb:fedora.im>
16:14:03
Now that I've said that I'll see it every time
<@lruzicka:matrix.org>
16:14:10
I would go with punt.
<@farribeiro:matrix.org>
16:14:27
!hi
<@zodbot:fedora.im>
16:14:28
Fábio Ribeiro (farribeiro) - he / him / his
<@adamwill:fedora.im>
16:14:29
yeah, i guess we can punt for more data
<@adamwill:fedora.im>
16:14:39
i.e. get more people to test
<@conan_kudo:matrix.org>
16:15:15
!hi
<@zodbot:fedora.im>
16:15:17
Neal Gompa (ngompa) - he / him / his
<@kparal:matrix.org>
16:16:23
the significant impact is that it blocks me from doing effective release validation! 😄
<@frantisekz:fedora.im>
16:16:41
how have you set up your vm Kamil Páral ?
<@kparal:matrix.org>
16:16:42
just waiting 45sec to run a terminal is somewhat annoying
<@adamwill:fedora.im>
16:16:50
Kamil Páral: one thing i didn't entirely understand - does this have ongoing effects or is it just 'it's weird for one minute after boot'?
<@frantisekz:fedora.im>
16:16:50
I haven't managed to trigger it
<@lruzicka:matrix.org>
16:17:00
Perhaps Kamil will be able to convince everybaby.
<@kparal:matrix.org>
16:17:05
why am I always the one affected by race conditions, I don't know
<@kparal:matrix.org>
16:18:03
eh, matrix seems lagging today
<@kparal:matrix.org>
16:18:34
I don't see any long-lasting effects. Except that a few user session services crashed, that might affect something
<@nielsenb:fedora.im>
16:18:48
Glad it's not just me
<@kparal:matrix.org>
16:18:54
all defaults in virt-manager
<@adamwill:fedora.im>
16:19:50
so, i'm feeling -1 or punt-y on this
<@adamwill:fedora.im>
16:19:51
brb
<@frantisekz:fedora.im>
16:20:03
hmm, I am using EFI vms for 99 % of my tests, but I doubt that'd be it :/
<@frantisekz:fedora.im>
16:20:08
-1 || punt
<@nielsenb:fedora.im>
16:21:15
After reading the AMD SFH bug, anything seems possible
<@kparal:matrix.org>
16:22:07
I just reproduced it again. It also often makes the boot longer, waiting for abrt to time out.
<@kparal:matrix.org>
16:22:26
As long as it's just me, I guess not a blocker
<@kparal:matrix.org>
16:22:40
or if the frequency is low, which seems to be the case on your machines
<@kparal:matrix.org>
16:22:59
I'll download the latest image and try again tomorrow
<@geraldosimiao:matrix.org>
16:23:00
!hi
<@lruzicka:matrix.org>
16:23:00
I am still for a punt and let's see what it brings.
<@zodbot:fedora.im>
16:23:02
Geraldo S. Simião Kutz (geraldosimiao) - he / him / his
<@lruzicka:matrix.org>
16:23:11
Better than dismiss too soon.
<@nielsenb:fedora.im>
16:23:30
Yeah, I would probably be happier punting than voting it away
<@adamwill:fedora.im>
16:24:22
ok
<@adamwill:fedora.im>
16:24:58
proposed !agreed 2316066 - punt (delay decision) - the impact of this seems fairly muted and it seems like everyone but kamil struggles to reproduce it. however, we'll punt to allow for more testing on more hardware
<@geraldosimiao:matrix.org>
16:25:05
ack
<@decathorpe:fedora.im>
16:25:10
ack
<@farribeiro:matrix.org>
16:25:13
ack
<@kparal:matrix.org>
16:25:16
yesterday image, same problem
<@frantisekz:fedora.im>
16:25:31
ack
<@kparal:matrix.org>
16:25:48
ack
<@nielsenb:fedora.im>
16:26:16
ack
<@conan_kudo:matrix.org>
16:26:42
ack
<@adamwill:fedora.im>
16:27:25
!agreed 2316066 - punt (delay decision) - the impact of this seems fairly muted and it seems like everyone but kamil struggles to reproduce it. however, we'll punt to allow for more testing on more hardware
<@kparal:matrix.org>
16:27:30
František Zatloukal: happen in UEFI VM as well
<@kparal:matrix.org>
16:27:36
František Zatloukal: happens in UEFI VM as well
<@adamwill:fedora.im>
16:27:51
!info Ticket vote: FinalBlocker (+7,0,-0) (+chrismurphy, +ngompa, +asciiwolf, +geraldosimiao, +nixuser, +derekenz, +nielsenb)
<@adamwill:fedora.im>
16:27:51
!info Proposed Blocker, kernel, NEW
<@adamwill:fedora.im>
16:27:51
<@adamwill:fedora.im>
16:27:51
<@adamwill:fedora.im>
16:27:51
!topic (2314331) AMD SFH Driver Causes Memory Errors/ Page Faults
<@adamwill:fedora.im>
16:28:07
this one has a lot of votes, but i kinda figured we should still discuss it
<@adamwill:fedora.im>
16:28:15
since it's hardware-specific and also affects stable releases
<@lruzicka:matrix.org>
16:30:37
What does this driver work for? Would I be able to reproduce it on any AMD device?
<@adamwill:fedora.im>
16:30:49
i couldn't quite figure that out yet
<@adamwill:fedora.im>
16:31:23
modinfo says it's "AMD(R) PCIe MP2 Communication Driver"
<@frantisekz:fedora.im>
16:31:39
as per https://docs.kernel.org/hid/amd-sfh-hid.html
<@adamwill:fedora.im>
16:31:40
google says "AMD Sensor Fusion Hub (SFH) is part of an SOC starting from Ryzen-based platforms"
<@frantisekz:fedora.im>
16:31:43
it'd be on all ryzens
<@nielsenb:fedora.im>
16:31:54
I think orientation / light sensor stuff
<@lruzicka:matrix.org>
16:32:42
Thanks.
<@adamwill:fedora.im>
16:33:04
has anyone seen this on an AMD framework or anything?
<@nielsenb:fedora.im>
16:33:24
I *think* it's mobile only
<@jannau:fedora.im>
16:34:05
Is this this kernel regression https://lore.kernel.org/all/58fac815-e630-4ece-9dba-18dc255a0d1b@hixontech.com/ ?
<@jannau:fedora.im>
16:34:05
I think that is finally in linux-stable
<@adamwill:fedora.im>
16:35:12
yes, it's an upstream kernel issue in 6.11, it seems like
<@nielsenb:fedora.im>
16:35:50
I thought the issues started cropping up in late 6.8
<@nielsenb:fedora.im>
16:36:03
I can't find the ticket referred to in that mailing list post though
<@nielsenb:fedora.im>
16:36:12
So I can't find the bisect result
<@adamwill:fedora.im>
16:36:56
oh, yeah, although in fedora it seems like it's not in 6.10? i dunno
<@nielsenb:fedora.im>
16:36:58
https://bugzilla.kernel.org/show_bug.cgi?id=219331
<@nielsenb:fedora.im>
16:37:01
Derpt
<@adamwill:fedora.im>
16:37:11
anyhow, we don't need to fix it here
<@adamwill:fedora.im>
16:37:34
i'm kinda +/-0 on it, but i can see the argument for +1 to commonbugs it at least
<@nielsenb:fedora.im>
16:37:38
Anything with commit 2105e8e00da4673266e217653292fef6acefde03
<@jforbes:fedora.im>
16:37:54
I expect it will come in this week
<@jforbes:fedora.im>
16:38:22
but also not in next, so unsure
<@nielsenb:fedora.im>
16:38:45
Started with 6.9 I guess
<@kparal:matrix.org>
16:38:49
the slight difference over stable releases is that their livecds are safe (are they?), only kernel updates can cause corruption. And once we submit a fixed kernel, even updates will be safe. OTOH live F41 will be unsafe forever.
<@frantisekz:fedora.im>
16:39:25
yea, let's punt
<@nielsenb:fedora.im>
16:39:29
Yeah, I really don't like the thought of shipping something with a known corruption bug
<@kparal:matrix.org>
16:39:52
punt? it has +7 already
<@nielsenb:fedora.im>
16:40:30
Laptops are probably regularly updated, but still, install, bork your data, then check common bugs (or yell on Discourse) seems like a likely flow
<@nielsenb:fedora.im>
16:40:54
Though I guess that assumes people update from install media
<@kparal:matrix.org>
16:41:19
yes, they should be safe, just checked kernel versions, they are all prior 6.9
<@geraldosimiao:matrix.org>
16:41:20
yeah, thats what I thougt "OTOH live F41 will be unsafe forever."...
<@adamwill:fedora.im>
16:42:09
yeah, that's a fair point
<@adamwill:fedora.im>
16:42:18
if nobody is wildly -1 on this i guess we can just go with the ticket vote
<@kparal:matrix.org>
16:42:41
I'm +1 or close to +1
<@geraldosimiao:matrix.org>
16:43:00
I'm still +1 (as voted on the ticket)
<@jforbes:fedora.im>
16:43:01
I expect this will be fixed one way or the other before the 15th
<@lruzicka:matrix.org>
16:43:20
I am not anything near -1, so I am good with anything above that.
<@nielsenb:fedora.im>
16:43:40
I think I remain +1
<@adamwill:fedora.im>
16:44:04
ok
<@jforbes:fedora.im>
16:44:16
We have a fix, and knurd is tracking it in regressions. So either that fix will go in, another fix will go in, or we can pull the proposed fix if necessary for a short term until the final fix is in
<@farribeiro:matrix.org>
16:44:27
+1
<@adamwill:fedora.im>
16:44:39
proposed !agreed 2314331 - AcceptedBlocker (Final) - this is accepted as a violation of "All known bugs that can cause corruption of user data must be fixed or documented at Common Issues"
<@frantisekz:fedora.im>
16:44:41
thanks for chiming in jforbes
<@geraldosimiao:matrix.org>
16:44:47
ack
<@farribeiro:matrix.org>
16:44:50
Ack
<@nielsenb:fedora.im>
16:44:50
ack
<@adamwill:fedora.im>
16:44:52
proposed !agreed 2314331 - AcceptedBlocker (Final) - this is accepted as a conditional violation of "All known bugs that can cause corruption of user data must be fixed or documented at Common Issues", for affected AMD laptops
<@adamwill:fedora.im>
16:44:55
patched
<@frantisekz:fedora.im>
16:44:55
ack
<@frantisekz:fedora.im>
16:45:09
pack
<@kparal:matrix.org>
16:45:23
ack
<@lruzicka:matrix.org>
16:46:06
ack
<@adamwill:fedora.im>
16:46:43
!agreed 2314331 - AcceptedBlocker (Final) - this is accepted as a conditional violation of "All known bugs that can cause corruption of user data must be fixed or documented at Common Issues", for affected AMD laptops
<@adamwill:fedora.im>
16:46:46
ok, moving on to:
<@adamwill:fedora.im>
16:46:57
!topic Proposed Final freeze exceptions
<@adamwill:fedora.im>
16:47:03
<@adamwill:fedora.im>
16:47:03
!topic (2316735) Update appstream-data before F41 Final
<@adamwill:fedora.im>
16:47:03
!info Ticket vote: FinalFreezeException (+2,0,-0) (+asciiwolf, +nielsenb)
<@adamwill:fedora.im>
16:47:03
!info Proposed Freeze Exceptions, appstream-data, NEW
<@adamwill:fedora.im>
16:47:03
<@adamwill:fedora.im>
16:47:08
looks like an easy +1
<@geraldosimiao:matrix.org>
16:47:34
yeah +1
<@frantisekz:fedora.im>
16:48:10
+1
<@farribeiro:matrix.org>
16:48:11
+1 FE
<@lruzicka:matrix.org>
16:48:30
-1
<@nielsenb:fedora.im>
16:48:37
I remain FinalFE +1
<@lruzicka:matrix.org>
16:48:40
+1
<@adamwill:fedora.im>
16:50:10
proposed !agreed 2316735 - AcceptedFreezeException (Final) - this cannot be as fully addressed with a post-release update and is a low-risk change
<@frantisekz:fedora.im>
16:50:18
ack
<@nielsenb:fedora.im>
16:50:21
ack
<@geraldosimiao:matrix.org>
16:50:23
ack
<@farribeiro:matrix.org>
16:50:59
Ack
<@adamwill:fedora.im>
16:51:46
!agreed 2316735 - AcceptedFreezeException (Final) - this cannot be as fully addressed with a post-release update and is a low-risk change
<@adamwill:fedora.im>
16:51:50
oookay, that's all the proposals
<@adamwill:fedora.im>
16:52:40
let's take a quick trip through:
<@adamwill:fedora.im>
16:52:46
!topic Accepted Final blockers
<@adamwill:fedora.im>
16:52:53
!info reminder: we're reviewing status, not re-voting (usually)
<@adamwill:fedora.im>
16:53:01
<@adamwill:fedora.im>
16:53:01
!topic (2311936) pyanaconda.modules.common.errors.storage.UnknownDeviceError: Volume0_0
<@adamwill:fedora.im>
16:53:01
<@adamwill:fedora.im>
16:53:01
!info Accepted Blocker, anaconda, ON_QA
<@adamwill:fedora.im>
16:53:16
i think we just need confirmation that this is fixed?
<@frantisekz:fedora.im>
16:54:19
I was able to take a really quick look at this one, I wasn't able to install f41 to fw raid at all (even with the proposed fix applied)
<@frantisekz:fedora.im>
16:54:29
but, I had something like 15 mins for it
<@adamwill:fedora.im>
16:54:44
hmm, okay...please try it again and update the bug
<@nielsenb:fedora.im>
16:54:55
Wonder if you're hitting the "anaconda crashes immediately when encountering an unclean fs" bug
<@adamwill:fedora.im>
16:55:28
František Zatloukal: lruzicka or i can re-run the openqa test on staging to produce a new image at any time if it's necessary to avoid other bugs
<@lruzicka:matrix.org>
16:55:29
I was hitting it quite often but I have not tried for some time now.
<@frantisekz:fedora.im>
16:55:32
no, it was exactly this: https://bugzilla.redhat.com/show_bug.cgi?id=2311936#c17
<@frantisekz:fedora.im>
16:55:50
"no usable configuration" on fw raid
<@nielsenb:fedora.im>
16:56:26
I see it was lruzicka in the ticket saying it just crashes now, sorry
<@nielsenb:fedora.im>
16:56:30
Skimmed too fast
<@adamwill:fedora.im>
16:57:13
!action František Zatloukal and lruzicka to work on re-testing this and updating the bug
<@adamwill:fedora.im>
16:57:28
<@adamwill:fedora.im>
16:57:28
!info Accepted Blocker, anaconda, NEW
<@adamwill:fedora.im>
16:57:28
<@adamwill:fedora.im>
16:57:28
!topic (2316058) anaconda fails to start and *fails to show an error dialog* when a device requires fsck
<@adamwill:fedora.im>
16:57:37
i think the 'fails to start' part of this should be ON_QA now right?
<@adamwill:fedora.im>
16:57:44
i feel like the 'fails to show an error dialog' part should be split off
<@nielsenb:fedora.im>
16:58:17
Is there an image with the fails to start fix?
<@kparal:matrix.org>
16:58:22
should I split it off and propose it as another blocker?
<@nielsenb:fedora.im>
16:58:28
I have a machine with a dirty dirty NTFS partition
<@kparal:matrix.org>
16:58:33
but I don't know how to validate it once the bug is fixed 🙂
<@adamwill:fedora.im>
16:58:43
Kamil Páral: split it off, whether you propose it as a blocker is up to you :P
<@adamwill:fedora.im>
16:59:07
eh, it shouldn't be too hard to do a custom image with that fix regressed or some other intentional crasher stuffed into early anaconda code
<@adamwill:fedora.im>
16:59:46
Brandon Nielsen: if i'm following the breadcrumbs right, current stable nightlies should have the fix
<@adamwill:fedora.im>
16:59:52
so if you're still seeing this with those that'd be bad
<@adamwill:fedora.im>
17:01:10
are you?
<@nielsenb:fedora.im>
17:01:33
I'll have to test when I have a recent image downloaded
<@nielsenb:fedora.im>
17:01:53
Pretty sure I still saw it with the last nominated compose...but I can't remember to say for sure
<@adamwill:fedora.im>
17:03:42
please do re-test and comment on the bug
<@nielsenb:fedora.im>
17:03:53
Will do
<@nielsenb:fedora.im>
17:04:39
It's downloading
<@adamwill:fedora.im>
17:04:44
looks like it's been fixed since 1004.n.0 at least
<@kparal:matrix.org>
17:05:18
I can test right now, will try
<@nielsenb:fedora.im>
17:05:43
Last I tested with 9-27
<@kparal:matrix.org>
17:06:09
once the race conditions let me to actually start anaconda
<@adamwill:fedora.im>
17:07:12
well, if it's fixed for you but broken for brandon we have an issue :D
<@kparal:matrix.org>
17:07:39
I'll tell you in 45+ seconds
<@nielsenb:fedora.im>
17:08:42
Still downloading
<@nielsenb:fedora.im>
17:08:48
Then there will be the writing to USB
<@adamwill:fedora.im>
17:08:54
yeah, we can wait for your result
<@kparal:matrix.org>
17:08:55
it's fixed for me
<@adamwill:fedora.im>
17:08:58
let's say:
<@adamwill:fedora.im>
17:09:17
!action Kamil Páral to split this bug in two and update status of both appropriately, Brandon Nielsen to add his feedback
<@adamwill:fedora.im>
17:09:50
<@adamwill:fedora.im>
17:09:50
<@adamwill:fedora.im>
17:09:50
!topic (2282171) gsk: vulkan renderer causes gtk4 apps to crash on resize operations on Raspberry Pi 4 and 400
<@adamwill:fedora.im>
17:09:50
!info Accepted Blocker, bcm283x-firmware, ON_QA
<@adamwill:fedora.im>
17:10:05
so...it seems like Peter Robinson thought he had a fix for this but the official build didn't work out for some reason?
<@adamwill:fedora.im>
17:10:21
are you around, pete? any news?
<@frantisekz:fedora.im>
17:11:06
the uboot bin copied out manually worked, but the one from updated rpm didn't :/
<@adamwill:fedora.im>
17:14:07
well, i guess we can let this one keep going on the bug
<@adamwill:fedora.im>
17:14:33
!info work on this is progressing, the fix did not work in an official build for some reason, Peter Robinson and František Zatloukal will continue to work on and test it
<@adamwill:fedora.im>
17:14:46
!topic (2315638) Fedora 41 aarch64 KDE disk image is oversize
<@adamwill:fedora.im>
17:14:46
!info Accepted Blocker, distribution, ASSIGNED
<@adamwill:fedora.im>
17:14:46
<@adamwill:fedora.im>
17:14:46
<@adamwill:fedora.im>
17:15:19
!info we changed the size, but that caused the build to fail. I sent a patch for blivet which should fix that - https://github.com/storaged-project/blivet/pull/1301 - so once that's downstream we can try again
<@adamwill:fedora.im>
17:17:13
!action adamw to make sure the blivet fix gets sent out as an update and then get the size drop reapplied
<@adamwill:fedora.im>
17:17:31
i think that's the lot
<@adamwill:fedora.im>
17:17:33
!topic Open floor
<@adamwill:fedora.im>
17:17:44
!agreed happy birthday to lruzicka
<@adamwill:fedora.im>
17:17:46
everybody sing now
<@lruzicka:matrix.org>
17:18:01
You are so kind, thank you :D
<@frantisekz:fedora.im>
17:18:07
🎂🎂🎂
<@decathorpe:fedora.im>
17:18:11
🎶
<@adamwill:fedora.im>
17:19:20
any other, less important business? :D
<@adamwill:fedora.im>
17:19:32
wait, shouldn't it be 🍺🍺🍺
<@adamwill:fedora.im>
17:19:48
has czechia invented the beer cake yet? I feel it's inevitable
<@frantisekz:fedora.im>
17:20:05
just a note maybe: https://bodhi.fedoraproject.org/updates/FEDORA-2024-3d2d44c3a7 would be nice to test / get in time for f41
<@frantisekz:fedora.im>
17:20:40
I haven't heard of it... yet, it'll come, the ultimate destiny of a country
<@adamwill:fedora.im>
17:21:02
yeah, sure, folks please test+karma that update
<@adamwill:fedora.im>
17:26:35
i guess that's all!
<@adamwill:fedora.im>
17:26:38
thanks for coming, everyone
<@adamwill:fedora.im>
17:26:46
see you next time
<@adamwill:fedora.im>
17:26:51
!endmeeting