16:01:15 #startmeeting f19final-blocker-review-4 16:01:15 Meeting started Mon Jun 10 16:01:15 2013 UTC. The chair is tflink. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:01:15 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:01:15 #meetingname f19final-blocker-review-4 16:01:15 #topic Roll Call 16:01:15 The meeting name has been set to 'f19final-blocker-review-4' 16:01:47 * kparal in active lurking mode 16:02:36 active lurking mode? 16:03:40 that's way better than a passive lurking mode. I might even participate in the discussion :) 16:04:54 :-/ 16:05:32 * jreznik is here 16:06:01 no worries, I'm 90% present 16:06:40 but used one free hour without qa meeting to play football and got hit by the ball to the head and is not sure he will make it today till the end 16:06:51 * jreznik hopes his brain is still in the scull 16:07:12 jreznik: sport is dangerous 16:07:12 I'm here for the next 10 minutes ;) 16:07:25 jreznik: way more dangerous than having meetings 16:07:31 no matter how many times this happens, I still think of american football first when someone says football 16:07:39 you'd think I'd know better by now 16:07:42 * jreznik is going to grab some ice 16:08:00 tflink: do you mean handegg? http://4.bp.blogspot.com/-u_EoVz2CPPc/T7jzJqF0RkI/AAAAAAAADak/AklIgH7tRx8/s1600/football-handegg.jpg 16:08:00 kparal: some meetings are far more mentally dangerous 16:08:07 is it not only americans that do not call football by it's real name 16:08:31 jreznik: :D 16:08:57 jreznik: something like that, I suppose. 16:09:11 morning folks 16:09:13 Viking-Ice: I thought that it was known as football pretty much everywhere outside the US 16:09:14 sorry to be late 16:09:22 .fire adamw for being late 16:09:22 adamw fires adamw for being late 16:09:35 tflink, yeah and you call it soccer 16:10:04 Viking-Ice: there are other countries that call football something different? 16:10:46 jreznik: the irony about that is that american football is directly derived from the "rugby football" code, as is still played in the UK and supervised by the "rugby football" union 16:11:05 i would just love to see some of those smug 'handegg' posters go into a pub full of rugby players and suggest calling their game handegg. i'd pay money. 16:11:21 adamw: :D 16:11:31 tflink, not that I'm aware of we all call it football here while american football is kinda of womens version of rugby 16:12:25 * kparal likes adamw's proposal and wants to watch 16:12:38 Viking-Ice: that's rather sexist, no? 16:12:55 anyhow, I suppose it's about time to get started with the _real_ fun 16:13:05 any volunteers for secretarialization? 16:13:19 tflink, depends on how you look at it 16:14:31 sure 16:14:43 adamw: thanks 16:14:55 #topic Introduction 16:14:58 * jreznik can write only with one hand, holdimg bag with ice... no help from me 16:15:02 Why are we here? 16:15:02 #info Our purpose in this meeting is to review proposed blocker and nice-to-have bugs and decide whether to accept them, and to monitor the progress of fixing existing accepted blocker and freeze exception bugs. 16:15:19 er, do we have enough people to even do the review? 16:15:29 me, you, kparal, viking, jreznik? 16:15:47 I count adamw and I, 90% of kparal but no jreznik and Viking-Ice for another 10 minutes or so 16:16:02 true, and tflink/adamw/viking are all QA 16:16:14 * kparal boosts his capacity to 100% 16:16:30 * tflink continues with boilerplate, we can figure out people after that's done 16:16:33 nirik: around? 16:16:39 #info We'll be following the process outlined at: 16:16:39 #link https://fedoraproject.org/wiki/QA:SOP_Blocker_Bug_Meeting 16:16:43 #info The bugs up for review today are available at: 16:16:44 #link http://qa.fedoraproject.org/blockerbugs/current 16:16:46 somewhat, but updating/rebooting machines. ;) 16:16:54 #info The criteria for release blocking bugs can be found at: 16:16:55 #link https://fedoraproject.org/wiki/Fedora_19_Final_Release_Criteria 16:16:57 #link https://fedoraproject.org/wiki/Fedora_19_Beta_Release_Criteria 16:17:01 #link https://fedoraproject.org/wiki/Fedora_19_Alpha_Release_Criteria 16:17:02 tflink: /me will try to help 16:17:04 #info Up for review today, we have: 16:17:37 #info 15 Proposed Blockers 16:17:37 #info 12 Accepted Blockers 16:17:37 #info 15 Proposed Freeze Exceptions 16:17:37 #info 18 Accepted Freeze Exceptions 16:17:41 ew 16:17:45 I need to run and pick up my new phone ( the shop closes at 17:00 45 minutes from now ) but can join again when I home ( which takes around 45 minutes to get to ) 16:17:55 later 16:17:58 Viking-Ice: enjoy the new toy :) 16:18:10 Galaxy s4 I hope so ;) 16:18:39 I think we have enough for the unanimous ones 16:19:00 * dgilmore is here 16:19:24 hi dgilmore, thanks 16:19:47 cool, we have enough non-disabled people for 3 acks :) 16:20:07 if there are no objections, we'll get started with the proposed blockers 16:20:28 tflink: :) im disabled still. dont get the cast off till wednesday 16:20:35 but lets start 16:20:53 dgilmore: ah, forgot about that. I was referring to jreznik's football injury :) 16:21:23 #topic (966761) storage configuration failed: Not enough free space on disks for automatic partitioning 16:21:27 #link https://bugzilla.redhat.com/show_bug.cgi?id=966761 16:21:29 #info Proposed Blocker, anaconda, NEW 16:22:42 methinks we forgot to do the needed digging on this one 16:22:57 I see no attempts to reproduce 16:23:39 well, we were waiting on the needinfo for mark, i think, not reproduction attempts... 16:23:54 tflink: he said he was able to reproduce 16:24:07 https://bugzilla.redhat.com/show_bug.cgi?id=966761#c11 16:24:10 i guess we do need to see if anyone else can hit it now, though 16:25:49 yeah 16:25:49 * kparal running it 16:26:03 proposed #agreed - This has been reproduced on ppc but it needs to be reproduced on PA before taking it as a blocker. will revisit when more information is available 16:26:23 unless we want to wait for kparal's run to finish 16:26:50 wasn't there also a question about whether ks was sequential or not - ie, the order or part and clearpart? 16:27:43 ack 16:27:50 ack 16:27:54 possibly, but that's probably just me guessing stuff 16:28:27 other ack/nak/patch? 16:28:29 ack 16:28:39 #agreed - This has been reproduced on ppc but it needs to be reproduced on PA before taking it as a blocker. will revisit when more information is available 16:28:54 #topic (972561) Anaconda fails find to a dependent package and crashes with Errno 256: "No more mirrors to try" on TC-2 DVD install. MATE and minmal tested so far. 16:28:57 #link https://bugzilla.redhat.com/show_bug.cgi?id=972561 16:29:00 * kparal is experiencing technical difficulties 16:29:00 #info Proposed Blocker, anaconda, NEW 16:29:01 and before I forget 16:29:11 #chair adamw jreznik kparal dgilmore 16:29:11 Current chairs: adamw dgilmore jreznik kparal tflink 16:29:36 I got usual "no disks selected" 16:31:13 i hit a simmiliar issue doing a arm network install but it was transient due to me hammering my internet connection at the time 16:31:20 i suppose all these ks issues could all be manifestations of the same underlying bug... 16:31:53 yeah, isn't one of them already accepted as a blocker? 16:32:19 this sounds like an issue with an individual system ATM - reject or punt for more reproducers? 16:32:20 tflink: yes, 969327 16:33:02 "CPU memory: 512GB"? 16:33:54 ask him to append logs in the least 16:34:34 im good with waiting to collect more data 16:34:38 minimal install started for me just fine 16:34:47 isn't file part of @base? 16:35:55 if so, -1. if this was widespread, we'd of heard more about it by now 16:36:05 it could be a low memory condition 16:36:18 but I couldn't reproduce it with the same amount of memory (though I can reproduce his other 512MB RAM bug) 16:36:21 MATE also started installing fine 16:36:39 so, punt. i asked him for more info in irc last night, but I can do it in the bug today 16:36:47 i saw that issue, but on a network install 16:37:18 he might have used online repos? 16:37:28 updates-testing? 16:37:35 ask to clarify 16:37:38 possible, yeah 16:37:55 * adamw fires a quick net install 16:37:58 could just be a weird io issue 16:38:22 proposed #agreed 972561 - This sounds like a transient error, but would like to see more information from reporter before making a decision on blocker status. Will revisit once more information is available 16:38:23 it's a somewhat strange error in general 16:38:30 Packages/f/file... is clearly a path on the DVD 16:38:37 but "No more mirrors to try" is a remote repo error 16:38:55 but anyway, we need logs. 16:38:57 ack 16:38:57 ack 16:39:10 ack 16:39:56 #agreed 972561 - This sounds like a transient error, but would like to see more information from reporter before making a decision on blocker status. Will revisit once more information is available 16:40:05 #topic (890302) installer does not also try non-default device types when trying to create new /boot with full disk(s) 16:40:08 #link https://bugzilla.redhat.com/show_bug.cgi?id=890302 16:40:10 #info Proposed Blocker, anaconda, NEW 16:41:52 is boot-from-btrfs a valid layout? from the anaconda dev comments, it sounds so 16:42:33 yeah, i'd read it that way. though it seems that this should be trivial to work around 16:42:43 create the subvol as just about any other mount point and then change its mount point 16:43:14 might be good to invite some anaconda dev here? 16:43:39 * adamw does that thing 16:43:40 I'm also noticing "I asked for /boot, without specifying a Desired Capacity value (left it blank.)." 16:44:12 well, that ought to work okay 16:44:17 it's a pretty normal thing to do for btrfs subvols 16:45:52 hi dlehman 16:45:56 thanks 16:46:00 we're on https://bugzilla.redhat.com/show_bug.cgi?id=890302 16:47:19 * bcl says .* -1 16:47:25 we don't try multiple device types, ever 16:47:40 bcl: ack, now let's go for a drink 16:48:08 so as far as blocker status goes, this seems to qualify, but i suggested it'd be easy to work around: create the mount point as / or /home and then just change it 16:48:09 we don't create /boot on btrfs without the user explicitly asking for that exact thing for that specific mountpoint 16:48:32 dlehman: which they can only do when changing the properties of an existing mount point, not when creating one, right? 16:48:37 right 16:49:03 we don't even need to offer btrfs /boot at all IMO 16:49:11 -1 16:49:12 so this is the experience as designed 16:50:01 i'm okay with -1 on the basis that it can be worked around, at least. 16:53:55 welp, we can argue about whether it's a bug to be fixed or not separately, what matters here is blockeriness 16:53:58 ack 16:54:07 ack 16:54:18 ack 16:55:15 #agreed 890302 - RejectedBlocker - While inelegant, this is as designed and is easily workaround-able by modifying an existing non-boot mountpoint. Therefore, it is rejected as a blocker for F19 final 16:55:26 #topic (971191) DVD install option unavailable in TUI 16:55:26 #link https://bugzilla.redhat.com/show_bug.cgi?id=971191 16:55:26 #info Proposed Blocker, anaconda, NEW 16:57:04 is there any argument for separate /boot on btrfs anyway? 16:57:13 the Refresh option in TUI is mostly confusing 16:57:17 dlehman: that's a point, i spose. but we're on the next thing now :) 16:57:29 kparal: it's kinda needed for the hub because it changes state behind your back 16:57:32 but anyhoo 16:57:44 this bug is "DVD install option unavailable in TUI" 16:57:44 adamw: well, don't print if you're not ready, it's as simple as that 16:58:03 the same approach as for GUI can't be used for TUI 16:58:06 kparal: then the user won't see anything for a while at start of install and won't be able to do stuff in other spokes while mirror discovery is happening 16:58:10 but anyhoo 16:58:16 "Please wait..." 16:58:18 yes 16:58:31 but the visual clues are simply not there 16:58:44 kparal: geez, take a hint. 16:58:47 on this specific bug...we need to be a bit more specific about the required functionality in each interface in the criteria, but for now i'd say we should only require pretty minimal functionality from the text installer 16:58:57 adamw: I'm +1 for this. 16:59:08 but i dunno, i'm willing to have my mind changed 16:59:22 I don't know why you'd be using text with a DVD, but if you are we sure ought to actually do the install from it. 16:59:25 bcl: so as long as we're having the full-fat text install interface, it should actually work? 16:59:34 I think text install from DVD used to work, didn't it? 16:59:44 oh, did it? i may be getting confused 16:59:58 i was thinking that at 18 all we did was minimal netinstall. if dvd has always worked then sure, +1 16:59:58 in F18 I believe so 17:00:08 I can check 17:02:15 yes, in F18 it installs from DVD, rather than downloading packages 17:02:28 * kparal will be back in 5 minutes 17:03:11 ok, +1 17:03:16 proposed #agreed 971191 - AcceptedBlocker - Violates the following F19 alpha release criterion for text installs using DVD media: "When using the DVD image, the installer must be able to use the DVD as a package source." 17:03:18 +1 17:03:31 ack 17:05:04 ack 17:06:29 * tflink assumes ack from kparal 17:06:32 * kparal is back 17:06:43 just a note, samantha said she doesn't consider this bug to me about install source 17:06:58 but that's a minor issue I guess 17:07:01 ack 17:07:04 #agreed 971191 - AcceptedBlocker - Violates the following F19 alpha release criterion for text installs using DVD media: "When using the DVD image, the installer must be able to use the DVD as a package source." 17:07:29 kparal: she says she's aware of it 17:07:31 ~ 1 hour in, 4 bugs covered 17:07:37 #topic (972547) Anaconda hangs and crashes on netinstall on a system with 512MB of RAM allocated 17:07:40 #link https://bugzilla.redhat.com/show_bug.cgi?id=972547 17:07:42 #info Proposed Blocker, anaconda, NEW 17:08:16 nope 17:08:32 "Whether in regular graphics mode or basic graphics mode" -- what is that? 17:08:44 kparal: nomodeset 17:08:50 oh I see 17:08:59 I didn't know DVD/netinst has it as well 17:09:01 Note this is an attemp to do a 512M GUI install. text works fine (and actually I've had 512 work at various times, but it is slow due to lots of swapping) 17:09:06 -1 per c#3 and c#5 17:09:24 -1 17:09:29 -1 17:09:37 tflink: we did spend the first half hour arguing about football 17:09:57 -1, but we should adjust the docs if needed 17:09:58 i'd be -1, but we can consider twiddling with the minimum requirements wording 17:10:07 i wouldn't want to raise the hard floor in the installer because a DVD install works 17:10:45 bcl: for me, 512MB graphical DVD reliably works, 512MB graphical netinstall reliably fails; presumably the load of reaching a mirror and doing mirror-y...stuff...before swap is available kills it 17:11:02 proposed #agreed 972547 - RejectedBlocker - It is possible to do an install with the text interface with 512M ram or with the DVD. All installation methods are not guaranteed to work with minimum ram and thus, this is rejected as a release blocking bug for F19 final. 17:11:09 also, he might be installing without a swap part 17:11:10 ack 17:11:16 ack 17:11:18 kparal: it fails before you can do partitioning 17:11:26 ack 17:11:39 #agreed 972547 - RejectedBlocker - It is possible to do an install with the text interface with 512M ram or with the DVD. All installation methods are not guaranteed to work with minimum ram and thus, this is rejected as a release blocking bug for F19 final. 17:12:58 do we want to do the anaconda FEs while the devs are here? 17:13:31 fine with me 17:13:40 ok 17:13:49 not a bad idea 17:14:02 actually, lets do this blivit proposed blocker first 17:14:04 #topic (969182) DeviceCreateError: ('Could not commit to disk /dev/mapper/mpatha, (py_ped_disk_commit)', 'mpatha3') 17:14:08 #link https://bugzilla.redhat.com/show_bug.cgi?id=969182 17:14:10 #info Proposed Blocker, python-blivet, NEW 17:15:20 this sounds like something that would break any multipath system 17:15:23 +1 17:15:44 there is a verified non-invasive fix 17:15:46 dlehman: am I reading this right or is there any reason to think this would be ppc-specific 17:15:55 not arch-specific AFAIK 17:16:01 +1 17:16:04 +1 17:16:14 +1 17:16:50 +1 17:16:52 proposed #agreed 969182 - AcceptedBlocker - Violates the following F19 final criterion for systems with multipath storage: "The installer must be able to complete an installation using any network-attached storage devices (e.g. iSCSI, FCoE, Fibre Channel)" 17:17:30 ack 17:17:38 ack 17:17:41 ack 17:17:41 brb call of nature 17:17:43 ack 17:17:45 and yes, let's do anaconda proposed FEs 17:18:12 #agreed 969182 - AcceptedBlocker - Violates the following F19 final criterion for systems with multipath storage: "The installer must be able to complete an installation using any network-attached storage devices (e.g. iSCSI, FCoE, Fibre Channel)" 17:18:20 #topic (966253) SELinux is preventing /usr/sbin/ntpdate from read, write access on the chr_file /dev/mapper/control. 17:18:23 #link https://bugzilla.redhat.com/show_bug.cgi?id=966253 17:18:26 #info Proposed Freeze Exceptions, anaconda, NEW 17:19:49 so, this is just a harmless warning, not enforced, right? 17:20:12 this sounds kinda blockery to me 17:20:26 unless you're doing a live install 17:21:12 anaconda expects to be in permissive or disabled. 17:21:28 bcl: even during a live install? 17:21:38 especially during live install. 17:21:44 (well any install) 17:22:26 We should figure out how to fix this, but the problem is the environment isn't matching what we do for boot.iso 17:22:52 tflink: one of the things 'liveinst' does is set enforcing off. 17:22:58 when I boot live, I'm in Enforcing mode 17:23:03 ah 17:23:04 kparal: ^^^ 17:23:20 yes, you're right. just checked 17:23:53 does this happen for kde or gnome? I would assume that it isn't limited to secondary DEs 17:24:00 yeah, we should check 17:24:25 the criterion we have is actually a 'polish criterion' - it's not intended to make sure that no AVCs are breaking anything, it's just that we think showing AVCs during install / firstboot / first log in is really bad 17:24:30 and more encouragement to the 'turn off selinux' brigade 17:24:32 oh, so this is just a report, not a crash. 17:24:50 so if this avc is popping up during release-blocking desktop live installs I'm +1 *blocker* even if it doesn't actually break anything 17:24:59 -1 17:25:10 turn off the report daemon on live :) 17:25:19 hah, nice try :) 17:25:23 (that being said, I'll look into fixing it) 17:25:58 the 'turn off selinux' brigade point is the important one: it _does_ look pretty bad for selinux if we can't even get our own house in order so that AVCs don't pop up when you're installing the OS. 17:26:46 ;) 17:26:53 * adamw starts a quick desktop live install 17:29:11 +1 17:29:40 well, i'm not seeing this yet in a minimum-possible-actions live install... 17:29:48 at 60% of 'installing software' 17:32:07 completed the install, didn't get any notifications, and SELinux Alert Browser says "No Alerts" 17:32:13 so +1 FE, no blocker? 17:32:18 so I guess we need more details to pin down whether this happens reliably 17:32:21 +1 FE at least for now 17:32:27 +1 FE for now, sure 17:32:29 (assuming the fix isn't too crazy) 17:33:36 proposed #agreed 966253 - AcceptedFreezeException - This is a polish issue; there should be no AVC denials during live install. If this were on a primary DE, it would be a blocker so accepted as a FreezeException. A tested fix would be considered past freeze. 17:33:50 ack 17:33:59 ack 17:34:03 ack 17:34:20 #agreed 966253 - AcceptedFreezeException - This is a polish issue; there should be no AVC denials during live install. If this were on a primary DE, it would be a blocker so accepted as a FreezeException. A tested fix would be considered past freeze. 17:34:29 #topic (972592) Text install claims 'GNOME Desktop' is the default package set, but actually installs minimal (Final TC2) 17:34:32 #link https://bugzilla.redhat.com/show_bug.cgi?id=972592 17:34:34 #info Proposed Freeze Exceptions, anaconda, NEW 17:35:33 this could qualify for a blocker 17:35:52 yeah, i didn't want to over-propose, but if you folks think so... 17:36:00 i also didn't check if you can actually get a GNOME install going at all 17:36:35 * kparal testing quick 17:36:45 depends on whether you can acutally install gnome desktop from text, I think 17:37:15 no, it says gnome but installs minimal. 17:37:50 -1 from me. it does what you told it to, the label is just wrong. 17:38:13 bcl: -1 blocker or -1 FE? 17:38:25 bcl: and, um, I didn't 'tell it to do' anything. this is about the *default* behaviour 17:38:25 -1 blocker 17:38:46 oh, I thought this was a minimal kickstart... 17:38:51 no, just interactive install. 17:39:49 "Software selection" spoke lists "GNOME Desktop", but if you just complete "Install Destination" and set a root password and timezone and start the install, without going into that spoke at all, you get minimal. 17:39:53 damn, it's downloading packages. I'm still waiting on the number of packages to be installed 17:39:55 oh, there's another one like this then where you want minimal, get miminal, but the label is wrong. probably related. 17:40:20 adamw: +1 blocker then. If you meant to do gnome and it said it would and it didn't... 17:40:42 adamw: when I picked GNOME manually, I get GNOME 17:40:51 just the default label is wrong, it seems 17:41:05 also sounds like a dupe of another bug? 17:41:15 bcl: we can argue that if you go into the spoke and explicitly pick it and it works, then the bug can be 'worked around' 17:41:34 +1 FE, not as sure about blocker 17:41:34 let's not argue for too long, though, it should be fixable and we should fix it, so let's pick one and go for it... 17:41:35 adamw: 972182 is what I was thinking of. 17:42:03 I'm also not strongly opinionated about the blocker call, since it's easy to work around 17:42:18 bcl: yeah, probably related. 17:42:35 i guess it's just stuck always showing 'GNOME Desktop' as the initial text for that spoke whatever the actual case is, or something. 17:43:04 i'm definitely +1 FE, don't really mind whether we make it blocker or not. 17:43:23 we could just accept FE for now and if somehow we get close to final and this isn't fixed, raise the alarm? 17:43:35 ok 17:43:58 adamw: when you make your note go ahead and assign to sbueno 17:44:06 proposed #agreed 972592 - AcceptedFreezeException - While this could lead to installing a different package set than intended, it is relatively easy to workaround (manually select package set) and a tested fix would be considered past freeze. 17:44:35 ack 17:44:38 ack 17:44:38 ack 17:44:43 #agreed 972592 - AcceptedFreezeException - While this could lead to installing a different package set than intended, it is relatively easy to workaround (manually select package set) and a tested fix would be considered past freeze. 17:44:49 ack 17:44:51 #topic (972265) When using a kickstart that specifies nothing about repo, 'closest mirror' doesn't seem to be usable 17:44:53 bcl: I think I already did 17:44:54 #link https://bugzilla.redhat.com/show_bug.cgi?id=972265 17:44:57 #info Proposed Freeze Exceptions, anaconda, NEW 17:45:39 this is a dupe of my older bug report? 17:45:43 "url" is required 17:46:15 no, this isn't about having it set properly in the kickstart 17:46:29 it's about intentionally using a kickstart which only defines a package set, and wanting to set everything else interactively 17:46:36 ah 17:46:37 remember, this is what we're telling people who want multiple desktop installs to do 17:46:56 and anyone who wants to set a package set at install time that you can't get through the GUI 17:47:03 so i think it ought to be a case we care about 17:48:10 +1 FE for preventing loud people with pitchforks 17:48:14 hrm. seems an awful lot like the missing url variation. 17:48:37 bcl: see a few lines above :) 17:48:40 bcl: it may be similar / the same to the code, but the 'user experience' end is a bit different, i guess 17:48:57 Yeah. I'll have to ponder that. 17:49:15 note there's another issue with this workflow which we'll probably hit next, that's rather more of a PITA to deal with 17:49:23 sneak preview! 17:49:42 * kparal claps his hands in anticipation 17:49:55 * bcl cringes 17:49:58 so, i'm +1, i don't disagree with myself :) 17:50:04 +1 here as well 17:50:18 proposed #agreed 972265 - AcceptedFreezeException - This makes the recommended workflow for people installing multiple DEs and other non-standard package sets more difficult than it needs to be. A tested fix would be considered past freeze. 17:50:21 of course, if anaconda decide in their Wisdom that this can't be fixed, we'd probably defer, but assuming there's something to be fixed, doing it for Final seems indicated. 17:50:22 ack 17:50:42 ack 17:52:02 ack 17:52:07 er, sorry, i already acked. 17:52:17 adamw: you forgot your moustache 17:52:21 whoopsie 17:52:30 * adamw dons moustache and beret 17:52:30 le ack 17:52:35 adamw: here you go - http://4.bp.blogspot.com/_lwmLZvKDhXQ/TDbCYd0GYOI/AAAAAAAAADY/E6c0FLzSiow/s1600/moustache+clearer+closer+crop.jpg 17:52:48 #agreed 972265 - AcceptedFreezeException - This makes the recommended workflow for people installing multiple DEs and other non-standard package sets more difficult than it needs to be. A tested fix would be considered past freeze. 17:52:48 hah! 17:53:09 if I'm ever in italy, I want to try finding a can of that 17:53:23 #topic (972266) Installation Destination spoke behaves strangely when installing from a minimal (packages only) kickstart 17:53:26 #link https://bugzilla.redhat.com/show_bug.cgi?id=972266 17:53:29 #info Proposed Freeze Exceptions, anaconda, NEW 17:53:44 il acko 17:53:57 yup, this is the other one. 17:54:04 ouch 17:54:07 same kind of bug with the 'installation destination' spoke, but rather more of a pain to deal with. 17:54:14 technically workaround-able 17:54:59 yeah, but you have to perform some contortions. the bootloader bit is especially tricky. 17:55:08 weird. 17:55:27 +1 FE 17:56:04 +1 here as well 17:56:17 since some organizations (Red Hat) prepare simple kickstarts with some basic data for their employees which can then do a network installation, this might be even blocker for them 17:56:43 +1 FE 17:56:54 I think there's usually some network preconfigured, ntp service and something like that, the rest is configured interactively 17:57:09 oh, just for the record, these aren't 'single-source' bugs: robatino saw the same thing 17:57:16 in fact he saw it first and I just confirmed it (and investigated a bit more) 17:57:41 +1, obviosly 17:58:05 proposed #agreed 972266 - AcceptedFreezeException - This makes an interactive install started with incomplete kickstart far more complicated than it needs to be, even though it is techinically workaround-able. A tested fix would be considered after freeze. 17:58:17 ack 17:58:27 ack 17:58:27 ack 17:58:34 #agreed 972266 - AcceptedFreezeException - This makes an interactive install started with incomplete kickstart far more complicated than it needs to be, even though it is techinically workaround-able. A tested fix would be considered after freeze. 17:58:37 ack 17:58:44 #topic (970048) pre-selected language is cropped on the initial screen 17:58:47 #link https://bugzilla.redhat.com/show_bug.cgi?id=970048 17:58:50 #info Proposed Freeze Exceptions, anaconda, NEW 18:00:16 that's an odd polish bug 18:00:35 +1 FE as long as its earlier in freeze 18:00:49 how risky is fixing this? 18:00:52 I'm not sure about taking a fix for this late in freeze unless it's something simple 18:00:53 pretty trivial, but +1 for pre-official-freeze or early-official-freeze 18:01:13 isn't there an accent in the text string in question somewhere? 18:01:17 that could be throwing it off 18:01:29 there is 18:01:35 well, scrolling to the top is never really a nice way to do things. 18:01:55 Čeština (Česká Republika) 18:01:57 is this just broken for auto detection or manual selection as well? 18:02:01 yeah, it has a whole bunch of those funky accents you crazy czech kids use =) 18:02:04 proposed #agreed 970048 - AcceptedFreezeException - This is a polish issue that looks bad on the first screen presented to a user outside of the US. A tested fix would be considered past freeze, but unlikely to be accepted late in freeze 18:02:17 Viking-Ice: i'd assume only auto-selection because that's the only case where anaconda is trying to position the selected language specially like this 18:02:28 Viking-Ice: otherwise the widget just behaves like a pretty normal list 18:02:36 oh, I see, it moves it to the top entry instead of scrolling to its location. 18:02:41 yeah, if I boot a VM and select 'Czech' it just gets highlighted 18:02:46 no positioning weirdness 18:02:48 ack 18:02:51 ack 18:03:07 ack 18:03:08 #agreed 970048 - AcceptedFreezeException - This is a polish issue that looks bad on the first screen presented to a user outside of the US. A tested fix would be considered past freeze, but unlikely to be accepted late in freeze 18:03:09 bcl: right, i think it changed since 18. 18:03:22 OK, that's all the anaconda bugs 18:03:32 adamw: yes, a bunch. 18:03:40 back to the non-anaconda and non-blivet blockers! 18:03:43 cool. see ya. 18:03:50 bcl: thanks for the help 18:04:17 #topic (971741) php and cups-php incompatibility 18:04:17 #link https://bugzilla.redhat.com/show_bug.cgi?id=971741 18:04:17 #info Proposed Blocker, cups-filters, ASSIGNED 18:04:54 hmm looks like this could be fixed with a zero day 18:05:14 if you update from the network 18:05:21 and not from DVD 18:05:26 upgrade, not update, no? 18:05:31 upgrade, right 18:05:51 yeah, this could be fixed with an update 18:06:02 there's the media-based upgrade case, though isn't that still basically broken? 18:06:39 I don't think that cups-php is on the dvd, though 18:06:40 -1 18:06:44 it's fundamentally broken, but that's not probably your question :) 18:06:55 tflink: as i'm reading it, cups-php is what you have one the system *to be upgraded* 18:07:10 adamw: right 18:07:17 it requires a specific version of cups and cups-libs (packages that are certainly on the DVD) and they aren't there in f19 18:07:17 but the package doesn't exist anymore 18:07:44 so any time you're upgrading an f18 system with cups-php installed, you'll hit this (unless it gets fixed, obviously) 18:07:59 that's how I understand it as well 18:08:01 oh, but cups-php isn't in the default package set, so -1. whee. 18:08:10 we only support default package set upgrades. 18:08:15 er, 'block on' 18:08:25 FE? 18:08:28 minimal, default GNOME, default KDE, anything else takes a hike 18:08:31 eh, 0-day 18:08:37 i'd be FE if media upgrade worked, but as it doesn't...-1/-1. 18:08:50 -1/-1 18:09:25 I suppose this breaks only php printing as well ( PHP::PRINT::IPP? ) 18:09:53 proposed #agreed 971741 - RejectedBlocker, RejectedFreezeException - While unfortunate, cups-php isn't part of any default package sets and thus does not qualify as a release blocking issue. This can be fixed as an update post-release and thus is rejected as a FreezeException for F19 final as well. 18:10:02 ack 18:10:14 ack 18:10:21 ack 18:10:24 ack 18:10:33 we have 50 minutes left until the 3 hour mark 18:10:36 Viking-Ice: well, it'd just stop upgrade working until you futzed it somehow (probably by manually removing cups-php...which thinking about it is just what we want to do anyway, so there's even an easy workaround) 18:10:39 how many blockers left? 18:10:42 #agreed 971741 - RejectedBlocker, RejectedFreezeException - While unfortunate, cups-php isn't part of any default package sets and thus does not qualify as a release blocking issue. This can be fixed as an update post-release and thus is rejected as a FreezeException for F19 final as well. 18:10:47 9 18:10:56 #topic (971763) disable updates-testing 18:10:56 #link https://bugzilla.redhat.com/show_bug.cgi?id=971763 18:10:56 #info Proposed Blocker, fedora-release, ASSIGNED 18:10:59 crap 18:11:02 #undo 18:11:02 Removing item from minutes: 18:11:04 #undo 18:11:04 Removing item from minutes: 18:11:11 #topic (971763) disable updates-testing 18:11:11 #link https://bugzilla.redhat.com/show_bug.cgi?id=971763 18:11:11 #info Proposed Blocker, fedora-release, ASSIGNED 18:11:48 my bad in requesting too early 18:11:59 but we can formally take this as +1 18:12:13 WFM 18:12:13 yup we should not leave testing enabled for final ;) 18:12:15 +1 18:12:20 +1 18:12:27 sure 18:12:56 i think releng have this on one of their 'todo lists', but no harm having a blocker bug to track it too 18:12:56 +1 18:13:42 +1, /me will track it too 18:13:43 proposed #agreed 971763 - AcceptedBlocker - This is a normal part of the final release process and violates the following F19 final release criterion: "No notices or alerts about pre-release status should be present" 18:14:13 ack 18:14:18 er 18:14:19 patch 18:14:32 use criterion #25 18:14:39 ack 18:14:51 point 18:15:03 proposed #agreed 971763 - AcceptedBlocker - This is a normal part of the final release process and violates the following F19 final release criterion: "A Package-x-generic-16.pngfedora-release package containing the correct names, information and repository configuration for a final Fedora release (as opposed to a pre-release) must be present on ISO media while the appropriately versioned Package-x-generic-16.pnggeneric-release package must be 18:15:07 .fire tflink criteria fail 18:15:07 adamw fires tflink criteria fail 18:15:10 gah 18:15:18 chop out the icon names. sorry about that. 18:15:27 i think i'll drop the wikification in the Final criteria, if I didn't already. 18:15:34 it doesn't really gain us anything. 18:15:42 proposed #agreed 971763 - AcceptedBlocker - This is a normal part of the final release process and violates the following F19 final release criterion: "A fedora-release package containing the correct names, information and repository configuration for a final Fedora release (as opposed to a pre-release) must be present on ISO media while the appropriately versioned generic-release package must be available in the online release repository" 18:15:53 ack 18:16:01 adamw: but the wiki standards ... 18:16:04 :-D 18:16:08 re-ack 18:16:14 ack 18:16:19 ackity ack ack 18:16:34 #agreed 971763 - AcceptedBlocker - This is a normal part of the final release process and violates the following F19 final release criterion: "A fedora-release package containing the correct names, information and repository configuration for a final Fedora release (as opposed to a pre-release) must be present on ISO media while the appropriately versioned generic-release package must be available in the online release repository" 18:16:46 #topic (972250) Release notes in Lost&Found KDE menu 18:16:46 #link https://bugzilla.redhat.com/show_bug.cgi?id=972250 18:16:46 #info Proposed Blocker, fedora-release-notes, ASSIGNED 18:19:08 +1 FE 18:19:14 +1 FE 18:19:15 oh, proposed blocker 18:19:36 so, as the non-revised criteria stand, this should be a +1 18:19:39 yeah, but the relevent criterion is being proposed for de-blockerification 18:19:43 but i'm of the opinion we're going a bit far in that criterion 18:19:59 I dont think this warrant us blocking the release over 18:20:05 I don't really see this in our current criteria 18:20:17 hum, now i read it more closely you may be right 18:20:19 * adamw looks 18:20:42 yeah...it doesn't really hit the old 'menu sanity' criteria anyway. it's not appearing twice, it's just not in the right place. 18:20:46 I think we used to have "nothing in Other menu", but it's no longer there 18:21:03 so -1 blocker 18:21:08 we also have "The final branded release notes from the Documentation team must be present on ISO media" but it is _there_ 18:21:22 kparal: i think we killed that one when Shell came in, as it was pretty GNOME 2-specific. 18:21:37 so yeah, even by the old criteria this is -1, but +1 FE since we're here. 18:21:56 -1/+1, Kevin maybe has a point but I'd say we have bigger issues sometimes :) 18:22:28 proposed #agreed 972250 - RejectedBlocker, AcceptedFreezeException - This doesn't violate any of the F19 release criteria and thus is not a blocker for F19 final. However, a tested fix would be considered past freeze. 18:22:31 yeah, the release notes are kinda important so it'd suck a bit to release like this, but i wouldn't hate myself. 18:22:35 ack 18:22:39 ack 18:22:42 ack 18:22:45 #agreed 972250 - RejectedBlocker, AcceptedFreezeException - This doesn't violate any of the F19 release criteria and thus is not a blocker for F19 final. However, a tested fix would be considered past freeze. 18:22:57 #topic (971021) Some icons in collection bubbles are not visible 18:22:57 #link https://bugzilla.redhat.com/show_bug.cgi?id=971021 18:22:57 #info Proposed Blocker, gnome-shell, ASSIGNED 18:23:58 I'm +1 to this one 18:24:04 ( being a blocker ) 18:24:08 the problem here is that you can't reach some app icons 18:24:14 seriously, "it meets blocker bug criteria" but he doesn't say _which_ one? 18:24:20 they are cut off 18:24:36 well, we might use " All applications listed under the Applications menu or category must start successfully " 18:24:40 the apps can't be started 18:25:22 if there are just 3 rows of them, you at least see the name. if there would be 4 lines, you would probably not see the first row at all 18:25:30 yeah, i wouldn't mind wiggling in under that criterion 18:25:41 it was written for GNOME 2, this seems like a reasonable application of it to GNOME 3 18:25:47 let me see what I wrote in the 'revised' criteria 18:26:05 Personally I dont understand why they dont just list all the icons in the same place 18:26:12 hum, "All applications that can be launched using the standard graphical mechanism of a release-blocking desktop after a default installation of that desktop must start successfully and withstand a basic functionality test.", so it sounds like it doesn't quite hit that. 18:26:40 we can never cover every glitch that can happen 18:26:42 adamw: how so? 18:26:54 how do you start the apps you can't click on 18:27:03 touche 18:27:11 the re-worded criterion doesn't actually require that all apps 'can be launched' 18:27:18 it just says that all apps that 'can be launched' must work 18:27:26 oh, point 18:27:35 i didn't actually think about a case like this bug, where some apps should be launchable but aren't 18:27:36 it should say ever app that it shipped 18:27:43 mean is 18:27:49 so that's an inadequacy in my rewording, i guess 18:28:17 Viking-Ice: the problem is that some are intentionally _not_ listed, like gnome-session-properties. but i can work on the wording later. 18:28:25 what the hell is this 'colllection bubble' thing anyway? 18:28:29 and how do you get to it? 18:28:30 shipped and presented? 18:29:00 adamw: a new feature of the new GNOME 18:29:04 oh, I see it. 18:29:04 it's there in all 18:29:09 adamw: it's in the new menu instead of categories 18:29:18 categories are old-school, it seems 18:29:41 all the smart phones just list all icons/apps they decided to go another route with those collection bubbles 18:29:41 it seems to be Utilities in English, not Tools.\ 18:29:56 Viking-Ice: you can actually group apps like that on Android 18:29:57 anyhoo 18:30:14 i guess I'm +1 under the spirit of the old criterion and I should improve the re-wording to cover this case 18:30:28 +1 18:30:35 still +q 18:30:39 mean 1 18:31:30 proposed #agreed 971021 - AcceptedBlocker - Causes certain apps to be non-launcable from the gnome-shell overlay and violates the following F19 final release criterion: "All applications listed under the Applications menu or category must start successfully" 18:31:47 maybe say 'spirit of the following F19...', but sure, ack 18:31:52 ack 18:32:01 ack 18:32:04 ack 18:32:12 adamw: either way is fine with me 18:32:40 #agreed 971021 - AcceptedBlocker - Causes certain apps to be non-launcable from the gnome-shell overlay and violates the following F19 final release criterion: "All applications listed under the Applications menu or category must start successfully" 18:32:47 #topic (961533) grub2 menu entry shows Schr?dinger?s Cat after a 'yum update' 18:32:50 #link https://bugzilla.redhat.com/show_bug.cgi?id=961533 18:32:53 #info Proposed Blocker, grub2, NEW 18:33:42 eh, aesthetic issue, -1/+1 18:33:56 it *works*, right? 18:34:09 yep 18:34:09 yeah it works agreed -1/+1 18:34:22 I didn't think that we had a fork of grub2 18:34:30 Arguable we should just have Fedora $Number in the grub menu 18:34:35 ( or just Fedora ) 18:35:11 tflink: i think mads is referring to the use of grubby 18:35:21 but imbw. 18:35:21 proposed #agreed 961533 - RejectedBlocker, AcceptedFreezeException - This is a minor polish issue that doesn't violate any of the F19 release criteria but a tested fix would be considered after freeze. 18:35:25 ack 18:35:27 adamw: or grub-efi 18:35:36 we don't use grub-efi any more, we use grub2's efi mode. 18:35:36 ack 18:35:48 ack 18:35:48 adamw: I know, but that's the closest thing I could think of 18:35:51 btw does anyone care about the release name in the grub menu? 18:35:52 25 minutes left, how many more blockers to go? 18:36:01 Viking-Ice: i'm still on the 'ditch the damn release names already' team 18:36:05 but hey 18:36:07 5 blockers left 18:36:08 ack 18:36:14 #agreed 961533 - RejectedBlocker, AcceptedFreezeException - This is a minor polish issue that doesn't violate any of the F19 release criteria but a tested fix would be considered after freeze. 18:36:27 I have a hard stop at the top of the hour, FWIW 18:36:34 adamw: the question is - do we really want to show them everywhere? 18:36:36 adamw, I'm on that team as well ;) 18:36:43 if we keep going past that, someone else is going to have to take over the meeting 18:36:47 it's fun to have release name but... 18:37:02 #topic (971046) GError: Timeout was reached 18:37:02 #link https://bugzilla.redhat.com/show_bug.cgi?id=971046 18:37:02 #info Proposed Blocker, polkit, ASSIGNED 18:37:04 let's try to finish those 5 and leave it at that 18:37:04 jreznik: well, my opinion contains an obvious sub-opinion on that topic =) 18:37:20 actually, it's 4. I forgot about the blivit bug that we already covered 18:37:39 this turns out to be PPC-specific, I think 18:37:45 down to a crash in policykit on PPC or something 18:37:47 so -1 blocker, +1 FE 18:38:10 yup 18:38:38 yep, same here: -1/+1 18:39:36 proposed #agreed 971046 - RejectedBlocker, AcceptedFreezeException - This would be a release blocker on PA but appears to be PPC specific. Thus, rejected as a blocker but a tested fix would be considered after freeze. 18:39:42 ack 18:40:08 ack 18:40:17 ack 18:40:20 aclk 18:40:46 #agreed 971046 - RejectedBlocker, AcceptedFreezeException - This would be a release blocker on PA but appears to be PPC specific. Thus, rejected as a blocker but a tested fix would be considered after freeze. 18:40:53 #topic (964006) cloud-init hostname service failing on initial boot 18:40:56 #link https://bugzilla.redhat.com/show_bug.cgi?id=964006 18:40:58 #info Proposed Blocker, selinux-policy, ASSIGNED 18:42:44 it sounds like cloud images are showing AVC denials on setting the hostname 18:42:55 there appears to be no explanation of why this is nominated as a blocker 18:43:09 yeah, I was about to chide matt about that 18:43:16 matthew miller simply set it as a blocker without explanation 18:43:30 does anyone want to try and interpret why we should block on this or shall we just punt and ask? 18:43:43 adamw: my explanation wasn't good enough? 18:44:11 Dan is already on it ( which usually means it get fixed yesterday ;) ) 18:44:29 tflink: well, what's the practical consequence of that? 18:44:43 tflink: consider most of us have zero experience with doing anything in EC2. or at least, I don't. 18:44:52 however this is cloud spesific right 18:44:54 it slows down, at least 18:45:06 slows down boot, rather 18:45:16 doesn't look like it prevents boot 18:45:33 but we do have "In most cases, there must be no SELinux 'AVC: denied' messages or abrt crash notifications on initial boot and subsequent login" 18:46:01 if I understand this correctly it's that clouds own tool that's triggering this 18:46:31 yeah, I think this would be cloud image only (ec2, openstack etc.) 18:46:45 and it's not blocking - fe? 18:46:49 and as such it should be fixable via zero day update 18:46:52 -1/-1 18:47:07 not sure it's so fixable with an update 18:47:24 -1/+1 as described above 18:48:11 workaround would be to boot into permissive mode and update 18:48:13 tflink: the AVC criterion as noted earlier is a polish criterion for the 'common case' install, i'm not sure how bad an AVC during a cloud deployment looks to users or what effect it'd have on our image... 18:48:23 only 12 minutes left 18:48:23 Viking-Ice: how exactly do you do that on ec2? 18:48:36 which is why I don't want to spend too much time figuring out the justification for this one 18:49:14 tflink, I would assume through console 18:50:00 ( I have no idea how Amazon does things or we along with them ) 18:50:06 proposed #agreed 964006 - RejectedBlocker, AcceptedFreezeException - While this does slow down boot for cloud images, it does not violate any F19 release criteria and is rejected as a release blocking bug for F19 final. However, a tested fix would be considered after freeze. 18:50:41 Viking-Ice: I don't think you can do that and even if you could, if you needed to do that for _every_ cloud instance, I'd never use the F19 AMIs 18:50:58 I suspect that most other people wouldn't, either 18:51:10 tflink, I see 18:51:19 ack 18:51:19 i can ack that, but maybe with a note that it can be re-proposed if we misread it 18:51:30 it's akin to telling desktop users that they have to adjust their boot options every time they reboot 18:51:48 #agreed 964006 - RejectedBlocker, AcceptedFreezeException - While this does slow down boot for cloud images, it does not violate any F19 release criteria and is rejected as a release blocking bug for F19 final. However, a tested fix would be considered after freeze. 18:52:10 #topic (971392) systemd update breaks polkit-kde-auth 18:52:10 #link https://bugzilla.redhat.com/show_bug.cgi?id=971392 18:52:10 #info Proposed Blocker, systemd, NEW 18:52:46 I just tried with Czech l10n, still works for me 18:53:06 and nobody from kde sig was able to reproduce it neither 18:53:29 the strange thing is - when kde polkit agent started on command line - it just works 18:53:31 reject, ask him to repropose if more info/reproducers show up? 18:53:38 agreed 18:53:47 yes 18:53:49 so in really worst case, we have workaround 18:54:20 but seems like jiri is the only guy affected by this one for some reason 18:54:26 kparal, was this Jiri own laptop ? 18:54:31 ( which he upgraded ) 18:54:37 it was a clean install 18:54:50 proposed #agreed 971392 - RejectedBlocker - While this sounds like it could be a blocker, other people are having difficulty reproducing the issue and thus, this is rejected as a release blocking bug for F19 final. Please re-propose as a blocker if more people are able to reproduce the issue. 18:55:03 ack 18:55:10 ack 18:55:16 patch, maybe add there's known workaround 18:55:38 ack 18:55:53 #agreed 971392 - RejectedBlocker - While this sounds like it could be a blocker, other people are having difficulty reproducing the issue and there is a known workaround. Thus, this is rejected as a release blocking bug for F19 final. Please re-propose as a blocker if more people are able to reproduce the issue. 18:56:00 #topic (955236) [abrt] yum-3.4.3-83.fc20: cli.py:1945:removeGroups:AttributeError: 'InstalledGroup' object has no attribute 'groupid' 18:56:03 #link https://bugzilla.redhat.com/show_bug.cgi?id=955236 18:56:05 #info Proposed Blocker, yum, MODIFIED 18:56:05 still no dev input on this 18:56:09 reject? 18:57:12 is this not fixed ( per maintainers comment ) 18:57:20 yeah, per what we said at the last meeting, it seems like this just isn't important enoguh to accept. 18:57:34 Viking-Ice: the update isn't stable yet (or possibly not even submitted as an update yet) 18:57:47 adamw, 0 day anyway 18:57:53 * adamw double checks 18:57:57 he's removing not installing 18:58:06 proposed #agreed 955236 - RejectedBlocker - This doesn't violate any of the F19 release criteria and is thus rejected as a release blocking bug for F19 final. 18:58:18 yeah, there's no -93 update yet at all 18:58:19 so you would do that before upgrade or after installation 18:58:19 ack 18:58:31 Viking-Ice: the question was whether there was db corruption on failure or not 18:58:45 but we're now figuring that if there was, someone would have said so by now 18:58:47 ack 18:58:49 adamw: I thought there was a -94, though 18:59:04 yum-3.4.3-91.fc19 is the most recent build in bodhi. 18:59:09 was that the last blocker? 18:59:22 yep 18:59:41 right on time :) at least for blockers 18:59:49 #agreed 955236 - RejectedBlocker - This doesn't violate any of the F19 release criteria and is thus rejected as a release blocking bug for F19 final. 19:00:06 we didn't touch the accepted blockers, but we can do that on wednesday 19:00:16 #topic Open Floor 19:00:17 yup 19:00:20 yep 19:00:24 Anything else that should be brought up today? 19:00:44 yep, and I'll be checking the status too 19:00:44 just please comment on the final blocker revision so we can get it into production 19:00:50 i'll work on the stuff that came up during the meeting 19:00:58 * jreznik has to run, will comment on revision 19:01:07 adamw, we really ought not to change it until we release F19 ;) 19:01:53 Viking-Ice: eh, maybe, i wanted to get them all done but you're right that it's a bit late now 19:02:05 hmm, my head seems to be better now, but my mouth definitely not - I can't open it and what's worst - I'm hungry but I can't bite :))) 19:02:05 i did manage to get alpha and beta into proudction before we really got rolling on validation for those milestones 19:02:11 jreznik: diet! 19:02:26 adamw, yeah 19:02:57 Viking-Ice: we can always basically use the 'old' ones but keep the 'new' ones in mind too, as we did for the GNOME menu thingh 19:03:32 alrighty, apart from that, anything else? 19:03:42 nothing from me 19:04:06 nothing from me, thanks guys 19:04:45 * adamw sets Quantum Fuse for a number of minutes that has been redacted by the NSA 19:04:57 and not whistle blown yet 19:05:01 =) 19:06:31 alrighty, thanks for coming, folks 19:06:38 let's get the testin' rolling 19:06:41 #endmeeting