19:02:22 <jwb> #startmeeting
19:02:22 <zodbot> Meeting started Fri Jan 11 19:02:22 2013 UTC.  The chair is jwb. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:02:22 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:02:33 <jwb> #meetingname fedora-kernel
19:02:33 <zodbot> The meeting name has been set to 'fedora-kernel'
19:02:45 <jwb> #meetingtopic Fedora Kernel Meeting
19:02:52 <jwb> #topic init
19:02:55 <jwb> who's here?
19:02:59 * jforbes 
19:03:11 <brunowolff> I'm here.
19:03:11 * nirik is around
19:03:36 <jwb> ok, let's get rolling
19:03:42 <jwb> #topic Rawhide
19:03:49 <jwb> jforbes, want to cover this one?
19:04:14 <jforbes> Sure, so rawhide is running at 3.8-rc3 at this point, and there has been little activity since then
19:04:26 <nirik> side note here... if kernel folks want at fudcon, I would be happy to show them what the debug kernel looks like on my laptop. ;) (if you want to try and isolate things to make it more usable)
19:05:03 <jforbes> For people interested in testing it, we really appreciate it.  If for some reason the debug kernels are not your cup of tea, you can run the nodebug kernels
19:05:24 <brunowolff> I am running rc3 on a machine now.
19:05:24 <jwb> #info Rawhide is on 3.8-rc3
19:05:26 <nirik> sure. I would run debug ones if the hit was reasonable.
19:05:43 <nirik> but it's so slow I can't get anything done and it burns my lap. ;)
19:05:46 <jforbes> nirik: yeah, jwb and I will both be at fudcon. It would be interesting to see what could be done to speed that up for you without killing useful info
19:06:00 <brunowolff> I started running 3.8 kernels once there was a nodebug build.
19:06:25 <jwb> #info nodebug kernels continue to be provided in the side repo for those interested
19:06:32 * nirik is also seeing a iwlwifi regression in 3.8... it's very slow. Might be a fix merging soon tho
19:06:43 <jforbes> Right, and we have been doing nodebug builds for every regular build, so hopefully we will get some good feedback there.
19:06:58 <jforbes> nirik: have you seen a possible fix, or are you just speculating?
19:07:15 <nirik> speculating... knurd mentioned he saw the same thing and pointed to:
19:07:28 <nirik> knurd> see https://patchwork.kernel.org/patch/1927911/ , which is heading upstream: http://article.gmane.org/gmane.linux.kernel/1419214
19:07:55 <jforbes> Ahh, that could cause a problem, yeah
19:07:59 <nirik> also, bug https://bugzilla.redhat.com/show_bug.cgi?id=863424
19:09:14 <jwb> anything else on rawhide?
19:09:25 <jforbes> Now that we are into rc3, I expect things will settle a bit. and bugs will be fixed.
19:09:31 * nirik has nothing. Otherwise working ok.
19:09:35 <jwb> jforbes, ha
19:09:41 <jforbes> The only other issue is an S390 build issue I am working on
19:09:56 <jforbes> jwb: I didn't say *all* bugs
19:10:14 <jwb> jforbes, oh, i missed read "and" as "all
19:10:14 <jwb> "
19:10:24 <jwb> apparently my eyes are tired today
19:10:31 <jwb> ok, let's move on to f18
19:10:33 <jwb> #topic F18
19:10:39 <jwb> jforbes i think you're still up
19:10:53 <jforbes> For F18. 3.7.1 is in updates testing, 3.7.2 is building locally right now
19:11:34 <jforbes> I should have it into updates testing for the weekend. We have been getting feedback, so clearly people are testing.  I think most of the rebase gotchas are worked out now
19:11:44 <jwb> #info F18 release kernel is 3.6.10-4.fc18
19:12:19 <jwb> #info F18 0-day update kernel is 3.6.11-3.fc18, but a 3.7.2 rebase should be in updates-testing shortly
19:12:25 <jforbes> Feedback from the latest 3.7.1  (which allows 3rd party modules to build) has been all positive
19:12:36 <jforbes> At least from the people who actually tested it
19:12:50 <jwb> maybe we should put out a call for testing?
19:12:59 <jforbes> I will when 3.7.2 goes out
19:13:09 <brunowolff> Yeah, it would be nice if pwalter would stop -1 kernels because he doesn't recognize the policy.
19:13:15 <jwb> jforbes, cool
19:13:18 <nirik> we should have a f18 updates later today...
19:13:29 <jforbes> We also got a bit of 3.7 testing over the holidays when rawhide sat at 3.7.0 and rawhide-nodebug was sitting on 3.7.1 stable
19:13:31 <jwb> #action jforbes to put out a wider call for testing of 3.7.2
19:13:44 <jwb> nirik, we've disable auto-karma for the time being
19:13:53 <jwb> so it won't go there automagically
19:13:57 <nirik> ok. sounds good.
19:14:15 <nirik> if you want to push it before release, let me know and I can make sure and do a updates push for it.
19:14:24 <jforbes> I think I will turn on autokarma for 3.7.2, there has been good feedback on the last 3.7.1
19:15:04 <jwb> jforbes, i'd be OK with that i think.  though when it comes to f17, i'd want it off again.  the sucky thing is autokarma isn't per-update, it's per-package iirc
19:15:15 * nirik has 2 f18 machines on 3.7.1 working fine.
19:15:25 <jforbes> The problems we did have were packaging issues that were fixed in rawhide in the early rcs and not brought back to the F18 rebase.  those are hashed out, and there was a decent amount of testing with 3.7.1 in nodebug over the holidays
19:15:45 <nirik> we want to make sure f18 is newer than f17 for upgrades... if at all possible
19:15:56 <jforbes> jwb: Yeah, but the other problem is you can't push without karma automation for a very long period of time
19:16:07 <jwb> jforbes, nah.  i can push 3.7.1 right now
19:16:39 <jwb> nirik, nope.  we tried to do that during the NTH process and were told it didn't matter
19:16:43 <jforbes> nirik: I am not sure that really matters, if you upgrade and still have the F17 kernel, no harm done.  You wont be able to secure boot until you get an F18 kernel, but it isn't a regression
19:16:47 <jwb> nirik, so... we aren't holding things up anymroe
19:17:03 <nirik> huh, ok.
19:17:13 <jwb> i was a bit... discouraged.
19:17:21 <jwb> anyway, anything else on f18?
19:17:25 <jforbes> There are not major userspace features which rely on a kernel features not in F17 already
19:17:29 <davej> for the most part, we keep them in sync anyway. it's only freezes that screw it up
19:17:29 <jforbes> nothing else on F18
19:17:56 <jwb> fwiw, i have been artificially bumping baserelease to a higher number in f18
19:18:12 <jwb> on a rebase that is
19:18:29 <jwb> anyway, let's move on
19:18:31 <jwb> #topic F17
19:18:45 <davej> actually, let's stay on that for a sec..
19:18:50 <davej> would it be worth us formalising that ?
19:18:52 <jwb> #undo
19:18:52 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x12b94650>
19:19:07 <davej> like make f17 always be 100 baserelesaes higher.  f18 200 higher etc. ?
19:19:43 <jwb> davej, we can.  it's a trivial way to keep upgrade path working while they're in sync.  but if we have one stuck in freeze at e.g. 3.6 and the stable branches are at 3.7, it's broken
19:19:43 <jforbes> davej: I think that will get the be a problem, when F17 goes EOL, we have 200-400, and it just keeps climbing
19:20:06 <davej> ok, never mind.
19:20:38 <jforbes> I guess we could manage it by dropping them to the 100-300 baseline after the first rebase post EOL
19:20:41 <jwb> jforbes, you'd reset it on rebase.  so once the oldest goes EOL, you use the "oldest" number for the new oldest release when you rebase
19:20:45 <jwb> right
19:20:55 <jwb> the only case it doesn't help is the freeze situation
19:21:03 <davej> heh, flashback to the trick with cvs ident we used to do..
19:21:11 <jwb> davej, yeah :)
19:21:39 <jwb> move on?
19:21:40 <jforbes> I am okay with it if that helps everyone with the upgrade path
19:22:11 <davej> it sounds like it would still have corner cases, but it might be something to think about
19:22:20 <davej> ok, move on.
19:22:40 <jwb> hm.  apparently #undo didn't undo my earlier #topic
19:22:44 <jwb> so wth did it undo?
19:22:50 <davej> heh
19:23:02 <jwb> #action jforbes to put out a wider call for testing of 3.7.2
19:23:06 <jwb> just in case ;)
19:23:18 <davej> I like that it printed out an address. that's useful for debugging.
19:23:24 <jwb> ok so F17 is sitting at 3.6.11 right now
19:23:38 <jwb> what i think will be the last 3.6 update is sitting in updates testing at the moment
19:23:51 <davej> yeah
19:23:54 <jwb> i plan to rebase it to 3.7 either during or shortly after FUDCon
19:24:13 <jwb> waiting on 2 things really, 1) a bit more testing in f18, 2) f18 release
19:24:18 <jwb> mostly 1
19:24:31 <jwb> anyone have an issues with that plan?
19:24:57 <davej> I thought I was doing 17 this month? (but no objections if you want to do it)
19:25:06 <jwb> no, you're doing f16
19:25:19 <davej> ah, the red-headed stepchild
19:25:22 <jwb> i'm f17.  jforbes is f18/rawhide
19:25:58 <jwb> it's OK everyone.  we totally have this under control.  nothing to see here...
19:26:00 <jwb> ;)
19:26:21 <davej> yeah, I'm not entirely with it today.
19:26:34 <jforbes> Sorry, box rebooted for some reason
19:26:44 <davej> though getting sick now is preferable to during travel to lca in 2 weeks I guess.
19:26:54 * knurd , the journalist, takes a closer look at the last lines
19:26:59 <knurd> ;-)
19:27:02 <jwb> #info final F17 3.6.11 update is currently in updates-testing.  F17 will rebase to 3.7.x right around FUDCon
19:27:30 <jwb> the F17 bug count is huge
19:27:48 <jwb> i'm guessing that 3.7 will help some, but honestly i don't expect it to drastically decrease
19:28:04 <jwb> i plan on doing a mass bug update again for the rebase, so we'll see
19:28:07 <davej> I'm guessing we'll get 100 tops that will go away on a rebase.
19:28:21 <jwb> yeah.  i'm thinking more like 20-30
19:28:22 <davej> there's just too much 'weird shit' still there that's really bothering me
19:28:31 * jsmith tends to agree
19:28:54 <jwb> davej, which means most of them are still going to be there on f18
19:28:56 <jwb> fun times.
19:29:11 <davej> jwb: I'm convinced that DRI is a big contributor.
19:29:30 <jwb> #action jwb will do a mass bug update when F17 is rebased to 3.7.x.  Advanced apologies for all the email
19:29:54 <jwb> davej, yes.  seems dri, sound/usb are the ones that are triggering it lately
19:30:24 <jwb> anything else on f17?
19:30:30 <jwb> oh
19:30:39 <jwb> davej we had talked about turning intel iommu back on
19:30:51 <davej> yeah, let's do that for 18/rawhide.
19:31:10 <jwb> ok.  i'll stick something about it in the meeting minutes by hand
19:31:16 <davej> I don't think rawhide will get us enough coverage to really see anything interesting
19:31:21 <jwb> right
19:31:31 <jforbes> Okay, I will do it with this 3.7.2 update
19:31:35 <jwb> jforbes, great
19:31:40 <davej> and depending how it works out in 18. we'll consider it for 17 when we rebase
19:31:54 <jwb> hopefully things work better now. :\
19:31:59 <jwb> ok, move on?
19:32:17 <jwb> #topic F16
19:32:26 <jwb> davej you want to take this one?
19:32:32 <davej> not much to report here. EOL can't come soon enough.
19:32:52 <davej> unless we see something really critical come in, hopefully we can let it bitrot.
19:33:07 <jwb> #info F16 will remain on 3.6.11-X.  serious bug fixes will be backported until EOL
19:33:15 <jwb> #info People are encouraged to update to F17 or F18
19:34:04 <jwb> #topic Open Floor
19:34:12 <jwb> anyone have anything?
19:34:16 <davej> yeah.
19:34:27 <davej> debug options. been doing some thinking.
19:34:48 <davej> Right now, we have this binary state of debug on/off (or slow/usable as some people see it)
19:35:09 <davej> I think it might be worth trying to narrow that down by doing a debug off build, and then every day, we add back some more options.
19:35:30 <nirik> davej: I'm happy to work with you guys at fudcon to try and isolate whats causing so much slowness.
19:35:30 <jwb> i think it'd be worth doing if we had some people impacted signed up to test daily
19:35:42 <davej> jwb: I think nirik just did
19:35:42 <nirik> if there's some perf we could run or something.
19:36:11 <jwb> davej, i was thinking more like a set of people representing i915, nouveau, and ati
19:36:15 <davej> ah
19:36:22 <davej> yeah, that might be a good thing
19:36:23 <jwb> or whatever particular configs are most impacted
19:36:29 * nirik nods. intel here.
19:36:36 <jwb> seems people mostly gripe about desktops with compositing turned on
19:36:40 <jforbes> I will put out a call for testers and bring it up at Fudcon, then we can start the week after fudcon
19:36:45 * jsmith is using intel at the moment as well
19:36:50 <knurd> I still have one machine where that suddenly got slower a lot with debug kernels during 3.3-rc or so; I could try testing things, too, but daily might get complicated, as it's my main machine at work
19:36:50 <jwb> jforbes, yeah, good idea
19:37:21 <davej> additionally, having just some of the debug options on instead of all of them might turn up some bugs that we may not have seen with the kitchen sink config, so it might be worth mixing things up perioidcally anyway
19:37:28 <jwb> knurd, if you can find a simple testcase that runs well without debug and poorly with debug, that might be enough to start with
19:37:34 <brunowolff> I update kernels pretty often, but I think I might be seeing a different slowness than some other people. In my case the biggest impact is that yum updates take much longer.
19:37:52 <jwb> brunowolff, some of the debug options definitely impact FS performance
19:37:59 <jwb> which... i have no idea
19:38:06 <nirik> Ideally I would love to see the base rawhide kernel more usable day to day. I think that would get more people on rawhide (even tho they can use kernel-nodebug repo, some folks just don't realize it)
19:38:37 * nirik sees very slow graphics, but also general cpu goes through the roof... driving cpu temp way up
19:38:57 <knurd> jwb: something better than glxgeards should do the trick; we discussed that in https://bugzilla.redhat.com/show_bug.cgi?id=735268 , but that bug got forgotten
19:40:00 <jwb> #info post-FUDCon, we'll look at incrementally enabling the debug options in rawhide.  hopefully that will help narrow down which impact machines the most
19:40:25 <jwb> #action jforbes and jwb to discuss this at FUDCon as well
19:41:11 <knurd> if someone gives me a quick reminder how to best run perf to the root of the slowdows then I can try to get a trace; maybe we can then narrow down the list of debug options that we need to test
19:41:21 <davej> I'm mostly puzzled as to why this only seems to affect certain people really badly.  I see some performance loss, but it's still perfectly usable.
19:41:31 <jwb> davej, are you using compositing?
19:41:33 <davej> yeah
19:41:40 <jwb> hrm
19:41:50 <nirik> davej: what card/desktop?
19:41:52 <jwb> knurd, i'm not sure we know what to do with perf
19:42:00 <davej> nirik: intel, xfce
19:42:05 <knurd> jwb: k, then I'm not alone :-)
19:42:08 <nirik> huh, same here. ;)
19:42:17 <jwb> knurd, btw, i think we still need to get back to your patch to change how we do debug options in general
19:42:34 <jwb> iirc, you sent back another quick patch after i made some comments, and then we all went on holiday
19:42:51 <knurd> jwb: it#s on my todo list; i wanted to let you guys do the rebase to 3.8 in rawhide first before I wanted to recreate the patch
19:42:54 <davej> from memory, I don't recall anything really objectionable in that patch
19:43:01 <knurd> IOW: I can start working now :-)
19:43:19 <jwb> no, it seems like a nice cleanup, my comment aside
19:43:25 <knurd> davej: jwb found something; should be not that hard to fix
19:43:42 <jwb> ok, great
19:44:24 <jwb> anything else?
19:45:25 <jwb> ok, let's end.
19:45:30 <jwb> good participation today.  thanks everyone
19:45:33 <jwb> #endmeeting