12:00:05 <JustinClift> #startmeeting Gluster Community Weekly Meeting
12:00:05 <zodbot> Meeting started Wed Mar  4 12:00:05 2015 UTC.  The chair is JustinClift. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:00:05 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
12:00:14 <JustinClift> #topic Roll call
12:00:19 <JustinClift> Who do we have here? :)
12:00:24 * partner background noise
12:00:29 <JustinClift> ;)
12:01:15 <partner> but looks quiet :o
12:01:32 <JustinClift> Yeah.  Several people are travelling and stuff atm
12:01:56 <JustinClift> k, lets give it till say 5 past, and then see who's turned up
12:02:13 <JustinClift> partner: Is there anything specific you're interested in?
12:03:27 * JustinClift listens to the crickets
12:03:29 * msvbhat is present
12:03:39 <JustinClift> Who, we're not alone :)
12:03:52 * tigert is here
12:04:05 * misc is here but see his stomach slowly leaving
12:04:06 <JustinClift> Cool
12:04:07 <dlambrig> “Spurious failure report for master branch -	2015-03-03”
12:04:12 <partner> JustinClift: not really, hanging around but due to change at the job i no longer maintain gluster :/
12:04:20 <JustinClift> partner: :/
12:04:21 <dlambrig> perhaps can bring that up...
12:04:33 <JustinClift> dlambrig: Sure.  Add it to the etherpad here: https://public.pad.fsfe.org/p/gluster-community-meetings
12:04:44 <JustinClift> k, lets start
12:04:55 <JustinClift> #topic Agenda items from last week
12:05:03 <JustinClift> hchiramm: You around?
12:05:41 * hchiramm in
12:05:41 <JustinClift> k, guess not
12:05:45 <JustinClift> Cool
12:05:57 <JustinClift> "hchiramm and spot will check the blog duplication backend"
12:06:01 <JustinClift> How'd that go?
12:06:14 <hchiramm> TBD..  sorry :(
12:06:24 <tigert> I attempted to look at that too
12:06:30 <tigert> it resulted in .. ahem, some mayhem :)
12:06:36 <hchiramm> :)
12:06:38 <JustinClift> Ahhh yeah.
12:06:47 <JustinClift> Is the problem itself fixed now tigert ?
12:06:51 <tigert> suspected something was pulling the same feed second time
12:07:04 <tigert> the duplicates? I did not find the cause
12:07:41 <tigert> there was some other blog too that was duplicating in addition to nixpanic
12:07:49 <firemanxbr> \o/
12:08:04 <JustinClift> k.  I'll mark it down as in progress or something... do we need to bring in someone else, if we know anyone who's more of an expert with this stuff?
12:08:21 <tigert> could some other feed be syndicating the same stuff second time?
12:08:22 <JustinClift> Actually... firemanxbr: Do you have skill with WordPress rss stuff?
12:08:33 <JustinClift> tigert: Oh, interesting thought.
12:08:42 <JustinClift> tigert: I don't know.  Maybe? :)
12:08:44 <tigert> since there is just one subscription to nixpanic
12:08:50 <firemanxbr> JustinClift, no :(
12:08:52 <tigert> but maybe one of those feeds is a planet in itself?
12:09:00 <JustinClift> firemanxbr: No worries.  Asking just in case. :)
12:09:04 <firemanxbr> JustinClift, I'm using sphinx-doc or tinkerer :D
12:09:16 <tigert> I tried to look but did not quite catch anything until I managed to flood everyting with old posts from nix :P
12:09:21 <JustinClift> #action tigert to continue investigating the blog post duplication
12:09:24 <JustinClift> ;)
12:09:30 <tigert> JustinClift: the nasty thing is the feed itself is fed to social media
12:09:55 <tigert> so it would be good to unhook that, and then rebuild the planet from the feeds and check everything, and then re-hook them
12:10:00 <tigert> to avoid the chain reaction
12:10:06 <JustinClift> tigert: You may need to do something funky regardless, like disable all feeds except on, and see if a new post from nixpanic turns up, then enable the next, and so on
12:10:16 * krishnan_p is here
12:10:27 <tigert> yeah
12:10:28 <JustinClift> And ignore the mayhem that results until you find the solution ;)
12:10:37 <JustinClift> Ok, next agenda item
12:10:40 <tigert> and to do that, you want to disconnect the social media waterhose first
12:10:43 <tigert> yeah
12:10:56 <JustinClift> "hchiramm will share the outcome of the non-mailinglist packagng discussions on the mailinglist (including the Board)"
12:11:04 * krishnan_p is reading to know what he has missed so far
12:11:05 <JustinClift> Email sent?
12:11:10 <hchiramm> JustinClift, there is no conclusion yet
12:11:15 <JustinClift> k.
12:11:20 <JustinClift> "In progress then"
12:11:26 <hchiramm> yep
12:11:32 <hchiramm> we had a meeting to discuss it
12:11:51 <JustinClift> #action hchiramm will share the outcome of the non-mailinglist packagng discussions on the mailinglist (including the Board)
12:12:00 <JustinClift> "hagarth will update the feature page for (k)vm hyperconvergence"
12:12:11 <JustinClift> hagarth isn't here atm.  Anyone know?
12:12:24 <misc> hyperconvergence look cool for a name
12:12:36 <JustinClift> There's stuff on the page, but it's the same stuff as last week
12:12:40 <JustinClift> I'll mark it as "in progress"
12:12:55 <krishnan_p> misc, there is an industry around. It's another marketing term that gives away nothing about what's inside :P
12:13:01 <JustinClift> #action hagarth will complete the feature page for (k)vm hyperconvergence
12:13:05 <krishnan_p> s/around/around it
12:13:13 <JustinClift> "spot to reach out to community about website messaging"
12:13:26 <JustinClift> tigert: Have you seen ^
12:13:26 <JustinClift> ?
12:13:46 <JustinClift> I've been ignoring the website thread completely, so not aware of any messages to it
12:14:44 <JustinClift> k, that's a "I'm not here"
12:14:50 <JustinClift> Next...
12:15:00 <JustinClift> "hagarth to carry forward discussion on automated builds for various platforms in gluster-infra ML"
12:15:09 <JustinClift> I don't remember seeing ^
12:15:11 <JustinClift> Anyone?
12:15:31 <JustinClift> Next...
12:15:41 <JustinClift> "ndevos should send out a reminder about Maintainer responsibilities to the -devel list"
12:15:54 <JustinClift> Don't remember seeing this either?
12:16:05 <JustinClift> Next...
12:16:15 <JustinClift> "telmich will send an email to the gluster-users list about Gluster support in QEMU on Debian/Ubuntu"
12:16:18 <tigert> JustinClift: I havent heard of spot
12:16:39 <JustinClift> tigert: Spot is Tom Callaway
12:16:43 <tigert> I know that :)
12:16:53 <JustinClift> Ahh
12:16:55 <tigert> I mean, I havent heard of him re: the website / gluster stuff
12:16:55 <JustinClift> s/of/from/
12:17:02 <JustinClift> Gotcha :)
12:17:11 <tigert> yeah, grammar helps :)
12:17:53 <JustinClift> Looking over gluster-users messages since last week quickly, doesn't seem actioned yet
12:18:09 <JustinClift> Oops, need to catch up on action items.  1 sec
12:18:23 <JustinClift> #action spot to reach out to community about website messaging
12:18:31 <JustinClift> #action hagarth to carry forward discussion on automated builds for various platforms in gluster-infra ML
12:18:40 <JustinClift> #action ndevos should send out a reminder about Maintainer responsibilities to the -devel list
12:18:47 <JustinClift> #action telmich will send an email to the gluster-users list about Gluster support in QEMU on Debian/Ubuntu
12:18:57 <JustinClift> "JustinClift to engage the board, asking for their direction and input for both 3.7, and 4.0 releases"
12:19:09 <tigert> JustinClift: I assume spot is busy with other stuff, hence I am trying to poke the community about the webstite stuff
12:19:11 <JustinClift> Discussed this with JimJag and Spot
12:19:22 <tigert> so I could pick that item
12:19:45 <JustinClift> tigert: It'll take a bunch of persistence and repetition.  Don't be discouraged by that, and pls keep at it :)
12:20:02 <tigert> pokage is ongoing, but not much response, apart from one good suggestion about the test suite being more visible
12:20:07 <JustinClift> #action tigert to reach out to community about website messaging
12:20:13 <tigert> :)
12:20:23 <tigert> JustinClift: that is ongoing
12:20:34 <tigert> please reply to the mails (and stop ignoring the thread :))
12:20:55 <JustinClift> tigert: Maybe lead in with catchy email subject lines like "GlusterFS Website v2"
12:21:05 <tigert> perhaps
12:21:10 <JustinClift> And just and URL, and the question "thoughts?" after it
12:21:17 <JustinClift> That'll get some kinda response :)
12:21:38 <JustinClift> If people don't say anything, just friggin deploy it
12:21:39 <tigert> like "Revamping GlusterFS website"? ;^)
12:21:45 <JustinClift> Sure
12:22:02 <JustinClift> k, next item
12:22:14 <JustinClift> Ahh, alread on the board item
12:22:49 <JustinClift> So, discussed board engagement with JimJag and Spot.  JJ is already working on it.  I need to get some info to him though soon.
12:22:59 <JustinClift> "hagarth will post more details about an overlay xlator (container targetted) when he has a bit more clarity (and post his hacks)"
12:23:23 <JustinClift> #action JustinClift to get JimJag the info he needs for progressing the board engagement
12:23:41 <JustinClift> Has anyone heard about an overlay xlator?
12:24:06 <krishnan_p> not me
12:24:11 <JustinClift> hchiramm krishnan_p dlambrig: ^ ?
12:24:20 <JustinClift> k
12:24:26 <JustinClift> I guess it's still to be done
12:24:27 <hchiramm> not me
12:24:27 <dlambrig> overlay? no
12:24:34 <bene2> I've heard of OverlayFS.  Is there a feature page for it?
12:25:11 <JustinClift> bene2: No idea.  The action item is for hagarth to post details about an overlay xlator
12:25:25 <JustinClift> Since none of us know yet, I'm getting it's still to be done
12:25:38 <JustinClift> Moving it to next week, when hagarth should be around for the meeting :)
12:25:41 <JustinClift> #action hagarth will post more details about an overlay xlator (container targetted) when he has a bit more clarity (and post his hacks)
12:25:45 <dlambrig> sounds like a cointainer thing, that redhat’s fetish du jour
12:25:54 <JustinClift> Yeap
12:26:01 <JustinClift> With feathers and everything ;)
12:26:14 <JustinClift> #topic 3.6
12:26:23 <JustinClift> So... who's is this to tell us about?
12:26:31 <hchiramm> raghu, ^^
12:26:43 <raghu> I have merged few more changes.
12:26:56 <JustinClift> Cool
12:27:18 <JustinClift> raghu: What's your feeling about the current 3.6 branch at the moment?
12:27:19 <raghu> But I remember niels telling me to wait till a dependency issue on el5 is resolved before making beta2
12:27:48 <JustinClift> raghu: k.  So we need to wait for that, and try and get as much other fixes / stability stuff into it in the meantime?
12:27:57 <raghu> JustinClift: For 3.6.3 I think, there are still some more patches that can be accepted (they in review)
12:28:04 <JustinClift> Cool.
12:28:24 <JustinClift> When's a practical time to expect that?  Are the reviews for them competing with dev time for the 3.7 stuff?
12:28:59 * JustinClift is wondering if we should set a rough ETA for it some time after 3.7 feature freeze, so people have clear priorities
12:29:02 <raghu> I think I will wait for one more week. And can decide a timeline for 3.6.3 in the next community meeting
12:29:09 <JustinClift> Cool :)
12:30:04 <JustinClift> raghu: I've got it down as "raghu suggests we wait anothe week to see how things go, and decide on eta at next meeting"
12:30:11 <JustinClift> raghu: Is that ok?
12:30:25 <JustinClift> s/anothe/another/
12:30:27 <raghu> JustinClift: yeah, that looks fine
12:30:36 <JustinClift> Cooll  Anything else for 3.6?
12:30:37 * overclk is here
12:30:45 <JustinClift> overclk: Cool :)
12:30:51 <raghu> JustinClift: nope.
12:30:55 <JustinClift> :)
12:30:57 <JustinClift> Moving on
12:31:04 <JustinClift> #topic 3.5
12:31:24 <JustinClift> ndevos isn't here atm.  Is anyone able to cover 3.5 in his place?
12:31:42 <hchiramm> I dont think so.. :)
12:32:01 <JustinClift> np, we'll leave 3.5.x item then
12:32:03 <JustinClift> Moving on
12:32:08 <jdarcy> Good morning.
12:32:13 <JustinClift> Heya
12:32:18 <JustinClift> #topic 3.4
12:32:49 <JustinClift> kkeithley isn't here, guessing we skip this item unless someone has something for it?
12:33:11 <JustinClift> Moving on
12:33:25 <JustinClift> #topic Gluster next
12:33:31 <JustinClift> jdarcy: That's you?
12:33:47 <jdarcy> Well, Vijay is 3.7
12:34:05 <JustinClift> Sure, but he's not here atm
12:34:06 <jdarcy> Not a whole lot to report on 4.0 except the new server-side-AFR results
12:34:16 <jdarcy> Did I mention those last week?
12:34:16 <JustinClift> Ahhh... and?
12:34:31 * krishnan_p is waiting for the blog which he was mentioning ...
12:34:31 <JustinClift> You said some initial very positive stuff about um... server-server NSR?
12:34:35 <JustinClift> Yeah
12:34:50 <jdarcy> It's a stepping stone to NSR - most of the data flow, leader election etc. but not the log-based part.
12:34:59 <JustinClift> We were promised a blog about it :D
12:35:10 <JustinClift> Can we hold you to a blog date? :)
12:35:18 <jdarcy> On one set of tests (Digital Ocean) it was ~2x regular AFR performance by any measure.
12:35:22 <firemanxbr> sorry my question, but we have planning for 3.6.x ? (example 3.6.3) I can submit one patch :)
12:35:27 <jdarcy> Yes, give me an AI for that.
12:35:36 <JustinClift> jdarcy: ETA
12:35:37 <JustinClift> ?
12:35:43 * jdarcy already has one from the 4.0 meeting, but more incentive doesn't hurt.
12:35:59 <jdarcy> I can probably knock that out today, now that I'm back from visiting my family.
12:36:02 <JustinClift> #action firemanxbr to submit a patch for 3.6.3
12:36:19 <JustinClift> firemanxbr: Discuss outside of the meeting about it, with raghu :)
12:36:38 <firemanxbr> JustinClift, okay, thnkz :D
12:36:53 <JustinClift> #action jdarcy to write blog post today about initial server-side-AFR results - including Digital Ocean stuff
12:37:10 <jdarcy> That's it for 4.0
12:37:18 <JustinClift> Thanks. :)
12:37:39 <JustinClift> Other items time
12:37:42 <JustinClift> #topic "Upcoming talks at conferences: https://public.pad.fsfe.org/p/gluster-events"
12:37:46 <JustinClift> Who's it that? ^
12:37:54 <JustinClift> s/it/is/
12:38:41 <JustinClift> Well, we have an events etherpad
12:38:53 <JustinClift> Looks like VAULT is the next major thing coming up
12:39:23 <JustinClift> hchiramm had a GlusterFS meetup about a week ago, with good attendance and lots of places looking to get involved
12:39:32 <JustinClift> Good stuff hchiramm :)
12:39:42 <JustinClift> Next item...
12:39:49 <hchiramm> JustinClift, thanks!!
12:39:58 <JustinClift> :)
12:40:09 <JustinClift> #topic GSOC 2015
12:40:26 <JustinClift> We were rejected.
12:40:27 <JustinClift> Meh
12:40:31 <JustinClift> Next item
12:40:40 <JustinClift> #topic Website practical plans
12:40:42 * firemanxbr :(
12:40:45 <JustinClift> tigert: That's you
12:41:56 <firemanxbr> I have one sugestion about our website, We can use mediawiki, similar other projects, for example: ovirt.org and freeipa.org ?
12:42:24 <JustinClift> firemanxbr: Discuss with tigert on the mailing list
12:42:40 <JustinClift> firemanxbr: He has some discussion happening about this stuff, and is needing more input
12:42:45 <JustinClift> So, "more input" him :)
12:42:52 <JustinClift> Please :)
12:43:02 <firemanxbr> ok
12:43:16 <misc> firemanxbr: ovirt is looking at leaving mediawiki...
12:43:35 <bene2> what problem(s) are we trying to solve?
12:43:44 <JustinClift> Paging tigert....
12:43:45 <firemanxbr> misc, humm, okay, sorry my interruption.
12:43:49 <JustinClift> tigert: You're up... ;)
12:43:49 <misc> bene2: the website is messy
12:43:52 <misc> for example
12:44:14 <misc> http://blog.gluster.org/ still show the old website
12:44:39 <misc> the prcoess to deploy the website is a bit messy, because we have wordpress, mediawiki, 2 statics website
12:44:49 <misc> ( one with middleman, one with awestruct )
12:44:53 <JustinClift> k, I think tigert has stepped away from the computer for a bit
12:45:06 <JustinClift> We'll move this item to next meeting ;)
12:45:11 <misc> or we have on http://www.gluster.org/ wiki and documentation
12:45:25 <misc> the wiki being some documentation, and documentation being documentation, etc, etc
12:45:46 <misc> so I think that's the problem tigert want to fix
12:46:03 <JustinClift> #action tigert will share practical plans for ReVamping the Mutha F**kn Stonkin AweSUM next GlusterFS Web Presence
12:46:16 <JustinClift> ^ this is what happens if you miss your item ;D
12:46:27 <hchiramm> http://blog.debs.io/3 misc
12:46:42 <JustinClift> k, open floor
12:46:59 <JustinClift> Anyone else have stuff they want to bring up / discuss / etc? :D
12:47:31 <dlambrig> there seem to be spurious errors in the tests, was that discussed? (I had to step out)
12:47:47 <JustinClift> dlambrig: Not dicussed yet
12:47:53 <JustinClift> But yeah, worth mentioning
12:48:01 <firemanxbr> I have one item about infra: I can create FreeIPA 4.1.2 on CentOS 7 for our project ?
12:48:08 <JustinClift> dlambrig: Are you meaning the spurious regression failures?
12:48:17 <misc> #url http://blog.debs.io/3
12:48:23 <JustinClift> firemanxbr: firemanxbr Wait a sec, dlambrig is in first :)
12:48:39 <dlambrig> yes, we submit a fix, and see these errors unrelated to our patch,
12:49:00 <dlambrig> I suppose I’ll raise a flag to draw attention to this
12:49:06 <JustinClift> dlambrig: Yeah.  We've been suffering from this for a while, and it's out of control atm
12:49:10 <dlambrig> and ping people in Bangalore.
12:49:19 <JustinClift> 1 sec, knowck at front door
12:51:54 <JustinClift> back.  Grocery delivery
12:52:05 * firemanxbr lol
12:52:21 <JustinClift> dlambrig: So, we've brought up 15 extra VM's to cope with the 75% spurious failure rate we're seeing in the regression runs
12:52:57 <JustinClift> We've blown out the Rackspace sponsorship budget, but they've covered us for the outage so far
12:53:01 <JustinClift> (thankfully :>)
12:53:02 <bene2> that doesn't sound like a root cause analysis, does it?
12:53:10 <JustinClift> No, it doesn't
12:53:11 <dlambrig> that sounds like a step in the right direction,
12:53:34 <JustinClift> So, I sent that email to the dev list last night, so we can prioritise
12:53:56 <JustinClift> This isn't the first time this has happened btw
12:54:00 <dlambrig> I saw you got one reply.
12:54:06 <JustinClift> We had similar um.. 1 or two releases ago
12:54:07 <JustinClift> Yeah
12:54:29 <JustinClift> http://www.gluster.org/pipermail/gluster-devel/2015-March/044046.html
12:54:52 <JustinClift> If we can knock off the most common ones, that generally goes a long way towards fixing things
12:55:00 <JustinClift> But, we've never got it down to 0%
12:55:09 <JustinClift> Which I reckon we really need to
12:55:23 <dlambrig> hm. well, I’ll ping some other devs , but in the meantime, submitting a “clean fix” will be hard. i.e. you will see failures ?
12:55:52 <JustinClift> dlambrig: The approach I've been using is a brute force one...
12:56:01 <JustinClift> (effective, but ick)
12:56:34 <JustinClift> dlambrig: Run the tests 2-3 times.  If it fails the some one over and over, it's not spurious failure
12:56:49 <JustinClift> If it changes all the time... it likely is
12:56:51 <dlambrig> right..
12:57:04 <lalatenduM> unless we make the failure rate low, we will have difficulty finding people to run and maintain the regression test
12:57:05 <lalatenduM> s
12:57:16 * JustinClift nods
12:57:27 <JustinClift> It's not awesome work looking after this stuff.  Just saying.
12:57:57 <dlambrig> seems like it should be a priority. ok, point made :)
12:57:59 <JustinClift> But it's also going to take a bunch of dev time and effort getting root causes analysis done for everything
12:58:10 <JustinClift> dlambrig: Oh yeah.  It should definitely be a priority
12:58:23 <JustinClift> The previous time around, we found all kinds of interesting problems
12:58:36 <JustinClift> We got rid of the low hanging fruit (eg making longer timeouts :>)
12:58:48 <dlambrig> lower timeouts is a bandaid
12:59:00 <JustinClift> And then we found stuff like.. bad design that needed fixing, and other Actual Problems
12:59:09 <jdarcy> Perhaps when a test fails responsibility should be equally assigned to the patch and test authors.
12:59:19 <JustinClift> The Actual Problems stuff is what we need to get fixed this time around too
12:59:43 <dlambrig> has it gotten worse in the last month or two?
12:59:46 <JustinClift> jdarcy: Doesn't seem real practial, because tests fail for *everything* ;)
13:00:00 <JustinClift> dlambrig: Well, we have the 3.7 feature freeze rush atm
13:00:15 <JustinClift> So, there's a lot more um... desire/push for dev's to get stuff through
13:00:24 <JustinClift> At the same time, we have massive failure rate
13:00:33 <JustinClift> So, the extra VM's is my way of coping ;D
13:00:45 <JustinClift> It's not a long term solution tho
13:00:49 <jdarcy> Right, if there are tests that are failing constantly then their authors should be on the hot seat.  They're the ones blocking the deliverable.
13:00:57 * JustinClift nods
13:01:00 <misc> that's interesting, because that's kinda the same type of tension between dev and ops
13:01:05 <dlambrig> well, with respect to action items, I can contact other devs about it and escalate the issue a bit.
13:01:10 <dlambrig> but no promises :)
13:01:13 <JustinClift> dlambrig: Please do
13:01:39 <misc> (and there is a few conf dedicate on solving this kind of issue between the desire to have stable thing and the push for feature)
13:01:59 <JustinClift> jdarcy: Next email out of Spurios failure results, I should probably put each git blame-d patch author's name next to each failing test
13:02:14 <JustinClift> s/patch author/test author/
13:02:20 <JustinClift> Gah, we're over time
13:02:26 <jdarcy> misc: Well, it's the same if the tests are failing because the thing they're testing for isn't working (which does happen).  In this case I think tests are failing because of their own internal inadequacies.
13:02:51 <misc> jdarcy: the fact that test is code by itself also bring a interesting recursion on the topic :)
13:02:52 <JustinClift> #action dlambrig to contact other devs about the spurious failures
13:03:05 <jdarcy> JustinClift: I think that's a good idea.  A little accountability can go a long way.
13:03:06 <JustinClift> k, next item super quickly then we'll end this
13:03:14 * JustinClift ndos
13:03:16 <JustinClift> nods even
13:03:21 <JustinClift> firemanxbr: Your item, quickly :)
13:04:05 <JustinClift> ...
13:04:10 <firemanxbr> My idea is storage our ssh keys and we have more services to infra: ntpd, certmonger,
13:04:25 <firemanxbr> ldap, roles for access to all servers
13:04:53 <firemanxbr> freeIPA is very great identify manager, and running simple in one vm
13:04:58 <JustinClift> firemanxbr: Actually, it's not a bad idea.  Let's discuss it fully on gluster-devel (where the devs are) + CC gluster-infra?
13:05:10 <jdarcy> SGTM
13:05:24 <firemanxbr> I can create one vm for tests ?
13:05:30 <JustinClift> Sure
13:05:33 <firemanxbr> ok
13:05:45 <JustinClift> k, wrapping up the meeting everyone, thanks for attending :)
13:05:51 <JustinClift> #endmeeting :)