12:03:03 <ndevos> #startmeeting
12:03:03 <zodbot> Meeting started Wed Feb 18 12:03:03 2015 UTC.  The chair is ndevos. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:03:03 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
12:03:09 <ndevos> hello all!
12:03:30 <ndevos> The agenda can be found here:
12:03:31 <ndevos> https://public.pad.fsfe.org/p/gluster-community-meetings
12:03:36 <hchiramm> hello :) \o
12:03:37 <ndevos> #topic Roll Call
12:04:10 * msvbhat is present
12:04:13 <gothos> o/
12:04:14 <ndevos> we have raghu and hchiramm, anyone else?
12:04:19 <ndevos> aha!
12:04:19 * Debloper is present
12:04:31 * overclk is there
12:05:09 <ndevos> #topic Last weeks action items
12:05:21 <ndevos> #info subtopic: ndevos should publish an article on his blog
12:05:30 * ndevos still needs to do that
12:05:35 <hchiramm> thanks ndevos :)
12:05:58 * JustinClift here now too (sorry a bit late)
12:05:58 <ndevos> #info subtopic: hchiramm will try to fix the duplicate syndication of posts from ndevos
12:06:16 <ndevos> yes, that can only be checked when I posted something....
12:06:29 <ndevos> #info subtopic: hchiramm will share the outcome of the non-mailinglist packagng discussions on the mailinglist (including the Board)
12:06:42 <ndevos> hchiramm: was a final result reached?
12:06:53 <hchiramm> ndevos, yet to conclude the solution..
12:07:06 <hchiramm> discussion is 'on' ..
12:07:08 <JustinClift> Is this the thing that was fixed by Spot and JMW?
12:07:16 <JustinClift> Ahhh no, that was multiple Tweets...
12:07:29 <hchiramm> this is abt the blog duplication
12:07:44 <JustinClift> Yeah, sorry
12:07:48 * JustinClift should get coffee
12:07:50 * ndevos silences JustinClift
12:07:53 * hchiramm :)
12:08:02 <ndevos> #info subtopic: hagarth to open a feature page for (k)vm hyperconvergence
12:08:13 <hchiramm> ndevos, he is on pto
12:08:23 <ndevos> I wonder if I won my bet? did hagarth create a page for it?
12:08:31 <hchiramm> afaik, he is not :)
12:08:45 <ndevos> hchiramm: yeah, I know he's out, thats why I'm in charge :D
12:08:48 <JustinClift> We get to find out next week?  <-- you have to imagine me saying this, as I'm currently silent :)
12:09:08 <hchiramm> ndevos, that info is for others , not for u :)
12:09:21 <ndevos> ah, ok :)
12:09:22 <ndevos> #info subtopic: spot to reach out to community about website messaging
12:09:44 <ndevos> I did not see an email about this?
12:09:53 <ndevos> did someone else see one?
12:10:05 <hchiramm> I didnt
12:10:22 <ndevos> okay, I'll remind him
12:10:34 * msvbhat brb
12:10:44 <ndevos> #action ndevos will contact spot about open standing action items on the weekly agenda
12:10:57 <ndevos> #info subtopic: hagarth to carry forward discussion on automated builds for various platforms in gluster-infra ML
12:11:21 <ndevos> I also did not notice anything about that
12:11:43 <ndevos> #info subtopic: ndevos should send out a reminder about Maintainer responsibilities to the -devel list
12:12:00 <ndevos> argh, forgot about that, will try to get that done later today
12:12:18 <ndevos> #info subtopic: telmich will send an email to the gluster-users list about Gluster support in QEMU on Debian/Ubuntu
12:12:37 <ndevos> telmich: are you paying attention?
12:13:07 <ndevos> was this email sent? I can not remember seeing one
12:13:48 <ndevos> I guess I did not miss it then :-/
12:13:57 <ndevos> #info subtopic: jimjag to engage the board, asking for their direction and input for both 3.7, and 4.0 releases
12:14:34 <ndevos> jimjag: has there been any discussion on the (private) board list about it?
12:14:44 <ndevos> or JustinClift, could you chime in?
12:15:26 * ndevos feels a little lonely - is there anyone reading this?
12:15:38 <gothos> yes :)
12:15:54 <ndevos> well, at least I'm not alone :D
12:15:54 <raghu> yes.
12:16:14 <ndevos> #topic GlusterFS 3.6
12:16:21 <jimjag> ndevos: nope
12:16:24 <ndevos> raghu: you're up!
12:16:36 <ndevos> jimjag: okay, maybe something by next week?
12:16:44 <jimjag> ndevos: +1
12:17:08 <ndevos> jimjag: okay, thanks!
12:17:16 <raghu> Not much progress this week. I wanted to take some patches in. But many patches were failing regression tests (dont know whether its a spurious or not)
12:17:35 <ndevos> yeah, regression tests were a pain the last days
12:17:51 <ndevos> but I think JustinClift spent some time on checking it out
12:18:40 <raghu> So as of now, in this list of 3.6 patches (http://review.gluster.org/#/q/status:open+branch:release-3.6,n,z), there is no patch which has both +1 from a reviewer and passed regression tests
12:19:16 <raghu> apart from that I closed (changed the status properly to be precise), many bugs from release-3.6 branch
12:19:31 <JustinClift> Sorry, just had to fix something.  Back now.
12:19:41 <JustinClift> ndevos: I can ping jimjag about the board
12:19:55 <raghu> ndevos: yeah. I retriggered some tests thinking the failures might be spurious. But they failed again.
12:19:56 <ndevos> JustinClift: jimjag ponged already
12:19:57 <JustinClift> I haven't seen anything about stuff mentioned on it tho (yet)
12:20:06 * JustinClift reads that ;)
12:20:24 <ndevos> JustinClift: how are the regression tests looking?
12:20:38 <JustinClift> Not in a good state
12:20:46 <JustinClift> We have failures in 3.6 release branch
12:20:51 <JustinClift> I've pinged Avra about one of them
12:20:57 <JustinClift> I need to ping Jeff about the other
12:21:33 * JustinClift still needs to test our other branches more, and figure out whats a "real" failure vs something wrong with the slave nodes that needs fixing
12:21:37 <JustinClift> So... "in progress". :/
12:21:48 <ndevos> raghu: were those two tests the issues for the failures you saw, or were there more?
12:22:02 * partner late :/
12:22:36 <JustinClift> raghu: http://www.gluster.org/pipermail/gluster-devel/2015-February/043882.html
12:22:49 <JustinClift> This gives the two failure names^
12:22:57 <JustinClift> tests/bugs/bug-1045333.t
12:23:05 <JustinClift> tests/features/ssl-authz.t
12:23:06 <raghu> ndevos: I saw different failures
12:23:29 <ndevos> #info the release of 3.6 is getting delaybed because of regression test failures
12:23:49 <raghu> some times it was in ec
12:23:52 <JustinClift> Yeah.  We're probably right to delay it too, until we know the cause.
12:24:26 <JustinClift> Also saw a failure in tests/bugs/bug-1176062.t, but it wasn't consistent
12:24:39 <JustinClift> I'll have more idea of what's a real failure vs not a real failure later today
12:24:46 <ndevos> hmm, yes, but figuring out what suddenly causes this should be top priority, right?
12:24:49 * JustinClift is going to run some bulk regression tests again
12:25:09 <JustinClift> I'm happy to spin up VM's in Rackspace for people to investigate as needed
12:25:10 <raghu> ndevos: Some tests failed were from the ones that you mentioned now.
12:25:16 <ndevos> #action JustinClift keeps on investigating the regression test failures
12:25:17 <raghu> from ssl-authz.t
12:25:52 <ndevos> raghu: and you did not merge any ssl changes?
12:26:27 * raghu checking
12:26:47 <JustinClift> I kinda hope it's some new dependency that needs adding to the slaves, and not a real failure
12:27:13 <JustinClift> It's possible
12:27:36 <raghu> ndevos: not recently. I think the last ssl patch was from october-2014.
12:27:52 <ndevos> raghu: okay, thanks
12:28:21 <ndevos> well, we'll leave it in the hands of JustinClift to track down and involve developers
12:28:36 <ndevos> raghu: anything else for 3.6?
12:28:50 <raghu> ndevos: nope.
12:29:06 <ndevos> #topic GlusterFS 3.5
12:29:07 <jdarcy> Failures in ssl-authz.t are almost certain to be because of increased parallelism, not SSL per se.
12:29:29 <ndevos> jdarcy: I do not think the epoll changes are in 3.6?
12:30:01 <ndevos> anyway, very little progress on 3.5 too
12:30:36 <JustinClift> jdarcy: Just emailed you.  Would you have time to login to slave31 and take a look or something?
12:30:37 <ndevos> mainly due to the failing regression tests and the increased time they are waiting in the queue before they get run
12:30:48 <jdarcy> ndevos: They're not, but socket multi-threading was implemented when SSL was and gets turned on by default when SSL is.  First test might be to turn off own-thread even when SSL is turned on.
12:30:59 <jdarcy> JustinClift: Sure.
12:31:29 <JustinClift> Both slave30 and slave31.cloud.gluster.org.  They're using our standard Jenkins user/pw (I can dig it up for you if needed)
12:31:40 <JustinClift> slave31 has only ever run the one regression test
12:31:43 <ndevos> jdarcy: hmm, right, but I wonder why things start to fail only just recently - but we'll follow the conversation on the -devel list
12:32:26 <JustinClift> jdarcy: Also, keep an eye out for "these newly setup slaves are not setup correctly" problem.  Just in case.  They're new slaves, and the configuration script needed some updates.
12:32:33 <JustinClift> I *think* it's good... but keep it in mind.
12:32:34 <ndevos> I still hope to do a beta1 for 3.5 later this week, but only if the testing becomes more stable
12:33:05 <ndevos> #info beta1 for 3.5.4 might be delayed a week due to regression test issues
12:33:05 <JustinClift> Hmmmm....  I wonder if our memory usage in Gluster has grown or something?
12:33:23 <ndevos> #topic GlusterFS 3.4
12:33:29 <ndevos> kkeithley: please have a go at it
12:33:30 <JustinClift> Maybe there's a memory leak in our code base or something that's causing us to run out of ram on the nodes, so weird behavour...
12:33:39 <kkeithley> nothing, still working through permutations of perf xlators
12:33:53 <kkeithley> trying to find a combination short of *all* that leaks
12:33:57 * gothos hasn't seen any problematic memory behavior
12:34:11 <kkeithley> In the client
12:34:37 <kkeithley> some people see the client glusterfs daemon grow until it trips the OOM killer
12:34:57 <kkeithley> And I see it growing continuously in my test harness
12:35:29 <ndevos> is that with the default configuration?
12:35:33 <kkeithley> anyway, if i get ahead of my work for 3.7, I'll take a look at other patches for 3.4.7. IIRC I've seen a couple patches go by for 3.5
12:35:35 <kkeithley> 3.4
12:35:48 <kkeithley> yes, that's the out-of-the-box defaults
12:36:08 <ndevos> hmm, ok
12:36:38 <JustinClift> Is there a way to grab the entire host memory for a VM at a given time, and then analyse it to find out what's going on where?
12:36:59 <kkeithley> not that I'm aware of
12:37:00 <ndevos> *cough* vmcore *cough*
12:37:04 <gothos> kkeithley: just rechecked all our servers, we actually have one where the gluster process is using about 8GB RES the other servers are around 1.5GB
12:37:16 <JustinClift> kkeithley: I'm pretty sure I saw a debugging tool that might suit this actually.  Not open source, but they did offer us a license.
12:37:38 <kkeithley> not servers, client-side fuse mounts. The glusterfs daemon on the clients
12:37:40 <JustinClift> Lets discuss in -dev later
12:37:55 <kkeithley> sure
12:38:05 <gothos> yes, that's is what I meant, since we have that running on our servers
12:38:19 <ndevos> I think you would be able to use gcore to capture a coredump of the glusterfs process too?
12:38:47 * ndevos just does not have an idea where to look into such a core and find the leaking structures...
12:39:02 <ndevos> kkeithley: anything else for 3.4?
12:39:23 <kkeithley> I'm running a debug build, so mem-pools are disabled. Should see it with valgrind, but so far no smoking gun
12:39:27 <kkeithley> no, nothing else
12:39:39 <ndevos> #topic Gluster.next
12:39:48 <ndevos> #info subtopic: 3.7
12:40:00 <bene2> I have a couple of questions
12:40:01 <ndevos> jdarcy: are you following the 3.7 progress?
12:40:25 <jdarcy> ndevos: Cache tiering somewhat, the rest hardly at all.
12:40:26 <ndevos> bene2: sure, ask them, but hagarth isnt there, so I'm not sure if we have all the answers :)
12:41:04 <bene2> 1) is there anyone working on md-cache with listxattr capability described in http://www.gluster.org/community/documentation/index.php/Features/stat-xattr-cache
12:41:57 <jdarcy> Vijay told me that someone was, in the form of using md-cache on the server, but the patches were still private.
12:42:15 <overclk> jdarcy, myself and raghu have the initial patchset ready for BitRot
12:42:53 <jdarcy> overclk: Ahh, good.  It would be nice to get those into the pipeline.
12:42:56 <ndevos> overclk: ah, nice, did you sent out an email with a pointer to the patches or git repo?
12:43:27 <bene2> 2) is anyone working on cluster.lookup-unhashed?  I met with Raghavendra G in Bengaluru and he did not see any obstacle other than possible problems with clients caching out-of-date directory layouts.  Any other thoughts on that?
12:43:33 <ndevos> bene2: yes, I also only heard about the plan to put md-cache on the bricks, but do not know who would be working on that
12:43:41 <overclk> jdarcy, ndevos actually I was planning to send out the patch to review.g.o before this meeting started..
12:43:58 <overclk> .. but checkpatch.pl restricted me to do so :)
12:44:14 <ndevos> #action overclk will send out initial BitRot patches for review this week
12:44:31 <overclk> ndevos, make that today (max tomorrow) :)
12:44:35 <jdarcy> bene2: AFAIK nobody's actively working on that.  I think you're right that there's little controversy, though.  Just needs more eyeballs.
12:44:39 <raghu> can we disable checkpatch.#!/usr/bin/env python
12:44:39 <raghu> that are sent as rfc?
12:45:14 <raghu> oops sorry....I means disabling checkpatch.pl checking for patches that are sent as rfc
12:45:31 <ndevos> I think that would make sense, but I dont know how to disable checkpatch :-/
12:45:51 <overclk> raghu, if that's possible, I will send the patches right away.
12:46:04 <raghu> for rfc patches, BUG id would not be given. We can use that info
12:46:20 <kshlm> Some rfc.sh trickery could achieve that
12:46:29 <raghu> yeah.
12:46:57 <ndevos> raghu: ah, if it is called from rfc.sh, you could use 'git-review' to push the change ;)
12:47:06 <bene2> jdarcy: is the controversy on lookup-unhashed about the need for it or the method of implementation?  Because with JBOD support we are going to need something similar to this patch very soon IMHO.
12:47:26 <jdarcy> According to Vijay in the last 4.0 meeting, the person working on server-side md-cache is himself.
12:48:10 <ndevos> okay, I guess that makes a nice topic to move to
12:48:15 <ndevos> #info subtopic: 4.0
12:48:16 <raghu> ndevos: ok. We can try it
12:48:45 <jdarcy> bene2: The only thing really approaching controversy is some implementation details.  Nothing that should take long to resolve if we can tear people away from $thisweekscrisis.
12:49:28 <jdarcy> Not much going on for 4.0, mostly some talk about multiple networks and my own continuing struggle to revive NSR from the near-dead.
12:49:47 <jdarcy> We agreed on a meeting time.  Is that progress?
12:50:00 <ndevos> yeah, I'd call that progress
12:50:16 <ndevos> 4.0 is scheduled for 2016, isnt it?
12:50:27 <jdarcy> I believe so.
12:50:50 <ndevos> and 3.7 feature freeze is the end of this month, so I expect to see more work on 4.0 soon
12:51:09 <jdarcy> Has there been any talk at all about 3.8?
12:51:22 <ndevos> oh, the "Gluster Design Summit" proposal has been sent ouy
12:51:24 <ndevos> *out
12:51:33 <ndevos> that would encourage working on 4.0 too :)
12:51:41 <jdarcy> Indeed.
12:52:12 <ndevos> #topic Other Agenda Items
12:52:24 <ndevos> #info REMINDER: Upcoming talks at conferences: https://public.pad.fsfe.org/p/gluster-events
12:52:41 <ndevos> please update that etherpad with anything you think Gluster should be present at
12:52:58 <ndevos> #info subtopic: GSOC 2015
12:53:07 <ndevos> kshlm, JustinClift: anything to report?
12:53:48 <kshlm> Unfortunately no progress has been yet.
12:53:56 <ndevos> :-/
12:54:16 <kshlm> I need to get hold of spot
12:54:28 <jdarcy> Have the Red Hat Summit folks made their decisions yet?
12:55:06 <ndevos> I do not know if the presentation proposals already have been accepted/rejected?
12:55:24 <bene2> I haven't heard anything from them yet.
12:55:43 <jdarcy> I don't remember getting my annual "ha ha you loser" email either.
12:55:53 <ndevos> kshlm: if spot does not respond to email/irc, I suggest you call him
12:56:02 <kshlm> I'm preparing a proposal myself, and will have it on the mailing lists soon.
12:56:30 <ndevos> kshlm: really, give the guy a call :)
12:57:20 <ndevos> #topic Open Floor
12:57:52 <ndevos> nobody added a topic to the agenda, but maybe someone has an item to discuss?
12:57:58 <overclk> ndevos, yes
12:58:04 * ndevos \o/
12:58:24 <overclk> ndevos, would it benefit to have a hangout session on bitrot  (at least the flow and a small demo) with the community?
12:58:38 <kkeithley> overclk: +1, yes please
12:58:44 <ndevos> overclk: yes, that would be AMAZING
12:59:07 <ndevos> overclk: when I presented at FOSDEM, BitRot was one of the topics that were most interesting for users
12:59:08 <msvbhat> overclk: Yes, please
12:59:11 <overclk> ndevos, how about early next week?
12:59:25 <ndevos> overclk: sure, whatever works for you?
12:59:46 <overclk> ndevos, Tuesday (24th Feb)
13:00:09 <ndevos> overclk: sounds good to me, but I dont think I can really help with that :)
13:00:17 <ndevos> JustinClift: you know about hangouts?
13:00:41 <JustinClift> Not much
13:00:42 <overclk> ndevos, well someone just need to send out a hangout invite to the community..
13:00:47 <ndevos> overclk: or, I think hchiramm can organize things lika that?
13:00:54 <JustinClift> Have used them before, but never recorded them
13:01:12 <ndevos> overclk: is there a reason you would not send the invite yourself?
13:01:20 <overclk> ndevos, np. I guess it I'll send it myself.
13:01:44 <overclk> ndevos, yeh. but last time I guess davemc use to initiate it and chair the meeting..
13:01:55 <ndevos> overclk: I'm happy to help you out with that, but I also have never done that before :)
13:02:10 <overclk> ndevos, NP. I'll take care of that.
13:02:17 <kkeithley> spot is probably overloaded --- just do it
13:03:00 <ndevos> #action overclk will schedule a BitRot Google Hangout with some technical details and a small demo
13:03:02 <kkeithley> but let him know you're doing it, just so he knows
13:03:43 <ndevos> -- any other topics?
13:03:48 <overclk> ndevos, sure.
13:03:59 <overclk> ndevos, nothing more from me as of now..
13:04:27 <kkeithley> ndevos:  do we want to mention our travel plans?
13:04:41 <ndevos> kkeithley: sure, remind me?
13:05:02 <kkeithley> ndevos and I are coming to BLR April 6-10.
13:05:36 <ndevos> oh, yes, *those* travel plans :)
13:05:42 <kkeithley> ;-)
13:06:31 <ndevos> #info kkeithley and ndevos will be visiting Bangalore, 6-10 April
13:06:50 <ndevos> I do not think many are impressed, kkeithley
13:06:55 <kkeithley> lol
13:07:34 <ndevos> I guess thats it for today, thanks for joining!
13:07:34 <kkeithley> ugh, I wrote "lol"
13:07:44 <ndevos> #endmeeting