12:01:33 <JustinClift> #startmeeting Weekly GlusterFS Community Meeting
12:01:33 <zodbot> Meeting started Wed Sep 10 12:01:33 2014 UTC.  The chair is JustinClift. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:01:33 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
12:01:56 <JustinClift> Roll call!
12:01:58 <JustinClift> sowhodowehavetoday?
12:01:58 * kkeithley is herfe
12:02:01 <kkeithley> here
12:02:03 <JustinClift> Hi Herfe
12:02:04 <JustinClift> :)
12:02:27 * jdarcy is here, kinda.
12:02:31 * davemc is here <yawning>
12:03:04 <JustinClift> hagarth mentioned he'd be joining later
12:03:09 <jdarcy> Happy 5am davemc.
12:03:19 <JustinClift> hchiramm: You around?
12:03:26 <davemc> and today will end around 9PM
12:03:54 <JustinClift> #topic Action items from last meeting
12:04:02 <JustinClift> #info https://public.pad.fsfe.org/p/gluster-community-meetings
12:04:11 <JustinClift> "JustinClift to email the GlusterFS Consultants data to davemc"
12:04:13 <JustinClift> Done
12:04:30 <JustinClift> Well, forwarded the emails with the places to include.
12:04:42 <JustinClift> "davemc to get the GlusterFS Consultants and Support Company's page online"
12:04:50 <davemc> by end of week
12:05:09 <JustinClift> np
12:05:22 <JustinClift> #action davemc to get the GlusterFS Consultants and Support Company's page online by the end of this week
12:05:48 <JustinClift> hchiramm doesn't seem to be around yet, so will leave his item until later
12:05:52 <JustinClift> "kkeithley to release first GlusterFS 3.4.6 beta on monday"
12:05:54 <JustinClift> Done. :)
12:06:01 <JustinClift> http://supercolony.gluster.org/pipermail/gluster-devel/2014-September/042170.html
12:06:10 <JustinClift> "hagarth to figure out the new meeting time and let us know :)"
12:06:13 <JustinClift> Done.
12:06:18 <JustinClift> http://supercolony.gluster.org/pipermail/gluster-devel/2014-September/042188.html
12:06:23 <JustinClift> New slot is 8:30 PM IST / 15:00 UTC / 8:00 AM US PDT, starting next meeting (17th Sep)
12:06:33 <JustinClift> "hagarth to send out a response to andersb_'s email"
12:06:50 <JustinClift> hagarth mentioned he'll likely get that done later on today
12:07:10 <JustinClift> #topic 3.4
12:07:29 <JustinClift> 3.4.6beta1 was released Monday. :)
12:07:43 <kkeithley> \o/
12:07:51 <JustinClift> Thanks guys.
12:08:07 <JustinClift> I didn't realise the RPMS were on the download server yet
12:08:15 <JustinClift> Have we sent email about that?
12:08:18 <kkeithley> about 30 mintues ago.
12:08:25 <kkeithley> Yes, Lala sent email
12:09:08 <kkeithley> We still have two BZs outstanding that need to be fixed for beta2. Raghavendra is actively working on one. Is pranithk here?
12:09:18 <kkeithley> Maybe he's the right one to work on the other
12:09:48 <kkeithley> he's not here :-(
12:10:59 <JustinClift> Sorry, internet connection here went dodgy for a bit
12:11:02 * JustinClift is back now
12:11:40 <JustinClift> kkeithley: Should we chase up pranithk, or is already aware/active on the BZs?
12:12:22 <kkeithley> #action kkeithley to chase pranithk about https://bugzilla.redhat.com/show_bug.cgi?id=1127140
12:12:23 <glusterbot> Bug 1127140: unspecified, unspecified, 3.4.6, gluster-bugs, NEW , memory leak
12:12:30 <JustinClift> Heh, that works. :)
12:13:11 <JustinClift> lalatenduM: Any interest in creating a blog post about the 3.4.6 beta1 rpms being available?
12:13:27 <JustinClift> Ahhh, not I'm not sure if lala is her
12:13:28 <JustinClift> here
12:14:02 <JustinClift> #action JustinClift to create blog post about GlusterFS 3.4.6beta1 rpms being on the download server
12:14:11 <JustinClift> I'm do it right after this meeting, so I don't forget
12:14:25 * JustinClift is full of tpyos today ;)
12:14:44 <JustinClift> Anything else we need to consider here for 3.4 ?
12:15:03 <kkeithley> not AFAIK
12:15:12 * jdarcy sends JustinClift a copy of "Just My Typo"
12:15:23 <JustinClift> :)
12:15:26 <JustinClift> #topic 3.5
12:15:40 <ndevos> still awaiting patches...
12:15:58 <JustinClift> ndevos: Is beta for 3.5.3 this week looking ok, or should we move that?
12:16:25 <ndevos> it will probably get later, unless the patches get posted and reviewed today
12:16:52 <ndevos> the end of next week is more realistic, if the patches have landed by then
12:17:30 <ndevos> bugs in NEW are troublesome: https://bugzilla.redhat.com/showdependencytree.cgi?maxdepth=1&hide_resolved=1&id=glusterfs-3.5.3
12:17:35 <JustinClift> ndevos: Cool, no worries. :)
12:17:53 <ndevos> also the ones in ASSIGNED, btw :)
12:18:04 <JustinClift> Heh.  Maybe we should say in 2 weeks then?
12:18:32 <JustinClift> Or should we attempt to be extra pushy on the people who are assigned to them bugs?
12:18:34 <ndevos> well, depends on the devs :)
12:18:57 <ndevos> 2 weeks is fine with me too, and if we're lucky its earlier?
12:19:03 <JustinClift> ndevos: Sure
12:19:42 <JustinClift> A few of those bugs in the list are assigned to gluster-bugs instead of a person.  That does sound like a problem. :(
12:20:00 <JustinClift> ndevos: What's the right way to get those assigned to someone?
12:20:03 <ndevos> oh, yes, indeed
12:20:37 <ndevos> JustinClift: wait, only the ones in NEW are assigned to gluster-bugs
12:20:57 <JustinClift> Well, there are 6 bugs with NEW.
12:21:09 <ndevos> it means that they are up for grabs for developers/maintainers, they should have been triaged by now
12:21:11 <JustinClift> Of those, 4 are gluster-bugs
12:21:26 * lalatenduM is late to the party :)
12:22:51 <JustinClift> ndevos: Hmm, I'll but out and leave you to it. :)
12:23:08 <JustinClift> lalatenduM: Do you want to create a blog post about the GlusterFS 3.4.6 beta1 rpms being available on the download server?
12:23:18 <ndevos> #info During next weeks Bug Triage Meeting we'll have a topic on "what happens after a bug has been Triaged"
12:23:20 <lalatenduM> JustinClift, sure
12:23:39 <lalatenduM> JustinClift, not sure if I have a account for it , need ur help for it
12:23:44 <JustinClift> lalatenduM: Cool.  That'll get it out via Twitter too, which also seems to help
12:24:00 <davemc> when is that triage mtg?
12:24:12 <JustinClift> lalatenduM: Do you have to get that sorted out with me right after this meeting?
12:24:19 <JustinClift> ndevos: Yesterday wasn't it?
12:24:31 <lalatenduM> JustinClift, sure , we can do that
12:24:48 <ndevos> JustinClift: yesterday, and next week Tuesday again
12:24:50 <davemc> thats okay, i have a tardis on my desk
12:25:13 <ndevos> davemc: https://public.pad.fsfe.org/p/gluster-bug-triage
12:25:17 <JustinClift> #action lalatenduM to create blog post about GlusterFS 3.4.6beta1 rpms being available on the download server
12:25:27 <davemc> ndevos tks
12:25:50 <JustinClift> #action JustinClift and lalatenduM to ensure lalatenduM has a blog account
12:26:28 <lalatenduM> :)
12:26:36 <lalatenduM> thats better
12:26:40 <JustinClift> #info New rough ETA for 3.5.3 beta rpms is end of next fortnight.  eg ~26th Sep
12:27:03 <JustinClift> k, I think that's all for 3.5?
12:27:24 * ndevos doesnt know of anything more
12:27:29 <JustinClift> #topic 3.6
12:27:33 <hagarth> hang on
12:27:41 * JustinClift hangs
12:27:45 * ndevos stands
12:27:50 <hagarth> anything blocking 3.5.3 apart from dht at the moment?
12:28:02 <JustinClift> https://bugzilla.redhat.com/showdependencytree.cgi?maxdepth=1&hide_resolved=1&id=glusterfs-3.5.3
12:28:11 <ndevos> hagarth: one fuse patch from pranith
12:28:20 <hagarth> ndevos: ok
12:28:56 <hagarth> ndevos: if patches land sooner, can we target a beta release earlier?
12:29:03 <JustinClift> This doesn't look like DHT: https://bugzilla.redhat.com/show_bug.cgi?id=1099270
12:29:04 <glusterbot> Bug 1099270: high, unspecified, ---, rgowdapp, ASSIGNED , Gluster 3.5.0 NFS server crashes under load
12:29:46 <hagarth> right, it is related to nfs.drc
12:30:19 <hagarth> ndevos: what do we propose to do with this?
12:30:35 <hagarth> do we intend fixing nfs.drc anytime soon?
12:31:11 <ndevos> hagarth: I think many things have been fixed already, just needs more testing
12:31:21 <hagarth> ndevos: ok
12:31:38 <hagarth> should we move it to 3.5.4 if we need more time?
12:31:46 <JustinClift> This doesn't seem like DHT either: https://bugzilla.redhat.com/show_bug.cgi?id=1117888
12:31:47 <glusterbot> Bug 1117888: medium, unspecified, ---, gluster-bugs, NEW , Problem when enabling quota : Could not start quota auxiliary mount
12:31:58 <ndevos> yes, DRC can wait
12:32:08 <JustinClift> Hmmm, maybe we should discuss this on gluster-devel mailing list, instead of going through during the meeting?
12:32:21 <hagarth> JustinClift: yeah, we can do that. sounds like a good idea.
12:33:03 <kkeithley> speaking of gnfs, have we documented plans to migrate from gnfs to nfs-ganesha, and in what time frame that will happen?
12:33:05 <JustinClift> #action ndevos and hagarth to discuss the outstanding 3.5.3 blocker BZ's, and which ones to move to 3.5.4
12:33:26 <JustinClift> gah, that should say "on gluster-devel"
12:33:31 <hagarth> kkeithley: not before 3.7
12:33:49 <JustinClift> #action ndevos and hagarth to discuss on gluster-devel the outstanding 3.5.3 blocker BZ's, and which ones to move to 3.5.4
12:34:14 * JustinClift should investigate if there are MeetBot commands for adjusting action items
12:34:19 <JustinClift> k, anything else for 3.5?
12:34:45 <hagarth> nothing else for me
12:34:59 <hagarth> JustinClift: put an action item on yourself for that one? ;)
12:35:03 <JustinClift> Meh
12:35:05 <JustinClift> :)
12:35:14 * JustinClift avoids the ai for that
12:35:34 <JustinClift> #topic 3.6 (attempt 2)
12:35:48 <JustinClift> hagarth: This one's yours
12:35:56 <hagarth> release-3.6 is in a slightly bad state owing to a gerrit problem
12:36:08 <hagarth> I will clean up the branch later today
12:36:27 <hagarth> good news on 3.6 is that we have got good traction on dht, aft and geo-replication backports
12:36:42 <JustinClift> Cool. :)
12:36:46 <JustinClift> Btw: http://www.gluster.org/community/documentation/index.php/Planning36
12:36:48 <hagarth> we still need some ec and uss fixes to be in
12:36:59 <JustinClift> Cool
12:37:07 <lalatenduM> hagarth, probable date for beta release? may be after 1 week I think
12:37:10 <kkeithley> xavih is begging for ec code reviews
12:37:25 <hagarth> I think we can start doing test days from next week for 3.6
12:37:35 <hagarth> and the culmination of the test days could be a beta release
12:37:51 <hagarth> I am still looking to have 3.6.0 out before the end of this month
12:38:09 <hagarth> me and hchiramm plan to send out schedule for 3.6.0 test days later this week
12:38:15 * JustinClift will believe that when he sees unicorns flying
12:38:28 <JustinClift> 3.6.0 before end of month
12:38:32 <JustinClift> that is
12:38:36 <hagarth> JustinClift: you may have to start looking for unicorns soon ;)
12:38:41 <JustinClift> But hey, open to being wrong. ;)
12:38:49 <lalatenduM> :0
12:38:52 <lalatenduM> :)
12:39:12 <hagarth> kkeithley: yes, I have reviewed parts of ec
12:39:20 <hagarth> intend merging them soon
12:39:22 <JustinClift> With the test day, we're going to be providing people with rpms from the nightly build beforehand or something prepared differently or ?
12:39:32 <jdarcy> I've been reviewing parts of EC as well.  So has Dan.
12:39:41 <JustinClift> Cool
12:39:45 <hagarth> JustinClift: yes, that's the plan. We are also contemplating having a series of test days.
12:39:50 <JustinClift> Cool
12:40:05 <hagarth> JustinClift: more like one test day per feature types, will send out details soon.
12:40:17 <kkeithley> I think xavih was looking for reviews of the fixes he has queued up in gerrit actually
12:40:39 <hagarth> kkeithley: yes, we are referring to the same. just noticed a +1 from jdarcy on one of the patches :).
12:40:47 <JustinClift> When we feel the needed pieces are all in (eg ec, dht, afrv2, whatever), we should ping Corvid Tech and see if they can hammer it for a bit
12:40:53 <hagarth> JustinClift: sure
12:41:34 <hagarth> that's the update on 3.6
12:41:40 <JustinClift> k
12:41:46 <hagarth> any other thoughts/questions on 3.6?
12:42:06 <JustinClift> Just for my clarity here, are we thinking that these bits will all be in 3.6 and ready for "test day" within a week yeah?
12:42:20 <hagarth> JustinClift: right
12:42:24 <JustinClift> Cool
12:42:34 <lalatenduM> kkeithley, regarding 3.6 spec file for fedora , will need ur help after I take all changes from source to review it
12:42:45 <kkeithley> lalatenduM: sure
12:42:49 <lalatenduM> kkeithley, there are so many changes to it :(
12:42:51 <JustinClift> #action JustinClift to ping David Robinson (Corvid Tech) to see if they will have time to test 3.6 beta
12:43:00 <kkeithley> ugh
12:43:06 <kkeithley> c'est la vie
12:43:29 <hagarth> JustinClift, davemc: we probably need to sync up on launch plans for 3.6 offline.
12:43:38 <JustinClift> AI it
12:43:45 <davemc> hagarth +1
12:43:55 <davemc> hagarth++
12:43:57 <glusterbot> davemc: hagarth's karma is now 2
12:43:58 <hagarth> #action hagarth, JustinClift and davemc to sync up on 3.6.0 launch plan offline
12:44:10 <lalatenduM> kkeithley, yup
12:44:27 <JustinClift> k, that sounds like 3.6 item is covered
12:44:36 <JustinClift> No-one else got anything?
12:45:02 <hagarth> did we discuss the time slot for next week's meeting?
12:45:15 <JustinClift> #topic time slot for next weeks meeting
12:45:27 <jdarcy> What ever happened to the search for a Forge replacement?
12:45:31 <JustinClift> It turns out that moving back to the old time won't work for davemc nor myself
12:45:46 <hagarth> JustinClift: how about 1400 UTC ?
12:45:56 <davemc> so I believe we can stay at this time
12:46:06 <JustinClift> I'd prefer 1400 :)
12:46:18 * JustinClift is the opposite of a morning person
12:46:23 <davemc> it's early can someone do the math for pacific time
12:46:36 <hagarth> davemc: 7:00 AM before Nov 2
12:46:39 <JustinClift> davemc: 7:00
12:47:00 <JustinClift> However, the present time slot isn't killing me.
12:47:08 <JustinClift> So, I'm not really super fussed ;)
12:47:11 <davemc> I can make 7 work. might be a bit late from the 6Am meeting
12:47:27 <hagarth> if this slot works for davemc, we can continue with this.
12:47:42 <JustinClift> "this slot" == ?
12:47:48 <JustinClift> Context is confusing
12:47:55 <JustinClift> 5am one, yeah?
12:47:58 <hagarth> this slot = 1200 UTC
12:48:05 <hagarth> right ;)
12:48:06 <kkeithley> http://www.worldtimebuddy.com/    14:00UTC=10AMEDT=7AMPDT=730PMIST
12:48:12 <davemc> I'm fine with it.  The only hiccup might be the daylight savings changeover
12:48:16 <ndevos> !date -d '14:00 UTC'
12:48:54 <JustinClift> Sounds like we're keeping the 12UTC slot then, with potential adjustment when DST happens for [whichever TZ it happens for]
12:49:18 <hagarth> JustinClift: sounds good, let us see in November about the new time slot
12:49:19 <JustinClift> Anyone object?
12:49:25 <ndevos> 12:00 UTC is fine for me
12:49:35 <JustinClift> k, carried :)
12:49:44 <JustinClift> hagarth: #info it :)
12:49:52 <JustinClift> Moving on...
12:50:06 <JustinClift> #topic Problems with Gerrit
12:50:10 <JustinClift> Already covered...
12:50:20 <JustinClift> #topic Something has been killing Rackspace VM's
12:50:34 <kkeithley> sounds like a B-movie title
12:50:34 <JustinClift> So far it seems to be related to the release-3.6 problem
12:50:35 <hagarth> #info community meeting slot to be revisited in November. Till then 1200 UTC is going to be the start time for weekly community meet.
12:50:44 <JustinClift> hagarth: tx
12:51:07 <hagarth> JustinClift: let us await for the 3.6 problem to get sorted out
12:51:14 <JustinClift> If the rackspace VMs keep dying after the release-3.6 problem is fixed, then we'll need to investigate more
12:51:17 <JustinClift> Yeah
12:51:26 <JustinClift> #topic Longevity Cluster is running
12:51:35 <JustinClift> kkeithley: This is yours yep? :)
12:51:41 <kkeithley> yup, just FYI
12:51:51 <JustinClift> What's a Longevity Gluster do?
12:52:05 <JustinClift> Description page somewhere?
12:52:20 <JustinClift> Ahhh: http://download.gluster.org/pub/gluster/glusterfs/dynamic-analysis/
12:52:21 <kkeithley> It's an 8 server 4x2 dist/replicate cluster with I/O running continuously
12:52:27 <kkeithley> http://download.gluster.org/pub/gluster/glusterfs/dynamic-analysis/longevity
12:52:39 <hagarth> kkeithley: awesome!
12:52:42 <kkeithley> sampling the RSZ/VSZ of all the gluster processes
12:53:08 <kkeithley> hourly sampling
12:53:19 <ndevos> kkeithley: what kind of IO is it doing?
12:53:36 <JustinClift> kkeithley: Can we do something similar for 3.6.0 beta when that's ready, or would that be too fast moving a target to do a "longevity" thing on?
12:53:40 <kkeithley> It's bengland's fsstress
12:53:56 <ndevos> over fuse and nfs?
12:53:59 <kkeithley> yeah, we can switch to 3.6 when we want
12:54:08 <JustinClift> kkeithley: Cool
12:54:09 <hagarth> JustinClift: might be good to have rc go through that
12:54:10 <kkeithley> right now only over fuse. I'll add NFS at some point
12:54:18 <ndevos> cool
12:54:23 <kkeithley> I'd like to generate .csv files and graphs
12:54:35 <JustinClift> hagarth: Yeah, that's the kind of thing I'm thinking.  "Throw everything we can think of at pre-GA 3.6.0"
12:54:41 <lalatenduM> kkeithley, which OS you are using?
12:54:52 <kkeithley> servers are RHEL7
12:55:02 <JustinClift> #action kkeithley to put 3.6.0 beta / rc on the Longevity Cluster
12:55:20 <JustinClift> On a related topic...
12:55:45 <kkeithley> someone needs to look at the memory leak on the client glusterfs though. It might be the same leak that's in 3.4.
12:55:59 <JustinClift> With libvirt back in the day, one thing we found useful in a similar way was to do stuff (scripted) like "create and destroy a VM" 10k times overnight
12:56:06 <ndevos> kkeithley: http://download.gluster.org/pub/gluster/glusterfs/nightly/glusterfs-3.6/ contains el7 builds too ;)
12:56:26 <JustinClift> It showed up a few resource leaks and similar that otherwise wouldn't have been seen in short duration tests
12:56:55 <JustinClift> Does this longevity cluster do stuff like "create and destroy a volume" 10k times
12:56:58 <JustinClift> ?
12:57:00 <ndevos> kkeithley: that memory leak was in fuse, right?
12:57:42 <kkeithley> If it's in FUSE....   Where ever it is, it's making the glusterfs process grow
12:57:51 <JustinClift> http://review.gluster.org/#/c/8541 ?
12:58:00 <lalatenduM> ndevos, kkeithley, memory leak issue, do you have the bugzilla link?
12:58:19 <kkeithley> https://bugzilla.redhat.com/show_bug.cgi?id=1127140
12:58:19 <glusterbot> Bug 1127140: unspecified, unspecified, 3.4.6, gluster-bugs, NEW , memory leak
12:58:35 <lalatenduM> kkeithley, thanks
12:58:54 <JustinClift> Is it related to: https://bugzilla.redhat.com/show_bug.cgi?id=1126831
12:58:55 <glusterbot> Bug 1126831: medium, high, ---, gluster-bugs, NEW , Memory leak in GlusterFs client
12:59:08 <kkeithley> The point though it to let it run as long as possible.  I don't want to restart it frequently
12:59:34 <JustinClift> kkeithley: Yep, gotcha
12:59:36 <kkeithley> 1127140 is the clone of 1126831 for 3.4.x
12:59:44 <JustinClift> k
12:59:49 <kkeithley> IIRC
12:59:51 <JustinClift> Anyway, moving on. :)
13:00:15 <kkeithley> or not
13:00:18 <JustinClift> #topic Replacement for Gluster Forge
13:00:22 <JustinClift> we have a few seconds :)
13:00:35 <JustinClift> I took a look at Gitorious v3
13:00:44 <JustinClift> compared to Gitorious v2, and compared to GitLab
13:00:57 <JustinClift> In summary, Gitorious v3 is a bunch nicer than v2
13:01:02 <JustinClift> So we'll likely upgrade to that
13:01:29 <JustinClift> I'd really like to move us off the Gitorious software platform though, as I don't think it's the best environment possible for our Community
13:01:48 <JustinClift> I'd personally like to get us onto GitHub, as that's where teh largest amount of potential contributors is
13:02:14 <JustinClift> But, lets discuss in detail on gluster-devel and gluster-users I guess, as we need practical migration/backup/recovery planning
13:02:28 <JustinClift> And really, there might be something better. ;)
13:02:34 <jdarcy> SGTM.
13:02:42 <hagarth> +1
13:02:56 <JustinClift> #action JustinClift to bring up Gluster Forge replacement on gluster-devel and gluser-users
13:03:11 <JustinClift> It'll come after the backup stuff is done though
13:03:17 <JustinClift> 1 major task at a time
13:03:24 <JustinClift> Anyway, thanks everyone for attending
13:03:30 <JustinClift> Let's finish today's meeting. :)
13:03:34 <JustinClift> #endmeeting