12:01:59 <rastar> #startmeeting Weekly Community Meeting 31/Aug/2015
12:01:59 <zodbot> Meeting started Wed Aug 31 12:01:59 2016 UTC.  The chair is rastar. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:01:59 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
12:01:59 <zodbot> The meeting name has been set to 'weekly_community_meeting_31/aug/2015'
12:02:20 <rastar> #topic Rollcall
12:02:35 * kshlm \o_
12:02:37 * ndevos _o/
12:02:46 * aravindavk is here
12:03:01 * jdarcy o/
12:03:09 * kkeithley is here
12:03:17 * rjoseph o/
12:03:33 <nigelb> o/
12:03:41 * jiffin1 is here
12:04:02 <rastar> today's meeting will be co-chaired by jiffin
12:04:07 <rastar> #chair jiffin
12:04:07 <zodbot> Current chairs: jiffin rastar
12:05:04 <rastar> will wait for two more mins
12:05:37 * msvbhat is here
12:06:34 <rastar> Hello everyone. We will start now.
12:06:44 * partner here
12:06:50 * atinm is here too
12:06:55 <samikshan> o/
12:07:06 <rastar> #topic Next week's meeting host
12:07:42 <rastar> Do we have any volunteer?
12:08:09 <kshlm> Me.
12:08:20 <kshlm> I already put my name in the pad.
12:08:24 <rastar> #info kshlm will host the next meeting.
12:08:38 <rastar> #topic GlusterFS-4.0
12:09:22 <jdarcy> Nothing from me.  On vacation, plus multiplexing work.
12:09:33 <kshlm> Not much from me either.
12:10:10 <kshlm> I've been doing some refactoring of the transaction framework. And I need to push it out.
12:10:23 <kshlm> Also, last week gRPC-1.0 was announced.
12:10:36 <kshlm> I'm thinking of giving it a go once again.
12:11:08 <rastar> Cool.
12:11:22 <rastar> Moving on.
12:11:32 <rastar> #topic GlusterFS-3.9
12:12:09 <rastar> aravindavk: any updates?
12:12:20 <rastar> i don't see pranith or dblack
12:12:22 <aravindavk> today is last day to get patches in
12:12:43 <aravindavk> Pranith sent mail to followup with feature owners
12:13:09 <rastar> I see that most of the owners replied back.
12:13:25 <rastar> If any of you haven't replied, please do.
12:13:28 * obnox arrives late
12:14:12 <rastar> Just an update from testing side
12:14:55 <rastar> We have tests/features/lock_revocation.t hanging on NetBSD
12:15:06 <rastar> which has caused the build pipeline to increase.
12:15:33 <rastar> If you have a patch that needs to be merged in 3.9, please monitor NetBSD results and follow up.
12:15:42 <rastar> Contact nigelb or kshlm or rastar
12:15:52 <rastar> keep aravindavk aware about it
12:16:10 <ndevos> do we know since when this test started to hang?
12:16:35 <rastar> not exactly, but the guess is around 3 weeks
12:16:59 <hagarth> are we marking this as a bad test?
12:17:01 <nigelb> I can get a number post-meeting.
12:17:06 <rastar> hagarth: that is done.
12:17:10 <hagarth> rastar: ok
12:17:11 <nigelb> But pretty sure it's longer than 3 weeks.
12:17:12 <jdarcy> hagarth: Just merged the patch to do so.  http://review.gluster.org/#/c/15374/
12:17:15 <rastar> hagarth: patch needs to be merged though
12:17:21 <rastar> thanks jdarcy
12:17:30 <hagarth> great, thanks jdarcy!
12:17:37 <rastar> I have posted possible fix at http://review.gluster.org/#/c/15375
12:17:45 <kshlm> The patch that added it was merged in mid-july
12:17:52 <jdarcy> I just hit the merge button.  Others did the work.  ;)
12:18:05 <rastar> that is based on a hunch that dd wasn't complete while the test proceeded to append
12:18:27 <rastar> anyways, it was just an update to people blocked by it.
12:18:28 <hagarth> rastar: many test unit deletions with 15375?
12:18:32 <atinm> http://www.gluster.org/pipermail/maintainers/2016-August/001262.html
12:18:37 <hagarth> jdarcy: teamwork ftw! ;)
12:18:40 <atinm> so I reported it on 19th August
12:18:55 <rastar> hagarth: that is a test, so that I can re-run it many times in less time.
12:19:09 <rastar> hagarth: if that works, I will update the patch to restore all tests.
12:19:12 <hagarth> rastar: ok, good idea :)
12:19:22 <rastar> hagarth: easier than having a NetBSD VM on laptop.
12:19:33 <hagarth> rastar: right
12:19:58 <rastar> any more updates?
12:20:56 <rastar> Moving on.
12:20:57 <atinm> my understanding (after discussing with rastar) is that the test is actually a bad one and needs improvement, there is nothing wrong in the code, right rastar ?
12:21:13 <rastar> atinm: not sure yet.
12:21:26 <rastar> if 15375 passes we will be sure
12:21:37 <atinm> rastar, yes that's what I assumed too :)
12:21:46 <ndevos> it looks a little bad that a single test can hange a whole VM...
12:22:06 <ndevos> I would expect the cleanup function to get the VM back into running
12:22:11 <rastar> ndevos: it is deadlock test
12:22:13 <atinm> that's how the test case is written
12:22:18 <rastar> ndevos: so kind of expected.
12:22:49 <ndevos> sure, but should we not improve the cleanup to be able to recover from it?
12:23:13 <rastar> ndevos: it does not proceed to cleanup
12:23:14 <jdarcy> BTW during the multiplexing work I found a case where cleanup was not getting called.  I should send a separate patch for that.
12:23:29 <rastar> ndevos: there is a line with & at the end
12:23:43 <rastar> ndevos: which is a dd process. cleanup is not aware of the process.
12:23:56 <jdarcy> Perhaps for the most general case we should run tests under a timer.  What's the longest any test legitimately takes?
12:24:12 <rastar> jdarcy: around 600 seconds
12:24:23 <ndevos> rastar: we need to improve cleanup then, there is a 'jobs' command that would show background processes, I think
12:24:53 <rastar> #action improve cleanup to control the processes that test starts
12:24:55 <jdarcy> Not sure if "jobs" works from a non-interactive shell.  Let me test.
12:25:08 <nigelb> It's with great fear and self-loathing I've added a `pkill gluster` into the Jenkins job. I'd love to be able to remove it.
12:25:09 <rastar> let us not delay any more.
12:25:57 <rastar> please use the bug 1369401 for more suggestions
12:25:58 <glusterbot> Bug https://bugzilla.redhat.com:443/show_bug.cgi?id=1369401 high, high, ---, bugs, NEW , NetBSD hangs at /tests/features/lock_revocation.t
12:26:14 <rastar> #topic GlusterFS-3.8
12:26:47 <ndevos> all on track
12:27:01 <ndevos> nothing that I heard needs special mentioning
12:27:17 <ndevos> schedule is to release 3.8.4 around the 10th of September
12:27:30 <rastar> Nice, boring release management is a good thing.
12:27:35 <partner> we're having major issues with all the 3.8 version when used with libglusterfs and libvirtd.. :(
12:27:53 <ndevos> partner: got a bug reported for that?
12:28:01 <kshlm> partner, Do you mean libgfapi?
12:28:07 <partner> ie. trying to use glusterfs as a volume backend for openstack..
12:28:14 <partner> sorry, libgfapi yes
12:28:30 <partner> no ticket yet since we don't know what to blame..
12:28:57 <partner> but that combo does not work fine with latest'ish centos 7 (7.2), works better with 7.1, no segfaults there
12:29:16 <partner> just fyi, no need to dig into deeper here
12:29:24 <ndevos> partner: if you think it is a gluster issue, file a bug early and keep updating its progress, we can move bugs between components if needed
12:30:03 <rastar> partner: check it is related to https://bugzilla.redhat.com/show_bug.cgi?id=1352632
12:30:03 <glusterbot> Bug 1352632: unspecified, unspecified, ---, bugs, CLOSED CURRENTRELEASE, qemu libgfapi clients hang when doing I/O
12:30:06 <rastar> *if
12:30:13 <ndevos> partner: also make sure to use the lastest 3.8 release, there have been some fixes for gfapi
12:30:18 <rastar> it is fixed in 3.8.1
12:30:28 <partner> we've tried 3.8.1 through 3.8.3
12:30:47 <partner> trying to find some working combo by downgrading other components such as libvirtd
12:31:03 <rastar> partner: ok, please file and bug then and we will assign it to right group when we triage.
12:31:15 <partner> yup, thanks
12:31:37 * post-factum is late
12:31:40 <rastar> nothing else for 3.8 I guess.
12:31:47 <rastar> post-factum: hi!
12:31:53 <post-factum> rastar: o/
12:32:04 <rastar> moving on 3.7
12:32:12 <rastar> #topic GlusterFS-3.7
12:32:43 <kshlm> I tagged 3.7.15  yesterday.
12:33:08 <kshlm> .15 has about 30 fixes over .14.
12:33:12 <kkeithley> fedora packages are building now.  Debian, Ubuntu, SuSE shortly
12:33:20 <kshlm> I'm waiting on packages to show up before doing the announcement.
12:33:25 <kshlm> kkeithley, Thanks!
12:33:26 <rastar> Yay!
12:33:33 <kshlm> That's it.
12:34:00 <rastar> #topic GlusterFS-3.6
12:34:33 <kshlm> We've finished the 3.6 bug-screen.
12:34:53 <kshlm> Thank you everyone who helped.
12:35:09 <rastar> All set for EOL then.
12:35:16 <kshlm> Yep.
12:35:39 <rastar> #topic Project Infrastructure
12:36:44 <rastar> atinm: do you want to notify nigelb about lost smoke reports?
12:36:58 * misc want to remind people about the planned reboot on the 1st
12:37:28 <rastar> misc: reboot of?
12:37:35 <nigelb> formicary, I believe.
12:37:52 <misc> the mail I sent on gluster-dev, jenkins and gerrit
12:37:58 <misc> to snapshot them and for security update
12:38:13 <atinm> rastar, sure
12:38:15 <rastar> we don't have update on mail with subject Reboot of jenkins/gerrit for upgrade and snapshot
12:38:34 <atinm> nigelb, I guess we discussed about it couple of days back when you figured out we have a missing mkdir -p
12:38:37 <rastar> from any of the maintainers
12:39:14 <rastar> ok, no objections for reboot.
12:39:31 <rastar> #info Reboot of jenkins/gerrit for upgrade and snapshot on September 1st
12:39:35 <nigelb> atinm: yeah, but that was outright failure rather than lcak of reporting, right?
12:39:45 <kkeithley> in the middle of the night? For some definition of middle and night?
12:40:01 <atinm> nigelb, yes, there are two different issues
12:40:15 <nigelb> atinm: okay, file a bug and I'll dig tomorrow.
12:40:28 <atinm> nigelb, I believe we don't know about the reason on why it failed to vote back
12:40:35 <nigelb> Indeed not.
12:40:54 <nigelb> I've been working on a prototype for somethign to show test failure trends
12:40:57 <atinm> rastar, if you have the link, can you file it? I've lost the other one which happened two days back
12:40:57 <nigelb> I got this far - http://www.gluster.org/pipermail/gluster-infra/attachments/20160826/ad3e45eb/attachment-0001.png
12:41:14 <rastar> atinm: I will check with itisravi
12:41:15 <nigelb> Needs more work for sure, but it's getting there during free hours between debugging failures in the CI system.
12:41:17 <misc> kkeithley: well, more doing during end of the day for me
12:41:47 <rastar> nigelb: that is really nice.
12:41:48 <kkeithley> is there a reason why the "vote string" (i.e. the ssh command used to register a vote) that used to be at the end of every regression and other test is no longer there?
12:41:54 <atinm> nigelb++
12:41:55 <glusterbot> atinm: nigelb's karma is now 2
12:42:12 <kkeithley> misc: that's probably the middle of the night for someone
12:42:17 <nigelb> kkeithley: Only regressions had them, as far as I know.
12:42:27 <nigelb> and they should still have it.
12:42:36 <kkeithley> do they? okay. nm then
12:42:43 <nigelb> smoke tests are voted directly by gerrit after aggregating multiple tests.
12:42:48 <nigelb> *directly by jenkins
12:42:54 <misc> kkeithley: japan, yep
12:43:22 <nigelb> rastar: I'm still not happy with it. Needs more work. Hopefully, I can have a demo by the end of the month.
12:43:43 <nigelb> kkeithley and I have been talking about getting strfmt test from centos-ci into our ci as a smoke test.
12:43:49 <rastar> nigelb: it will always be "in progress". It is a nice start though.
12:43:58 <nigelb> there's a thread on gluster-devel that we've kicked off. So, please voice your opinions there.
12:44:12 <nigelb> It'll run on master to begin with and will be non-voting until we get all the failures to pass.
12:44:21 <nigelb> and then it'll start being a normal smoke test which will vote on failure.
12:44:55 <nigelb> And the last update: I'm talking to centos CI folks to see how we can figure out some sort of regression tests for performance.
12:45:03 <nigelb> I'd like to run it as close to a production setup as possible.
12:45:17 <nigelb> i.e. with 4 servers and 4 clients or something like that.
12:45:33 <nigelb> (on hold because I've been away yesterday and I'm technically away today)
12:45:37 <ndevos> hmm, the strfmt test passes currently? https://ci.centos.org/view/Gluster/job/gluster_strfmt/
12:45:41 <nigelb> When I have something, I'll report back on glsuter-devel.
12:46:15 <rastar> ok, thanks nigelb
12:46:16 <kkeithley> I just noticed a lot of warnings in a scratch build log of 3.7.15
12:46:20 <nigelb> ndevos: hrm, kkeithley said they were creeping up.
12:46:21 <kkeithley> strfmt warnings
12:46:52 <kkeithley> how does the test in CentOS CI decide there were no warnings?
12:47:09 <obnox> different compiler version?
12:47:13 <kkeithley> e.g. used %lu for a size_t
12:47:17 <nigelb> grep, I think?
12:47:27 <nigelb> grep -E ".*: warning: format '%.*' expects( argument of)? type '.*', but argument .* has type 'ssize_t" build.log | tee -a warnings.txt
12:47:30 <nigelb> grep -E ".+: warning: format '%.+' expects( argument of)? type '.+', but argument .+ has type 'size_t" build.log | tee -a warnings.txt
12:47:38 <ndevos> https://github.com/gluster/glusterfs-patch-acceptance-tests/blob/master/centos-ci/gluster_strfmt/run-test.sh
12:48:26 <rastar> ok, next toic
12:48:28 <rastar> *topi
12:48:30 <kkeithley> and who of our devs ever looks at the CentOS CI results?
12:48:30 <rastar> *topic
12:49:15 <ndevos> kkeithley: that test sends an email to the maintainers list if it fails, with the warnings in an attachment (+URL to them)
12:49:18 <rastar> we need to do something about that.
12:49:47 <kkeithley> okay
12:49:52 <rastar> ndevos: kkeithley should we cc devel?
12:50:07 <kkeithley> uh, -devel is already pretty noisy
12:50:17 <nigelb> (this is why I want to make it smoke)
12:50:24 <nigelb> so the relevant dev gets notification instantly
12:50:30 <ndevos> rastar: should not be needed, maintainers are expected to catch these before they merge the patches - this is just a 'oops, fix it!' mail
12:50:57 <rastar> nigelb: I like that idea.
12:51:22 <rastar> ok, next topic
12:51:31 <rastar> #topic Ganesha
12:51:35 <ndevos> nigelb: yeah, but that means more load on the slaves, and it needs to be a 32-bit build
12:51:42 <kkeithley> 2.4 RC1 was tagged on Friday
12:52:36 <rastar> no other updates then.
12:52:40 <kkeithley> expect GA in 2-3 weeks.  There's some 11th hour clean up of some of the headers so that they'll compile with C++, for reasons that aren't related to glusterfs.
12:53:10 <kkeithley> but if you want to know, ping me in -devel
12:53:15 <rastar> #info NFS-Ganesha GA around 3rd week of September
12:53:17 <kkeithley> or I can tell you here
12:53:44 <kkeithley> if anyone is interested
12:53:47 <rastar> #info NFS-Ganesha 2.4 GA around 3rd week of September
12:54:06 <rastar> #topic Samba
12:54:20 <obnox> samba 4.5rc3 was tagged no Aug 29
12:54:30 <obnox> not that it currently matters much gluster.
12:54:37 <kkeithley> semi-related, for 3.10 I expect we will make gnfs a build option and separate RPM package
12:55:18 <obnox> performance work (md-cache...) is ongoing in gluster to speed up samba
12:56:05 <obnox> in samba proper, multi-channel work is going on, driven by people also in this community.
12:56:10 <rastar> #info Samba 4.5rc3 tagged on Aug 29th
12:56:35 <rastar> jiffin will take over now.
12:57:00 <jiffin> rastar: thanks for ur time
12:57:20 <jiffin> #topic Last weeks AI
12:57:23 <rastar> thanks jiffin
12:57:52 <jiffin> #topic pranithk/aravindavk/dblack to send out a reminder about the feature deadline for 3.9
12:58:01 <ndevos> It feels like we're in a football match where jiffin gets some play minutes just before the end wistle
12:58:11 <aravindavk> Pranith sent mail
12:58:19 <jiffin> aravindavk: i saw that
12:58:26 <jiffin> can u paste me the link
12:58:30 <rastar> ndevos: I thought my next meeting was midway this one. It is in 2 minutes from now. :)
12:58:56 <jiffin> ndevos: substitution rastar out jiffin in
12:59:11 <jiffin> aravindavk: for the mail?
12:59:15 <kkeithley> it's okay, jiffin will get the last batsman out and win the game
12:59:28 <aravindavk> jiffin: searching
12:59:28 <ndevos> #link http://www.gluster.org/pipermail/gluster-devel/2016-August/050627.html
12:59:41 <jiffin> ndevos: thanks
13:00:01 <jiffin> #info pranithk send out a reminder about the feature deadline for 3.9 in ML
13:00:20 <jiffin> there are no more AIs
13:00:28 <jiffin> moving to next topic
13:00:31 <ndevos> and also the release check-list at http://www.gluster.org/pipermail/gluster-devel/2016-August/050675.html
13:00:45 <jiffin> #topic Open Floor
13:01:02 <jdarcy> Just posted some brick-multiplexing status to gluster-devel, if anyone's interested.
13:01:21 <jdarcy> Should show up for others in about half an hour.  ;)
13:01:34 <jiffin> thanks jdarcy for the update
13:02:08 <kkeithley> it just arrived.  But the email I sent to -devel 35 minutes ago still hasn't
13:02:19 <jiffin> great
13:02:36 <nigelb> kkeithley: it just did, for me.
13:02:37 <ndevos> kkeithley: I have the email from you
13:02:43 <ndevos> and jdarcy's too
13:02:44 * jiffin wonders why it is taking long
13:03:45 <jiffin> times is already up
13:03:54 <jdarcy> Mailing-list split brain.
13:04:11 * jdarcy almost typed "splat brain"
13:04:19 <jiffin> so do we need to discuss anything else?
13:04:33 <kkeithley> strfmt already discussed
13:04:52 <kkeithley> longevity is just for people to see the latest numbers, no need to discuss
13:05:02 <kkeithley> ditto for RHEL/CentOS RPMs
13:05:05 <jdarcy> Anyone planning extended vacation in September?
13:05:13 <kkeithley> there was a Gluster Summit topic, but now it's gone
13:05:15 <jiffin> k then, i shall wind up the meeting
13:05:44 <jiffin> kkeithley:i had put that topic
13:05:51 * ndevos will not be on 'vacation' as such, but travelling almost the whole month
13:06:02 <jdarcy> Good to know.
13:06:04 <kkeithley> ndevos and I will be in BLR starting the 12th
13:06:12 <nigelb> Ooh.
13:06:17 <nigelb> I'll be in town that weekend.
13:06:32 <jiffin> IMO we can discuss it next week, after publishing the schedule
13:06:35 <kkeithley> I arrive the 10th, not sure when ndevos arrives
13:06:46 <ndevos> 11th, late in the evening
13:07:06 <jiffin> just providing gentle reminder
13:07:11 <jiffin> #topic Announcements / Reminders
13:07:32 <jiffin> If you're attending any event/conference please add the event and yourselves to Gluster attendance of events: http://www.gluster.org/events (replaces https://public.pad.fsfe.org/p/gluster-events)
13:07:41 <jiffin> #end-meeting
13:07:49 <nigelb> no hyphen
13:07:59 <jiffin> sorry for typo
13:08:05 <jiffin> #endmeeting