12:05:48 <jdarcy> #startmeeting
12:05:48 <zodbot> Meeting started Fri Feb  6 12:05:48 2015 UTC.  The chair is jdarcy. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:05:48 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
12:06:02 <kshlm> krishnan_p, this is just the first meeting :)
12:06:08 <jdarcy> #meetingname GlusterFS 4.0
12:06:08 <zodbot> The meeting name has been set to 'glusterfs_4.0'
12:06:15 <jdarcy> #topic Roll call
12:06:26 * shyam1 is here
12:06:26 * hagarth is here
12:06:27 * kshlm is here
12:06:29 * xavih is here
12:06:32 * krishnan_p is here
12:06:32 * kkeithley_ is here
12:06:36 <dlambrig> Dan is here
12:06:40 <jdarcy> me is here (obviously)
12:06:46 * ndevos is here
12:06:52 <jdarcy> #addchair hagarth
12:06:57 <jdarcy> #addchair krishnan_p
12:07:13 <jdarcy> Awesome turnout.  :)
12:07:26 <jdarcy> #topic Introduction and Goals
12:07:40 <jdarcy> OK, just a couple of words here.
12:08:09 <jdarcy> What I'm hoping here is first for "interested parties" to get to know each other.  Making good progress there.  :)
12:08:16 * telmich is here
12:08:57 <jdarcy> Second, I hope to get a good picture of what people are *really* interested in working on, and can really hope to get done sometime this summer.
12:09:09 <jdarcy> Anyone have any other goals they'd like to add?
12:09:23 <telmich> yes
12:09:34 <jdarcy> telmich: OK, go ahead.
12:09:54 <telmich> if it is not too much out of scope, I would like to add a small topic of "running glusterfs in production" - status & best practices
12:10:23 <telmich> this can be related to or unrelated to 4.0 and if it is too much out of scope, it's no trouble kicking it for this meeting
12:10:43 <jdarcy> telmich: If we can, we'll try to address that, but it might need more of a follow-on in #gluster.
12:11:08 <hagarth> telmich: we can also talk about this in the community meeting on Wednesdays
12:11:20 <telmich> hagarth: sure thing
12:11:43 * ndevos thinks its a great idea, many users will benefit from that
12:11:45 <jdarcy> One caveat on goals: there's still some uncertainty about resource levels and availability.  Hopefully what comes out of this will be *input* to the Powers That Be who control such things (both Red Hat and elsewhere).
12:12:07 <jdarcy> Moving on...
12:12:12 <hagarth> ndevos: +1
12:12:17 <jdarcy> #topic Sub-project Status
12:12:34 <jdarcy> Working off the page at http://www.gluster.org/community/documentation/index.php/Planning40
12:13:09 <jdarcy> Let's start with thousand-node glusterd.  krishnan_p, can you fill us in on your current half-year-or-so vision?
12:13:22 <krishnan_p> OK, let me try
12:13:42 <gothos> o/
12:13:57 <krishnan_p> We have tried to come up with two proposals to adddress the consistent configuration store problem
12:13:59 <jdarcy> gothos: Is that a raised hand?
12:14:11 <gothos> jdarcy: that is a hello
12:14:28 <krishnan_p> Both can be looked up in gluster-devel archives.
12:14:33 <jdarcy> gothos: Ah.  Hi!  :)
12:15:00 <jdarcy> krishnan_p: You guys have lots of ideas.  Which particular two?
12:15:01 <krishnan_p> The proposal involving an existing consistent store like etcd or an "in-house" implementation of the same seems like the way to go
12:15:14 <krishnan_p> jdarcy, let me share the links in a bit
12:15:20 <jdarcy> OK.
12:16:10 <krishnan_p> 1) http://www.gluster.org/pipermail/gluster-devel/2014-November/042944.html
12:16:54 <krishnan_p> I am having trouble locating the other one (the one worth pursuing) :(
12:17:12 <krishnan_p> jdarcy, can I add this during the course of this meeting and not make everyone wait on me?
12:17:26 <jdarcy> Sure.
12:17:48 * kshlm will check if I can find the other.
12:17:51 <krishnan_p> We are planning to revive the discussion on gluster-devel after 3.7
12:18:01 <jdarcy> So the other one is more along Paxos/Raft DB lines?
12:18:10 <krishnan_p> jdarcy, yes.
12:18:36 <krishnan_p> Currently, we have Kaushal and Atin working with me on this. I hope to see more joining us.
12:19:20 <kshlm> The other proposal http://www.gluster.org/pipermail/gluster-devel/2014-September/042121.html
12:19:40 <jdarcy> krishnan_p: Do you think it's likely you'll have something prototyped (at least) this summer/fall?
12:19:46 <jdarcy> Thanks kshlm.
12:20:06 <krishnan_p> jdarcy, that is what we aspire to do.
12:20:19 <jdarcy> Yay!  That seems to be one of our more solid sub-projects.
12:20:23 <krishnan_p> The prototype could use one of consul or etcd to do the heavylifting of consensus
12:20:48 <krishnan_p> jdarcy, fall seems more doable. Summer would be too ambitious
12:20:56 <kshlm> We began very basic tests with when the mail was sent, and have some idea of how to proceed.
12:21:00 * jdarcy is itching to write our own, but that's just being a codemonkey.  Not necessarily a good idea.
12:21:39 <jdarcy> Anyone want to speak up on http://www.gluster.org/community/documentation/index.php/Features/dht-scalability?  shyam1 maybe?
12:21:47 <shyam1> yup
12:21:47 <krishnan_p> jdarcy, thousand-node-glusterd has lot at stake. Just to handle that amount of details makes me run away from writing my own consensus algorithm implementation :)
12:21:56 <jdarcy> krishnan_p: Fair enough.
12:22:10 * jdarcy (Sorry for the aggressive pace.  Hoping to overlap some topics.)
12:22:21 <shyam1> We have a design document that details the core idea for the next generation distribution and its benefits
12:22:28 <shyam1> Here, http://www.gluster.org/community/documentation/index.php/Features/dht-scalability#Status
12:22:48 <shyam1> The main goals are scalability and consistency in DHT and to change the base requirement of needing a directory on every subvol of DHT (part of the scale issue)
12:23:06 <shyam1> There are some further thoughts in some of the DHT problem areas as presented in the docs above, but no code/prototype yet
12:23:22 <shyam1> The next plan would be to, start a prototype and visualize the benefits that DHT2 can demonstrate over existing DHT
12:23:46 <shyam1> Currently myself and Du are sort of thinking about this (i.e not actively working, spending large amounts of time etc.)
12:24:01 <jdarcy> This one also has a pretty long fuse.  Any thoughts on pieces that might be separately prototype-able in a short/medium timeframe?
12:24:43 <shyam1> I would state, we can prototype the separation of DIRS and DATA, and start from there... Other areas can come in later as we expand the problem space
12:25:03 <jdarcy> Excellent.
12:25:24 <shyam1> For example cluster expansion use cases can be prototyped/coded a little later
12:25:51 <shyam1> On timelines, I would really like to see this move out of concept and into a strong prototype by Summer
12:26:03 <krishnan_p> shyam1, by cluster-expansion are you referring to add-brick and rebalance?
12:26:09 <shyam1> yes
12:26:13 <jdarcy> Yay!
12:26:19 <jdarcy> Next is Next is http://www.gluster.org/community/documentation/index.php/Features/sharding-xlator
12:26:36 <jdarcy> Lots of people consider this important, but AFAIK nobody has it at the top of their list to actually work on.  Correct?
12:26:47 <shyam1> yes
12:26:53 <hagarth> no
12:27:04 <hagarth> Kruthika and Pranith have been working on this
12:27:16 <jdarcy> Oh.  Well, that's why we have these meetings then.  ;)
12:27:26 <hagarth> Kruthika did add this to the 3.7 feature list - so I expect some action on sharding soon.
12:27:39 <dlambrig> +1
12:27:41 <shyam1> I believe we need to consider this (sharding) in lieu with DHT2 so that we can get a more cogent story...
12:27:55 <jdarcy> So should this be moved to 3.7?
12:27:56 <shyam1> I know xlators are independent etc. but just stating.
12:28:05 <hagarth> jdarcy: yes, we can park it in 3.7 for now
12:28:30 <jdarcy> I consider pulling stuff forward into 3.7 to be awesome.
12:28:48 <jdarcy> OK, http://www.gluster.org/community/documentation/index.php/Features/caching
12:29:03 <jdarcy> xavih: Do you think that spec's ready for general consumption?
12:29:29 <xavih> we have an initial approach, but haven't had time to complete it
12:29:43 <xavih> I'm working on some changes for ec on 3.7
12:29:58 <xavih> I expect to have something ready after Feb 10 to publish
12:30:24 <jdarcy> xavih: Do you think you'll be in a position to focus on caching after that, or are there other demands that will take precedence?
12:30:48 <ndevos> is the cache invalidation done like what is being worked on with the upcall infrastructure?
12:31:01 <ndevos> its what we need for nfs-ganesha...
12:31:14 <ndevos> like this: http://www.gluster.org/community/documentation/index.php/Features/Upcall-infra
12:31:18 <xavih> ndevos: that is one of the things I've seen. It seems there are duplicate features...
12:31:41 <ndevos> xavih: ah, right, the upcall work is targetted for 3.7
12:31:57 <xavih> jdarcy: I'll have some work on ec before 3.7. After that we are considering caching and dfc v2 (new design)
12:32:14 <ndevos> dfc?
12:32:30 <xavih> ndevos: an old idea we tried to implement long ago
12:32:34 <jdarcy> I look forward to seeing that.  Should we add that to the 4.0 page perhaps?
12:32:45 <xavih> ndevos: we need more time to have a good proposal
12:32:51 <ndevos> xavih: ah, ok
12:33:02 <xavih> jdarcy: I'll try to finish a design document a publish it
12:33:14 <jdarcy> All great.  Thanks, xavih.
12:33:16 <xavih> jdarcy: after that we can decide if it can be included in 4.0
12:33:28 <jdarcy> Next up: http://www.gluster.org/community/documentation/index.php/Features/data-classification
12:34:03 <dlambrig> cache tiering work will provide a good foundation,
12:34:03 <jdarcy> dlambrig , I know it's not the same, but can you briefly tell us where we are with tiering?
12:34:11 * jdarcy was too slow typing.
12:35:12 <dlambrig> I think volume level tiers are a good direciton once cache tiering is completed.
12:35:43 <dlambrig> Also, using python in glusterd seems like a good implementation choice, that can be investigated.
12:36:13 <jdarcy> You mean for the volfile-generation stuff?
12:36:20 <dlambrig> cache tiering is on track for 3.7, this includes the fixes to readdirp aforementioned in the gluster-dev discussions.
12:36:47 <dlambrig> Jeff, yes, monkeying with volfiles using python is easier than C, IMO
12:37:04 * krishnan_p wonders why not Go/Rust, but that opens a can of worms, or not ...
12:37:31 <dlambrig> KP, I do not know the string manipulation libraries in Go but definately worth a look
12:38:00 <jdarcy> dlambrig: Post 3.7, do you think you'll be available/interested to push toward the more general kind of data classification, or is that still in need of volunteers?
12:38:42 <dlambrig> I will stay involved with tiering for gluster. I may not be full time, but will stay involved and would like to see the feature get better and better.
12:38:55 <jdarcy> krishnan_p: I think for something occasional and well bounded like generating volfiles, we have more flexibility, and I already have a library (from HekaFS) for manipulating volfiles in Python.
12:39:20 <jdarcy> Yay, thanks!
12:39:29 <krishnan_p> jdarcy, I started typing that before I knew it was only for volfile manipulation :(
12:39:52 <dlambrig> python +1
12:39:52 <jdarcy> All I can say about http://www.gluster.org/community/documentation/index.php/Features/SplitNetwork is that it's important but still looking for volunteers.  I'd love to do it myself but don't think I'll be able.
12:40:03 <jdarcy> Next up: http://www.gluster.org/community/documentation/index.php/Features/new-style-replication
12:40:41 <jdarcy> I've put together a few pieces, including a new code-generation infra (so we don't have several thousand lines of repetitive code for every fop) and a full data-logging xlator.
12:41:08 <jdarcy> http://review.gluster.org/#/c/9411/ is the codegen infra.
12:41:24 <jdarcy> https://forge.gluster.org/~jdarcy/glusterfs-core/jdarcys-glusterfs-data-logging is the data-logging part
12:41:28 * krishnan_p thinks codegen is awesome!
12:41:48 <jdarcy> Hoping to get NSR as it was a year ago revived in the next few weeks, and fully (?) done this summer.
12:42:02 * hagarth will drop off in about 10 minutes from now
12:42:16 <jdarcy> krishnan_p: Yeah, I want to do stubs, syncops, RPC, etc. that way instead of not-quite-identical separate code for every fop.
12:42:28 <hagarth> jdarcy: that would be fantastic
12:42:37 <krishnan_p> jdarcy, I am waiting for that day!
12:42:49 <jdarcy> Also, adding new fops might require touching fewer than twenty files.
12:43:05 <jdarcy> Next up: http://www.gluster.org/community/documentation/index.php/Features/better-brick-mgmt
12:43:08 <krishnan_p> jdarcy, that requires a separate feature page IMO
12:43:15 <jdarcy> krishnan_p: OK.
12:43:29 <jdarcy> #action jdarcy to add separate feature page for codegen infrastructure
12:43:53 * krishnan_p thanks jdarcy for adding a feature page for codegen
12:44:22 <jdarcy> BBM is also a bit of an orphan.  Again, I think it's really important but probably won't be able to do it myself.  Anyone have any thoughts on whether to keep it?
12:44:56 <ndevos> I think its a much needed featuer
12:45:03 <ndevos> feature even
12:45:13 <jdarcy> Overlapping with that, I think the other items (small-file perf and other, except for maybe stat/xattr cache) are also orphans.
12:45:36 <hagarth> jdarcy: maybe we can ciruclate a list of features needing volunteers on -devel
12:45:43 <jdarcy> Anybody know of any activity or interest in picking those up?
12:45:43 <krishnan_p> hagarth, +1
12:45:47 <jdarcy> hagarth: Good idea.
12:45:59 <dlambrig> was small file mostly a wish list of small fixes from Ben , or a large stand alone project
12:46:04 <jdarcy> #action jdarcy to poll gluster-devel for volunteers on "orphan"  subprojects
12:46:15 <jdarcy> dlambrig: It was culled from Ben's list.
12:46:48 <hagarth> jdarcy: I will try to keep knocking off ideas in small-file-perf. I can be an interim owner for that.
12:47:13 <jdarcy> hagarth: I think there has been *some* progress on stat/xattr cache (md-cache on server).  Who's doing that?
12:47:24 <hagarth> jdarcy: I have patches sitting in my local repo
12:47:47 <hagarth> krishnan_p: merging volgen re-factor will help expedite this ;)
12:48:17 <jdarcy> Volgen refactor?
12:48:18 <krishnan_p> hagarth, I will merge it. I got carried away by some 3.7 related activities
12:48:30 <hagarth> jdarcy: a basic brick graph generation refactor
12:48:40 <krishnan_p> jdarcy, http://review.gluster.com/#/c/9521/
12:48:42 <hagarth> could not withstand the ugliness that existed there
12:48:47 <jdarcy> Most excellent.  That code's getting really hard to work on.
12:49:28 <jdarcy> I'll have to see how that interacts with http://review.gluster.org/#/c/9387/
12:49:40 * krishnan_p appreciates any help anyone can offer in cleaning up code in glusterd. Its not a one person's work.
12:49:54 <jdarcy> +1
12:50:05 <jdarcy> In the interests of time . . . meeting times and formats.
12:50:14 <jdarcy> #topic Meeting Times and Formats
12:50:20 <jdarcy> Does this time work for people?
12:50:36 <jdarcy> I'm concerned that it's inconsiderate for BLR people who might want to start their weekends.
12:50:48 <hagarth> move it to Thursday maybe or setup a doodle poll?
12:51:07 <krishnan_p> jdarcy, Any day other than Friday should be OK for me.
12:51:14 * jdarcy is tempted to do a poll on whether to use Doodle.  A meta-poll.
12:51:19 <hagarth> jdarcy: lol
12:51:28 <jdarcy> #action jdarcy to set up a poll for meeting times
12:51:38 <hagarth> with that, I shall drop off now. need to head to my talk in devconf.cz.
12:51:41 <hagarth> later everyone
12:51:49 <krishnan_p> hagarth, bye
12:51:52 <jdarcy> hagarth: Have fun!
12:52:07 <jdarcy> How about frequency?  Every other week, maybe?
12:52:20 <ndevos> sounds good to me
12:52:28 <shyam1> sounds good to me too
12:52:35 <krishnan_p> sounds good to me too.
12:52:41 <jdarcy> Also,, I'd say let's stick with IRC for now.  Seems to be working, and leaves a record.
12:52:50 <jdarcy> Nicely searchable, and everything.
12:53:23 <jdarcy> #topic Collaboration Tools
12:53:26 <krishnan_p> jdarcy, IRC works
12:53:39 <jdarcy> Are people reasonably satisfied with using the wiki, or do we need/want something else?
12:54:04 <jdarcy> https://freedcamp.com/ looks really nice, but non-OSS and needs each person to be invited
12:54:20 <jdarcy> Haven't found a way to make a project fully public.
12:54:26 <ndevos> wiki should do, right? have all the information at one place?
12:54:36 <shyam1> Well wiki is useful, when better things are needed the wiki can serve as a link redirection point
12:55:12 <shyam1> I would like to know future aims of this meeting, operational/technical discussions, based on which collaboration medium may change
12:55:39 <jdarcy> shyam1: I think mostly sharing status and issues that have arisen during the last $interval
12:55:45 <shyam1> I mean if we were presenting a proposal, IRC would not cut it
12:55:52 <shyam1> Ah ok, then IRC is just fine
12:56:32 <jdarcy> Also calls to action - please review this proposal/patch, that kind of thing
12:56:36 <ndevos> whatever media, everything should be recorded and made available for everyone in the community
12:56:43 <jdarcy> ndevos: +10
12:57:07 * ndevos *hates* phone conferences other project(s) do
12:57:18 <jdarcy> OK, so wiki for now, people are *always* welcome to bring other alternatives to our collective attention.
12:57:38 <jdarcy> #topic Open Floor
12:57:53 <jdarcy> Anyone have anything else they want to discuss before we close
12:57:55 <jdarcy> ?
12:58:05 <ndevos> telmich's topic?
12:58:58 <jdarcy> A good topic, but probably more than a few minutes' worth.  Move to #gluster perhaps?
12:59:01 <ndevos> maybe we should have something like a "best practise" or "advised usage" for each of the features?
12:59:07 <jdarcy> Or #gluster-somethingelse?
12:59:29 <jdarcy> User stories?  ;)
12:59:47 <ndevos> user stories with techincal details on the configuration and all
13:00:19 <jdarcy> That's a pretty good idea.  Should I start a wiki page to collect those?
13:00:43 <ndevos> I guess we need something lika that, yeah
13:00:47 <jdarcy> We *could* put those in the existing feature pages, but that might be kind of fragmented.
13:01:02 <ndevos> yes, and not each feature is really user visible
13:01:09 <jdarcy> ndevos: Would you mind taking the AI for that?
13:01:16 <ndevos> or, features need to be combined somehow
13:01:35 <ndevos> hmm, not really, I do not have time for that in the next few wekks
13:01:37 <ndevos> *weeks
13:01:52 <jdarcy> #action jdarcy to create wiki page for user stories, send out CFP
13:02:29 <ndevos> thanks!
13:02:30 <jdarcy> Anything else?
13:02:45 <krishnan_p> nothing else from me
13:03:02 <ndevos> no, nothing
13:03:04 <jdarcy> OK then, thank you thank you thank you everyone.  :)
13:03:13 <jdarcy> #endmeeting