16:08:15 <sgallagh> #startmeeting Cockpit (2014-12-08)
16:08:15 <zodbot> Meeting started Mon Dec  8 16:08:15 2014 UTC.  The chair is sgallagh. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:08:15 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:08:28 <mvollmer> .hello mvo
16:08:29 <zodbot> mvollmer: mvo 'Marius Vollmer' <marius.vollmer@gmail.com>
16:08:31 <sgallagh> #chair mvollmer stefw andreasn sgallagh
16:08:31 <zodbot> Current chairs: andreasn mvollmer sgallagh stefw
16:08:34 <stefw> .hello stefw
16:08:35 <zodbot> stefw: stefw 'Stef Walter' <stefw@redhat.com>
16:08:38 <sgallagh> .hello sgallagh
16:08:40 <mvollmer> sgallagh, thanks!
16:08:41 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
16:09:08 <mvollmer> #topic agenda
16:09:21 <mvollmer> ok, what's on the agenda?
16:09:29 <mvollmer> * metric channel
16:09:39 <sgallagh> If we get a moment, I'd like to talk about the (currently nonfunctional) Coverity scans.
16:09:47 <mvollmer> sure
16:09:49 <stefw> * Template HTML rendering
16:09:52 <mvollmer> * coverity
16:10:13 <mvollmer> * dashboard discovery?
16:10:55 * stefw nods
16:11:26 <stefw> * interframe caching/sharing
16:11:48 <danofsatx-work> .hello dmossor
16:11:50 <zodbot> danofsatx-work: dmossor 'Dan Mossor' <danofsatx@gmail.com>
16:12:51 <mvollmer> danofsatx-work, welcome!
16:13:01 <danofsatx-work> thank ye, thank ye ;)
16:13:14 <mvollmer> :)
16:13:17 <stefw> o/
16:13:21 <mvollmer> ok, first topic?
16:13:23 <danofsatx-work> Figured I lurk here often enough I may as well speak up
16:13:58 <mvollmer> #topic metric channels
16:14:07 <mvollmer> stefw, do you want?
16:14:34 <stefw> sure
16:14:41 <mvollmer> ok, great.
16:14:56 <stefw> so we're working on a new protocol channel payload for transferring metric samples ... to populate graphs and such.
16:15:19 <stefw> it's important that this is done the right way, because of performance
16:15:44 <stefw> so i've sketched out a basic channel payload and structure and handed it over to mvollmer who really wants to get this done soon.
16:15:55 <stefw> in addition this work is about reading data from PCP where we can
16:16:05 <stefw> such as logged CPU/memory/disk metrics etc.
16:16:25 <stefw> not everything can be consumed from PCP
16:16:30 <stefw> http://www.pcp.io/
16:16:32 <mvollmer> (not yet)
16:16:40 <stefw> because it doesn't yet have all the right metrics available to us
16:16:58 <stefw> so we'll continue to have our own internal gathering for certain metrics, such as per container CPU/memory/disk usage
16:17:25 <stefw> currently we've been gathering these resource metrics in cockpitd
16:17:50 <stefw> and unbundling this from cockpitd brings us one step closer to running cleanly on Atomic or RHEL 6 where cockpitd does not compile.
16:17:59 <stefw> (due to dependencies)
16:18:08 <stefw> i think that's about it, mvollmer do you have any more about this?
16:18:29 <mvollmer> the main new feature will be meaningful historical data
16:18:43 <mvollmer> where we configure pcp to log interesting stuff always
16:18:56 <mvollmer> so we can display data from the last week or so
16:19:08 <mvollmer> this means the graphs will be scrollable and zoomable etc
16:19:13 <mvollmer> so big changes in the UI
16:19:45 <mvollmer> i am reading up on the pcp api and have stefw's code running here.
16:20:15 <stefw> oh cool ... it runs?
16:20:18 <mvollmer> so PoC for kernel.all.cpu.user works
16:20:21 <stefw> weee
16:20:37 <mvollmer> yes, great work if you only have compiled it but never run it so far.
16:20:39 <mvollmer> :-)
16:21:00 <stefw> thanks
16:21:14 <mvollmer> I made some changes here https://github.com/mvollmer/cockpit/commits/metrics-channel
16:21:23 <stefw> so these metric channels are a blocker for doing the graphs right ... which is a blocker for putting stuff into components ... our big goal
16:22:34 <mvollmer> and it's also a great feature by itself. :-)
16:22:40 <stefw> indeed
16:22:58 <mvollmer> i am surprised by how well people seem to like the silly 5 minute graphs that we have. :-)
16:23:12 <sgallagh> mvollmer: They demo *really* well
16:23:34 <mvollmer> sgallagh, imagine zooming out and scrolling back to last week!
16:23:53 <mvollmer> with important events marked on the graphs
16:23:53 <sgallagh> <keanu reeves>Whoa!</keanu reeves>
16:24:01 <mvollmer> :-D
16:24:06 <mvollmer> so, anyway that's the goal.
16:24:21 <sgallagh> cool
16:24:40 <mvollmer> kind of a big thing, so it will take a bit unfortunately.
16:25:17 <mvollmer> ok, next topic?
16:25:20 <stefw> yup
16:25:40 <mvollmer> #topic template html rendering
16:26:01 <stefw> i've been looking at what kind of HTML client side templating to use
16:26:16 <stefw> since we're componentizing everything, we have some liberty to experiment with differcent things in different components
16:26:32 <stefw> in fact, if certain components are very complex and have different needs they can use different templates
16:26:39 * stefw pulls up an example of client side templates
16:27:13 <stefw> https://github.com/janl/mustache.js/#include-templates
16:27:22 <stefw> you can see the {{ value }} syntax there for example
16:27:25 <stefw> that's what this is about
16:27:42 <stefw> angularjs contains templating and data-binding for updating the rendered template when data changes
16:27:53 <stefw> but it is very complex and hard to imagine using it for simple things
16:28:00 <stefw> so i've been looking at mustache.js
16:28:12 <stefw> mustache is a standard template form usable in all sorts of different languages
16:28:25 <stefw> i have the basic docker containers listing rendering using mustache
16:28:44 <stefw> and i have some code to do incremental html updates, rather than blowing away the entire screen whenever any little thing changes
16:29:12 <stefw> this work is related to the dashboard work, because i would like to render the 'Servers' and 'Events' section using templates
16:29:22 <stefw> however i am first experimenting with this in a component before bringing it into the shell
16:29:34 <mvollmer> I like this a lot
16:30:37 <mvollmer> I guess the browser also does clever incremental updates somehow, blowing away a whole list and constructing it again doesn't seem to be terrible, performance wise.
16:30:51 <stefw> it does cause flicker though
16:30:53 <mvollmer> and doesn't flicker or stuff like that.
16:30:57 <mvollmer> does it?
16:31:00 <mvollmer> hmm...
16:31:03 <stefw> i guess it depends
16:31:12 <stefw> it does break state on elements, selection, etc.
16:31:27 <stefw> if the user has half clicked a button, and that button gets replaced ...
16:31:38 <mvollmer> anyway, my point is, it's not perfect and there might be flicker and other artifacts, like menus closing when they get recreated randomly
16:31:49 <stefw> but anyway, incremental updates are just 50 lines of code
16:31:51 <stefw> nothing fancy
16:31:56 <mvollmer> yes, great.
16:32:13 <mvollmer> i guess incremental rendering of templates is much harder
16:32:19 <stefw> and by doing the actual incremental diff near on the outputted DOM
16:32:24 <stefw> we can avoid diffing mountains of stuff
16:32:27 <mvollmer> or impossible if the output is a string anyway
16:32:29 <stefw> we almost always have *way* more in our model
16:32:43 <stefw> the output of mustache.js is indeed a string
16:32:53 <stefw> but i'm pretty happy with diffing the parsed DOM
16:33:02 <stefw> because it's way faster than diffing all the internal JSON and objects etc...
16:33:09 <stefw> lots less data
16:33:11 <mvollmer> so, this sounds quite  right for us.
16:33:16 <stefw> i think so
16:33:24 <stefw> i'm not against someone using angularjs in a cockpit component
16:33:29 <stefw> and i would like to see what obstacles are encountered
16:33:36 <stefw> but it is very heavy weight ... far more than we need
16:34:05 <mvollmer> yes, I'm afarid we would have even more cargo-cult code than now.
16:34:09 <stefw> indeed
16:34:20 <stefw> my main concern would be the barrier to entry
16:34:36 <stefw> in order for someone to contribute to cockpit they would need to learn angularjs
16:35:24 <mvollmer> i have the feeling that angular makes it quite hard to find the end of the string to start pulling on
16:35:32 <stefw> right
16:35:50 <mvollmer> everybody knows how to follow some function calls down the stack
16:36:51 <stefw> alright, well i'll keep everyone posted on this
16:36:56 <mvollmer> cool
16:37:03 <mvollmer> #next-topic
16:37:07 <mvollmer> :)
16:37:23 <mvollmer> #topic Coverity
16:37:29 <sgallagh> /me waves
16:38:21 <sgallagh> OK, so I had to stop my regular Coverity scans a while back, because changes to the upstream make-srpm tool resulted in being unable to create SRPMs that could be built by the coverity tool
16:38:33 * stefw thought he fixed that
16:38:41 <sgallagh> I just tried again this morning
16:39:13 <sgallagh> http://koji.fedoraproject.org/koji/taskinfo?taskID=8322634
16:39:39 <sgallagh> Ah, actually it looks like this may be a different problem than before.
16:39:50 <stefw> yes
16:40:00 <stefw> this has to do with koji not allowing network access
16:40:10 <sgallagh> (As it shouldn't)
16:40:21 <stefw> you should run autogen.sh before you run make-rpms
16:40:22 <sgallagh> We need to have the Node modules you need packaged and BuildRequires:
16:40:31 <stefw> then you'll download the necessary stuff and make-srpm will include it
16:40:34 <stefw> sgallagh, no
16:40:41 <stefw> these are not BuildRequires dependencies
16:40:45 <stefw> these are development dependencies
16:40:50 <stefw> our tarballs do not require these
16:40:52 <sgallagh> ah
16:40:53 <stefw> to build
16:40:54 <sgallagh> ok
16:41:07 <stefw> we do not expect packagers or distributors of cockpit to have to mess with node.js
16:41:25 <sgallagh> ok
16:41:33 <stefw> if you run autogen.sh before you run make-srpm you'll get all the necessary node.js modules in the srpm
16:41:40 <sgallagh> Let's table this discussion for the moment. I thought we were still in the same shape as before.
16:41:50 <sgallagh> I need to spend a little more time here.
16:41:57 <stefw> i think i fixed this issue, and then it got broken again
16:42:02 <sgallagh> Hopefully I'll have those scans back up today.
16:42:10 <stefw> mvollmer, maybe make-srpm should do the 'npm install'?
16:42:20 <stefw> so that we avoid this situation?
16:42:30 <mvollmer> should we have make-srpm fail when the modules are missing?
16:42:42 <mvollmer> or that way around.
16:42:55 <mvollmer> yea, it shouldn't run autogen.sh
16:42:56 <stefw> i guess we could check for the presence of the directory
16:43:04 <stefw> and just run 'npm install' if it doesn't exist
16:43:16 <stefw> make sure we don't slow down CI just because of this use case
16:43:16 <mvollmer> yes
16:43:43 <sgallagh> No, don't bother.
16:43:51 <sgallagh> My test this morning was flawed.
16:44:01 <sgallagh> In my automated env, it will be running autogen.sh
16:44:03 <mvollmer> would it make sense to make a branch in gt that is equivalent to a tarball, just for coverity?
16:44:07 <sgallagh> (since it's always doing a clean checkout)
16:44:16 <stefw> mvollmer, how do you do that?
16:44:34 <stefw> if you want that you just run 'make dist'
16:44:54 <sgallagh> Yeah, there's no need to have 'make distdir' checked in
16:44:55 <mvollmer> so coverity can do all these things itself, right?
16:45:21 <sgallagh> Right, I was operating under a bad assumption. I thought we were still in the situation where 'make-srpm' only worked as a feeder into 'make-rpms'
16:45:27 <mvollmer> ok, #undoidea
16:45:28 <sgallagh> But it seems that stefw fixed this ahead of time
16:45:45 <stefw> yes, we sed the cockpit.spec with the right info
16:46:04 <stefw> so that the SRPM builds independent of any cockpit scripts
16:47:01 <stefw> next topic?
16:47:40 <stefw> #topic Dashboard Discovery
16:47:58 <stefw> The above work on templates, and the JSON cache work (later agenda) item is my current status on the Dashboard discovery
16:48:15 <stefw> these are dependencies, and things we need to figure out before completing the dashboard
16:48:20 <stefw> one update though
16:48:39 <stefw> i think that the 'Events' or 'Alerts' table should come from the same plugin information sources as the machines and objects for the dashboard
16:48:50 <stefw> so these various sources can provide all that information
16:48:56 <mvollmer> yes, makes sense
16:48:57 <stefw> but that's theoretical ... and i haven't implemented it yet
16:49:19 <mvollmer> what about the graphs?
16:49:31 <mvollmer> they could also be populated from the discovered objects
16:49:42 <stefw> right, certain machines will get put into the graphs
16:50:08 <stefw> but i don't think the dashboard plugins will have too much knowledge of that
16:50:23 <stefw> i do imagine that the plugins will include a list of cgroups that are related to the object
16:50:27 <stefw> but not much more than that
16:50:33 <stefw> sgallagh, by the way this is all relevant to rolekit
16:50:38 <stefw> rolekit would be one such source of information
16:50:55 <sgallagh> /me reads scrollback
16:51:02 <sgallagh> (Attending several simultaneous meetings)
16:51:12 <stefw> sgallagh, https://github.com/cockpit-project/cockpit/pull/1548/files#diff-d29b701b0d40938876d4a0f7ea6d037dR1
16:51:47 <sgallagh> So this is plugin stuff, then?
16:51:57 <stefw> yes, there are different kinds of plugins
16:52:06 <stefw> we have component plugins that present a full page UI
16:52:22 <stefw> and these are discovery plugins for finding machines and objects and information about them
16:52:42 <stefw> rolekit will likely have a discovery plugin
16:52:46 <sgallagh> ok
16:52:50 <stefw> and a number of component plugins for its various roles
16:52:55 <sgallagh> /me nods
16:52:58 <stefw> as well as a component plugin for adding a new role
16:53:25 <sgallagh> Sorry, I can't parse that last part.
16:53:52 <stefw> well, the way i see it for each role that is deployed, we'll be able to show details of that role's state
16:53:52 <mvollmer> (/me has just read the whole mustache reference manual I think.  Very nice and simple stuff.)
16:54:17 <stefw> sgallagh, and likely there will be one or more user interfaces for adding a new role(s)
16:54:32 <sgallagh> ok
16:55:47 <stefw> #topic Interframe Caching/Sharing
16:56:11 <stefw> As part of the dashboard work, i've also been working on a way to use window.sessionStorage to share cached information, and track updates to it.
16:56:43 <stefw> for example if we have the dashboard looking up a list of containers, then we don't want other things performing the same task, and doing 2 or 3 times as much work every time docker tells us that a change happened.
16:57:08 <stefw> this caching happens at a low level
16:57:18 <stefw> and is used internally to the plugin code
16:57:36 <stefw> callers of the various docker plugins have no idea that information is being shared between instances of the plugins in the dashboard, and the component user interface
16:57:50 <stefw> but the docker code uses this capability to be efficient
16:58:02 <stefw> early pull request here: https://github.com/cockpit-project/cockpit/pull/1563
16:58:33 <stefw> from the get go, this allows us to share the list of cockpit packages between components, not loading it over and over again
16:58:46 <stefw> but very likely server information like hostname etc... would be similarly cached
16:59:16 <mvollmer> the pretty hostname?
16:59:16 <stefw> the JsonCache code automatically arbitrates between instances of its caller, and one of them gets to own and perform updates to the cache
16:59:18 <stefw> mvollmer, yes
16:59:23 <mvollmer> right
16:59:41 <stefw> if the caller instance that is updating the cache goes way, it clears the cache, and another caller takes over
17:00:09 <stefw> the actual data cached is JSON
17:01:28 <stefw> alright, next topic?
17:01:42 <stefw> or are we done with the agenda?
17:02:02 <stefw> how do we move to open floor?
17:02:08 <sgallagh> #topic Open Floor
17:02:14 <stefw> andreasn, are you around, anything to add?
17:02:22 <sgallagh> FYI: http://www.phoronix.com/scan.php?page=article&item=fedora_21_cockpit
17:03:10 <danofsatx-work> wow...more dev-y-ish than I expected. I understood about half of what was presented today.....
17:03:15 <stefw> again, all the 'Ooops' in the corner means stuff went wrong
17:03:30 <stefw> danofsatx-work, sorry bout that
17:03:41 <stefw> we're in heavy development mode right now
17:03:46 <stefw> but happy to discuss other topics too
17:04:32 <danofsatx-work> no probs ;)
17:04:41 <danofsatx-work> It's a learning experience for me
17:06:37 <stefw> all done?
17:06:46 <mvollmer> i'd say so.
17:07:23 <stefw> #endmeeting