15:01:18 <acozine> #startmeeting Docs Working Group aka DaWGS
15:01:18 <zodbot> Meeting started Tue Jun 29 15:01:18 2021 UTC.
15:01:18 <zodbot> This meeting is logged and archived in a public location.
15:01:18 <zodbot> The chair is acozine. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:18 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:18 <zodbot> The meeting name has been set to 'docs_working_group_aka_dawgs'
15:01:23 <acozine> #topic opening chatter
15:01:27 <acozine> who's around?
15:01:28 <lmodemal> Hello everyone o/
15:01:34 * samccann waves
15:01:38 <acozine> #chair lmodemal samccann
15:01:38 <zodbot> Current chairs: acozine lmodemal samccann
15:02:11 <Xaroth> Every time I see the DING DING DING I imagine loud church bells ringing.
15:02:12 <abadger1999> Bom dia
15:02:28 <acozine> heh, that's what we're going for
15:02:32 <lmodemal> Xaroth :)
15:02:36 <acozine> #chair Xaroth abadger1999
15:02:36 <zodbot> Current chairs: Xaroth abadger1999 acozine lmodemal samccann
15:03:05 <acozine> dmsimard: gundalow aminvakil briantist cyberpear felixfontein tadeboro zbr you folks chatting docs today?
15:03:08 <samccann> For whom the bell tolls... it tolls... for DOCS!
15:03:13 <briantist> o/
15:03:29 <zbr> o/ (but no subject on my side)
15:03:52 <acozine> official agenda: https://github.com/ansible/community/issues/579#issuecomment-866184625
15:03:58 <acozine> #chair briantist zbr
15:03:58 <zodbot> Current chairs: Xaroth abadger1999 acozine briantist lmodemal samccann zbr
15:04:45 <abadger1999> Dmsimard is enjoying a vacation in nature this week
15:04:57 <acozine> that sounds lovely
15:05:17 <acozine> though strangely he still shows up as connected . . .
15:05:30 <acozine> maybe he's still in cell-phone range
15:06:04 <acozine> I try not to ping people whose nicks are greyed out, I figure that means they're asleep
15:06:36 <felixfontein> hi!
15:06:43 <acozine> hey felixfontein
15:06:46 <acozine> #chair felixfontein
15:06:46 <zodbot> Current chairs: Xaroth abadger1999 acozine briantist felixfontein lmodemal samccann zbr
15:06:59 <acozine> welcome, welcome everyone
15:07:12 <acozine> let's start with action items from last week
15:07:17 <acozine> #topic follow up on action items
15:07:56 <acozine> first one was mine - follow up with core, galaxy, and other folks about semantic markup
15:08:26 <acozine> I reached out on slack, plus samccann sent out an email with details, but we haven't heard anything back
15:09:11 <acozine> so it's hard to judge whether folks know about the plan and are supportive or at least not opposed, or whether they just aren't aware
15:09:22 <abadger1999> I mentioned it to the technical architects for the core and galaxy trans.
15:09:30 <acozine> oh, excellent
15:09:35 <acozine> thanks!
15:09:49 <felixfontein> I'm /4
15:09:54 <felixfontein> oops :)
15:09:55 <abadger1999> They may on Thursdays. I can tell them you've emailed people but haven't heard back if you want
15:10:01 <abadger1999> *meet
15:10:40 <acozine> my hope is that the plan is non-controversial
15:10:48 <samccann> #info no feedback yet on semantic markup from core and galaxy teams
15:10:51 <abadger1999> Yeah
15:11:28 <samccann> the controversy is what do these other teams do with the new markup options when they start showing up
15:11:43 <acozine> #info summary of semantic markup plan https://github.com/ansible/ansible/issues/75054
15:11:59 <samccann> Like V(foo) would look just like that in galaxy-ng etc if they don't support it
15:12:20 <samccann> and not sure what ansible-doc output would look like etc
15:12:23 <abadger1999> We may need an issue for tracking all changes that should be put in  front of all docs consumers. I was alerted to one from the network team a couple weeks ago too
15:12:26 <acozine> true, but isn't that already true? with `M(foo)`?
15:13:00 <samccann> I'd have to specifically check. I would have thought galaxy a least knew it was 'a thing' and didn't just output it as M(foo).
15:13:11 <samccann> sorry galaxy-ng
15:13:22 <felixfontein> would be great, because M() has been around ... forever :)
15:13:28 <acozine> yep
15:14:37 <acozine> okay, sounds like we need to have another action item for "keep requesting feedback on semantic markup plan from core and galaxy folks, especially about the display of the new options in galaxy-ng and ansible-doc"
15:15:32 <acozine> #action acozine to keep requesting feedback on semantic markup plan, especially about the display of the new options in galaxy-ng and in ansible-doc output"
15:16:49 <acozine> next action item from last week was review and merge PR 75058, which we did
15:17:22 <acozine> third action item from last week was  @abadger1999 to review and (hopefully) merge antsibull PRs 284, 283, 285, and then 282
15:18:09 <abadger1999> All of those were merged :-)
15:18:11 <acozine> all of which are merged
15:18:17 <acozine> thank you abadger1999!
15:18:40 <abadger1999> The big one is that we are now capable of building devel docs the way we want.
15:18:50 <felixfontein> \o/
15:18:53 <acozine> that is an awesome step forward
15:19:00 <felixfontein> unfortunately the devel docs no longer build, due to memory exhaustion...
15:19:05 <acozine> thanks felixfontein for expanding the docs capability there
15:19:25 <felixfontein> which probably means that the docsite for today's release won't build either
15:19:39 <samccann> ooch
15:19:39 <acozine> yeah, let's talk about the build . . . but let's get through the last action item
15:19:45 <acozine> clear the decks
15:19:54 <acozine> fourth and last action item from last week:
15:20:09 <acozine> with a new version of antsibull (potentially today?), getting Bump sphinx_ansible_theme to v0.7.0+ ansible#75059 (with == 0.7.0 and not >= 0.7.0) merged would be good, as well as getting Update docs requirements list ansible#74956 merged (probably first, so it doesn't collect more potential conflicts)
15:20:30 * acozine looks at those
15:20:40 <acozine> okay, we merged 74956
15:20:57 <acozine> but not 75059, at least, not yet
15:21:30 <felixfontein> there's https://github.com/ansible/ansible/pull/75100 which backports some things to stable-2.11
15:22:10 <felixfontein> (oh, just one thing; the multiple things backport PR was https://github.com/ansible/ansible/pull/75058 and that's already merged)
15:22:33 <acozine> okay, 75059 looks ready to merge, I'll merge it unless anyone has last-minute objections?
15:22:53 <samccann> go for it
15:23:12 <acozine> okay, merged
15:23:38 <samccann> woot!
15:24:06 <acozine> i also merged 75100
15:24:32 <felixfontein> thanks!
15:25:02 <acozine> onward and upward!
15:25:25 <acozine> #topic devel docs update and build EOF/OOM issues
15:25:41 <samccann> FOOM BOOM!
15:25:49 <acozine> looks like https://github.com/ansible-community/antsibull/issues/54 is still open
15:26:01 <acozine> can we close it?
15:26:24 <felixfontein> I *think* we can close it
15:26:43 <felixfontein> just did it :)
15:27:04 <acozine> ready or not, here we come!
15:27:07 <samccann> hahah
15:27:43 <abadger1999> Thanks :-)
15:27:56 <acozine> felixfontein: can you add a link from that issue to the PRs that did the work?
15:28:28 <acozine> I have the feeling that Future Me may be grateful for the online breadcrumbs there
15:28:49 <acozine> ah, I found it
15:29:09 <felixfontein> thanks for adding it!
15:29:47 <acozine> sure thing
15:29:51 <acozine> okay, build issues
15:30:36 <abadger1999> So.... if this is the same problem (I think it's likely, but we can test in a VM where we control how much memory it has) we need to ask gundalow and the people who are now managing the jenkins issue if we can fund a worker (or two) with more memory for building docs.
15:30:54 <acozine> yeah
15:31:38 <samccann> It works in my VM locally. Shall I start lowering the RAM it has and see what happens?
15:31:40 <acozine> the problem we saw before was this: Sphinx builds index pages from lists kept in memory, and it runs out of memory trying to build https://docs.ansible.com/ansible/devel/collections/index_module.html
15:31:55 <abadger1999> samccann: If you have time, yep, that's what I did before.
15:32:22 <samccann> ok sure, I can kick it off in the background. it has 8G right now. Do you recall what the threshold was the last time?
15:32:23 <acozine> do we have any other options for addressing this problem?
15:32:37 <abadger1999> ... once we added breadcrumbs
15:32:50 <acozine> because I suspect the problem will continue to surface
15:32:57 <acozine> abadger1999: oh, is it the breadcrumbs?
15:32:59 <abadger1999> If we can't get more memory, we can toggle breadcrumbs off again
15:33:05 <acozine> ah, my mistake
15:33:07 <samccann> #info last time we had jenkins build memory problems it was because  Sphinx builds index pages from lists kept in memory, and it runs out of memory trying to build https://docs.ansible.com/ansible/devel/collections/index_module.htm (after we added breadcrumbs)
15:33:17 <samccann> #undo
15:33:17 <zodbot> Removing item from minutes: INFO by samccann at 15:33:07 : last time we had jenkins build memory problems it was because  Sphinx builds index pages from lists kept in memory, and it runs out of memory trying to build https://docs.ansible.com/ansible/devel/collections/index_module.htm (after we added breadcrumbs)
15:33:41 <samccann> #info the sphinx memory problem stems from building breadcrumbs. Workaround if necessary is to disable them again
15:34:03 <abadger1999> Breadcrumbs contribute because we add a lot more documentation nodes to the table of contents.  Those don't show up on the index pages (they're :hidden:) but they do contribute to the menory usage.
15:34:05 <samccann> last time we disabled by dropping down the antsibull version I think, but that was a few versions ago
15:34:16 <acozine> ah
15:34:23 <acozine> so it's a combination?
15:34:27 <samccann> So can we actually do that now? What else do we lose?
15:34:28 <abadger1999> Yeah, I can resurrct the toggle code.
15:34:48 <samccann> Is there a release today?
15:34:57 <abadger1999> If this is the same issue, we should just lose breadcrumbs.
15:35:03 <abadger1999> Yes, there's a release today.
15:35:23 <acozine> I think breadcrumbs are valuable to users
15:35:51 <abadger1999> We don't know if it will push the stable docs build into out of memory, though.  devel docs might be building with newer versions of collections than the stable build
15:36:01 <acozine> though I don't have data to support that
15:36:10 <acozine> is there any other way we could work around this problem?
15:36:11 <felixfontein> I wonder how much the costs is of paying for a VM with more memory compared to the working hours you (as paid employees) spend on discussing this and working around it :)
15:36:16 <acozine> heh
15:36:17 <samccann> people asked for breadcrumbs... even I use breadcrumbs now.
15:36:24 <samccann> AAHAHAHA
15:36:26 <acozine> I use them too
15:36:31 <samccann> bonus points to felixfontein++
15:37:19 <acozine> I just hate to go back every N months and ask for more memory
15:37:21 <abadger1999> We can see if we can add more RAM to the jenkins instance building docs without purchasing more RAM.  Spredzy had said the hosts were running low on memory to allocate, though.
15:37:26 <acozine> and it feels like this will keep happening
15:37:40 <samccann> ok so we have two action items - one to test the VM locally with lower memory and see where it breaks.  The other is getting more memory somehow for the build
15:37:54 <abadger1999> We can also look at implementing breadcrumbs in code we write instead of using sphinx's builtin way of doing it
15:38:06 <samccann> and a third one to put a toggle or something so we can disable breadcrumbs again so the build works for today's release etc?
15:38:23 <acozine> maybe we could also test at the higher level of memory and see how many more collections/modules/plugins that amount of memory can handle
15:38:39 <abadger1999> <nod>
15:38:41 <samccann> not sure how to do that ^^ ?
15:38:55 <samccann> are you talking in Jenkins or locally in a VM?
15:39:02 <acozine> locally
15:39:34 <acozine> a bit of jury-rigging would be required - add some fake collections, re-test
15:39:41 <abadger1999> We can add things to ansible.in or ansible-VERSION.deps locally and then retest.
15:40:08 <abadger1999> (modifying those files can be done locally too... no need to check them into the real ones on github)
15:40:10 <samccann> is it as easy as including the same collection more than once? I don't have any 'dummy' collections I could bring in
15:40:28 <abadger1999> It would have to be distinct collections from galaxy.
15:40:37 <acozine> because if each extra GB of RAM gets us 20 more collections, that's one thing; but if each extra GB of RAM gets us 2 more collections, that's something else again
15:40:43 <samccann> ah gotcha.  that makes sense
15:40:53 <abadger1999> But there are many collections on galaxy which are not in ansible so that shouldn't be a problem.
15:41:03 <abadger1999> Yeah.
15:41:15 <samccann> #action samccann to test local VM with less memory, then with more memory and more collections to see how much each extra GB of ram gets us
15:41:42 <abadger1999> I don't know the data structures sphinx is keeping preciely but I think it will dpend on number of plugins/modules inside of each collection.
15:41:43 <samccann> #info modify the ansible.in or ansible-VERSION.deps locally to add more collections from galaxy
15:42:15 <samccann> #action abadger1999 to find a way to turn of breadcrumbs so we can get the builds working again, esp for today's release
15:42:28 <samccann> does that sound right on the actions?
15:42:34 <abadger1999> (ansible.in for devel docs. ansible-VERSION.deps for stable docs)
15:42:37 <abadger1999> yep.
15:42:49 <samccann> #info (ansible.in for devel docs. ansible-VERSION.deps for stable docs)
15:42:52 <acozine> abadger1999: ah, okay, so testing collections with lots of modules
15:42:53 <abadger1999> And talk to gundalow/Spredzy once we know for sure that memory is our issue
15:43:16 <samccann> #action acozine to talk to gundalow/Spredzy once we know for sure that memory is our issue this time
15:43:30 <acozine> if there are any "slow periods" on the Jenkins server, we can schedule the docs builds for those times
15:43:31 <samccann> there. sprinkling action items like they are candy today!
15:45:27 <acozine> heh
15:47:41 <acozine> okay, we're at 45 minutes in
15:47:45 <samccann> cyb-clock-clone finally wakes up to say we are 47 min into the meeting and 18 into the current topic
15:47:53 <acozine> shall we tackle another item?
15:47:58 <samccann> sure
15:48:56 <acozine> #topic new module documentation fields
15:49:06 <acozine> this was a long-running topic that we can also close out
15:49:19 <acozine> original proposal was https://github.com/ansible/proposals/issues/68
15:49:44 <acozine> implementation was https://github.com/ansible/ansible/pull/73707
15:49:55 <samccann> cool!
15:50:33 <acozine> I'm reading the PR now
15:50:43 <acozine> does anyone else have a better understanding of what it does?
15:51:21 <abadger1999> There are several changes being proposed by the core team and networking team to documentation strings and ansible-doc...... have any of those hit your radar?
15:52:08 <acozine> looks like it creates docs fragments for module attributes like supporting tags, async, become, and other common Ansible behaviors
15:52:21 <acozine> abadger1999: I'm not sure
15:52:37 <abadger1999> Oh... including that one.
15:52:48 <felixfontein> acozine: bcoca is apparently in the process of changing the format of the attributes
15:52:48 <abadger1999> We'll need to update antsibull to account for hte new fields.
15:53:22 <acozine> we'll need to update the page about documenting modules as well
15:53:39 <samccann> #info new attributes PR - https://github.com/ansible/ansible/pull/73707
15:53:42 <felixfontein> (I added validation for them in https://github.com/ansible/ansible/pull/74602, and he said to me on IRC that he is going to change the structure first...)
15:53:45 <abadger1999> felixfontein: I think ALLOW_EXTRA is a bad idea (as acozine pointd out in that PR) ?
15:54:13 <samccann> ok so that pr was merged, and now he's changing it?
15:54:20 <abadger1999> felixfontein: Probably should revert the original, then, so that no one starts using it?
15:54:21 <felixfontein> abadger1999: it was only thought from my side as to get it working first, so as the next step - in the same PR - proper validation could be added
15:54:35 <felixfontein> abadger1999: but it got merged without validation, and then I created https://github.com/ansible/ansible/pull/74602 to fix it
15:54:54 <abadger1999> felixfontein: <nod> Yeah, it seems like he merges things without getting specific approval.  He takes any comments as approval.
15:54:55 * acozine remembers this conversation now
15:55:32 <felixfontein> if the state of attributes is resolved very soon, I don't mind waiting. but if it will take more months, then I don't really like it
15:55:57 <felixfontein> and it looks like it already took almost two months since my PR :)
15:56:26 <samccann> sigh
15:56:27 <samccann> https://docs.ansible.com/ansible/devel/collections/ansible/builtin/import_playbook_module.html#ansible-collections-ansible-builtin-import-playbook-module
15:56:35 <samccann> it breaks docs output is my guess
15:57:22 <briantist> right, date matches `Last updated on Jun 25, 2021`
15:58:30 <samccann> ok we only have 2 min left, what are the next steps for this? Revert? Wait for something better to come along?
15:58:33 <felixfontein> it definitely does. but adding support to antsibull only makes sense if it won't get modified again
15:58:44 <abadger1999> Okay, I'm +1 to revert.
15:58:50 <acozine> yeah, me too
15:59:19 <felixfontein> I'd like to know what his plans are ... whether it is going to get finalized in the next week (say), or later. if later, revert (for now) is the better option.
15:59:31 <abadger1999> Proper steps should be: discuss with other stakeholders, nail down the format, implement format in core, galaxy, antsibull, (and navigator possibly), then merge all at once.
15:59:41 <briantist> I would also question why no automated tests caught it? seems like it would be useful
15:59:52 <briantist> but also +1 on reverting until any issues are figured out
15:59:58 <abadger1999> felixfontein: There's still lag where we have to implement functionality in antsibull and docsite will be broken until then.
16:00:01 <felixfontein> briantist: the tests were disabled in that PR
16:00:13 <briantist> ah :(
16:00:18 * acozine smacks palm to face
16:00:44 <felixfontein> abadger1999: true, but the lag will be short if things are merged in the correct order
16:01:16 <abadger1999> felixfontein: I don't think it will necessarily be short.
16:01:39 <abadger1999> felixfontein: Did I miss it or is there docs pages explaining what each key means in that PR?
16:02:21 <felixfontein> I don't think there is yet
16:02:26 <acozine> abadger1999: no changes to docs/docsite in that PR at all
16:02:36 <abadger1999> Yeah
16:02:38 <samccann> #info this PR breaks the docs output since antsibull isn't recognizing these fields yet. But PR owner plans more changes, so need to decide if we wait for that or revert until it's complete
16:02:51 <abadger1999> So that also has to be implemented before it's ready.
16:02:52 <samccann> #info PR also has no documentation for what it is supporting
16:03:25 <abadger1999> i: this feature isn't going to be ready just when docsite build doesn't fail.  New content needs to be added as well.
16:03:34 <abadger1999> s/i:/ie:/
16:04:57 <acozine> yep
16:05:00 <samccann> we're 4 min past the hour. What are the action items here?
16:05:26 <abadger1999> We probably need to vote to either revert or leave it broken until next week.
16:05:39 <acozine> yeah
16:05:46 <acozine> at least it only breaks devel, not latest
16:05:52 <acozine> I'd vote to revert
16:05:53 <samccann> The current broken state only affects the modules he added attributes to, correct?
16:06:17 <acozine> we still haven't merged the revert for the last breaks-the-build PR
16:06:23 <acozine> ugh
16:06:41 <acozine> okay, let's vote
16:06:44 <abadger1999> yeah.  it's been multiple weeks since nitzmahone said he'd take care of that one too.
16:06:54 <abadger1999> I'll talk to nitzmahone today.
16:07:13 <abadger1999> (and I'll include you too acozine and samccann )
16:07:15 <samccann> ok so it looks like he only modified 3 modules for this. Assuming no one else knows bout it and also added to their modules
16:07:46 <samccann> so from a docs perspective, I feel okay leaving them broken for another week if we think this is going to be fixed
16:08:23 <abadger1999> We won't be able to plan fixing it until next week, though.
16:08:59 <acozine> because of this week's release?
16:09:02 <samccann> ok so my thoughts - it only breaks what he did. If we put a comment in the proposal and the PR to say do not use this feature as it breaks other things
16:09:10 <samccann> maybe that's enuf?
16:09:17 <abadger1999> it's not a matter of it will be fixed next week.  It's a matter of we'll be able to start planning to fix it next week.
16:09:30 <briantist> I think the devel docs are useful enough that they shouldn't remain broken
16:09:42 <samccann> agreed, and I think he has to fix them
16:09:50 <abadger1999> (1) We don't know what the format actually is since he's going to be changing it at some point in the future.
16:09:52 <samccann> But I'm not sure we should be responsible for all his reverts
16:10:08 <abadger1999> (2) We then need to figure out how to show the information in the docsite build.
16:10:48 <samccann> (3) and gauge any impact on other things that display docs, like galaxy-ng, ansible-navigator etc
16:10:50 <abadger1999> (3) We need to implement the parsing/validation [easy] and the display [harder]
16:11:48 <samccann> cyb-clock-clone sez we are 11 minutes over the hour and 24 min into this topic
16:11:56 <acozine> sigh
16:12:03 <abadger1999> yeah.... my steps don't really account for the other teams who need to be aware and able to comment/modify this either.
16:12:32 <acozine> okay, let's take a vote on what folks prefer
16:12:55 <acozine> though we can't guarantee that it will happen, at least we'll know the  general preference of the DaWGs
16:12:59 <acozine> VOTE: a) prefer to revert PR 73707 b) prefer to fix it with a follow-up PR or PRs (keep in mind that we cannot guarantee a timeline for either option)
16:13:12 <abadger1999> (a) +1
16:13:17 <acozine> a
16:13:21 <briantist> (a) +1
16:13:32 <acozine> #chair
16:13:32 <zodbot> Current chairs: Xaroth abadger1999 acozine briantist felixfontein lmodemal samccann zbr
16:14:01 <samccann> c) leave it and let him fix his own cruft, at least until it becomes a problem beyond the three existing builtin modules. Assign any issues raised on those to him.
16:14:02 <lmodemal> (a) +1
16:14:10 <abadger1999> We shouldn't have in-progress formats that people can use by mistake.  We shouldn't be adding or changing formats without discussion amongst all the stakeholders.
16:14:10 <acozine> samccann: heh
16:15:07 <samccann> since we had this discussion at least once before, why aren't we following the process abadger1999 mentions?
16:15:27 <samccann> we as in the global we including the pr owner and their manager etc
16:16:13 <acozine> that's a very good questions
16:16:17 <acozine> er, question
16:16:18 <abadger1999> I do not know.  I've mentioned it to the architects but I donot know if they're relaying it to their teams or not.
16:16:42 <samccann> is anyone else flashing to Science Fiction and the mysterious The Architects!
16:16:50 <acozine> heh, yep
16:17:00 <felixfontein> sorry, have been afk
16:17:06 <acozine> okay, I'll see if there's anything I can do to break through the logjam
16:17:25 <acozine> #action acozine to follow up on the problems with PR 73707
16:17:31 <samccann> #action samccann abadger1999 acozine to followup on how we can havea better process in place to stop changes to ansible-doc etc until the full pipeline has agreed to support said changes
16:17:51 <acozine> okay, quick open floor just in case
16:18:01 <acozine> (in case someone has a burning issue, that is)
16:18:07 <acozine> #topic open floor
16:18:10 <abadger1999> In that vein, here's several PRs that need to go through all the docs stakeholders:
16:18:18 <abadger1999> https://github.com/ansible/ansible/pull/74873
16:18:20 <acozine> all questions, comments, ideas, welcome
16:18:26 <abadger1999> https://github.com/ansible/ansible/pull/74963/files
16:19:10 <abadger1999> This one.... We might want to look at it but I don't think it is problematic: https://github.com/ansible/ansible/pull/75116  (It changes ansible-doc output but if I'm reading it right, it doesn't affect json output)
16:19:32 <acozine> looks like we don't have a linkbot
16:19:34 <acozine> so . . .
16:20:34 <acozine> #info PRs that need input from docs stakeholders: https://github.com/ansible/ansible/pull/74873, https://github.com/ansible/ansible/pull/74963, and possibly https://github.com/ansible/ansible/pull/75116
16:20:49 <acozine> thanks abadger1999
16:21:24 <acozine> any other open floor items?
16:21:24 <abadger1999> No problem
16:21:48 <samccann> meawhile all  my VMs are dead. I won't be able to test memory issues until later this afternoon.
16:21:55 <acozine> oops
16:21:58 <samccann> (for the docs build)
16:22:01 <acozine> time to reboot?
16:22:11 <acozine> or do you mean the OOM error killed them all?
16:22:12 <samccann> i just did. Gnome Boxes just keeps crashing
16:22:16 <acozine> ah
16:22:31 * acozine sees little gnomes falling over across the lawn
16:22:33 <samccann> I'll try to remove one of the VMs from whereever boxes hides it and see if that helps
16:22:51 <samccann> but just so's y'all know
16:22:55 <acozine> heh
16:23:07 <acozine> but we were expecting INSTANT RESULTS!!!
16:23:20 <acozine> what do you mean we have to wait several hours???
16:23:37 <acozine> we will all survive . . .
16:23:40 <abadger1999> If you aren't able to revive it let me know and I'll put memory testing on my todo
16:23:48 <samccann> ok thanks
16:23:58 <samccann> meanwhile, are we ready to stick a fork in this meeting and call it done?
16:24:00 <acozine> yep
16:24:11 <acozine> as always, items for next week's agenda welcome
16:24:21 <acozine> thanks Xaroth abadger1999 briantist felixfontein lmodemal samccann zbr
16:24:36 <acozine> oh, forgot a link to the agenda again
16:24:47 <acozine> #info agenda items welcome, add a comment to https://github.com/ansible/community/issues/579
16:24:52 <acozine> thanks again, folks!
16:24:54 <acozine> #endmeeting