18:00:01 <nirik> #startmeeting Infrastructure (2016-08-25)
18:00:01 <zodbot> Meeting started Thu Aug 25 18:00:01 2016 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:01 <zodbot> The meeting name has been set to 'infrastructure_(2016-08-25)'
18:00:01 <nirik> #meetingname infrastructure
18:00:01 <zodbot> The meeting name has been set to 'infrastructure'
18:00:02 <nirik> #topic aloha
18:00:02 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore threebean pingou puiterwijk pbrobinson
18:00:02 <zodbot> Current chairs: abadger1999 dgilmore lmacken nirik pbrobinson pingou puiterwijk relrod smooge threebean
18:00:02 <nirik> #topic New folks introductions
18:00:18 <jflory7> .hello jflory7
18:00:19 <zodbot> jflory7: jflory7 'Justin W. Flory' <me@justinwflory.com>
18:00:23 <nirik> Morning everyone. ;) I'm a bit distracted today, the go/nogo meeting is still going on in #fedora-meeting-2, but I am here. ;)
18:00:38 * pbrobinson is vaguely around
18:00:40 <cydrobolt> o/
18:00:48 <clime> hi!
18:00:55 <smooge> here
18:01:10 * smooge slaps pbrobinson with a wet fish until he comes around
18:01:28 <smooge> nirik I can run the meetin
18:01:46 <nirik> smooge: if you want, feel free. ;) it's all in the gobby
18:02:04 <smooge> hi are there any new folks who want to say hi?
18:02:14 * sayan is here
18:02:18 <trishnag> .hello trishnag
18:02:19 <zodbot> trishnag: trishnag 'Trishna Guha' <trishnaguha17@gmail.com>
18:02:26 <smooge> Hello Trishna
18:02:35 <jflory7> sayan, trishnag: o/
18:02:39 <cydrobolt> hey trishnag
18:02:54 <sayan> jflory7: hello
18:02:58 <trishnag> smooge: Hi. I am not actually new :). but new in the meeting
18:03:05 <smooge> no problem
18:03:12 <trishnag> cydrobolt: jflory7 \o
18:03:26 <smooge> ok if no one else.. next up
18:03:44 <smooge> #topic announcements and information
18:03:44 <smooge> #info commops cloud instance fixed up for ssh port changes - kevin
18:03:44 <smooge> #info copr backend volume fscked, hopefully stable now - kevin
18:03:44 <smooge> #info Bunch of alpha issues found and sorted, hopefully go this week - kevin
18:03:44 <smooge> #info freemedia updated for f24 - kevin
18:04:08 * nirik is the only one who fills that in these days. ;)
18:04:13 <bowlofeggs> .hello bowlofeggs
18:04:14 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <randy@electronsweatshop.com>
18:04:18 <dgilmore> hola all
18:04:53 <smooge> nirik, sorry I haven't had anything to say announcement wise
18:05:05 <nirik> yeah, it's been pretty quiet with alpha freeze
18:05:06 <smooge> #info nothing too broke - smooge
18:05:27 <jflory7> Heheh
18:05:31 <jflory7> nirik++ smooge++
18:06:35 <smooge> nirik, the next item is listed ofr you but I will move it later so you can concetrate on meeting
18:06:39 <smooge> #topic "official" announcement on Trac sunsetting to Pagure - jflory7
18:06:46 <smooge> jflory7, you're up
18:06:53 <jflory7> Yep–
18:07:21 <nirik> so, we did somewhat announce this on the infra list... but I agree we should send something to hosted-announce and devel-announce and some blog posts might be nice
18:07:25 <jflory7> So I know that the general plan is to sunset Trac / FedoraHosted eventually in 2017 (as far as I am aware). I think an announcement may have been posted on the Infra list at some point.
18:07:42 <jflory7> nirik: I was going to suggest that exact same email could be updated and used as the announce email and also for the CommBlog
18:08:03 * jflory7 doesn't have a reference to the past announcement currently
18:08:11 <nirik> I can look for it...
18:09:42 <jflory7> Anyways, it was an idea I wanted to suggest to help communicate that big change and so teams could begin discussing migration plans (if they have not already).
18:10:00 <jflory7> To help make life easier™ later
18:10:10 <nirik> yeah. We should stress if there's shortcomings in pagure they should file RFE's so we can try and address them
18:10:30 <jflory7> +1
18:11:01 <jflory7> Was there a specific date or month for the sunset / obsoletion?
18:11:30 <smooge> Yesterday
18:11:31 <jflory7> If not, a month might be a helpful deadline for teams to work against
18:11:33 <jflory7> Heheh
18:11:45 <nirik> I think we talked about end of feb 2017
18:11:54 <nirik> ie, end of the Red Hat fiscal year
18:11:56 <jflory7> I thought I remembered that too but wasn't sure.
18:11:57 <smooge> I would go with that. Feb 28 2017
18:12:02 <nirik> (why its that way... I don't know)
18:12:35 <jflory7> Plus, with the Pagure importer tool being functional and ready ( cverna++ ), teams could try moving sooner than later.
18:12:53 <nirik> I was going to try that out this week, but I had some issues...
18:13:16 <nirik> cverna doesn't seem to be around this week. ;) I guess I should file a ticket on it
18:13:21 <jflory7> I don't have the cycles to write the announcement, but ideally, it should just be a matter of unburying the email, adding info / updating old info, and resending it to the lists and posting to the CommBlog.
18:13:28 * jflory7 can help with the CommBlog if someone does the copy
18:13:51 <nirik> I can do the email and send you a copy for the commblog
18:13:58 <jflory7> nirik++ Sounds like a plan to me.
18:15:14 <cydrobolt> sounds good
18:15:55 <smooge> ok anything else on this?
18:16:14 <smooge> #topic docker image mirroring proposal - bowlofeggs
18:16:44 <smooge> #info nirik will write email about pagure and jflory7 will post to commblog and other places
18:16:50 <smooge> bowlofeggs, you are up
18:17:27 <bowlofeggs> heyo!
18:17:30 * nirik has the email marked to read, but hasn't had time to yet
18:17:41 <bowlofeggs> i know i just wrote it, so probably not everyone has read it
18:17:56 <bowlofeggs> but the big thing to take note of is that i'm now pitching that we *not* use metalink responses
18:18:17 <bowlofeggs> because there's a feature that the schema 2 manifest has where URLs for the blobs can be listed
18:18:29 <bowlofeggs> however, that feature is *also* not supported by the docker client ☹
18:18:31 <bowlofeggs> but!
18:18:56 <bowlofeggs> there is a patch written for it (that was not accepted upstream) and an open upstream issue where it sounds like it might actually happen
18:19:09 <bowlofeggs> so i'm pitching the idea that we should work to get that patch accepted upstream
18:19:24 <bowlofeggs> and potentially carry it downstream until it is accepted upstream
18:20:00 <bowlofeggs> i also expanded on a few areas that people had questions about from my first post
18:20:05 <smooge> ok on the infrastructure point of things.. the biggest questions are what are mirrors going to be getting and what would they need to do?
18:20:44 <bowlofeggs> and smooge, i saw your post. i wasn't sure if you had read my reply from last week in response to similar questions from tibbs
18:21:09 <bowlofeggs> smooge: basically my answer to him was unfortunately that i don't know how many containers we are talking about
18:21:19 <bowlofeggs> smooge: i expect the base image will be a few hundred MB
18:21:29 <bowlofeggs> and we can probably use hardlinking to dedupe that layer
18:21:37 <bowlofeggs> but it's hard to guess what people will do with docker in fedora
18:22:11 <bowlofeggs> smooge: i think this is the post: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/WCOT3OMY3BB6TXCJYLPZOCZCYNQOLMUW/
18:22:12 <smooge> I realize that but someone is going to need to put together some sort of minimum/maximum we are going to mirror
18:22:35 <smooge> otherwise no one is going to mirror in the same way that they skip other things which don;'t have such answers
18:23:21 <smooge> this may not be something that you can personally answer.. but can you find someone who will?
18:23:43 <bowlofeggs> smooge: i don't know who could make those predictions. docker might become popular with the packagers, or it might not.
18:23:57 <bowlofeggs> do you have a suggestion on who might be good at predicting things like this?
18:24:27 <smooge> whether or not it becomes popular, there has to be a set of images that we are promising to deliver
18:25:04 <smooge> those would be the ones I expect that mirrors will take and they will skip the rest even if they are super popular with packagers
18:25:25 <nirik> well, it's a bit like asking how popular some packages will be...
18:25:28 <nirik> before we setup any packages
18:25:50 * nirik doesn't think it's going to be all that many, but could be wrong
18:26:40 <smooge> nirik, I am mostly wanting to be able to say " we are looking at having N GB for F-N it may go up after that but this is what is we are promising to deliver."
18:26:57 <bowlofeggs> afaik, there's only one promised image that we are delivering - cockpit
18:27:23 <nirik> yeah, for f25/ it wll be 1 image
18:27:31 <smooge> We currently say that we have N isos per release and we keep their size to M MB
18:27:40 <bowlofeggs> that image is probably small (<500 MB i'd guess?)
18:28:00 <bowlofeggs> well the isos don't compare as well, because packagers don't make them
18:28:21 <bowlofeggs> docker images are more analogous to rpms in this way i think
18:28:43 <bowlofeggs> i.e, how many rpms will be in Fedora 26 is hard to answer, but probably easier than this question i'd think
18:28:54 <bowlofeggs> because at least with rpms we have historical data to consider
18:28:58 <bowlofeggs> for docker, we don't
18:30:27 <smooge> we need to give some answers here. Mirrors do not have a large amount of disk space/bandwidth and we are asking them to give us more for something they could be using for something else.
18:30:40 <bowlofeggs> i'm certainly not opposed to anyone attempting to do this forecasting
18:30:51 <nirik> well, we can pull random numbers up, but I am not sure we have any basis for a WAG
18:31:18 <smooge> it isn't as much about forecasting as saying we are not going to put more than N GB in this tree. If there are more than that we will come up with a different mechanism
18:31:23 <bowlofeggs> smooge: do you know who might be more qualified to make such a forecast?
18:31:35 <bowlofeggs> oh i see
18:31:51 <bowlofeggs> so you want us to make a commitment and have some protections that we won't go beyond it?
18:32:11 <smooge> we have a soft committment that Fedora Enchilada will not be over 2 TB
18:32:43 <nirik> it used to be 1TB? when did we change it? ;)
18:32:54 <nirik> (well, I suspect we went over, but I don't know that we told mirrors. ;)
18:32:56 <smooge> nirik, I thought we changed it with F20 when we ran over
18:33:48 <bowlofeggs> smooge: does that mean there isn't code that ensures it stays in that limit, but just that we would notice it getting close and do something if it does?
18:34:45 <smooge> bowlofeggs, there isn't any code.. we usually get told by someone who is no longer mirroring us because we are too much disk space :)
18:34:59 <bowlofeggs> ah i see
18:35:26 <nirik> so perhaps we could look at the cockpit one and tell mirrors about f25 at least?
18:35:28 <smooge> at one point the compose tools were going to have it
18:35:41 <bowlofeggs> nirik: that i think would be quite doable ☺
18:35:50 * cverna is very late but there
18:35:51 <smooge> Also where do you plan to put these images
18:36:00 <bowlofeggs> nirik: though, i also don't think we'll have this mirroring ready for f25 actually
18:36:13 <nirik> bowlofeggs: ok, f26 then. ;)
18:36:22 <smooge> are these deliverables under /pub/fedora/linux or /pub/fedora-secondary or /pub/fedora-docker?
18:36:38 <nirik> cverna: hey, I was having some problems with the importer... can I ask you over on #fedora-apps (whenever you have time)
18:36:45 <bowlofeggs> smooge: i don't have a preference, any of those sound fine to me
18:38:33 <bowlofeggs> smooge: so in general, think container layers are bound to have a lot of duplicated bits (like, maybe a lot of them install libxyz in one of their upper layers)
18:39:07 <bowlofeggs> smooge: maybe we could say that these images will probably be larger than the RPMs, assuming that Docker packagers may use most of the RPMs (with some duplication)
18:39:11 <smooge> but are those visible outside of the image
18:39:16 <bowlofeggs> we can only really deduplicate the base layer
18:39:57 <bowlofeggs> smooge: not easily - the layers are just binary diffs
18:40:34 <bowlofeggs> so if you make a container and use libxyz and i do it too, but we don't do it at the same step in our repective Dockerfiles, the layer name you get and i get won't be the same
18:40:52 <bowlofeggs> at least i don't think it will
18:40:59 <bowlofeggs> i'm like 85% on that one
18:41:19 <smooge> sounds like a recipe for disaester but my geriatric medicine time is coming up
18:41:31 <bowlofeggs> haha
18:41:33 <bowlofeggs> yeah
18:41:46 <bowlofeggs> but then again, it's possible that packagers don't do much in this space too
18:42:15 <bowlofeggs> i could see it being huge, or relatively small
18:42:44 <smooge> ok so I don't really think we are going to get a lot of mirroring of this stuff but we need to put this in just so that the people can get it from a cdn or our other servers
18:42:55 <bowlofeggs> does mirror manager display the current sizes of its various modules to users?
18:43:06 <smooge> not that I know of
18:43:28 <smooge> we do a find du and update a file daily
18:43:29 <bowlofeggs> would it help this concern if we planned to add a feature like that to it?
18:43:46 <bowlofeggs> so admins could see something like "this module is currenly 1.7 TB"?
18:43:54 <bowlofeggs> ah
18:44:06 <bowlofeggs> so there is already a way for people to see that info
18:44:11 <smooge> well since most of them are getting it from rsync it would need ot be in the rsync info
18:45:14 <smooge> I think I may not be communicating clearly. Can we go over this 1:1 later?
18:46:56 <smooge> nirik, are you able to talk about the FLOCK followup or put this off til next meeting?
18:47:09 <nirik> actually we can just drop that.
18:47:13 <nirik> I forgot to do so
18:49:04 <bowlofeggs> smooge: sure. i think i understand and appreciate your concern. i just feel pretty unqualified to answer the question, and i don't really know who would be more qualified to do it ☺
18:49:36 <bowlofeggs> it's obvious that this concern is important to mirror admins, since Jason Tibbitts brought it up as well
18:51:08 <bowlofeggs> did anyone have any other thoughts wrt the change in plan?
18:51:14 <nirik> I think all we can do now is guess based on 1 project for f26 and see hoow much uptake it gets after that and extrapolate
18:51:21 <smooge> it is also going to be important to us because we didn't take it into consideration for atomic and that nearly killed our storage access
18:51:24 <bowlofeggs> i.e., the main change being moving from metalinks to that weird missing feature?
18:52:15 <nirik> I think it's fair for this to be a seprate target...
18:54:19 <smooge> ok we are coming up to the hour
18:54:37 <smooge> ok to drop the office hours today so the next meeting can start?
18:55:21 <clime> (there are also apprentices hours but I don't have much to ask I guess)
18:55:42 <clime> oh I guess, you meant that.
18:55:47 <smooge> yes sorry
18:56:12 <jflory7> smooge: Can I follow up in #fedora-admin on my request for apprentice question?
18:56:14 <smooge> #info will continue discussion of docker images to the list
18:56:29 <smooge> #topic apprentice items
18:56:46 <smooge> jflory7, you are wanting to get access to sysadmin-badges?
18:57:09 <jflory7> smooge: Yes, I was looking to become a more active participant to help with sysadmin-badges. :)
18:57:28 <nirik> we kinda handed that over to a group of badges interested folks. I have no objection adding you if you want...
18:58:10 <jflory7> nirik: I remember that conversation from April or May - at that time, I didn't think I had the chops for it but I feel much more confident about it now. :)
18:58:40 <smooge> jflory7, could you put in a ticket for that?
18:58:53 <jflory7> smooge: Yes, I can file one now.
18:59:55 <smooge> thanks
19:00:04 <smooge> ok anything else or time to close out the meeting?
19:00:06 <clime> nirik: I have a question. is there any (small) task I could take on now?
19:00:43 <nirik> clime: not sure off hand. Several things should open up after we get out of freeze I would think tho
19:00:55 <clime> nirik: ok
19:01:35 <clime> If anything pops up that I can help with, please, let me know
19:01:39 <nirik> can do
19:01:50 <aikidouke> clime - come on over to #fedora-noc and I can give you a couple of ideas
19:02:03 <clime> aikidouke: ok, thanks :)
19:02:22 <smooge> alright
19:02:35 <smooge> we are over so I am going to close this sucker out.
19:02:42 <smooge> Thank you all for coming
19:02:45 <smooge> #endmeeting