15:32:33 <dgilmore> #startmeeting RELENG (2014-10-06)
15:32:33 <zodbot> Meeting started Mon Oct  6 15:32:33 2014 UTC.  The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:32:33 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:32:43 <dgilmore> #meetingname releng
15:32:43 <zodbot> The meeting name has been set to 'releng'
15:32:43 <dgilmore> #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson
15:32:43 <zodbot> Current chairs: bochecha dgilmore masta nirik pbrobinson sharkcz tyll
15:32:44 <dgilmore> #topic init process
15:32:50 <nirik> morning
15:32:52 <dgilmore> who is here
15:32:56 <bochecha> good evening
15:33:03 * sharkcz is here
15:33:06 * pbrobinson waves
15:33:07 * sgallagh lurks
15:34:01 <dgilmore> #topic #6014 multiple builds in primary koji are lackign arm buildroot info
15:34:29 <dgilmore> this seems to be some fallout from the koji db move/outage last week
15:34:36 <nirik> yeah ;(
15:34:44 <nirik> sorry about that. I tried to be carefull... not sure what happened.
15:34:45 <dgilmore> I think we should just bump and rebuild any effected builds
15:35:01 <dgilmore> going forward we should just be sure to switch the hubs off for any migration
15:35:01 <nirik> sounds like a good workaround
15:35:08 <pbrobinson> +1
15:35:15 <nirik> yeah. ;(
15:35:53 <sharkcz> I've already rebuilt gcc-python-plugin, the rest seem to fixable by a hack in koji-shadow
15:36:05 <dgilmore> sharkcz: i would rather just rebuild
15:36:15 <sharkcz> don't disagree
15:36:47 <dgilmore> #action all rebuild all effected packages
15:36:53 * nirik grabs more coffee. back in a sec.
15:37:35 <dgilmore> anything else on this?
15:37:53 <sharkcz> nope
15:38:00 <dgilmore> #topic #5886 need method for distributing urgent fixes... urgently
15:38:03 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5886
15:38:31 <dgilmore> so we really didnt discuss this during the week like we said we would
15:39:15 <tyll> Hi there
15:39:34 <dgilmore> I think we are in agreement that we need something
15:40:20 <tyll> iirc nirik prepared a wiki draft page, but I did not yet take a look
15:40:35 <nirik> yeah, I didn't do much more on it after that.
15:40:42 <dgilmore> https://fedoraproject.org/wiki/Urgent_updates_policy
15:40:54 <nirik> there' some questions there we could discuss...
15:41:10 <dgilmore> maybe we should leave this for the end.
15:41:19 <dgilmore> I really do not want to go over an hour
15:41:24 <nirik> alright.
15:41:35 <dgilmore> I have a crap load I need to get done before 3pm tomorrow
15:41:42 <tyll> #link https://fedoraproject.org/wiki/Urgent_updates_policy
15:42:15 <tyll> #info nothing new
15:42:52 <dgilmore> I am kinda skipping over soem tickets were afaik there is nothing new
15:43:00 <nirik> sounds fine
15:43:01 <dgilmore> #topic #5775 switch to partitioned image for EC2; remove editing of menu.lst from image processing
15:43:07 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5775
15:43:21 <dgilmore> so this is really out of our hands
15:43:36 <dgilmore> oddshock's upload service is supposed to be doing this
15:43:42 <dgilmore> we just need to deploy it
15:43:50 <dgilmore> nirik: any idea where that is at?
15:43:57 <dgilmore> we really need it sorted for Beta
15:44:02 <nirik> there was some issues with x8664 or something...
15:44:09 <nirik> he was working on it... not sure current status
15:44:16 <dgilmore> okay
15:44:41 <dgilmore> I know ill have to setup the fedora account credentials in ansible-private
15:44:50 * nirik nods
15:44:50 * masta looks in
15:45:00 <masta> hey guys
15:46:50 <nirik> hey masta
15:46:56 <dgilmore> #topic #5963 Orphaned vulnerable packages in EPEL
15:47:02 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5963
15:47:31 <dgilmore> I guess we should just go through and retire them all
15:47:53 <tyll> I can do this
15:47:57 <dgilmore> but we do need to be sure of breakages
15:48:14 <nirik> I think we need to notify before we do things.
15:48:22 <dgilmore> nirik: yeah
15:48:27 <nirik> There's been a number of folks posting to epel-devel about packages disappearing.
15:48:31 <tyll> there was a notification on epel-devel when the ticket was filed
15:48:34 <dgilmore> something like we do when retireing packages in fedora
15:48:49 <nirik> dgilmore: right
15:49:02 <dgilmore> nirik: they were all packages incorrectly retired afaik
15:49:19 <nirik> do we have a list of them? should we unretire them?
15:49:21 <dgilmore> and with process fixes they got retired right
15:49:37 <dgilmore> I do not know we have a list of them all
15:49:44 <dgilmore> but I am sure we could get it
15:49:44 <tyll> nirik: they were retired in pkgdb but not blocked in koji
15:50:08 <tyll> now they are also blocked since we now block every retired package when buildbranched/rawhide runs
15:50:38 <nirik> for example someone was asking today about libgeotiff
15:50:49 <nirik> tyll: but that wouldn't affect epel?
15:51:27 <tyll> nirik: yes, it does, it does this for all branches that we allow packages to be retired
15:51:58 <tyll> nirik: it is just in these scripts to make sure everything is blocked before they run
15:51:58 <nirik> ah... that seems not ideal
15:52:24 <tyll> otherwise we would just have a second (or more) cron jobs that do the same
15:52:47 <dgilmore> nirik: not sure how it seems not ideal
15:53:00 <dgilmore> its acting on the maintainer retiring the package in pkgdb
15:53:02 <nirik> well, I guess they are retired...
15:53:37 <dgilmore> we may need to make some changes for epel
15:53:41 <nirik> I guess the problem was that some epel packages were orphaned, then retired, then blocked, without folks getting a chance to see what would be broken or if they wanted to take them
15:54:03 <dgilmore> to untag all builds and unblock in -build incase it moved to rhel
15:54:34 <tyll> nirik: libgeotiff was retired before we got pkgdb2
15:54:40 <dgilmore> nirik: yeah a lot of that was due to the maintainers not talking about what they are doing
15:54:42 <nirik> ok.
15:54:57 <dgilmore> things were left in zombieland
15:55:00 <tyll> also retiring is not something that is a privileged operation, everybody can do this for orphaned packages afaik
15:55:09 <dgilmore> we went and killed the zombies
15:55:20 <nirik> anyhow, for this ticket we should do the cycle like we do for fedora... 'these are going to be retired, these deps will be borken' ... wait a week... etc
15:55:35 <dgilmore> nirik: yeah for this one we should
15:55:38 <tyll> dgilmore: the build untagging and unblocking in the build tag is already happening for EPEL
15:56:55 <dgilmore> tyll: okay awesome :)
15:58:03 <tyll> I might be able to report dependencies for these packages this week
15:58:52 <dgilmore> #action tyll to work on reporting dependncies to the list for the packages listed in this ticket and follow the fedora orphaned packages practices
15:58:53 <tyll> but maybe I already did it once for all orphaned packages in one EPEL release
15:59:11 <nirik> thanks.
15:59:15 <dgilmore> tyll: if you wanted to do all orphaned I wouldnt object
15:59:39 <dgilmore> anything else?
15:59:42 <nirik> that would be ok with me too. ;)
16:00:02 <dgilmore> #topic Secondary Architectures updates
16:00:03 <tyll> dgilmore: ok, this was also discussed recently in EPSCo
16:00:04 <dgilmore> #topic Secondary Architectures update - ppc
16:00:14 <dgilmore> tyll: okay
16:00:19 <pbrobinson> So we have a test compose out for ppc
16:00:40 <dgilmore> \0/
16:00:44 <pbrobinson> awaiting IBM to feedback updates but intend to move to RC and possibly to Alpha this week
16:01:14 <pbrobinson> I now have working ability to sign secondary arches which should make things a little bit more straight forward
16:01:17 <dgilmore> we can only ship Release Tues-Thurs
16:01:25 <pbrobinson> OK
16:01:35 <dgilmore> ideally Tuesdays
16:01:38 <nirik> pbrobinson: did you get a chance to try those nfs mounts?
16:01:59 <pbrobinson> nirik: I was going to ask you which host/ip I should be using for the mount point :)
16:02:04 <nirik> let me know if they don't work. Sometimes the netapp is anoying and you have to flush it's export cache.
16:02:17 <nirik> ah, can get that for you.
16:02:42 <pbrobinson> nirik: yea, if you could ping me that I can test :)
16:03:59 <pbrobinson> I think there's a few issues with PPC but it's not looking too bad
16:04:13 <pbrobinson> sharkcz would know more on the package delta
16:04:44 <sharkcz> nothing major it seems - http://ppc.koji.fedoraproject.org/koji/serverstatus
16:05:15 <pbrobinson> sharkcz: cool, I'm doing a signing run as we speak
16:05:26 <sharkcz> pbrobinson: perfect :-)
16:05:38 <dgilmore> anything more ppc?
16:05:49 <pbrobinson> not from me
16:05:50 <sharkcz> no
16:05:52 <dgilmore> #topic Secondary Architectures update - s390
16:06:09 <dgilmore> sharkcz: hows the world according to s390
16:06:34 <sharkcz> I got first compose built with a hacked anaconda (broken deps in anaconda-gui), waiting for an official build
16:06:57 <dgilmore> okay, thats something
16:07:49 <sharkcz> packagewise we are 'older': 131, 'local_only': 13, 'remote_only': 995, 'same': 14618 - which I consider very well
16:07:59 <dgilmore> not too bad
16:08:44 <sharkcz> few perl modules still not rebuilt for the ABI, some staff needs vtk (same on ppc), and couple more
16:09:05 <sharkcz> so considered good :-)
16:09:21 <dgilmore> okay, so are you hoping to skip Alpha and ship Beta?
16:09:32 <sharkcz> yes
16:09:42 <dgilmore> okay
16:09:49 <dgilmore> anything else?
16:09:57 <sharkcz> nope
16:10:02 <dgilmore> #topic Secondary Architectures update - arm
16:10:12 <pbrobinson> so we're looking very good here as well
16:10:25 <pbrobinson> will likely do a RC tomorrow
16:10:34 <pbrobinson> once I'm up to date on package signing
16:10:49 <pbrobinson> and see if there was any fallout from people testing over the weekend
16:10:58 <pbrobinson> package wise it's all in pretty good shape
16:11:29 <nirik> BTW, I have all the arm*:builvm:buildhw:buildppc builders updated and rebooted... finally got the arm SOCs this weekend...
16:11:51 <pbrobinson> cool
16:12:01 <dgilmore> nirik: awesome
16:12:16 <nirik> and all the arm SOCs are working again...
16:12:26 <nirik> they sometimes get in a weird state that needs the chassis rebooted.
16:12:47 <dgilmore> :(
16:13:15 <nirik> (they see their 300GB drive as 8MB... which isn't nice :)
16:13:43 <nirik> we need to get the aarch64's ansibled sometime
16:14:12 <dgilmore> we need to install Alpha on them  :)
16:14:19 <nirik> yeah.
16:14:58 <masta> I think they are on pxe, should be doable
16:15:25 <dgilmore> It is doable
16:15:47 <pbrobinson> nirik: would be happy to work with you on the aarch64 ones
16:16:04 <nirik> cool. :)
16:16:21 <dgilmore> we need to move all secondary arch boxes to ansible
16:16:28 <dgilmore> fully ansibilised
16:16:32 <dgilmore> with fas for auth
16:16:33 <nirik> yep.
16:16:36 <masta> yeah, me too... happy to get involved on the builders
16:16:37 <pbrobinson> nirik: and ppc64*
16:16:58 <nirik> yeah. :)
16:18:00 <nirik> what else did we have? ;)
16:18:15 <dgilmore> s390 ;)
16:19:21 <dgilmore> okay
16:19:24 <dgilmore> #topic Open Floor
16:19:35 <dgilmore> I have one item
16:19:41 <nirik> did we want to discuss the urgent updates thing a bit? or leave it for next/in ticket?
16:20:20 <dgilmore> I am going to Australia tomorrow evening for 2 months, with a short jaunt to FUDCon LATAM
16:20:24 <dgilmore> I would like to move the meeting two hours earlier
16:20:47 <nirik> thats pretty early for me, but I can try and make it.
16:21:02 <bochecha> I'd be happy with that time (in fact, happier than with the current time)
16:21:04 <sharkcz> works for me
16:21:47 <masta> That is a bit early, but not unreasonably so. works for me.
16:21:59 <dgilmore> as it is 2:20am in australia right now
16:22:32 <tyll> I cannot attend if it is earlier, becaue I have to work then
16:23:01 <dgilmore> tyll: okay.
16:23:01 <dgilmore> I am trying to avoidfd having to be up half the night
16:23:26 * pbrobinson is find with the change
16:23:33 <masta> dgilmore: seems like a hard problem with folks all over the globe.
16:23:55 * pbrobinson runs
16:24:19 <dgilmore> tyll: is an hour earlier possible at all?
16:24:42 <tyll> dgilmore: only 30 minutes
16:24:53 <masta> dgilmore: I'd be fine with 8am east coast Australia, but that would likely exclude our EMEA friends.
16:25:16 <tyll> maybe we can start 1 hour earlier with secondary arches
16:25:18 <dgilmore> masta: yeah that would be midnight or 1am in europe
16:25:39 <tyll> I usually have not much to discuss about them
16:25:39 <dgilmore> Lets try that
16:26:34 <dgilmore> I will switch the agenda order around
16:26:41 <dgilmore> start an hour earlier
16:28:10 <dgilmore> okay
16:28:32 <dgilmore> we are getting to the hour
16:28:44 <tyll> btw. there is daylight changing time soon here
16:28:52 <tyll> at the end of the month
16:28:54 <dgilmore> lets talk about how to implement the fast updates repo on list
16:29:07 <nirik> ok
16:29:15 <dgilmore> tyll: yeah, there is no daylight savings in Queensland where I will be
16:29:15 <tyll> but I think it goes in the wrong direction and makes it worse again
16:29:28 <dgilmore> the other states in Australia that follow it just started it
16:30:59 <dgilmore> so lets talk about meeting time also
16:31:06 <dgilmore> see what can be worked out
16:32:31 <dgilmore> anything else?
16:32:37 <dgilmore> if not ill close up shop
16:32:43 <tyll> starting 2014-10-26 I will be in UTC+1 instead of UTC+2 so I won't be able to attend then
16:33:53 <masta> I might need help fixing my f21-secondary key, wont' accept my passphrase. Just a bit of help, and I think I'll be good to help secondaries where it counts... signed packages before composes
16:34:18 <masta> oef
16:35:40 <dgilmore> e about things like that
16:35:49 <dgilmore> masta: you can ping me anytime about things like that
16:36:00 <dgilmore> #endmeeting