<@tflink:fedora.im>
16:30:42
!startmeeting fedora-ai-ml-sig
<@meetbot:fedora.im>
16:30:43
Meeting started at 2026-07-02 16:30:42 UTC
<@meetbot:fedora.im>
16:30:43
The Meeting name is 'fedora-ai-ml-sig'
<@tflink:fedora.im>
16:30:47
!hi
<@zodbot:fedora.im>
16:30:48
tflink: Tim Flink (tflink)
<@tflink:fedora.im>
16:31:04
!link https://board.net/p/fedora-aiml-sig-meeting-agenda
<@tflink:fedora.im>
16:32:17
who all's here for some meeting fun?
<@jflory7:fedora.im>
16:32:41
!hi
<@zodbot:fedora.im>
16:32:43
Justin Wheeler: Justin Wheeler (jflory7) - he / him / his
<@xanderlent:fedora.im>
16:32:47
!hi
<@jflory7:fedora.im>
16:32:47
Howdy! o/
<@zodbot:fedora.im>
16:32:48
Alexander Lent: Alexander Lent (xanderlent) - he / him / his
<@trix:fedora.im>
16:34:01
i guess i can slow roll some status and get to the important stuff at the end.
<@tflink:fedora.im>
16:34:53
ok, let's get started
<@trix:fedora.im>
16:34:57
we are closing in on branching. i am looking at if ollama could be updated.
<@trix:fedora.im>
16:35:06
short answer is no.
<@tflink:fedora.im>
16:35:11
!topic Packaging updates
<@trix:fedora.im>
16:35:53
it depends on system llama-cpp and from what i can tell chasing the updates to 0.30.x llama-cpp is not stable.
<@trix:fedora.im>
16:36:08
nor does ollama work well with it.
<@jflory7:fedora.im>
16:36:11
I added discussion topics, late, but only if we have nothing else to cover
<@tflink:fedora.im>
16:36:16
not even the explicitly tagged version that ollama set?
<@trix:fedora.im>
16:36:30
correct.
<@tflink:fedora.im>
16:36:41
that's not a good sign :(
<@trix:fedora.im>
16:36:46
my guess is ollama is just going with the latest.
<@tflink:fedora.im>
16:37:13
!info ollama 0.30.x update is not likely to happen prior to F45 branch
<@trix:fedora.im>
16:37:16
so work for F46 will be to get closer to working set of llama-cpp.
<@trix:fedora.im>
16:38:18
another outcome is backends in ollama will go away, so ex/ vulkan in ollama will need to transition to llama-cpp and any other combo folks want to help with.
<@trix:fedora.im>
16:38:27
this is F46 .
<@trix:fedora.im>
16:38:39
so we got time.
<@trix:fedora.im>
16:39:06
poke me offline if you have a combo you want to support.
<@trix:fedora.im>
16:39:20
rocm and x86 i got.
<@tflink:fedora.im>
16:40:20
packaging help or more testing help?
<@trix:fedora.im>
16:40:28
both.
<@trix:fedora.im>
16:40:41
if for example you want llama.cpp on s390
<@tflink:fedora.im>
16:40:50
!info help to diversify the supported llama-cpp backends would be appreciated (both packaging and testing)
<@trix:fedora.im>
16:40:53
or opencl on ppc
<@jflory7:fedora.im>
16:41:09
Q: What Change proposals are going along with any of this, if any?
<@tflink:fedora.im>
16:41:30
this is post-F45, I think
<@trix:fedora.im>
16:41:35
yes.
<@tflink:fedora.im>
16:41:50
so, no change proposals at this time
<@trix:fedora.im>
16:41:58
for F45 nothing is changing.
<@jflory7:fedora.im>
16:42:11
How extensive is the work in transitioning with Vulkan? I am not super deep in the hardware side, but I'm wondering what the packaging effort looks like more from the inside
<@tflink:fedora.im>
16:42:15
I assume that ollama in F45 will continue to have a bundled llama-cpp
<@trix:fedora.im>
16:43:17
vulkan is a common interface so it has to work generally for all gpus. that is a lift if someone only has ex/ intel hw.
<@trix:fedora.im>
16:43:33
that gets to the testing.
<@jflory7:fedora.im>
16:43:39
Anyways, I am not raising a hand yet, but in Q3, I am planning to spend more time in this space, and I think trying to package some things might be a useful use of time and effort for me. Just trying to wrap my head around the various projects and upstreams I have watched for years, and how it ties into the deeper, low-level hardware integration work
<@jflory7:fedora.im>
16:44:16
Maybe some AI Test Days of some sort could be useful. Not likely now, I guess, but it is something we could possibly think about to drum up more interest and engagement
<@trix:fedora.im>
16:44:21
we do not want to oversell and undertest things.
<@jflory7:fedora.im>
16:44:34
Makes sense
<@tflink:fedora.im>
16:44:45
we have quite a bit of docs to write before we can host a test day, IMHO
<@jflory7:fedora.im>
16:44:50
AI Test Days sounds like something to keep in a backlog of ideas 🙂
<@tflink:fedora.im>
16:44:56
but that could be a good way to show how well things work
<@jflory7:fedora.im>
16:45:09
I figure you would know the ins and outs especially 🙂
<@tflink:fedora.im>
16:45:28
anything else on packaging updates or should we move on?
<@jflory7:fedora.im>
16:45:35
Or not, and have hardware feedback to capture and share 😄
<@trix:fedora.im>
16:45:38
that's a good lead into F45 pretesting topic i had.
<@jflory7:fedora.im>
16:45:46
tflink: I'm just digging for details, nothing urgent for meeting discussion. Can proceed
<@tflink:fedora.im>
16:45:58
!topic F45 Testing plans
<@trix:fedora.im>
16:47:47
as packages stop getting updates and only new packages coming in. i will be shift to testing ROCm and its integrations with ai bits and other things like blender. folks that need me to look at things poke me. i would like to know what other folks testing plans are so better testing.
<@tflink:fedora.im>
16:48:35
honestly, I'm just planning to help fill in gaps when I can and continue working more behind the scenes on getting better automated testing into place
<@trix:fedora.im>
16:49:27
that is maybe Justin Wheeler 's topic on gpu01 ?
<@tflink:fedora.im>
16:49:35
kinda, I suspect not
<@trix:fedora.im>
16:49:45
ok.
<@jflory7:fedora.im>
16:49:45
I have no testing plans of my own, but I did have an adjacent topic on `gpu01`
<@jflory7:fedora.im>
16:49:50
It is somewhat related but not directly
<@trix:fedora.im>
16:50:13
anyone else ? or we move on ?
<@tflink:fedora.im>
16:50:30
have you published the gaps in HW test coverage?
<@tflink:fedora.im>
16:50:34
for ROCm
<@trix:fedora.im>
16:50:45
not publically.
<@trix:fedora.im>
16:51:04
it comes down to who is testing and what hw we have.
<@trix:fedora.im>
16:51:33
i own a bunch of hw but not all the hw we build for ex/ gfx1153 is latest laptop hw. i don't have that.
<@tflink:fedora.im>
16:52:06
ok, just wasn't sure if you were looking for more volunteers there or not
<@trix:fedora.im>
16:52:17
yes, of course.
<@tflink:fedora.im>
16:52:21
anything else on this or should we move on?
<@trix:fedora.im>
16:52:31
can move on.
<@tflink:fedora.im>
16:52:54
Justin Wheeler: should we do your topics in order, then?
<@tflink:fedora.im>
16:53:04
or is there a priority that is different?
<@jflory7:fedora.im>
16:53:22
Nope, as written is fine 👍️
<@tflink:fedora.im>
16:53:30
!topic Action plan for `gpu01` and setting precedent/intention
<@jflory7:fedora.im>
16:53:52
OK, so!
<@jflory7:fedora.im>
16:54:00
I brought this up just as a discussion topic
<@jflory7:fedora.im>
16:54:18
I don't have clear outcomes or goals here, but I think that there is a sort of assumption that the `gpu01` is being managed by the AI/ML SIG.
<@jflory7:fedora.im>
16:54:24
I think both ideally and practically, that is true?
<@tflink:fedora.im>
16:54:40
honestly, that's unclear and a source of frustration
<@jflory7:fedora.im>
16:55:11
The slight misstep on realizing `gpu01` was intended for CI purposes when we had already started the RamaLama internship with this assumption months prior is one example of an "oops!"
<@jflory7:fedora.im>
16:55:29
I can imagine
<@jflory7:fedora.im>
16:55:36
It would help if Gordon Messmer were around?
<@jflory7:fedora.im>
16:55:44
Gordon Messmer: Ping, are you online right now?
<@tflink:fedora.im>
16:55:45
he said that he can't make it today
<@tflink:fedora.im>
16:55:50
his internet went out
<@jflory7:fedora.im>
16:55:59
Ahhh!
<@jflory7:fedora.im>
16:56:26
Then, some of this we can try to discuss without him, but obviously, we need him for some of the hardware allocation questions
<@jflory7:fedora.im>
16:56:40
But I think we can probably take some ownership
<@jflory7:fedora.im>
16:56:52
I think tflink is the person who even knows what all the pending hardware is better than anyone 🙂
<@jflory7:fedora.im>
16:57:15
So, we could create some docs in the public docs site, discuss the intended use cases, and put this somewhere public so it is less ambiguous
<@jflory7:fedora.im>
16:57:59
I am not sure what a Fedora Infra perspective might look like for sharing management of GPU nodes might look like. I was sorta wondering if we might want a `sysadmin-ai` group for SSH access and shell accounts on machines, but maybe that is getting ahead of myself
<@jflory7:fedora.im>
16:58:12
Documentation and stating ownership of the intention/plan seems like something in-scope for this group
<@jflory7:fedora.im>
16:58:14
WDYT?
<@tflink:fedora.im>
16:58:14
I didn't think it was ambiguous but I'm coming to learn that either miscommunication happens or memories are shorter than I'd like
<@jflory7:fedora.im>
16:58:34
Sometimes, things get lost in handovers and role changes and time.
<@jflory7:fedora.im>
16:58:45
It happens. But I think the public documentation under this SIG will be a nice place to start
<@tflink:fedora.im>
16:59:13
so you're asserting at least some ownership over the machine and its GPUs?
<@jflory7:fedora.im>
16:59:38
Errr, not me directly, but I think this SIG should be the community liaison with how this donated hardware arranged by y'all is used.
<@jflory7:fedora.im>
16:59:42
It feels like an even exchange.
<@jflory7:fedora.im>
17:00:01
We need to discuss this more with others, but at least documenting what has taken place in a public format will help in organizing a conversation.
<@jflory7:fedora.im>
17:00:31
I'd like people to come and chat here with ideas on what to do on these GPU tickets, especially if we are intending to use them for CI specifically, because it sort of appears to me that this is not universally understood
<@jflory7:fedora.im>
17:00:43
I'm not sure if my vision is just mine, or if it makes sense to others here too
<@jflory7:fedora.im>
17:01:00
Obviously, this is all racked in a RH DC somewhere and we need to work with Fedora Infra on things
<@jflory7:fedora.im>
17:01:14
But we can be delegated authorities on the actual utilization
<@jflory7:fedora.im>
17:02:24
Does this make sense to people, though? This is the main goal of the discussion, I guess. I implied in the Etherpad that if it feels like a useful discussion to move forward, I can get this into a ticket for later discussion
<@jflory7:fedora.im>
17:02:35
I know we will need Gordon involved there too
<@trix:fedora.im>
17:03:38
speaking a bit for AMD. when we ask AMD for hw for Fedora, a justification and entended use is part of the request. we need to show that use.
<@trix:fedora.im>
17:04:15
Tim and I both work for AMD and we both ask for hw. Sooo if you want some hw, poke me offline.
<@tflink:fedora.im>
17:04:28
that is the minimum, yes
<@tflink:fedora.im>
17:05:11
I'm running out of energy to fight this anymore
<@jflory7:fedora.im>
17:05:15
We need to have some public instrument to make obligations like this more transparent
<@jflory7:fedora.im>
17:05:23
And I don't want y'all to feel like it is so much of a fight 😕
<@jflory7:fedora.im>
17:05:34
There is some RH/AMD stuff which I think we can figure out offline in other spaces.
<@tflink:fedora.im>
17:05:40
fight, takeover ... there's a difference?
<@jflory7:fedora.im>
17:05:55
For the scope of this SIG though, I think inventorying the hardware and designating some use cases and scope are all things which will help.
<@jflory7:fedora.im>
17:06:20
To me, it feels like an appropriate delegation of authority for the SIG, which has largely facilitated the hardware into our environment.
<@jflory7:fedora.im>
17:06:45
So, let's split this out into two parts, one of which is more corporate bureaucracy headaches, and another about trying to capture what is out there somewhere, whether it is racked or unracked.
<@jflory7:fedora.im>
17:06:51
Does that seem like a fair next step for the SIG?
<@jflory7:fedora.im>
17:07:13
I can Meetbot-itize some of this, but I was trying to get a sense of "useful" or "not useful" from folks
<@jflory7:fedora.im>
17:07:19
I still feel a little new here 🙂
<@tflink:fedora.im>
17:08:35
I don't have much more to say at this point
<@trix:fedora.im>
17:08:57
if would be helpful if Gordon Messmer was here. we can move on.
<@jflory7:fedora.im>
17:09:05
OK, hold tight for a moment
<@jflory7:fedora.im>
17:09:14
I will log this in meetbot, and then we can move forward
<@jflory7:fedora.im>
17:09:18
and I will open that ticket
<@jflory7:fedora.im>
17:09:59
!info Context shared about the long-arc of AMD members of the SIG facilitating the donation of AMD hardware into Fedora data centers. This is an awesome collaboration, but it has come with a lot of bureaucratic pain points.
<@jflory7:fedora.im>
17:10:40
!halp There is frustration and some burnout over this topic, because it feels like it is difficult to get things done. More time needs to be spent on how to make collaborations like this easier and less painful. Some of this involves Red Hat and AMD specifically, and is less relevant for the SIG.
<@jflory7:fedora.im>
17:11:44
!info There is sufficient scope for this SIG to be the delegated authority for the intended allocation and use of this hardware, as it is racked into our infrastructure. Documenting what exists and was donated successfully, racked or not, will help shape community conversations around the use of this hardware, while also ensuring that obligatory requirements for the donations are met (i.e., particularly around automated testing).
<@jflory7:fedora.im>
17:12:05
!info @jflory7 will open a ticket in `ai-ml/tickets` to organize a discussion, together with some of the other docs-related work
<@jflory7:fedora.im>
17:12:54
!action @jflory7 Open a new `ai-ml/tickets` issue to propose a new "Hardware" module for the docs site, which documents the hardware strategically made available by members of this SIG for Fedora Infra.
<@jflory7:fedora.im>
17:12:56
[eof]
<@jflory7:fedora.im>
17:13:04
Thanks, just wanted to write some of that down
<@jflory7:fedora.im>
17:13:17
Let me know if any of my summaries were incorrect or incomplete
<@jflory7:fedora.im>
17:13:20
Next topic works for me
<@tflink:fedora.im>
17:13:38
it's your topic anyways
<@tflink:fedora.im>
17:13:39
!topic Skills Library curation and review
<@jflory7:fedora.im>
17:13:55
!link https://forge.fedoraproject.org/ai-ml/skills-library
<@jflory7:fedora.im>
17:14:05
!link https://forge.fedoraproject.org/ai-ml/skills-library/pulls/8
<@jflory7:fedora.im>
17:14:10
I see Clément Verna already merged this 🙂
<@jflory7:fedora.im>
17:14:43
But the point I was going to bring up here is that I would be interested in forming a "sub-team" of people in the SIG who want to spend time on creating, reviewing, and curating shared AI skills for the Fedora community
<@jflory7:fedora.im>
17:15:17
It is both a personal interest as I am seeing how the AI stuff is showing up more in the Fedora community, but of course, also a work interest as I am getting more time in Q3 to spend time on AI in the Fedora community
<@jflory7:fedora.im>
17:15:36
Skills feel like a neat way to create self-documenting best practices for important skills, relevant for humans and for bots.
<@jflory7:fedora.im>
17:15:43
I was going to ask if anyone wanted to join in this effort?
<@jflory7:fedora.im>
17:16:06
I think Clément Verna is implicitly tagged on for this too, but I wanted to formalize it with a reviewer group in Forgejo, and leave room for others to participate too
<@jflory7:fedora.im>
17:16:11
Any thoughts?
<@jflory7:fedora.im>
17:16:16
Or questions?
<@tflink:fedora.im>
17:17:09
it looks like that format is meant for claude code?
<@tflink:fedora.im>
17:17:27
nvm, I'm catching up as I write
<@jflory7:fedora.im>
17:17:33
!link https://agentskills.io/specification
<@jflory7:fedora.im>
17:17:42
No, very much agent-neutral
<@jflory7:fedora.im>
17:18:00
I think that is the whole appeal of our emerging library, TBH.
<@jflory7:fedora.im>
17:18:45
Also, I would love feedback on this idea… https://forge.fedoraproject.org/ai-ml/skills-library/issues/3
<@jflory7:fedora.im>
17:18:55
I am trying to think if I am going in the right direction at all with this approach
<@jflory7:fedora.im>
17:19:01
Feedback wanted 😛
<@jflory7:fedora.im>
17:19:32
I know Ankur Sinha (FranciscoD@fedora) (he/him/his) had a question about the same scenario which inspired me to open that issue
<@jflory7:fedora.im>
17:20:40
I'll Meetbot-itize while people think/process
<@jflory7:fedora.im>
17:21:36
!info There is growing interest in the topic of "AI skills." One of the cool things we have started to do is curate a library of skills following the Agent Skills specification, to create guided skills for AI agents from any LLM/model that can be collaborated on, improved, and shared with other Fedora community contributors.
<@jflory7:fedora.im>
17:22:08
!halp @jflory7 wants to start a "sub-team" in the SIG of "Skills Reviewers," who would be responsible for helping to create, review, and curate AI skills in the skills library repo.
<@jflory7:fedora.im>
17:22:58
!halp @jflory7 really wants to know whether the idea of installing an AI skills git submodule in an existing git repo for another project is a sensible approach, or if this is bad for unknown reasons. See `ai-ml/skills-library#3` for details.
<@jflory7:fedora.im>
17:23:44
!info @cverna is likely on board for this, given recent engagement and conversations. But we want to grow this group larger! Available for questions in #ai-ml:fedoraproject.org, just tag @jflory7:fedora.im and Clément Verna.
<@jflory7:fedora.im>
17:24:11
I guess this is all I have. An open call to participate, and I would love feedback from anyone to tell me whether my idea for better version control-backed skills is sensible or not.
<@tflink:fedora.im>
17:25:55
I don't understand the format well enough to say anything intelligent about this right now
<@tflink:fedora.im>
17:27:02
anything else on this topic?
<@jflory7:fedora.im>
17:27:03
Anyone who wants to help learn or figure that out with me, the door is open 🙂
<@tflink:fedora.im>
17:27:07
we only have 3 minutes left
<@jflory7:fedora.im>
17:27:22
But I'll try and formalize that team/review process structure more as a SIG process
<@jflory7:fedora.im>
17:27:31
Doesn't seem like anyone is meaningfully opposed
<@trix:fedora.im>
17:27:54
less is more.
<@tflink:fedora.im>
17:28:33
since we're pretty much out of time, let's move on
<@tflink:fedora.im>
17:28:42
!topic Next meeting chair
<@jflory7:fedora.im>
17:29:04
I would be happy to do the next meeting on July 16th. I'll be in PTO for the one on July 30th
<@tflink:fedora.im>
17:29:28
go for it
<@jflory7:fedora.im>
17:29:55
!info @jflory7 will chair 2026-06-16 meeting
<@jflory7:fedora.im>
17:30:07
Errr
<@jflory7:fedora.im>
17:30:13
!info @jflory7 will chair 2026-07-16 meeting
<@jflory7:fedora.im>
17:30:19
!halp Chair wanted for 2026-07-30 meeting
<@tflink:fedora.im>
17:30:25
time machine invented :)
<@jflory7:fedora.im>
17:30:36
Hah. I would rather not go back to the month before Flock please 😛
<@tflink:fedora.im>
17:31:01
!topic open floor
<@tflink:fedora.im>
17:31:16
anything else that folks want to bring up? otherwise, we're out of time and I'll close out the meeting
<@jflory7:fedora.im>
17:31:24
That's good for me
<@trix:fedora.im>
17:31:30
thanks all.
<@jflory7:fedora.im>
17:31:30
I have some action items to open some new tickets 👍️
<@tflink:fedora.im>
17:31:37
thanks for coming, everyone
<@tflink:fedora.im>
17:31:43
!endmeeting