Meeting started at 2024-08-15 16:30:57 UTC

The Meeting name is 'fedora ai-ml'

Hello 😊

!hi

Jonathan Steffan (jsteffan)

!hi

Tim Flink (tflink)

!hi

Mohammadreza Hendiani (man2dev)

let's get started, there's plenty on the agenda

tflink to change meeting name, move calendar from google to fedora, clarify that future meetings will all be on matrix unless otherwise scheduled

this is done as far as I know, if I missed something please let me know

Didn't we agree to this already

yes, the action item was on doing the changes

tflink to create ai-ml-sig pagure repo

this is also done

tflink to file ticket about clarifying whether the existing ai-ml FAS groups map to dist-gi

I added one more

this is also done and completed to the best of my knowledge

cool, if there are any others, please add them to the ticket

I think that's all of the previous meeting followup. anything else?

I think that's not we can close it is somthing is found we can reopen it

that list looks complete based on my recollection

That's it*

So are we closing the Issue?

What's this chatbot

At least I can multitask right now haha

yeah, we can close the issue.

moving on

You are doing two meetings? 😂

daMaestro: this is your item, take it away

lol yes, It's a town down lecture from google atm

**top down

okiedokie -- so this is half baked so don't jump in with questions too quickly because i don't have answers

what's the oven temp set too?? 350??

back in the day i had created searchfedora that was based on datapark search and it only indexed fedora resources. it was nice because when i wanted to find something fedora related i could do it fairly easily. that is long since something of the past as search engines got good. like really good. well... now they suck again

i can't find anything via goofoo about fedora anymore, so now i have bookmarks (fine, but grr)

yes so I have a suggestion, if you index the up to date fedora documentation you can use a technique called R-A-G (Retrieval automated generation) to generate in context responses from a chatbot that is going to access fedora embeddings documentation, and it's a specialized chatbot to give answers right?

I have a demonstration and code base of wizard of Oz chatbot acting in character using R-A-G I will give you a link so you can look up the code

it's a wizard of Oz dataset

* consider putting a web interface in front of it for a new fedorasearch

so i wondered... could we take the open source llm from instructlab and fine tune it so we could:

* talk to the fedora documentation and policies

* ask it questions like "how do i become an ambassador"

* ask it questions like "what steps do i need to take to review a package"

* ask it questions about programs on my compute (i.e. manpages, etc)

this was born of frustration i couldn't find valid docs on how to become an ambassador... and now living out of my bookmarks for common stuff i do

It's really difficult to determine what is better between fine tuning and RAG

Fine tuning requires the ludwig package

It's a good idea

iirc, instructlab is designed to allow for fine tuning

allow for/enable

yeah, ignore my suggestion of the technique. i was suggesting using the open source model from ibm and seeing if we could turn it into a cool specialized agent that knows *all about* fedora and is promptengineered in a way that we get high quality hits about our stuff and not some hat someone is selling on the internet

I mean the documentation, the wiki and all of day's information are not complete. It's one thing if it was the arch wiki That's up to date. Maybe if we subsidize their data based on previous talks that have been done by Fedora members, but I'm not sure about the legality of any of this.

anything submitted to the wiki is under the fpcl, I think

yeah, a majority of the hard work is going to be getting the data, cleaning it, and validating it

which is pretty permissive

yes, FPCL or some other openly licensed content is all i'd be suggesting

the legality is okay these models are open source and you are free to modify such as Mistral 8x7B but it costs money to host

sure, we'd have to work out the details of what is legal and what we have the resources to do but I think the idea was to see if there was enough interest to start looking into the idea and work on it

things like the wiki (with weighting on staleness), discourse (?), ask fedora (?, at least we have "answers" there), rpm package meta (the description and scrape the first page of the url for the package), all %doc data from packages, manpages, and heavily weight on the official docs

Yeah, I know that everything Fedora Post is under an open license, even the videos I think are under a CC License. But I don't know, some people have major ethical issues with these sort of tools. Seems like the kind of things that needs to be discussed outside of the AI SIG as well, for perhaps in the Fedor discussions? And madtodon? ...

yeah that's fine

i'd propose we get a prototype working before wider discussion

theory and product are different discussions

my biggest concern is the budget for the GPU server

fedora budget is only 250k a year

theory will lead to bikeshedding, product will lead to problem solving

we can deal with the budget if/when we get there

i'm suggesting we create something that can be ran locally

it could end up being a "download this through podman desktop and run it locally"

yup, and if it's actually useful it could be considered to be hosted in the future

this is easy

the hosting isn't the interesting part :-)

I have prototypes just sitting there on dagshub

The way I see it, the problem is shipping the data. What if the end user decides to use such a tool? For example, we don't train the data, we just have a tool that trains based on the man pages...

yeah, that will be part of the question of "can we do this" but I agree that if we start the conversation with purely theory, it won't be terribly productive

my biggest problem is it requires a serious computer to make those embeddings

it took 95 GB of RAM to make embeddings for the last dataset

if I could figure out how to chunk it somehow

we're getting a bit off into the weeds here

So basically we ship the tokenized data set as well as the tool to train it such as Lama-CCP ehat have you and the actual compilation and making of the AI is done on device.

yeah agreed the system needs to be precompiled embeddings and it can be done on device

we do have some access to hardware, especially for one-off or occasional tasks. I'm not worried about getting that taken care of

that's cool

i *think* i'm proposing the sig has a pretrained fine tuned model and end-users just run it locally -- and of course we open source everything we did to train it

that's more or less what I had understood

are there folks interested in working on this? I'm interested but I don't have much time to offer ATM

what is temp of an easybake oven? ;-)

I have a fine tuning notebook already but I have to get a dataset I haven't sucessfully gotten fine tuning working, RAG is much easier in difficulty using langchain

easybake oven these days would have LED lights.. so 35C ?

for a Q:A dataset

instructlab might be really helpful here - there is a format for adding data to the model

yes, i'm trying not to suggest solutions yet but instructlab is what i'd use if i did it

I have never used instructlab but I checked it on github it's definitely open possibility

https://github.com/sarthakkaushik/LLM-Functional-Code/blob/main/Ludwig_%2B_DeepLearning_ai_Efficient_Fine_Tuning_for_Llama2_7b_on_a_Single_GPU.ipynb

daMaestro: it sounds like there is some interest if you're willing to do the cat herding but that's up to you. do you want to discuss this more here or move to matrix/discourse to get into more details?

I will say that RAG is easier than fine tuning

if there is interest beyond me thinking "that'd be cool and i'd use it", i could sharpen up a phase 1 test

I need to learn fine tuning

Guys, how about another way to address this issue? The actual problem is that the information isn't easily available, right?

it's easily available, but you have to know where to find it and you have to know if it's still valid or not

If approved by the community and legal, why not just ship the dataset so that anyone can use any other chatbot they want to ask their questions.

well the way that ludwig works it's not a dataset

it's a model on hugging face they pull it off hugging face for "fedora 40" lets say

and just Q-A the bot

I think that would be part of this, if it happens. daMaestro said that part of this would be making things available in an open source way

but we're definitely getting off into the weeds and we have 3 topics left to cover

any objection to moving farther discussion to post-meeting matrix or discourse?

okay moving on

i agree that a significant amount of the work would be packaging the data we have into a high quality resource, and that would be valuable for others too

I take silence as OK with moving on

anything else on this topic before moving on?

i'll see if i can put this idea on rails, we can move on

I just wanted to do a quick check-in on the F41 ai-ml related features to see if anything was needed

I know that there have been pytorch 2.4 builds in rawhide and F41 - I assume that is on track?

it was until rocm 6.2 update.

@F40 it's fully featured, I haven't seen anything that I haven't been able to do so far Torch/Tensorflow works on NVIDIA GPU's, I don't have an AMD ROCm card to test a tensorflow pipeline, and I am not sure how to do ROCm

ah, do you foresee problems or need help with getting it done?

see my topic later on testing.

ok, does that apply to the rocm 6.2 update as well?

yes.

ok, let's just move on then

just a note that the deadlines for feature completeness are coming up soon

I think it's august 27 or so

llvm18 has no landed yet and that was called out as a risk.

yeah, that's always an issue - the schedules don't line up well and llvm always drops late

and it seems like rawhide updates have not really happend this week.

i am _still_ waiting for rocsparse to land in the mirror and i built that 2-3 days ago.

there are usually a few hiccups when branch happens but I wouldn't expect it to be that long

i will be on pto next week so.. maybe someone can finish the stack ?

yeah, I imagine that jeremy or I can finish the builds if needed but we can sync up offline

is there a video i am missing some of hte conversation

a video? for what?

I am miss reading

it sounds like work is still needed for the F41 features and some of it is waiting on llvm18 which hasn't landed in rawhide yet

but we'll get it figured out

llvm17 was an issue in F40. .. even if we get the stack in there is going to be 2 full rebuilds to do. one for rawhide, one for F41 now.

both of which take a non-trivial amount of time

fun

right.

anyhow, shall we move on to the testing topic or is there more to discuss in meeting?

move on.

Tom Rix: this is your topic, take it away

i am looking for ideas on how to improve the testing to our packages, i _think_ it is some manual testing on my part and when i get busy, there is no testing.

anyone else testing our packages ?

I do as I have time

but that has lessened as of late :(

i only test cpu and nvidia when i'm asked or am actively participating in an update

Well we had a week of flock that slowed things down

Very rarely because most times it needs a python package that we don't have

what are we missing?

Are you talking about AI Python packages?

are we to the point where we need to create something like the kernel test tool for the test days? something that installs everything needed and then runs some suite of simple tests? that way it should be easier for us to recruit testers that have the right hardware

Or torch packages?

https://pagure.io/kernel-tests

whatever might be missing to test ai-ml packages. I think that trix is mostly talking about torch and rocm ATM, though

yes.

There are a lot I'll make an issue about it and add to it each time I find one

yeah, I think that a big part of the problem is that we don't have test cases or much documentation. not sure we could do exactly what the kernel test stuff does but I also think that containers would be our friend

is there a fedora bot thing that could at least build cpu torch and its dependencies.. then run some cpu torch example/tests ?

bot thing? I'm not sure I understand

https://docs.fedoraproject.org/en-US/ci/

there are package-specific testing mechanisms that run in EC2 cpu-only instances

a build bot... something that would just build-n-test..

I'm pretty sure packit does this

nothing that does both that I'm aware of unless you count packit+testing farm

Is this CI usable as a jenkins build system where I can just pull code from a repo and run it?

not quite - it's a bit more specific that that. it needs to be attached to a fedora package for the fedora ci setup

thank you

I tried making one a while back but saw the spec had changed and so I put it on the back burner.

https://github.com/Man2Dev/pytorch-vk

I found a weird issue and haven't tried fixing it yet

as much as I'd like to hash some of this out now, we are running out of time

yup.

if folks have ideas on testing poke my on the ai/ml channel.

Tom Rix: do you have at least enough information to look at for the time being? otherwise, we can talk about this more after the meeting

tflink: after meeting.

let's talk.

ok, moving on

Yes please

do @ rooms ping in matrix half hour before meeting as to encourage new members or members who are interested in the sig to join in

Warning, it might get annoying real fast.

if it's a once a week ping it's not bad

I don't think it hurts so long as we're talking about one or two pings

once every two weeks is fine

Also, I used the ICS today to maintain conciousness of the meeting

Also, I used the *.ics today to maintain conciousness of the meeting

it sounds like folks are broadly OK with a ping before meetings

I'd say we can move forward with it and re-consider it if people find it to be too much

any objections?

OK then

Yap

which bring us to

we're already 2 minutes over. are there any other topics which have to be discussed now?

Everyone in the AI/ML SiG has been really helpful. I think that limited group members limits capabilities more than anything, getting more people into AI/ML SiG would help a lot

having clear project work will help with that

yes

yeah, very true. I've never had good experience with "ok, go find something to do" when it comes to attracting new people. it works some times, but not all that frequently :)

Not everyone is a self starter

yep, exactly

if there's nothing else, I'll end the meeting in 2 minutes

sounds good!

thanks for coming, everyone. sorry for going over a little bit

I'll send out minutes shortly