Meeting started at 2024-09-18 17:00:18 UTC

The Meeting name is 'fedora-server'

Neverthjeless, let's wait a moment for everybody to show up.

I'll post the agenda in 2-3 minutes.

Today we have a smaller line-up. Unfortunately, jwhimpel awill probably not be able to attend. So we should focus on non-Ansible related topics.

!hello

Emmanuel Seyman (eseyman) - he / him / his

!hello

Steve Daley (mowest)

OK, Welcome everybody. I think we should start.

Hm, zodbot is a bit ponderous today

Anything to add?

Nothing from me.

Still haven't found time to do anything ansible-related. I only finished handling EPEL10 requests yesterday

ah, I"m too early

No, nothing to add

Yeah, let's start.

I haven't done anything with Ansible either.

I'm still sorting out our open actions. I think, our docs meeting page is complete for now. I still have to make a summarizing list.

I hope, nex weekl ...... :-)

I've downloaded F41 Beta. Will take it for a spin this weekedn

I've downloaded F41 Beta. Will take it for a spin this weekend

eseyman, again a bit too quick :-)

As we already know. nothing to discuss here today.

Like I said above, I hope to submit PRs soon

Yeah thanks.

Just a note from me:

I just started to create an Ansible script to create an advanced, fully functional mailservice, that I use for some times on my servers.

A kind of learning by doing.

I think, I need at least 2-3 weeks to complete it.

But hopefully, it will be useful.

I would like to do a similar project "learn by doing" using Ansible to setup on F41 beta, a webserver and static documentation site that I update for my intranet.

I haven't started that project, and currently know nothing about Ansible.

Well, learning by doing is sometimes quite effective.

Please, have a look at out "Setup a WEB Server for static pages".

Agreed

Please, have a look at our "Setup a WEB Server for static pages".

OK, let's switch.

I need to download and do a test install and configuration. I could at least start the download here. Anyone have a quick download link?

I always use https://torrent.fedoraproject.org/

Downloaded. I hope to install it and poke it with a big stick this weekend

I would like to give some information and ideas.

In terms of systematization, I would like to have a test plan and a test overview, such as CoreOS has with the test weeks. For an example, see:

I am still not completely convinced by our testing efforts. Although there's actually nothing to complain about. Our distribution is actually very stable and reliable. But this is perhaps not so much due to our testing. And we have not noticed some errors in the past, e.g. the changes due to the changed LVM defaults.

Therefore I would like to achieve more systematization and especially a closer integration into or with the general Fedora QA process.

There you will find a (complete?) list of tests and a link to a test description for each test.

However, we don't need this for one test day, but for the entire period from the first branched version to the last beta.

According to Adam, the creation of such a test page is quite complex. I have tried to reproduce the essential structure in a wiki page.

It's a list of tests with a link to a test description (/)currently mostly empty pages), who has taken over to do the testing and the results. And the build id, because we can't completely test every build but should ensure to have a reasonable systematic testing.

We have such pages as part of Fedora QA as well, e.g.

OK, that about my ideas and my perception. And sorry for the long text. Perhaps we can take the opportunity here to discuss and determine possible better approaches for our testing efforts. <EOT>

But unfortunately I still don't really understand this page and don't know where exactly we can or should enter what. The tests listed there are all automated, therefore it is unclear to me what we are supposed to test and to enter in the tables there? Instead, it is necessary to observe and evaluate the automarted test results, af course.

Once I found a QA page which listed really nicely in a row all (or at least a lot) tests for server to be performed in different variations, e.g. install on UEFI and MBR, or install on various raid systems. Unfortunately, I'm unable to find this page again. It would certainly be a good starting point for us. And perhaps this page could make a separate wiki page, as I have suggested, unnecessary.

Sorry for the long text!

I've been mainly concerned about automated testing. I think we could do more here.

Yes, that is also Adam's suggestion. A first step is to formulate tests.

I saw your discussion with adamw and I believe that I also agree that it would be best to use automated testing for as much as possible.

Are the tests all bash scripts, or is there some other automation system that is used to run the tests, and do we know where this is documented?

Peter Boy: yes, as of right now we have full coverage for the server tests, so manual testing is not required

the reason we have the wiki pages is we do not have full automated coverage for all validation tests, so the wiki provides an interface for entering manual test results and a view of manual and automated test results combined

e.g. if you look at https://fedoraproject.org/wiki/Test_Results:Fedora_41_Beta_1.1_Installation it maybe makes more sense

the automated tests are almost all run by openqa. https://openqa.fedoraproject.org/group_overview/1

and yes, the Installation page - and also the Base page, https://fedoraproject.org/wiki/Test_Results:Fedora_41_Branched_20240915.n.0_Base - have some tests that apply to Server also

adamw: Agreed. But what we obviously need is a test of the end product, right?The "smoke test" of the installation. Or should we just stop doing any testing at all?

the Base ones are fully automated, but some of the 'default boot and install' tests on the Installation page should be run manually

the openqa tests do that (and more). but it is good to have human testing of the deliverables written to real physical media on real physical systems. that's the "Default boot and install" table on the Installation page

it'd be great if wg members can help with that testing when we get to final candidates (and for future releases)

and yes, the Installation page - and also the Base page, https://fedoraproject.org/wiki/Test_Results:Fedora_41_Beta_1.1_Base - have some tests that apply to Server also

it is also of course good to test whatever workflows you have that *aren't* covered in the validation tests

but anything really important we should add to the criteria and add formal tests for

I'm in favor of the WG doing test boot and installs using the install media burned to a USB. I tend to do that anyway during beta and release canidates.

I'm in favor of the WG doing test boot and installs using the install media burned to a USB. I tend to do that anyway during beta and release candidates.

mowest: if you're doing that, you can fill out the 'usb bios' and/or 'usb uefi' columns in the default boot and install table rows for server dvd and server netinst, which would be really useful

you can edit the wiki page directly, or you can use the `relval` tool i wrote

`dnf install relval`, then `relval report-results` - it gives you a silly CLI for entering these results, makes it a bit safer than editing wikitext directly

`dnf install relval`, then `relval report-results` - it gives you a silly CLI for entering these results, makes it a bit safer and easier than editing wikitext directly

Good to know, thanks adamw

adamw: We have long been on the todo list to complete the tests, Web Server, NFS Server, etc. Mostly probably automatable.

yep, they should be

adamw: Could you briefly post the manual tests that you consider necessary on the https://fedoraproject.org/wiki/Server/QA_Manual_Testing_Overview page? And delete the items on the list that you think are superfluous?

as i mentioned when we chatted, the process should be 'define these as blocking roles in the prd/tech spec, then cover them in the release criteria, then write wiki tests cases and add them to the tables, then automate them'

as i mentioned when we chatted, the process should be 'define these as blocking roles in the prd/tech spec, then cover them in the release criteria, then write wiki test cases and add them to the tables, then automate them'

looking at it, https://docs.fedoraproject.org/en-US/server-working-group/docs/server-technical-specification/#_4_server_roles probably needs an overhaul - it is written around the old, rolekit-based definition of 'server roles'

oh, or maybe it isn't,

i was probably reading to ofast :D

Peter Boy: sure

so if that's the list of roles we want to include/support/block on, great, step 1 is done, step 2 is write test cases and release criteria

https://fedoraproject.org/wiki/Server/QA_Manual_Testing_Overview <-- oh wow you reinvented release validation :D

so that definitely shouldn't exist, we should fold anything important it covers into one of the existing pages

there is a whole bunch of automation and tooling around the 'official' wiki matrices, documented at https://fedoraproject.org/wiki/Wikitcms

man, i need to take all the 'modular' stuff out of that page again

so looking it over briefly, we don't explicitly test systemd-nspawn in the existing validation tests

all the other things are covered, more or less, though with the combinations done a bit differently. (validation testing is a huge combinatorial explosion, you cannot test every variable with every other variable, you'd go insane)

adamw: Yes, maybe that page is superfluous. But we need a list that we can work off and that we can monitor to check that everything necessary has been done.

Peter Boy: that's what the official matrices are for. i guess you were maybe missing that some of the tests are on the Installation and Base pages.

software RAID, virt testing, install with various media, and upgrade tests are all on the Installation page

we don't specifically require all of them on all editions/spins because (see above re combinatorial explosions)

adamw: Adam, yes. I probably didn't / we didn't find the right pages.

but note openqa does actually run all these tests on server, as it happens - it uses the server DVD

upgrade tests we do split out by edition as they obviously tend to vary, so there is a specific row for server upgrade tests

adamw: Adam, can you give us a link where I can find the list we need to work through?

at any given time it is https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test

that will always point to the page for the compose currently 'nominated' for validation testing, so right now it's a random f41 nightly. when there's a final RC it'll point to the page for that

there is also https://fedoraproject.org/wiki/Test_Results:Current_Summary , which uses wiki magic to combine *all* the pages (Installation, Base, Server, Desktop, Cloud)

Sorry, needed some time to read through the tables.

My problem here: These pages contain a lot of items regarding workstation, mate, etc. not important for us.

It's hard to identify tasks for server WG

well, i guess https://fedoraproject.org/wiki/Server/QA_Manual_Testing_Overview could just carry a link to the current Server, Base and Installation pages, and a list of tests that you consider important for the Server WG to run from them

Yeah, that could be a good start.

Well, we are running out of time.

sorry to take over :D

Proposal: I will try to summerize our discussion on mailing list and we can probalby continue tghefre?

Don't be. This is important

sounds good. please poke me if you don't hear from me as i don't always remember to check all the mailing lists

adamw: Be sure, I will poke and poke and ....

I would like to finally find a definite path how to handle our testing. And I would like to leave behind that unsystematically probing around.

OK. let's switch to next topic.

I wrote a propsal / draft at

I did a lot of editing of the hackmd doc.

I believe we should have some goals for the survey to focus our attention on the types of questions to ask. What goals do you see for the survey?

I would like to find out from the survey: 1. Who is the audience? and 2. Where would the audience like us to focus our future efforts?

b) What is the main area of use.

a) What is server used for: Hardware or VM

mowest: I agree. I am also interested in

a) What is server used onr: Hardware or VM

mowest: I agree. I am also interested in

b) What is the main area of use.

mowest: I agree. I am also interested in

b) What is the main area of use.

a) What is server used on: Hardware or VM

Okay, so I think my edits help with a), but we could tweak the questions to get a better read of b), and I'm unsure if we have the right questions to get "future direction" that I mentioned above.

Unfortunately, I couldn't read your comments and edited so var.

Unfortunately, I couldn't read your comments and editing so far.

Are they not showing up, or you haven't had time since I did them earlier today?

lack of time :-)

lack of time is understandable, I just finished a few hours ago.

lack of time is understandable, I just finished editing a few hours ago.

So, I propose we continue discussion on hackmd and make this topic the first next meeting?

Okay, I will add to the hackmd doc our goals at the top, because I think this is important to consider as we edit the questions.

OK, so we will stop here now, continue on hackmd and discuss this first next meeting.

Anything here to discuss?

No I'm good

Same'

OK, so let's close

Next week looks likely that I will be able to attend at this point.

Many thanks to everybody. It was a most important discussion today.

Many thanks to everybody. It was a most important discussion today.an d

Many thanks to everybody. It was a most important discussion today.