Meeting started at 2024-10-16 17:00:05 UTC

The Meeting name is 'fedora-server '

I'll post the agenda in 2-3 minutes.

As usual, let's wait a moment for everybody to show up.

Hello, people!

hi!

!hi Emmanuel

Sorry, but Fedora Accounts user 'Emmanuel' does not exist

John has said he won't be here today. Steve will be 15 minutes late.

Yes, John is helping to fight the devastations of the hurricane.

So I think we should start for now.

Did I forget something urgent!

??

Well, obviously not

don't think so

Announcement:

So we have one week when the times differ. I currently prepared out Fedora calendar entries to switch time on N0v 3, so we Europeans have to adapt at one Wednesday. We did that the last time, too. And most groups who switch their meeting times us the US so we have a minimum of conflicting meeting times. But may be we want to do it this time the other way round?

nd of summer time or daylight saving time is comming. In Eurpo it is the Oct. 27, in US the Nov.3

don't really care, one way or another

End of summer time or daylight saving time is coming. In Europe it is the Oct. 27, in US the Nov.3

So we have one week when the times differ. I currently prepared out Fedora calendar entries to switch time on N0v 3, so we Europeans have to adapt at one Wednesday. We did that the last time, too. And most groups who switch their meeting times us the US so we have a minimum of conflicting meeting times. But may be we want to do it this time the other way round?

I do neither. So let's change over with the US change. And we two have to change our routine an one Wednesday.

Discussion on mailing list

Well, it's not so much about f41 anymore. That's done. But we have to find a solution we want to stick on in future.

I've only been vaguely following this discussion.

For F41, I've tested postgres and mariadb. Everything I've tried to do works

Plus, I can now deploy databases and users in a matter of seconds, now

!hello

Steve Daley (mowest)

Hi Steve!

hey, Steve

We are just discussing out test procedures. But I guess you were busy with out poll.

Well, I tested the installation media and adjusted the documentation. Everything OK so far, too.

Installation media on x86'_64

From what I have seen, Adam, seems to believe that all important testing could just be automated, and there wouldn't need to be any manual testing, but I don't know how we decide what to test or how those automatic tests get written. That is above my understanding.

if it's important, I really want it automated

Well, Adam says, we need to test at least the installation media, the "smoke tests". That is definitely part of the QA test program and you can't automate testing hardware installation.

And we need to test our special procedures, or try to automate those tests, too.

Yes, I'm not keen to do the boring testing again and again.

Ok, I did test a standard install of F41, and it went off without any issues at all.

Did you test on an SBC / raspberry by chancd?

Did you test on an SBC / raspberry by chance?

I have not.

yes, same thing installing via an iso in Boxes but I'm not sure that's where the failures are expected to be

I have not tested on my SBCs

Regarding installation I think the issue is hardware detection and system configuration accordingly. You can't test that on VMs, I suppose.

Hm, I have some SBCs unused, here. So I'll try to test some.

it's the niche/unusual stuff that I expect to fail, not the standard stuff

Well., our issues with LVM change were on standard stuff. And the issues with software Raid some releases ago, too.

true

So I think, the smoke tests are really useful and needed.

Well. adamw is obviously busy elsewhere. So we should continue on mailing list.

hey, sorry

it helps to ping me

Hi Adam

what's up?

I guess I would like to see us develop a list of "smoke" tests that can't be automated then, so I know what to look for when I do a test install of a release candidate or beta. My homelab uses of Fedora Server just need a working install, then everything else that I do on top, my son has automated with ansible playbooks.

adamw: I answered on mailing list, unfortunately a bit late.

mowest: as much testing as possible should be automated. we should always do a human smoke test that the actual bits we're intending to ship boot on real hardware, that is on the Installation page.

mowest: this is the "Default boot and install" tests on the Installation page, https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test

technically the requirement is that they be tested on a real USB stick attached to a real computer, which theoretically we could also automate. but we haven't yet, so that's all on the squishy pink organic test systems. :P

adamw: Could you read my reply? I listed 3 steps or 3 kind of tasks. I took them of some of your mails. Do you agree with that? Or can you modify it?

mowest: "but I don't know how we decide what to test or how those automatic tests get written" - we decide what to test based on what is in the PRD and tech spec for each edition, mainly. the server tech spec is at https://docs.fedoraproject.org/en-US/server-working-group/docs/server-technical-specification/ . the release-blocking test cases are derived from that

Peter Boy: i'll have a look

mowest: the wiki test cases are written first, those are generally written to be read by a human. we then write the automated tests to cover those test cases, where we can.

adamw: I would like to have a kind if "verified and authorized list of which tests we should perform" hopefully from you, You are our expert.

I'm the fool to try to organize it :-)

so i think the mismatch here is, that's what the current validation test pages are *already meant to be*

but it feels like you're struggling with that, maybe because of how big they are, maybe because it's difficult to pick out the things that are Server-specific and need human involvement?

adamw: yes, the issue, not only for me, is, that the relevant parts are very scattered and difficult to find for someone who does not work with them on a daily basis.

okay. it also kinda sounds like there's an element of 'we want to test more stuff than is currently listed'?

honestly, as of *right now*, the only things in the current validation matrix that *aren't* covered by automation are https://fedoraproject.org/wiki/QA:Testcase_freeipa_replication_advanced and the "Default boot and install" tests on the Installation page, so...that's the list

if you feel like there's stuff that's important to test, but that is *not* covered in the Installation, Server or Base pages, then we should nail it down and add it

Manually, we only want to test what is necessary.

How do we cover the issues, that are probably introduced by one of the changes?

As an example, the LVM issue which took us a long time

our answer to testing Changes has historically always been "do a Test Day"

OK, that's what I wanted for a decade, so to speak

And evalutating the changes should be one topic on our to-do list.

yeah, this is something i try to do in QA, but having domain-specific review of the changes is also a good idea

somewhere around branch point or beta freeze, look at the ChangeSet for the pending release and look into the details of any that seem scary

the ChangeSet can always be found at https://fedoraproject.org/wiki/Releases/41/ChangeSet (well, until we get around to redoing this whole process, maybe) - change '41' to be the release you're interested in. it gets updated periodically by the Change wrangler, which is amoloney, or sometimes me with my deputy Change wrangler hat

we usually wait until Beta is about to be released, which is way too late

yeah, there's no reason not to do it multiple times, and start earlier

Peter Boy: the LVM issue was a tricky one, i think there might be multiple lessons there. like 'make sure we specifically test Server on ARM written with the arm-installer script'. and 'maybe we should test more storage operations post-install'.

most of our storage testing at present is 'can we run an install to this type of storage successfully, and does it boot?' if we get to that point, we kinda give it a pass

we don't really have validation tests (human or manual) for, like, "take an installed system and resize stuff or move it around"

we don't really have validation tests (human or automated) for, like, "take an installed system and resize stuff or move it around"

we may have test cases like that lying around which were maybe used in test days in the past, but there's no mechanism to make people *run* them

adamw: yeah, the latter is one of our specific procedures. And that may become one topic of our task list.

so in this case we have https://fedoraproject.org/wiki/QA:Testcase_arm_image_deployment in the Installation matrix, which is supposed to be run for Final

but that test case just says "does writing it and booting it work"

so, this is a case where we might want to add a new test case, for instance

(or find an existing one and add it to the validation matrix somewhere)

at that point you also run into a bit of a 'combinatorial explosion' problem, though. say we write a test case that says "try resizing an LVM volume", do we apply it to *every* edition on every arch? that's a lot of testing. but if we don't, we might test this on Workstation on x86_64 and mark it as passed, and not catch that it was broken for Server on aarch64. it's a tricky area :|

I learnt a lot today, thanks to adamw . But, guys, we are running out of time. I try to do a new Summary on mailing list and we should continue on mailing list and next meeting.

yeah, sorry for rambling a lot :)

i have a bunch of stuff on my plate atm but i will try and find some time to look at some of the actionable stuff we talked about at least

maybe some pointers from the Server page to the relevant tests on Installation and Base would help. and the 'filtered download table' thing

adamw: the later would really help a lot!!!!!

honestly, as of _right now_, the only things in the current validation matrix that _aren't_ covered by automation are https://fedoraproject.org/wiki/QA:Testcase\_freeipa\_replication\_advanced and the "Default boot and install" and "AArch64 disk images" tests on the Installation page, so...that's the list

there's actually a crazy wiki thing i could do to show the tables from the other pages in the Server page - this is how the Summary page is done - but I'm a bit afraid that might cause weird effects so I'm not sure I want to do it :D

there's actually a crazy wiki thing i could do to show the relevant tables from the other pages in the Server page - this is how the Summary page is done - but I'm a bit afraid that might cause weird effects so I'm not sure I want to do it :D

We have to talk about the poll now. So ;:

adamw: Thanks a lot. We should have this discussion earlier and would have saved a lot of time!

ink https://pagure.io/fedora-server/issue/145

I have done the following: 1. Requested access to the Fedora Limesurvey instance through Gitlab. 2. Created the draft of our survey in my free account of Limesurvey [HERE](https://mowest.limesurvey.net/778249?lang=en)

that draft looks quite good

I think, the draft is perfect and we should use it unchanged.

did you mean to link this with the bot?

Unfortunately, to see the "draft" in my free Limesurvey, I'm limited to just 25 responses, so if more than the three of us look at it, I will run out of responses. But I believe I will be able to export and import it into our Fedora instance once I have access.

maybe add an "Other" option for "How are you using?"

it's the kind of question people might have unexpected responses to

other than that, looks great

Yeah, other is a good idea here

We have that for any other question with a list of alternatives.

I could do that, I didn't for the following reasons: We were using that question to get more of idea of what kind of users we have of Fedora Server, Homelab/hobby users, Small to Medium Business for business critical usage, Enterprise users and didn't want to have too many options. Secondly, I would have to figure out how to add an open ended "Other" that allows comments without adding comments to the other choices in Limesurvey, there may be a way, but I didn't see it.

For example the "Where do you run VM?" I couldn't figure out a way to give the Other a free form text field.

If I add "comment" field to Other in that question, it adds it for all of the choices which isn't really needed for the other choices.

I can do some more research to see if that is possible, because I would like to add that to two of the other questions.

mowest: Yeah, I agree, a kind of "forced choice". We may add "something else" without an open answer. Just to allow everyone to answer this question.

Peter Boy: I know I can do that in Limesurvey

OK, so we agree to add an option other without a free text field? Or do we want a real forced choice. I'm undecided.

I like the forced choice, but let me see if I can add an "Other" option with a text field without adding a text field to the others.

we are way over our time.

Proposal: I also write a summary on the mailing list and we continue next meeting.

Great let's say good bye for today. Thanks for all your work.

Make use of Fedora Server Channel as well, I'm often watching in there.

OK, I'll doi.

Bye everybody and thanks for coming!!!