Hi QA team,

@sumantrom and I have been talking over the last few weeks on how we can bring QA in to the change proposal process as early as possible, within reason. We have a few ideas, but need some fleshing out before we start a discussion about them, however one idea seemed straighforward enough to propose to QA, and that is to add a line in the Change Proposal template to optionally add a QA sponsor or point person or shepard. This would be similar to the FESCo shepard line and to get involved in the change proposal would be at QA's complete discretion.

The expectation is that QA would be willing to work with the change proposal owners who want to set up some test cases or plans and test days, and the QA person (or persons) would be associated with the change, but will not own the change.

Is this something you would be interested in doing? Im not sure what the uptake would be yet, but my next steps if I have your buy-in is:
* Update the change proposal template to include the <QA Shepard> point of contact.
* Send an email letting folks know that if they have a change they would like to propose, there is also an opportunity to work with QA and engage them in the proposal as an additional support for providing or validating tests/testing.

Please let me know your thoughts, if you have feedback you want to incorporate, or if you flat out dont think this would be a good idea :)

Thanks!
Aoife

= phenomenon =
GRUB2 supports several new md raid levels (0,4,5,6,10) and also lvm logical volumes as the /boot device. It would be nice to get some testing coverage for this, including combinations like root on lvm (no separate /boot) w/ md raid pvs.

After discovering https://bugzilla.redhat.com/show_bug.cgi?id=1221247 it seems reasonable we should test this more properly in the future. Currently we seem to have no test case about resizing partitions in manual part dialog. We probably should, and test it against at least a standard partition and a logical volume. Optionally, we can split standard partitions into ext4 and ntfs, to cover similar use cases as in our QA:Testcase_partitioning_guided_shrink .

We can ask to have a Quality team created on Discussion, which would be synced from FAS. QA members could then have a "Quality Team" (or similar) title displayed next to their name, a QA flair icon, etc. I think it would be nice. Instructions are here:
https://discussion.fedoraproject.org/t/fedora-accounts-discourse-group-sync-what-teams-need-to-know/82020

Does anyone want to handle that?

We need a test case written for checking whether media check failure is correctly presented on the screen when a boot media is corrupted. We currently don't have it and then regressions like bug 2246410 happen. See the bug for a description on how to create a corrupted media and how to check the failure.

The test case must be checked on bare metal, at least. Checking it in a VM might also be useful, especially with automation, but bare metal is the important part here.

Once the test case is written, also make sure it's included in our release validation matrices, and references/referenced from our release criterion.

Fedora Quality makes it much more obvious for people what our team does. We already have #quality channel on Matrix and #quality-team in Discussion (soon). Ubuntu has made the same move years ago.

Please see a relevant discussion here:
https://discussion.fedoraproject.org/t/rename-qa-team-to-quality-team/82416

We've captured some implementation thoughts in this Google Doc. A summary is below:

[RENAME] Any wiki pages and Fedora docs content where we used "QA".
* [RENAME] Changes and other program management docs where QA is cited
[ ] Any wiki pages and Fedora docs titles where we used "QA". This includes:
* [INVESTIGATE] Test case names - QA is a namespace for test case names, need to figure out if we can rename it and what it'll break (OpenQA reporting)
* [RENAME] Wiki templates
* [RENAME] SOPs
* [RENAME] Test days
* [RENAME] Lots of other pages under the QA category
* [RENAME] Quick Docs
[RENAME] QA calendar in Fedocal
[KEEP] "QA Contact" field in Bugzilla to "Quality Contact"
[RENAME] Domain name changes - QA dashboard, both in project title and URL, Blockerbugs
[INVESTIGATE] QA projects on Pagure - test on stg, talk to infra and pagure devs
[ ] QA mailing lists
* [KEEP] test* lists
* [TASK] Move qa-devel to Discussion, delete the list
[KEEP] #fedora-qa IRC channel
[RENAME] QA team meeting name and the meetbot template as well - don't change history, just future meetings
[RENAME] QA Telegram channel - figure out if the invite link is going to break
[INVESTIGATE] FAS groups: qa, qa-admin, qa-tools-sig, sysadmin-qa and maybe others. Also see this related ticket regarding the qa group. Investigate with infra.
[INVESTIGATE] qa-tools-sig packaging group in Pagure, FAS (see above), Bugzilla, mailing list, etc. Investigate with infra.
[RENAME] QA tag on Fedora Magazine, Fedora Community Blog
[TASK] Ask for a new logo for our wiki and dashboard

Anyone: Please extend this list if you find more places to rename.

As @roshi suggested in today's Fedora QA meeting, it'd be neat to have some kind of system for keeping track of validation tests that need running but haven't been run, and notifying us about them...somehow. This obviously overlaps with the testcase_stats subcommand of relval to some extent.

One rough thought I had: we could actually implement and/or use this in relvalconsumer - the fedmsg consumer that creates new release validation events - and have it include some short list of suggested tests to focus on in the announcement emails?

As noted by Christopher Beland on [https://www.redhat.com/archives/fedora-test-list/2009-June/msg00723.html fedora-test-list], evolution is near the top of most bugs filed by component in bugzilla. This ticket is intended to track drafting a debugging guide for evolution.

Similar guides are available at https://fedoraproject.org/wiki/Category:Debugging

Hey folks,

Keeping Fedora's long-term goal in mind, it might be a good idea to start blocking on a11y features. The packages are already in the compose and I would like to go ahead and propose that we block the basic functionality of these packages.

The packages in question will be
at-spi2-atk;https://fedoraproject.org/wiki/QA:Testcase_at_spi2_atk
at-spi2-core;https://fedoraproject.org/wiki/QA:Testcase_at-spi2-core
brltty;https://fedoraproject.org/wiki/QA:Testcase_brltty
orca;https://fedoraproject.org/wiki/QA:Testcase_orca
accerciser;https://fedoraproject.org/wiki/QA:Testcase_accerciser

This list might expand depending on how many people get involved with the a11y SIG in long run.

The a11y team and QA team ran a test week http://fedoraproject.org/wiki/Test_Day:2024-06-19_Fedora_41_A11Y

I will volunteer for release criteria writing when it comes to that.
Please let's discuss any pros and cons and concerns?

Hey Folks!

We have recently started testing features as developers build them in Rawhide. There are immense benefits for us to keep testing features and they will eventually come to us after the Branch. This is a good way to start finding bugs and figuring out scopes/test scenarios blocking the release. The Rawhide test days need some literature in the form of Wiki, to explain the existence of such in the first place.

The drafted wiki stands as https://fedoraproject.org/wiki/User:Sumantrom/Draft/QA:SOP_Rawhide_TestDay

The Rawhide Test Days will be using the same test day wiki template and use the same SOP of Test Day Management.

Contrary to what our current Cockpit test cases suggest, you can no longer log in as root:
https://bugzilla.redhat.com/show_bug.cgi?id=2269361

We should adjust the instructions. I think it affects this one:
https://fedoraproject.org/wiki/QA:Testcase_Server_cockpit_basic
and perhaps even this one?
https://fedoraproject.org/wiki/QA:Testcase_Server_cockpit_default

We probably need to say that in anaconda you must create a regular user (or add the user manually after server install).

We don't have an explicit test for doing a Secure Boot install. We should have one, for the most basic case (install to a system with a typical OEM SB configuration).

Beta is out now and we would want to publish the Heroes of Fedora. @coremodule previously you handled this. Do you intend to do it this time as well?
HoF is sadly the only way, we present/recognize the community members with a token of gratitude for the time, effort and commitment towards QA activities amongst other things in Fedora.
It's crucial that we keep true!

This ticket is all about proposing to better our recognition system. The idea is to have

a) Add more badges to our existing "quality" badge set
b) Revamp the old badges with some new design
c) Add relevant folks to have access to Fedora Badges front end
d) Create a Badge distribution framework - where contributors can go and claim their badge/ask for it (until such a time backend is fixed by Fedora Infra/CPE AKA @t0xic0der )

@nekonya3 , has agreed to help us with the goals. I spoke to her at Flock and will open badges tickets shortly!

We accepted https://bugzilla.redhat.com/show_bug.cgi?id=2241632 as an F39 Final blocker, but we have no formal test case for this. If we care about it so much, we probably should have one.

The test should be in https://fedoraproject.org/wiki/Category:Kickstart_test_cases and added to https://fedoraproject.org/wiki/Template:Installation_test_matrix#Kickstart . I'd call it something like QA:Testcase_kickstart_partial , I guess.

Then we should automate it in openQA, once it exists. I'll file a ticket for that too.

Hey folks,

The way we have scaled the test day model, we now have a lot of test days happening every week. The current test day wiki page which is something a lot of old folks look out for can still just hold one link .. is there a wiki trick by which we can redirect people to multiple different places?

If they aforementioned can't be done, would it be sensible to create a wiki page with a table and have all the happening test days updated there? That way we can use that wiki page in the /TOPIC bit of the IRC channel as well. Needless to say, that will mean we need to maintain another wiki page ( given we are not editing the current anymore in that case and whatever current points to gets edited instead).

@adamwill @kparal et all!

The page in question is https://fedoraproject.org/w/index.php?title=Test_Day:Current&redirect=no

During the F19 cycle, the anaconda text installer interface became a lot more capable; it now replicates almost all the functionality of the graphical UI, only really missing custom partitioning.

However, we still only have a single test case for it - https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text - which simply says 'run any old text install you like'. That's a bit weak.

We may want to re-evaluate the most sensible way to test the different installer interfaces; just having a single test case for each interface seems a bit of an odd approach. It seems more reasonable to run the functional test cases against multiple interfaces, though it may be unrealistic to try and run every test case against every possible arch/interface combination.

Along with the recent port from X.org to Wayland, anaconda switched its official remote desktop support from VNC to RDP.

We have release criteria, test cases, and openQA tests that require VNC to work and test that it does. We need to update all of those for this change.

Note that it seems there is no RDP equivalent of VNC's 'reverse connection' feature where you could have the system on which the install was running initiate the connection to the system from which you wanted to monitor it, which was useful for dealing with some network configurations where the server could not be reached from the outside. We will have to drop criteria and tests relating to that feature, I believe.

Tagging @jkonecny for info.

It is important that all the actions (or at least the majority of them) have their counter parts in OpenQA. Test it inside OpenQA.

Backspace is used to delete the latest character from the key buffer, but it does not get recorded when pressed together with a modifier. This should be fixed.

Sometimes people re-propose an older bug - either for a different milestone, or reopen a bug from an older release. When that happens, it seems that BlockerBugs App doesn't correct adjust Whiteboard. An example is here:
https://bugzilla.redhat.com/show_bug.cgi?id=1164492#c26

The bug was accepted in F21, and now proposed for F22 Beta. But AcceptedBlocker was not removed from the whiteboard.

I think we should clear all of AcceptedBlocker and RejectedBlocker from whiteboard in these cases, provided that the proposal has not been re-raised against an already blocking tracker bug.

A discussion ticket creation should automatically send a comment to Bugzilla, so that the developer/package maintainer and anyone watching knows about the discussion and can participate.

There are two different workflows.

• When the blocker is set through flags directly (e.g. Blocks:FXXBlocker), we will detect it in ~30 minutes, and then we will create the discussion ticket, and send the comment to Bugzilla.
• When a user uses BBA to propose the blocker, there is already a comment posted into Bugzilla ("User X proposed this as Y because Z"). It would be nice to add the discussion link to that very message, instead of having two separate comments. So this would mean creating the discussion ticket first, and then sending a comment to Bugzilla which contains both information together ("User X proposed this as Y because Z. You can vote on the proposal at URL").

Note: This idea came up in the blockerbugs discussion tickets feedback.

Bodhi updates are often not linked to accepted bugs. In that case, it's easy to miss that a blocker/FE fix is ready to be included/pushed. We can't do much about it, but can at least notify readers when the state is suspicious. If bugs are in ON_QA or VERIFIED state, but they don't have any linked update, that's suspicious. Let's add another alert section into /requests (we already have one for bug dependencies) which will list such bugs and ask the person to inspect them manually.

@adamwill

Moved from trac [[ https://fedorahosted.org/fedora-qa/ticket/313 | https://fedorahosted.org/fedora-qa/ticket/313 ]]:
"When bugs are marked as blocking existing proposed or accepted blocker bugs, they are not detected by the sync algorithm and thus are not detected as blocker/nth bugs

Current Example: ​[[ https://bugzilla.redhat.com/show_bug.cgi?id=852845 | 852845 ]] is blocking ​[[ https://bugzilla.redhat.com/show_bug.cgi?id=850775 | 850775 ]] which is an accepted blocker but doesn't show up on the blocker tracking page"

There was a sudden change in Bugzilla that limited anonymous queries to just 20 results. @tflink created a hotfix, which we applied in production, and on develop it is available as 0b612d2ed1. However, the issue is more complicated.

Bugzilla now returns max 20 results if you're anonymous, or optionally more, if you have a token (in ~/.config/python-bugzilla/bugzillarc, created e.g. with bugzilla login) and request a higher limit. However, if you're anonymous (e.g. the token expires), you again start getting max 20 results, without any warning. That means our approach in that code if len(last_query) == bugzilla_query_limit; then "this is probably not the last page" is flawed. If we created a token to be able to query faster, increased the requested limit, and then the token expired, we'd be getting fewer results than expected right with the first response and never tried a second page.

So we either need to come up with a better pagination implementation, or we need to agree that anonymous access and 20 responses for a call is enough for us and not worth the bother. In that case we should add a comment next to the BUGZILLA_QUERY_LIMIT = 20 line and say that this is not intended to be configurable and why.

This is also related to #184, switching to REST API might change things.

When voting for a proposed blocker, it is also possible to vote as 0Day +1 or PreviousRelease +1. However, the buglist view only considers Beta/FinalBlocker +1 votes, when showing the counts, and that's wrong. For example, in https://pagure.io/fedora-qa/blocker-review/issue/521 there was:

Current vote summary
0Day (+7, 0, -0)
+1 by catanzaro, mohanboddu, bcotton, kevin, lruzicka, geraldosimiao, ngompa

but https://qa.fedoraproject.org/blockerbugs/milestone/35/final/buglist showed +0, 0, -0 in the Review column.

Currently we only do a one-time sync of Bugzilla email and never update it. That's a problem for users who changed their email, or perhaps completely replaced their Bugzilla account with a new one (see #209 as an example). Add a button to the Propose form for which will allow users to re-link their Bugzilla account (unlink and link again), so that it can pull updated values.

I'd like to have two deployments on blockerbugs.stg:
a) one instance which uses staging services from stg.fp.o (so partner-bugzilla, bodhi.stg, pagure.stg). This one would be fully functional, i.e. automatically creating and updating discussion tickets, allowing blocker proposals, etc.
b) one instance which uses production services from fp.o (so bugzilla, bodhi, pagure). It would be "read-only", i.e. discussion tickets wouldn't be automatically created/updated, blocker proposals wouldn't be allowed, etc.

Instance A would allow us to test new or existing functionality, it would be our playground for testing. Instance B would help us verify that production data still display correctly (we could easily compare it to the real production instance). It would also answer the frequent question "should the stg instance run on stg data or prod data?" - the answer would be "let's have both" :-)

Of course the instances would run on a different URL, e.g. "qa.stg.fedoraproject.org/blockerbugs/" and "qa.stg.fedoraproject.org/blockerbugs-prod/".

In this ticket I'd like to identify the roadblocks which prevent this from happening. I currently thought of the following:
1. Is it possible to run the same application twice through WSGI? Of course on different URL paths.
2. Currently the app is hardcoded to read /etc/blockerbugs/settings.py. If we want to have two instances, they need to have a different config. We can provide a cmdline option/envvar to specify the config location. What is the best way to provide this to a WSGI app?
Note: Problems 1+2 might a non-issue, if we move blockerbugs to openshift, because running two completely separate instances might be very simple in there? Just a guess.
3. The config should be perhaps extended with READ_ONLY=True/False, that would make sure that even if we configure production URLs to the staging instance, a "write" operation is never attempted. It is not completely necessary at the moment, I think, but it could allow us e.g. listening to discussion ticket changes (and displaying their votes), but not updating them. Also this can be a way to disallow a blocker proposal (which we can't disallow any other way, at the moment, it seems).

What else I'm missing? Any other roadblocks, or implementation suggestions? Thanks.

@frantisekz @lbrabec @jskladan

Currently we're using Bugzilla's XMLRPC API through python-bugzilla. This API is deprecated and will be removed. Instead, the REST API should be used. python-bugzilla allows to use it, but our current queries seems incompatible with it and will require some changes (or fixes in python-bugzilla). Oraculum already uses this approach (REST API through python-bugzilla).

Figure out what needs to be changed and make the switch.

This change might also resolve performance issues mentioned in #183, i.e. allow us to do "quick" bugzilla syncs again (if we find it meaningful).

Once bugzilla starts emitting fedmsgs, replace current sync mechanism with a consumer for those messages.

https://lists.fedoraproject.org/pipermail/qa-devel/2014-December/001051.html

Along with #114, another case when Pagure issues should be closed (IMO) is if the proposed bug itself is closed. We don't treat closed bugs as 'proposed' or 'accepted' blockers elsewhere in the process (including the IRC template used for the synchronous blocker review meetings), so the async voting system shouldn't treat them as such either.

Of course, we should also be able to re-open the Pagure issue if the bug is re-opened...

Currently milestone updates in blockerbugs are done entirely manually through the admin web UI (usually by me). This is dull and sucky work and we should automate it.

There was some discussion in #114 , this is a separate ticket to spin it out. The way I think I'd conceive of this is just defining what the set of active and current trackers should be at change points, assuming that any tracker which does not yet exist in the app is automatically added, and any tracker not listed which is currently active gets deactivated:

1. At release N-1: current - N-Beta, active: N-Beta, N-Final, N+1-Beta, N+1-Final
2. At release N-Beta: current - N-Final, active: N-Final, N+1-Beta, N+1-Final

That's really it. Note the "N+1" trackers are also created at "N-1" release time, so that would need to be co-ordinated. That's currently scripted but not automated: all you have to do to create the tracker bugs and update the wiki page is run that script, but nothing currently automates it, I do it manually each cycle. Updating blockerbugs should obviously run after that script.

The only tricky parts here, I guess, are release detection and authentication. We do not currently have any great way to tell when Beta releases happen, that I know of. There are ugly hacky ways you could do it but I don't particularly like any of the ones I've thought of. There are kinda-OK ways to tell when official releases happen: this is a path I've been down extensively with fedfind and @frantisekz has been down with oraculum. Currently the way fedfind is doing it is just to refer to a bit of JSON I update manually. I can extend that to cover Beta releases, one way or another.

As for authentication, to automate blockerbugs work obviously the thing doing the work needs to be part of blockerbugs itself or have some kind of permanent auth capability. If we also want to automate the tracker creation, which we'd kinda have to do if we want to automate the blockerbugs work (as opposed to just scripting it), whatever does that needs permanent auth to Bugzilla and the wiki. I believe the openQA server has that at present, I don't know of anything else that does.

Instead of manually updating the voting decision in both places (discussion tickets and Bugzilla), it would be nice to be able to update it in just one place and it would propagate to the other one. My initial idea was to close the vote in Pagure and let it propagate to Bugzilla. However, @adamwill suggested the opposite way.

There are issues to consider:

• We only detect Bugzilla changes in ~30 minutes intervals, and so the vote in the ticket can change between a blocker was accepted in Bugzilla and before we detected it. The saved vote record (example) could then be incorrect. This doesn't occur when we close the vote in Pagure instead, because that is immediate (there can still be a race with multiple people sending a comment at roughly the same time, but the likelihood is much smaller). I place a quite high emphasis on having a trusted record of who voted how, and I'm not sure if this is OK.
• The voting system in Pagure has a superset of information compared to the system in Bugzilla. For example, we can reject a BetaBlocker while accepting the FinalBlocker in Pagure. That can't be done with the Bugzilla system. So if we want to close the vote in Bugzilla, we won't be able to propagate these finer details. (We could think about redoing the Bugzilla keywords system, but that's a big change in itself).
• It is definitely easier to provide a comment (especially multi-line) along with the decision in Bugzilla than to do it in Pagure and then pass it through. We currently have AGREED (Accepted|Rejected)TRACKER... [JUSTIFICATION] command, but that only allows a single-line justification, and for multi-line justifications the syntax would need to get more complex :disappointed:
• It's alluring to close the vote in the canonical location where the blocker data is stored (Bugzilla), rather than to do it in a different system and then hope all technology works as it is supposed to.

I'm not sure if there is a good solution here. Having two (or three) disjoint systems brings disadvantages. I'd love to hear your take on this, @adamwill (and anyone else's).

Note: This idea came up in the blockerbugs discussion tickets feedback.

We'd like to make the Prioritized Bugs Process more streamlined and more visible by using the Blocker Bugs app. Prioritized Bugs are not blockers, but are officially recognized as urgent and important and the FPL and FPgM work on getting resources to solve them. This gives us a mechanism for getting focus on serious problems which don't align with release criteria, and it takes pressure off of actual release blockers (we aren't tempted to use that hammer when it's not appropriate).

Our idea is to add Prioritized (maybe "Non-Blocker High Priority"?) as a milestone, and use the blocker bugs proposal process — but we'd still review at our separate fortnightly meeting. That should be very straightforward, I think. The only wrinkle is that for this milestone, we'd not want the Blocker or FE categorizations.

@bcotton @churchyard

From https://pagure.io/fedora-qa/blocker-review/issue/16 :

But https://bugzilla.redhat.com/show_bug.cgi?id=1861700 only has:

Whiteboard: AcceptedBlocker AcceptedFreezeException
Blocks:     F33FinalFreezeException, FinalFreezeException F34BetaBlocker 

How come?

The task here is to add support for automatic documentation deployment. There is an existing tool for deploying docs generated by sphinx [1] that might be worth looking into. Support for jenkins [2] seems to be missing as of now but is listed as something that will be done in the future. Other thing that needs figuring out is a location of the docs.

[1] https://github.com/teracy-official/sphinx-deployment
[2] http://jenkins.cloud.fedoraproject.org/job/blockerbugs/

Moved from trac [[ https://fedorahosted.org/fedora-qa/ticket/384 | https://fedorahosted.org/fedora-qa/ticket/384 ]]:
"Problem
At the moment, our documentation is out of date and somewhat incomplete.

Analysis
Update the docs and find an appropriate place for them to live."

The current code for linking FAS accounts to rhbz accounts asks for a username/password to verify that the current user is indeed the owner of said account.

Asking users for passwords isn't exactly ideal behavior and it would be better to send a verification link to the email address for verification purposes. Not 100% foolproof but definitely good enough for our purposes.

If there's a Bodhi update with a very long title (e.g. with 50 builds included), it can occupy a lot of space and make the UI elements harder to read and navigate. It's probably a good idea to shorten it, either by cropping after X characters, or by using the update id instead of update title in these cases (not always). This affects the bug list view, the updates view, and the requests view.

This is a follow-up of #275 and #276, where Adam requested that the "Releng commands" section in Requests doesn't need to be edited by hand (if you want to omit a FE), but instead you can use web-style interactive widgets (checkboxes, etc) to populate the releng section as you see fit. It would be more user friendly and less error-prone.

SQLAlchemy 2.0 brought an option to use dataclasses as ORM models. That brings nice improvements, like not having to define __init__() methods, possibly more accurate type hints, and more. See more documentation.

Unfortunately, at this moment, it's not supported by Flask-SQLAlchemy, see this issue.

Once the issue is resolved, consider switching our ORM classes to dataclasses.

Some of the buglist view are confusing. The light bulb and green circle icon are not immediately clear what they mean. It also seems that it's overkill to highlight any activity in any bug - usually the light bulb is displayed for half of the bugs.

Let's get rid of those two status icons and instead let's add a short clear label for new bugs proposed in the last 2 days, I'd say.

I think we could possibly even reuse the current data we have for the green circle icon ("blocker/fe status changed recently"), and just use it to display the new label. But this needs to be verified. It might turn out that it's not exactly that, and that it's not that easy to figure out when was the bug proposed.

It would save a lot of manual work if BBA could nominate an upstream bug as a blocker/FE. It would create a bug report in RH Bugzilla, link to the upstream bug, and set the correct fields for a nomination.

Here's a rough idea: Sometimes it happens that an accepted blocker bug gets closed, either by Bodhi automatically, or by the package maintainer who doesn't understand the blocker process ("reported/fixed upstream, closing"), or perhaps even the original reporter for some reason. If we don't follow such bugs closely (are CCed and paying attention whether it should really be closed at this point), it can easily happen that we lose track of important problems which are either not yet fixed, or might've been fixed, but we don't have any confirmation for it.

BBA could help identify these cases. It could look whether accepted (perhaps even proposed?) blockers were in the VERIFIED state before closing, and highlight those, which weren't. Those would probably appear on a new page in BBA. BBA admins (or some other privileged group) could then click a button to mark a particular bug as verified and remove it from the list, or we could use a flag in bugzilla, something like that.

The implementation needs some further thought, for sure. But the idea is good, I think.

In https://pagure.io/fedora-qa/blocker-review/issue/854 I voted for FinalBlocker while the bug was still proposed as a Beta blocker. After the milestone was adjusted in bugzilla (from blocks:BetaBlocker to blocks:FinalBlocker), and BBA refreshed it, I didn't see any "Voted" label in the Final milestone. Editing the Pagure ticket caused it to reload and display Voted properly.

It seems we only show Voted labels on Pagure ticket updates, but not when the bug (with an existing Pagure ticket) is moved between milestones.

When I try to link my bugzilla account to BBA, I get the following error:

Login failed: Application failed during request deserialization: 32000: You have attempted to access the API either using an unsupported method or using one or more unsupported parameters. You must use the 'Authorization' header to authenticate to the API and you must remove all unsupported parameters from the query. The unsupported parameters are: Bugzilla_login, Bugzilla_password, Bugzilla_token, Bugzilla_api_key. See https://bugzilla.stage.redhat.com/docs/en/html/api/core/v1/general.html#authentication for details on using the 'Authorization' header. at /usr/share/perl5/vendor_perl/SOAP/Lite.pm line 2855.

This might be caused by bugzilla auth changes:
https://listman.redhat.com/archives/bugzilla-announce-list/2022-February/msg00000.html

This might also be further complicated by the fact, that redhatters can no longer login using username+password only. When I try to do that in bugzilla, I get an error:

To access Red Hat Bugzilla Stage you are required to login using 2 Factor Authentication. To do so you can enter PIN and Token in the password field or you can use the Red Hat Associate SSO.
This is because you have an account with elevated permissions.

When I log in using username+pin+token directly in bugzilla, it works fine. But it doesn't work in BBA, same error.

So, we need to fix account linking, and in case of redhatters, we possibly need to tell them to use pin+token instead of a password (needs verification).

If a bug is proposed as a blocker and a freeze exception, voting just for one of the trackers makes the label "Voted" appear in buglist under both sections (proposed blockers and proposed FEs). Ideally "Voted" should appear only under that section which I actually voted for, and there should be standard "Vote" in the other one.

If bug states (NEW, MODIFIED, etc) were differentiated slightly, it could draw eyes to those which are currently in a test-able state. So MODIFIED and ON_QA labels could be colored, or perhaps in bold. And VERIFIED on the other hand could be gray, or similar, to indicate that your eyes can skip that entry. Just an idea to try out.

"External links" section in the Update tooltip (in Bug list view) is quite handy to explore other bugs/builds etc for the component. However, it is shown only when an update is available (and the popup appears).

Move the "external links" section into a different popup which could be accessed always, regardless of updates availability. Perhaps add a dotted underline for the component name and show the popup when hovering mouse cursor over the name. Or add a small (i) icon behind the component name and show the popup when the cursor is above it.

Also, add "Bodhi updates for this component" into the external links list. A bodhi icon was removed in a11c95b576f6, it can be revived from there.

Moved from trac [[ https://fedorahosted.org/fedora-qa/ticket/440 | https://fedorahosted.org/fedora-qa/ticket/440 ]]:

"I would like to be able to see also closed blocker bugs, not only the pending ones. The current setup can be the default and have a check box for showing the closed ones."

Moved from trac [[ https://fedorahosted.org/fedora-qa/ticket/318 | https://fedorahosted.org/fedora-qa/ticket/318 ]]:
"When bug A (proposed/accepted blocker) is closed as a duplicate of bug B (not proposed as a blocker), it disappears from the list. Unfortunately a lot of developers won't transfer the Blocks: and Whiteboard: fields, so we might lose track of some important bugs this way.

Displaying A or B in the list even when this happens would be nice. Displaying B is probably the preferable way. There could be a special icon indicating that B is displayed because of recursive dependencies and tooltip would specify it is because of bug A."

Moved from trac [[ https://fedorahosted.org/fedora-qa/ticket/356 | https://fedorahosted.org/fedora-qa/ticket/356 ]]:
"Problem
While the blocker proposal functionality does work, the process of actually proposing the blocker is very slow - it appears as if the page has just stalled for a while as the request is being submitted to bugzilla

Outside of being an annoyance, there is a possibility of users refreshing the page during submission, which would trigger a second proposal. This isn't the biggest issue ever but could end up generating extra comments in bugs.

Analysis
It would be better to show users a "bug is proposing" page until the proposal is complete - that way they wouldn't be tempted to refresh the page and if the page is done correctly, it wouldn't matter if the page was refreshed or not.

Enhancement recommendation
I have some ideas on how to go about fixing this but I need to explore the details before figuring out if any of them are even practical or how long the implementation would actually take."

Moved from trac [[ https://fedorahosted.org/fedora-qa/ticket/381 | https://fedorahosted.org/fedora-qa/ticket/381 ]]:
"Problem
The current bug and update sync algorithms are very fragile in that one error will stop the entire sync algorithm, leaving stuff out of sync with bodhi or bugzilla. This has caused problems in the past

Analysis
This could be solved by better error handling in the sync process such that an error is logged but doesn't bubble up to the main sync code and halt the entire process."

While the admin interface works well now, its UI could use some polishing. This ticket is a placeholder for UI enhancement ideas.

as per ​https://bugzilla.redhat.com/show_bug.cgi?id=1122081 , anne mulhern was unable to 'convert' an issue from being nominated as a blocker to being nominated as an FE - it wound up being nominated as both. it should be possible to convert among all possible states. "nominated as both" is technically viable, but should probably be simplified to "nominated as blocker", because we usually consider blockers as FEs if we reject them as blockers. so we probably only need two states.

I tried to reproduce [[ https://fedorahosted.org/fedora-qa/ticket/433 | ticket 433 ]]. So I put data into blocker proposal form, waited for quite a while (2 hours) and then clicked on "Submit Proposal" which resulted in error message "There were issues with your submission, please check below for more details." but there were no details provided. User input in forms were left intact. Clicking on "Submit Proposal" again successfully submitted the proposal.

One thing that comes to mind is that there might be login timeout in the bugzilla object through which the proposal is sent.

Try to reproduce the issue, investigate and propose a fix.

I was talking to Dennis, and this thought came up in the conversation. It might be helpful for releng if blockerbugs would send a fedmsg whenever the list of updates that fixes blocker or FE bugs changes. releng could write a consumer which listens for those fedmsgs and automatically fires new 'candidate' composes when appropriate.

imho, it'd be good idea if we copied proposal justification to the voting ticket, if the proposal was done via the blockerbugs application.

pytest failure if run on an openqa server already configured with non-default in config file

I am not familiar with pytest, so I basically called it on my PowerPC machine with not specific arguments, and it is reporting errors similar to exemples below (2)

Is there specific syntax/arguments to pass to handle the local hostname by pytest ?

(2) my PowerPC host is "abanc.test.toulouse-stg.fr.ibm.com"
and failures report it as a problem (with or without http prefix)

>               assert fake_schedule.call_args[1]['openqa_hostname'] == oqah
E               AssertionError: assert 'http://abanc...tg.fr.ibm.com' == 'openqa.fedoraproject.org'
E                 - http://abanc.test.toulouse-stg.fr.ibm.com
E                 + openqa.fedoraproject.org
tests/test_consumers.py:317: AssertionError
===
>               assert fake_schedule.call_args[1]['openqa_hostname'] == oqah
E               AssertionError: assert 'abanc.test.t...tg.fr.ibm.com' == 'openqa.stg.fedoraproject.org'
E                 - abanc.test.toulouse-stg.fr.ibm.com
E                 + openqa.stg.fedoraproject.org
tests/test_consumers.py:317: AssertionError

It'd be nice to get URLs into the ResultsDB groups when reporting, but the current design of resultsdb_conventions makes it a little tricky. We should try and figure out a nice way to achieve this, between the scheduler and conventions. It shouldn't be too hard.

It would be nice to have Fedora OpenQA support for running tests on merge requests to dist-git.

It would be nice to have Fedora OpenQA support for running tests on upstream PRs for upstreams that use packit. Both podman and anaconda have expressed interest in this.

Hello,

Opening this issue based on what was discussed in IRC (my username is "odra").

We are in the process of migrating/deploying ResultsDB to kubernetes/openshift which includes "ci-resultsdb-listener" as well.

I was wondering if "openqa results" and "ci-resultsdb-listener" are basically doing the same thing and if it would be possible to merge both into "ci-resultsdb-listener" by making it more configurable - this change should also include using basic auth requests in "ci-resultsdb-listener".

There were some concerns such as log format in "openqa results" being different than the format used in "ci-resultsdb-listener" which could be discussed in here instead of IRC.

PAWS is a cool tool for automating Windows deployments for testing purposes. We may be able to use it to automate the Active Directory tests for Server (that we currently have to run manually). We should try and find some time to work on that. We'd also have to check the legalities with someone, possibly, though I think it should be OK / legal to do this using a 90-day evaluation copy of Windows, which Microsoft seems happy to give away; each test would start from a stashed base qcow image and live for only an hour or so.

Not sure if this is worth adding the dependency, but might be nice to use Alpine's $persist, to keep track of which cards are open, or whether the story points graph is shown/hidden in order to keep the "state" between page reloads.

It is not needed right now, but once we implement the periodic background-refreshing of data, I could see the users being... not happy with the ui randomly opening/closing the cards and so on.

https://alpinejs.dev/plugins/persist

General

• ~~Rework the layout/navbar jinjax components to make more sense (e.g. ability to change the contents of the navbar's left side for each page)~~
• ~~Add links to to Pagure~~
• Show who is the ticket assignee in "Assigned" column (?)
• Show tags (?)
• ~~Remove the filters form~~ and replace with per-user configuration (saved in db)
  • select which repos in Backlog you want to see (or see on top?)
  • switch between "Assigned to me" and "All assigned tickets"
    • same with Finished (graph will always be just for the whole team numbers (?))
• UI to change story-points value
• UI to easily create tickets in fedora-qa/_private for Fedora testing/working on internal stuff
• UI to assign ticket from backlog "to me" (assigns and lets set SP value)
• UI to close ticket (assigned "to me")
• Add caching to filter_issues & prepare_plotdata
• Some HTMX/Javascript timer to re-fetch data in background
• Group private issues (per repo) instead of one-per-private ticket
• ~~Only show private tickets' contents to logged in users that have higher-than-none role~~
  • in the future, track actual ACLs/group correspondence for the private tickets' contents
• ~~UI show "all tickets" when logged out, show only tickets assigned to/finished by the logged in user~~
  • ~~Add switch to show "all tickets" even then (maybe url param assigned_to=* or smth like that)~~
• UI - play around with the Cards' padding - probably make padding-right 0px, and lower the padding-left to about half the current value.

CLI

• ~~Change users' role/permissions~~
• ~~run_cli update_issues -v reponame~~

Mega long-term

• Support multiple teams/boards (one user can be a part of multiple teams)
• Support Github/Gitlab

SSIA, see #11 for the whole context from @kparal

I'm not sure how feasible it would be, but it would be nice to have a listing of the most recent successful (which probably includes FINISHED_INCOMPLETE) rawhide and branched composes for the main compose, IoT, and CoreOS.

Bonus points if shows the age and highlights when the most recent is stale (e.g. yellow when it's more than 2 days old, red when it's more than 5 or something)

As per the UX feedback, the Landing page looks not very impactful, it has to be made simplified by modifying :

-The bottom button "Do you have some spare time....." needs to be with fewer words in it.
-Meeting Schedule, Blockers Table, Meeting Minutes can be in cards one below another for better readability.
-Configuration of widgets design change.
-Fix Technical wordings with layman terms.

I have been thinking about these changes. @lbrabec and @jskladan what can be the other changes? please suggest me so that we can proceed further.

This issue is a place to brainstorm and discus what should be changed in UI to improve user experience.

We already have a feedback from Renata Gegaj which identifies three problems:
- Lack of infographics
- Information Architecture
- Technical jargon

As developers we want to be able to develop quickly which means we want builds to be fast but with the current dockerfile dependencies specified in the package.json file are always fetched when the yarn command is run.
It will be performant to cache the package.json file and yarn.lock file so that dependencies are fetched only when this files change

After a chat on Flock, we decided this should be better!

@jibecfed helped!

The past week, I conducted interviews with different members of the QA community to get better context on specific features that would benefit the community. Based on my findings from the interviews(see my findings here - https://docs.google.com/presentation/d/1S7LwksjTDQFyeWfosu53b9pm2oCMogxbQ5rR6tLKu7I/edit?usp=sharing), I propose the following features be made for the dashboard to give it better value

Summary

A card with results of tests conducted recently on the platform and a link to the main test results page to see more.

Context

The card is to provide users a brief summary of recent reports created on the Test results page and a link that takes you to the test reports page to see more. The feature would be useful to the users on a daily basis as the activities of the QA team revolve mostly round the testing and the feature provides a brief summary of what is seen on the test results page. Ideally, when a user wants to start working for the day, the user would visit the dashboard to see what is happening. The user sees the list of the latest test results – if they passed or failed – then proceeds to the test results page to see more. The user can also see major metrics of tests done Instead of going the route of maybe having multiple tabs open and entering addresses, the user can easily just click a link located on the same card that will take the user to the main test results page immediately. Currently, such a feature does not exist on the dashboard.

Criteria

This feature will require access to an API that would retrieve details of test results as they come in. Without this access to such an API, keeping the list updated will become a daunting task as tests are running all the time.

I made a wire frame depicting the proposed changes here

This is also a feature that came as a result of the research I did here https://docs.google.com/presentation/d/1S7LwksjTDQFyeWfosu53b9pm2oCMogxbQ5rR6tLKu7I/edit?usp=sharing and I mentioned in #62

Summary

A card with top testers across QA teams every month.

Context

The card is to serve as a “card-of-fame” that honors people who have put in time in testing in the QA team. This is an aggregate of tests each person has done across board for a month, placed in a ranking. This serves as a way to encourage others to participate in tests and thank those that participated and earned the ranking. The ranking is done monthly to give people the chance to actually vie for a spot on the “card-of-fame”. Currently, the Fedora updates dashboard has something like that for testers of updates alone. Badges can be awarded according to the number tests completed and displayed beside the person’s name.

Criteria

There should be a counter of some sort that would keep track of the number of tests each user has done across board with access to the badges earned by each user.

Summary

A card with reports of bugs conducted recently on the platform and a link to the man bug reports page to see more.

Context

The card is to provide users a brief summary of recent reports created on the Bugs Report page and a link that takes you to the bug reports page to see more. The feature would be useful to the users on a daily basis too. People working on bugs will want to see the latest that have been reported in a bid to find something they can work on immediately. Ideally, if the user just resuming work for the day wondering what to work on, visiting the dashboard will give him an overview of how many bugs were filed recently, details of bugs etc. If the user sees a bug of interest, the user can get working on it immediately by clicking the listed bug for more information else, the user can click the link on the card to be redirected to the main bug reports page to see a comprehensive list of all the bugs reported. Currently, there is not a feature like this on the page.

Criteria

This feature will require access to an API that would retrieve details of the test results as they come in. Without this, access to an API the feature cannot be maintained properly.

It is good to have dark mode Support

The present font size of non-header text on the page is not legible for people with different levels of sight. There isn't a default font size set and this would allow different browsers display at font sizes which may not be adequate all the time.

As discussed in (#8) I am working on implementing redux-toolkit and code splitting. @lbrabec please do review the PR's I will raise against this issue and let me know if it's helpful and if it requires more changes.

After https://bugzilla.redhat.com/show_bug.cgi?id=2241632 was accepted as a blocker, we should really have a formal test case for that, and automate it. I've filed https://pagure.io/fedora-qa/issue/750 for creating a wiki test case; once that exists, we should automate the testing in openQA.

The test should prove that the following is supported.

• starts and stop
• checks for network
• sets background
• toggles notifications for an application
• adds and deletes an online account
• toggles sharing
• sets sound output loudness
• sets energy settings
• shows display information
• sets mouse speed
• adds a new key shortcut or redefines one
• adds keyboard layout
• toggles high contrast and bigger text
• adds, deletes, modifies a user
• redefines default applications
• sets date and time
• shows system info

After recent Fedora InstallFest event in Brno, we've seen that the common use case of how university students install Fedora has changed compared to past. Now, their notebooks run with UEFI. Also, very often students put in a second drive for Fedora installation. This ticket is a request to cover a frequent installation scenario with OpenQA.

The purpose of this test would be to finish a Fedora installation to a second disk, and leave the first disk untouched.

The installation would be performed on an UEFI system and the first disk would contain an existing partition layout including ESP (EFI system partition). Having the layout resemble default Windows disk layout including NTFS partitions would be a plus. The second disk would be expected to end up with its own ESP. The first disk (and especially its ESP) is expected to get untouched during the installation process. The system is expected to boot properly, when instructed to boot from the second disk (i.e. when the newly created "Fedora" boot option is set as the first one - I'm not sure if anaconda is supposed to do that automatically, would be great to find out).

This scenario would be nice to get automated using both guided partitioning and custom partitioning (of course, having blivet-gui covered as well wouldn't hurt either, but I don't think we need to exhaustively cover everything).

Detailed steps:
1. Prepare an UEFI system with 2 disks.
2. Prepare an existing GPT layout on the first disk, including ESP and some NTFS partitions. No free space.
3. Have the second disk empty.
4. Boot the installer.
5a. In guided partitioning, select only the second disk and perform the installation.
5b. In manual partitioning, select only the second disk, create a working layout and perform the installation.
6. Either confirm that the system is configured to automatically boot from the second disk's OS (using efibootmgr; in case anaconda does this automatically - and if it doesn't, it might be a bug/RFE), or use system firmware to configure "Fedora" to boot as the first option.
7. Boot the installed operating system, verify it boots correctly.
8. Verify that the first disk is untouched (same partition layout, same ESP contents).

I noticed in reviewing https://pagure.io/fedora-qa/os-autoinst-distri-fedora/pull-request/275 that the "help", "preferences" and maybe "about" needles are kinda redundant with several other app tests. e.g. the gnome/apps/gnome-text-editor/gte_preferences_submenu_light-* needles are the same as the system-monitor sysmon_menu_preferences needle. There are several needles which are all just the "Help" button from a standard GNOME app burger menu. And several app tests also have needles for the "About" item from the burger menu; sometimes this needle includes the application name, but unless we're really worried they might omit or misspell it, we could just have a generic needle that matches only the "About" text and which should work for all apps.

Can you please go through all the apps and consolidate these needles into generic ones where possible, stored in needles/gnome? Thanks.

Similarly to #301, we need this test to run on KDE, too. Either convert the existing test into KDE variations or make them KDE agnostic.

Background

Hi, I work on the Podman Desktop GUI for podman, and it would be very nice to have automated testing for Podman Desktop in Fedora OpenQA, particularly if Podman Desktop tests could be triggered by updates to podman, which is already in Fedora OpenQA.

There are a few complications to getting it into OpenQA:

• Podman Desktop is not in the Fedora flatpak repo. Podman Desktop is installed on Fedora via Flathub. OpenQA is geared towards flatpaks that are in the Fedora flatpak repo.
• Fedora flatpak repo inclusion requires an RPM. RPM-less Fedora Flatpak are basically not possible because of Fedora software requirements, reviews, etc. It is something that could be rethought but it would be a big thing.
• Podman Desktop is not (easily/sustainably) packaged as an RPM. Podman Desktop is an Electron application, which complicates packaging it as an RPM greatly. Unclear if the recent addition of ffmpeg (which is a req of electron?) makes packaging electron more possible.

Proposal

Would it be possible to investigate what it would take to pull a flatpak from flathub into Fedora OpenQA to enable testing of Podman Desktop?

@eischmann reminded me that we're still not testing GNOME's "enterprise login" feature. We test FreeIPA stuff a lot of other ways, but we don't test that. We really should! This is the feature where you can enrol to an enterprise domain and then login as a domain user all via gnome-initial-setup on the first boot, then GDM. We can add another test to the FreeIPA test group that would test this along with kickstart, Cockpit and console enrolment.

Server WG would like to have 2 new tests:
1)
Install Server on software raid using biosboot and GPT
Description at https://pagure.io/fedora-server/issue/87 Testcase 1 GPT

2)
Install Server on software raid using biosboot and MBR
Description at https://pagure.io/fedora-server/issue/87 Testcase 2 MBR

The former is a bit urgent as there is a accepted Change Proposal to switch everything to GPR with F37. Curently it doesn't work and ends up in a non-bootable system. With the switch and the bug we would make Server unusable for a unknown number of current users. There is also a bugzilla entry: https://bugzilla.redhat.com/show_bug.cgi?id=2092116

The latter used to work, at least up to F33/34. There is a report is does not anymore.

We have a freshly minted user switching criterion. This would be a great candidate for automation. We don't have a manual test case yet, but I requested one in https://pagure.io/fedora-qa/issue/630 .

While discussing https://bugzilla.redhat.com/show_bug.cgi?id=1331317 , it was noted that we could catch bugs of this type in openQA if we had tests which tried to boot the ISOs as disks (rather than as CD/DVD). I'm not sure off the top of my head if you can tell os-autoinst to do this, but if not, we could send patches. It'd be nice to catch this kind of problem in automation so we know when it breaks.

After recent Fedora InstallFest event in Brno, we've seen that the common use case of how university students install Fedora has changed compared to past. Now, their notebooks run with UEFI. Also, very often students put in a second drive for Fedora installation. This ticket is a request to cover a frequent installation scenario with OpenQA.

The purpose of this test would be to finish a Fedora installation to a second disk, but re-use the ESP from the first disk. This use case is purportedly not as recommended by anaconda/storage team as having a separate ESP on the second disk (already covered in #127), but this might be necessary with certain poor firmware implementations that ignore requests to boot from the second disk.

The installation would be performed on an UEFI system and the first disk would contain an existing partition layout including ESP (EFI system partition). Having the layout resemble default Windows disk layout including NTFS partitions would be a plus (including a similar ESP contents). The second disk would be expected to end up with Fedora's installation partitions, but no ESP. The first disk is expected to get untouched during the installation process except for ESP. The rebooted system is expected to boot into Fedora bootloader showing the new Fedora system and the pre-existing system (if we can fake it) and booting Fedora should work (it might be tricky to verify the pre-existing system would also boot ok).

This scenario would be nice to get automated using custom partitioning (of course, having blivet-gui covered as well wouldn't hurt either, but I don't think we need to exhaustively cover everything).

Detailed steps:
1. Prepare an UEFI system with 2 disks.
2. Prepare an existing GPT layout on the first disk, including ESP (ideally with something resembling valid content) and some NTFS partitions. No free space.
3. Have the second disk empty.
4. Boot the installer.
5. In manual partitioning, select both disks, and then create a working layout consisting of mounting the first disk's ESP as /boot/efi and creating all other partitions on the second disk (no ESP there). Perform the installation.
6. Confirm the system reboots into Fedora GRUB.
7. If possible, verify that the pre-existing system boot option is present.
8. Boot the installed operating system, verify it boots correctly.
9. Verify that the first disk is untouched except for ESP, which should receive Fedora-specific files.

Starting with Fedora 34, we have an AArch64 disk image and ISO for Fedora KDE. Can we have the Fedora KDE flavor added to Fedora AArch64 tests?

It was brought up today by @apol in the Fedora KDE Matrix room that we should be doing RTL testing for KDE in Fedora, after hearing a talk by @moceap about the state of RTL on Linux.

We already do this to some extent for Workstation (GNOME) and it'd be great to do for KDE as well.

These shouldn't need any changes (apart from initial install process) and should work on Silverblue as they are.

Backup and restore is another important feature of FreeIPA. ipa-restore must restore a fully working IPA server from a backup. You can easily test it in four steps:

• ipa-server-install
• ipa-backup
• ipa-server-install --uninstall --unattended
• ipa-restore /var/lib/ipa/backup/*

ipa-restore needs the DM password.

Fleet Commander is an interesting new-ish project which could well be a good target for openQA testing: it involves FreeIPA, GNOME and Cockpit, all of which we test, and which are pretty hard to test together 'as a user would' with anything else (though @stefw could probably do it...)

One note is that this would require nested virt on the worker hosts: part of the Fleet Commander workflow involves launching a VM, and as the SUT in openQA is itself a VM, that would be nested. This shouldn't be a big problem (well, except for RHBZ#1539330...) but it's worth keeping in mind.

This is to catch issues like the one reported here in the future: https://fedorahosted.org/rel-eng/ticket/6332

This might be a bit tricky, but it'd be an improvement.

Per https://bugzilla.redhat.com/show_bug.cgi?id=1446432 , the openQA 'minimal install' test is actually hitting a bug right now, we're just not noticing it. It actually installs the Server package set plus the minimal package set (so...the Server package set). But because anaconda makes no complaints and the installed system boots, the test 'passes'.

If we could somehow verify that the package set install tests installed the correct package set, that'd be good. Maybe checking the anaconda logs somehow?

This is a request from the anaconda devs. Ref https://bugzilla.redhat.com/show_bug.cgi?id=2238045 . There's an interactive flow in anaconda when a kickstart-driven install lists a package that doesn't exist, which had a bug. The anaconda team would like us to test that in openQA (it does seem like a natural fit). So the test would run an install with a kickstart that intentionally listed a missing package, and check that the expected UI flow appeared and could be successfully clicked through to complete the install.

It would be good to add some tests that check custom partitioning mode properly detects existing Fedora installs.

We could have various of the partitioning tests upload their disk images at the end and then add new tests that boot the installer with the uploaded images attached and check if custom partitioning shows an existing Fedora install properly...or we could add it just as an additional test step at the end of some/all of the install tests, boot back to the installer, go to custom partitioning, check it shows the installed system correctly...

anaconda team have said that having this tested automatically would be valuable to them.

@simo has asked if we could extend openQA's FreeIPA tests to cover krb5-authenticated NFS. This is a mechanism by which you can set up an NFS share within a domain and have the domain controller authorize access to it.

I think we ought to be able to do this reasonably easily by extending the domain controller role test to configure one or more such shares, and then extend one or more of the client tests to check the access control (make sure a user / machine which should be able to access the storage actually can, make sure one which shouldn't be able to can't, and so on). Alternatively adding an extra client test specifically for this might be useful/necessary, we'll have to try it and see.

@simo says @rharwood should be able to help us with this effort, including monitoring the results and updating the tests in future. @pschindl , @sumantrom , if either of you is interested in working on this, please feel free. I will try and get around to it in future if I can, but I have quite a few other things at higher priority ATM.

@puiterwijk asked me if it would be possible for us to add an openQA test that covers:

https://fedoraproject.org/wiki/QA:Testcase_ipsilonv1_ipa_idp_install

which is basically installing an Ipsilon server alongside a FreeIPA server and then checking that it works.

We're mostly trying to focus on release validation testing in openQA, but I do feel OK with adding extra tests that are specifically requested by people (e.g. the Atomic installer test), and this shouldn't be too hard to work in alongside the existing FreeIPA tests somehow. This is a 'wishlist' thing I intend to work on if I get the free time at some point.

Would it make sense to also see pull requests, bugzillas, updates etc. that I've opened/created against packages that I don't maintain?

Notes:

• Ideally this should be configurable separately in the filters, because I open a lot of bugzillas :D
• For packages I don't maintain, I would not be interested in other information (other bugzillas, other PRs, etc.), just my own.

WDYT?

I have just read this blog and it refers to packager dashboard. Based on that the blog claims that Fedora has 800 orphaned packages. And honestly, I don't know what to thing about what the link displays. Being e.g. POC of rubygem-actionpack, it certainly is not orphaned. Yes, it might be impacted by orphaned libspng, but that is all.

So is there chance to improve the page for "orphan" user?

Currently, the favicon is the react one

TMT is our (current) standard for storing CI tests for packages in a git repo. I think having tmt information on the packager dashboard would help increase packager awareness and increase coverage.

The low-hanging fruit is simply something like:

• "This package has no tmt tests. You can add them easily."

Followed by more sophisticated things like:

• ⚠️ This package's tests have failed since 2022-09-13!

... and whatever else.

When I take a look at https://packager-dashboard.fedoraproject.org/ndim, it shows the terminus-fontspackage with an FTBFS on S390 due to an unsatisfied BuildRequires for /usr/bin/grub2-mkfont.

However, the BuildRequires: /usr/bin/grub2-mkfont line is wrapped in %ifnarch s390 s390x (see https://src.fedoraproject.org/rpms/terminus-fonts/blob/rawhide/f/terminus-fonts.spec#_122), so on s390x, the build should just happen without the BuildRequires and without building the -grub2 subpackage, as there is no grub2 package on s390x.

This poses a few questions:

• Where does that FTBFS information come from?
• Is that FTBFS information accurate?
• How can I as a normal packager find out more about it? After all, koji builds do not show any s390x related issues.
• Do I need to (or should I) fix terminus-fonts.spec in some way?

I am not sure if this is an actual packager_dashboard problem, but packager_dashboard is the first place to expose this issue, so I though I would start the investigation here.

I have this bug that has been proposed as a freeze exception. The Dashboard nicely adds a FE marker near it and it even makes the package with a label:

However, from the bug row, I cannot tell whether the FE is just proposed or accepted, nor I can followup to the blockerbugs ticket. If my package had more FE bugs, I would not be able to tell them apart.

It would be really nice if the FE had some hover information with more data. Ideally, I would be able to click somewhere to get to https://pagure.io/fedora-qa/blocker-review/issue/490 (but I don't know where to put this link).

Currently, package reviews are not visible on the Packager Dashboard.
In order to make the Dashboard even more useful
as the place to check for outstanding work items,
it would be great to support some package review use cases as well.

One use case is displaying open review requests opened by me.
I would like it to work as follows:
1. All review requests opened by me that not CLOSED are shown,
as these all need work.
2. Status of fedora-review flag is shown,
because from that I know
if a reviewer has claimed the request (fedora-review: ?),
or if the review is completed, but still need to request the repository
(fedora-review: +, but the issue still open).
3. If there is a needinfo flag open for me,
because the means that the review won't proceed without my action.

Another use case is the reviewer point of view.
I would also like to see
1. All review requests that are not CLOSED that have been assigned to me,
as those are the ones where I am the reviewer.
2. It would be even better to also see those
where fedora-review flag has my name on it.
According to the process, these should be covered by the preceding point,
but keeping assignee and fedora-review in sync is a manual procedure
and accidents happen,
so it would be good check both.
3. Also here I would also like to see if there is a neednfo open for me,
because that is used to signal that the reviewee has addressed my comments
and I should do another round of review.

There is a build failure from 2020-09-15:

https://koschei.fedoraproject.org/package/lib3mf?collection=f33

Last successful build happened on 2020-09-12.

Yet the Dashboard says:

The last successful build was 2 months ago

I'd expect it to say:

The last successful build was 1 week ago

It appears that only "real" builds are considered (i.e. not strach builds by Koschei).

For a packager like me (with hundreds of packages) the dashboard is cluttered with issues that are not actionable, because they just cannot be resolved at this point in time. it would be very useful to be able to say "don't show this item for $Duration" (maybe with three options: one week? two weeks? a month?)

Maybe the "don't show this information until $timestamp" information can be stored locally in a cookie or something like that?

There is some glitch in the relval reporting. When I run relval report-results I am getting a CLI menu to choose from a compose, page, etc. When I know exactly, where the matrices are located, I take the shortest path to a result and I can report the result just fine.
However, if I make a mistake in selecting the test type, for instance, and I type back, it takes me back to selection of test types, but whatever I choose, I get the same selection of tables as before. For the situation to be even more confusing, if I make a choice, I will arrive at correct tables, see the image.

Relval automatically checks sizes of deliverables and reports bugs when they are oversize. But it looks like this correctly works just for ISO images. For compressed disk images (raw.xz), it should check the size of the uncompressed disk image, because that's what needs to fit to the target medium (an SD card, most commonly).

In the F39 cycle, some disk images went oversize and we didn't receive a bug report from relval. See https://bugzilla.redhat.com/show_bug.cgi?id=2247611 .

But I don't know if we can detect the uncompressed size without a direct access to the file. It looks like relval would have to download it first.

$ xz -l Fedora-Minimal-39-1.5.aarch64.raw.xz 
Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
    1     256    761,7 MiB  6 144,0 MiB  0,124  CRC64   Fedora-Minimal-39-1.5.aarch64.raw.xz

# try to get this info from just the file header

$ dd if=Fedora-Minimal-39-1.5.aarch64.raw.xz of=test.xz bs=1M count=1
1+0 records in
1+0 records out
1048576 bytes (1,0 MB, 1,0 MiB) copied, 0,00276787 s, 379 MB/s

$ xz -l test.xz 
xz: test.xz: Compressed data is corrupt

$ xz --ignore-check -l test.xz 
xz: test.xz: Compressed data is corrupt

From man xz:

  -l, --list
         Print  information about compressed files.  No uncompressed output is produced, and no files are cre‐
         ated or removed.  In list mode, the program cannot read the compressed data from  standard  input  or
         from other unseekable sources.

@adamwill please update your artisinally hand-crafted size limit file to reflect an increase in Workstation x86_64's limit, which is now reflected in the docs

Nobody in the Workstation chat objected to my promise to update this today.

I'm looking at this change:
https://pagure.io/fedora-qa/relval/c/04a9b9e0227c14dc8627fa3a85ea177707ab9cf5?branch=size-check-bz

And if I read it correctly, the bug is only created once per release, and it's only set to blocking when a release-blocking image is oversize at that very time. Which means, correct me if I'm wrong, that we're very likely to miss size-check errors again, when it goes like this:
1. During a release, the first time an image goes over size, it's a non-release-blocking image
2. relval files a bug but doesn't make it block Beta or Final
3. In the next compose, a release-blocking image goes over size
4. relval updates the bug comment, but doesn't block Beta or Final, so the bug doesn't pop up in blocker bug app
5. Somebody notices two days before Go/No-Go
6. People are upset

Unless I miss something, this is a very likely scenario (I'd even say 90+% likely, because spins are usually oversize and release blocking images are usually not). So we really need relval to set blocks: Beta Final each time a release-blocking image is detected to be over size. Of course there are some annoyances resulting from that approach. In order to make it better, I imagine we could have two different bugs reported, one for non-release-blocking and one for release-blocking. That could make the following human process fairly reasonable. Or, to make it completely simple, just don't check image size for anything that is not release-blocking. Spins are completely self-reliant and we don't necessarily need to file these bugs for them.

Recent update to mediawiki caused new or edited pages to be automatically put on your watchlist (unless you disable a checkbox when publishing). This can be disabled in user settings. I have it disabled in my settings, but pages created by relval compose still were auto-added to my watchlist. That's quite an annoyance for release validation pages :-)

Please make sure the created/edited pages are not added to user watchlist.

relval-2.2.1-1.fc27.noarch

I only entered the Virtual-install x86_64 result of the Installation part to this page[1]
but found the result is synced to Virtual-install of the aarch64 part

[1]https://testdays.fedoraproject.org/events/131

Write down a development setup documentation for the testdays-web app. Make sure the requirements.txt / setup.py / whatever is up to date, complete and working.

Already discussed as part of #34, let's create a separate ticket for this.

We currently have an admin interface, but we require no authentication. Anyone can do anything. That is of course bad, and we need to change it.

On the other hand, Sumantro points out that a forced authentication will make it harder for test day owners to update it, which is true. We should take that into consideration, perhaps there are ways to account for this (as an immediate idea, e.g. generating a url that would allow anyone opening it to edit one particular event).

Sometimes, people create a wrong event by accident. It's quite easy to do, just paste a wrong URL (see e.g. #32). Technically, there's no problem in having a wrong event in the database, we don't care. But users might get confused when they see it on the main overview page and start filling out something that wasn't meant to be filled out.

A good-enough solution could be to allow hide (and unhide) events from the main page, using the admin interface.

I was creating pages for gnome test week which comes at a short notice.
Firstly created the metadata page and passed it to /admin and it did gave me results page 149[0]
and then I decided to delete a bunch of test cases which and split the chunky rows and when I passed again it spit was results 150[1]

RCA

My understanding says, the /admin is looking at the link as the basis.. when results of 149 was created, I passed https://fedoraproject.org/wiki/Test_Day:2023-03-06_Gnome_44_Desktop_and_Core_Apps_TestdayApp_Metadata#TestdayApp_Metadata and the next time just https://fedoraproject.org/wiki/Test_Day:2023-03-06_Gnome_44_Desktop_and_Core_Apps_TestdayApp_Metadata .. I assumed the app would handle it and update but it didn't, just created a fresh new page.

Resolution

Delete 149 from the results page from db if we dont have time to update the codebase

Currently, there doesn't seem to be any restriction on comments. This means people can write whole bug descriptions or essays in there, and boy howdy, they sure do. As the desktop folks pointed out, this is not super useful for developers.

I think it'd probably be a good idea to put a fairly low character limit on the comment field - either a hard limit or a soft 'do you really want to submit a comment that long?' warning thing - and have the UI shown when you hit the limit encourage you to file a bug instead. Something like "Please don't write full bug descriptions as comments! Please submit a bug report following the instructions in the Test Day page, and link the bug report here instead."

We do have an associated consideration here of handling reports to places other than RH bugzilla; we want the limit to be long enough to allow e.g. links to gitlab.gnome.org without hitting the max. The tool I wrote to dump 'long' comments out of the table uses 150 characters as the cutoff for a 'long' comment, which is a number I more or less made up, but seems to work OK. A long gitlab.gnome.org link would be https://gitlab.gnome.org/GNOME/gnome-control-center/-/issues/775 , which is 65 chars...

While I understand the motive behind not exposing more functionality around testday events but sometimes if event creator mistakenly create wrong testday wiki page or suppose event got delayed and new wiki page need to be created then there should be some way to change event metadata URL.

I request if possible to support changing metadata URL. I have not seen code behind this project but if same metadata URL can generate updated event then why not allow to change metadata URL for the same event URL?

Follow-up from Issue #8:

(Some of?) the error messages presented to users are rather cryptic/off point, and the app can get to plain HTTP 500 when parsing "incorrect" data.

It might be worth reworking the error messages/app flow so it provides more helpful information.

Hello.
I did not attend the intel test day yesterday because I did not have a quick reference on where to download and (most importantly) how to compile the required programs properly.

In other words, in the case of both OpenCL basic and OpenCL apps, the suggestions in the "How to Test" section are insufficient for people who would like to contribute to the test, optimizing the time for system preparation and focus only on the test. In my opinion, suggestions such as
- "Compile and run the heavy computation OpenCL program" Link to howto
or
- "Compile and run a simple OpenCL application that performs a basic computation, for example, vector addition" Link to howto
should be replaced by a link with a complete tutorial on where to download the specific program and how to compile it properly on Fedora.

I tried following this tutorial Link github tutorial (and downloaded more software from the Intel site), but to no avail. Maybe it is not good for compiling in Fedora, maybe I got some steps wrong, I don't know.

I hope that tutorials for Intel OpenCL test will be included in the next Test Days. I would make some if I had the skills, but I can help and collaborate privately in making them if needed.

Cheers.

There is a nice "Getting Started" guide for FCOS here:
https://fedoramagazine.org/getting-started-with-fedora-coreos/

I believe it would be a good idea to extract a testcase from it related to running a basic container. We don't seem currently to have a similar test case written:
https://fedoraproject.org/wiki/Category:CoreOS_Test_Cases

So the testcase could talk about writing an ignition file that runs the "hello world" container with podman on start, create the FCOS instance in a VM and check that the "hello world" content is served.

Any happy testcase takers? :wink: CC @sumantrom @coremodule

In the Fedora 39 cycle, we've seen a big surge of FTI (fails to install) bugs being proposed as freeze exceptions, and this was a time-consuming activity for anyone involved in this process. I'd like to improve this process in the future, with the goal of dealing with them more efficiently (spending less time on it). I already started a discussion here:
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/thread/MWRG3YLVU356HABCOJBMAEHVEYKNW75N/

It seems that granting automatic freeze exceptions in certain cases is a viable approach that people would agree with it. I'm filing this ticket to make sure we don't lose track of this goal and make sure we finish it.

I'll take this task, propose policy changes, and make sure the ball keeps rolling.

@npmccallum asked us if we could help improve Bluetooth testing. The idea we came up with is to add some test cases covering particular Bluetooth workflows that can be included as Optional level tests in the desktop validation matrix.

Here's his list of flows to cover:

Headsets:
1. Pair device
2. Newly paired bluetooth device should be the default audio source (sound settings)
3. If the device supports A2DP, A2DP Sink should be default (sound settings)
4. Run the audio test ("Test Speakers" in Sound Settings)
5. Test music playback
6. If the device has a mic, make sure the mic works in a VOIP/meeting app (ex. Bluejeans)
7. Check any media buttons on the headset (play, pause, back, forward)
8. Power cycle the headset. Make sure device reconnects automatically. Run tests again.
9. Reboot system. Make sure device reconnects automatically. Run tests again.

Keyboards:
1. Pair device
2. Test each key
3. Test any media buttons present
4. Power cycle the keyboard. Make sure the device reconnects automatically. Run tests again.
5. Reboot system. Make sure device reconnects automatically. Run tests again.

Mice:
1. Pair device
2. Test mouse movement
3. Test all mouse buttons
4. Power cycle the mouse. Make sure the device reconnects automatically. Run tests again.
5. Reboot system. Make sure device reconnects automatically. Run tests again.

These should be turned into test cases - following the SOP , also use existing desktop test cases as references. Then the test cases can be added to the matrix. Thanks!

Nat, something that would help is if you can provide some info we can turn into a debugging guide along the lines of https://fedoraproject.org/wiki/How_to_debug_Wayland_problems - that will help people file better reports when they find bugs.

It would be nice if we could see interesting events happening (exposed by fedmsgs) printed out into our #fedora-qa freenode channel. Other groups/channels already use this.

After talking to puiterwijk, we need to open an infra ticket and provide a regexp that matches the fedmsg messages that we want. This is how the configuration looks like:
https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/fedmsg/irc/templates/ircbot.py

From top of my head, these events would be helpful:
1. Any commit, issue or pull request event happening in projects https://pagure.io/fedora-qa* and https://pagure.io/taskotron*.
2. Changes to fedocal QA calendar: https://apps.fedoraproject.org/calendar/QA/
3. Blocker bug or FE nominations. Can we intercept that?
4. Our meetings being started.
5. Nagios messages for Taskotron and OpenQA machines.
6. Emails to test-announce.
7. Bugzilla events for packages owned by qa-tools-sig.
8. Infra ansible commits into taskotron and openqa playbooks.
9. FAF report/problem reaching very high number of occurrences.
10. New blogposts on Fedora Quality Planet.
11. Wiki edits to important QA pages (testcases, SOPs, etc).

Wdyt? Anything else to add?

This is an idea from the F35 QA Retrospective, particularly from @ahmedalmeleh 's feedback (thanks Ahmed!)

The idea is to make a video to demonstrate common Bugzilla operations for new contributors - filing a bug, searching existing bugs, maybe adding attachments, things like that. I'm not sure of the best way to distribute it, I guess put it on a video sharing site and post to the mailing list and blogs about it, and include it in the pages we direct new contributors to?

We could do similar videos for other things (update testing, for e.g.) if this works out well.

@sumantrom volunteered to work on this, so I'll assign it to him. Thanks sumantro!

This is an idea from the F35 QA Retrospective, specifically from a suggestion by @sumantrom . He wrote "Onboarding session on Bugzilla and test case writing", and, well, that seems like a good idea, so let's do one :) I'll assign this ticket to @sumantrom himself, as he usually does the onboarding sessions.

As suggested by @sdharane , we could write a little script to check that the packages we request to be included in composes actually make it in.

The 'check if package X is in the compose' part of this should be trivial, as fedfind already has code for that. A bit harder is 'parse the compose request and pull out a list of packages to check for', since the requests are technically unstructured text; we usually paste a request generated by blockerbugs, but in theory we don't have to and we do sometimes hand-edit it and add text to it, sometimes saying things like "not including XXX" which could be tricky to parse. There's also the question of knowing which comments from the request ticket associate with the latest compose.

My initial plan for this would, I guess, be to parse every comment on the ticket, pull out every line that looks exactly like how blockerbugs generates them (and ignore any other line), parse all package NVRs out of those lines, reduce to the latest listed NVR for each package, and query for those in the most recent candidate compose.

This test day will focus on IPA-IPA Migration tool. Adding this ticket for tracking the effort.

Ref: https://freeipa.readthedocs.io/en/latest/designs/ipa_to_ipa_migration.html

Owner: Justin Forbes
Time Frame: 1st Dec - 7th Dec
Features: Kernel 6.12 rebase

As discussed at Flock, we will start hosting test days for each major kernel rebase (approximately every 8-10 weeks). It was discussed that we should extend this to a test week, to allow time for more participants. They have been incredibly helpful so far. As the 6.12 kernel is released, I would like to get a test week in before we push the rebase to stable releases. I will have ISOs for USB images so that users can test on bare metal or in VMs, and the kernel will be available in the stabilization repo for users who wish to install it on existing systems.

Hi, we would like to make a test days for Anaconda Web UI which is now part of the Fedora Workstation Rawhide. We were thinking if it would be possible to do make the Test Days next week. Would that be possible?

We are looking for a help from the Fedora QE to guide us through the process if possible.

There are two goals for this test days:
- Test bugs of the Web UI.
- Getting feedback from the testers for this new interface. We would like to know what is working and what they see as an UX issue.

We should run a Test Day for https://fedoraproject.org/wiki/Changes/AnacondaWebUIforFedoraWorkstation , which is slated for Fedora 40 (delayed from Fedora 39). Current F40 Workstation live images have the new installer workflow. We should particularly focus on the new design (available since 20240210.n.1 compose) where the 'custom partitioning' workflow is provided by Cockpit's storage module. See https://pagure.io/fesco/issue/3169 for more on that.

This tracks Podman v5.3.0 test week which will be happening 2024-10-28.

@lsm5 suggest the following.
a) are the dates working for y'all?
b) can you suggest any test case we should be adding?
c) anything else to keep in mind? like known issues?

Test Day Issue

This issue helps the QA team to take in test day request and put them on a release cycle. We might have a few follow up after a proposal is made.

Details

1. Name:
2. Dates/Range of Dates:
3. Owner:
4. Changeset wiki:

Misc

• Test that operating system updates are working properly
• Test that major updates (rebases) from F40 are working properly
• Test bootupd: https://fedoraproject.org/wiki/Changes/FedoraSilverblueBootupd

After having a word with the change owner (Timothée Ravier) we decided that having Kinoite test day is a good idea and we will have it around or after the F39 beta release. This issue is logged to track the progress for the said test day.

edit adamwill 2024-02-21: f39 is out already, let's try for f40

Test Day Issue
This issue helps the QA team to take in test day request and put them on a release cycle. We might have a few follow up after a proposal is made.

Details
Name: Fedora CoreOS Test Week
Dates/Range of Dates: Oct 7, 2024 - Oct 11, 2024
Owner:
Changeset wiki:
Misc
This is a tracker - FCOS test week for Fedora 41 release cycle
https://github.com/coreos/fedora-coreos-tracker/issues/1801

Owner: Justin Forbes
Time Frame: 29th Sep - 5th Oct
Features: Kernel 6.11 rebase

As discussed at Flock, we will start hosting test days for each major kernel rebase (approximately every 8-10 weeks). It was discussed that we should extend this to a test week, to allow time for more participants. They have been incredibly helpful so far. As the 6.11 kernel is released, I would like to get a test week in before we push the rebase to stable releases. I will have ISOs for USB images so that users can test on bare metal or in VMs, and the kernel will be available in the stabilization repo for users who wish to install it on existing systems.

Test Day Issue

This issue helps the QA team to take in test day request and put them on a release cycle. We might have a few follow up after a proposal is made.
Details

Name: i18n Test Week
Dates/Range of Dates: TBD
Owner: @paragn @pravins
Changeset wiki:

Misc

Hello.

I believe Bodhi updates for fedora-obsolete-packages should be treated as critical path updates. The package is quite critical and dangerous: it can accidentally Obsolete something that's needed and not retired.

I'd like the OpenQA gating tests to run/gate when fedora-obsolete-packages is updated.

However, I don't know if we can add the package to any critical path groups in comps -- after all, the package is not installable. This might also break assumptions from OpenQA tests.

E.g. the fedora-ci.koji-build.installability.functional test fails every time with this package.

Please help me accomplish my goal. Thanks

The podman 5.2 test days are going to be from 29th July to 31st July. This tracker will keep tab of the progress

cc @lsm5 @mohanboddu

Owner: Justin Forbes
Time Frame: 28th July - 3rd August
Features: Kernel 6.9 rebase

As discussed at Flock, we will start hosting test days for each major kernel rebase (approximately every 8-10 weeks). It was discussed that we should extend this to a test week, to allow time for more participants. They have been incredibly helpful so far. As the 6.10 kernel is released, I would like to get a test week in before we push the rebase to stable releases. I will have ISOs for USB images so that users can test on bare metal or in VMs, and the kernel will be available in the stabilization repo for users who wish to install it on existing systems. The 6.10 kernel will also be the release kernel for F41.

This issue helps the QA team to take in test day request and put them on a release cycle. We might have a few follow up after a proposal is made.
Details

Name: GNOME Test Week
Dates/Range of Dates: TBD
Owner: @kalev @otaylor
Changeset wiki:

Misc

Looking through the list of sysadmin-qa members, I see some names that might not need to be in there any more. We should probably clean it out. I'm not an admin of that group so I can't, but @kevin can. @tflink can maybe help with this?

We will be testing the Fedora a11y features in Fedora 41 (Rawhide) the idea will be to ensure coverage of the packages that have just been added to the Gnome Workstation Packageset [0].
The test cases will be user-focused and will cover basic scenarios for starters.

The a11y team will review the test cases and have full stakeholdership with QA in the test week.

[0] https://pagure.io/fedora-comps/pull-request/947#request_diff

Test Day Issue
This issue helps the QA team to take in test day request and put them on a release cycle. We might have a few follow up after a proposal is made.

Details
Name: Fedora CoreOS Test Week
Dates/Range of Dates: TBD
Owner: @aaradhak (please update as you feel fit)
Changeset wiki:
Misc
This is a tracker - FCOS test week for Fedora 41 release cycle

@coremodule , please update this tracker when you have details for test week

This instalment will be for testing DNF 5 system-upgrade functionality. This change is going to reflect as a part of a bigger changeset in Fedora 41[0] and a long series of test days [1]. This issue will keep tracking progress.

[0] https://fedoraproject.org/wiki/Changes/SwitchToDnf5
[1] https://fedoraproject.org/wiki/Changes/SwitchToDnf5#Testing_days

Our devs for this test days will be @egoode @jkolarik @jmracek

I met with Podman folks yesterday and we decided to do Podman Test Days for every minor release. The upcoming one is 5.1 and we are going to have a test day on 2023-05-23.

Kudos to @lsm5 and @mohanboddu for all the support!

Owner: Justin Forbes
Time Frame: 26th May - 1st June
Features: Kernel 6.9 rebase

As discussed at Flock, we will start hosting test days for each major kernel rebase (approximately every 8-10 weeks). It was discussed that we should extend this to a test week, to allow time for more participants. They have been incredibly helpful so far. As the 6.9 kernel is released, I would like to get a test week in before we push the rebase to stable releases. I will have ISOs for USB images so that users can test on bare metal or in VMs, and the kernel will be available in the stabilization repo for users who wish to install it on existing systems.

https://fedoraproject.org/wiki/Changes/ModernizeLiveMedia is a significant change to how live media are built. It would be good to have a test day for it, particularly to test more 'advanced' features of live images that are less commonly used and not tested by automation, e.g. overlays.

In this blocker proposal it was claimed that nautilus doesn't need to withstand a basic functionality test for archive management, because it's primarily a file manager and not an archive manager (even though it's the primary and only archive manager in Fedora Workstation default install).

It would be good to clarify the criterion and define requirements when an app serves multiple roles (and perhaps even when there's no app for a particular role). Perhaps the criterion could also be rewritten to be focused on roles (browsing the web) instead of apps (web browser), to make it even clearer.

Owner: Justin Forbes
Time Frame: 24th March - 30th March
Features: Kernel 6.8 rebase

As discussed at Flock, we will start hosting test days for each major kernel rebase (approximately every 8-10 weeks). It was discussed that we should extend this to a test week, to allow time for more participants. They have been incredibly helpful so far. As the 6.8 kernel is released, I would like to get a test week in before we push the rebase to stable releases. I will have ISOs for USB images so that users can test on bare metal or in VMs, and the kernel will be available in the stabilization repo for users who wish to install it on existing systems. As 6.8 is also going to be the release kernel for Fedora 40, extra testing is required.

There have been a series of critical packages being pushed stable too fast and breaking our distro. I failed to write them down in the past, but I'm starting today.

The latest example is grub2 from the last week. It spent mere 13 hours in updates-testing and then it was pushed because of +3 karma. Just 1 hour afterwards a negative comment was added, but it was too late. The update ended up in stable and probably broke booting to Windows for all our users on UEFI. The details are here:
https://bodhi.fedoraproject.org/updates/FEDORA-2022-8ffd58c713
https://bugzilla.redhat.com/show_bug.cgi?id=2115202

While we can blame the package maintainer for keeping the default "Stable by Karma = 3" for such a critical package like grub (can you imagine a more critical package?), instead of raising it to something like 15 or disabling it completely, I believe this is a policy failure. In particular this policy:
https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/
and especially this section:
https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/#karma-requirements

The defaults are completely wrong for critical packages like grub. Three thumbs-up in a short succession shouldn't mean that grub gets pushed immediately and automatically to millions of users. Yet, if the package maintainer doesn't pay attention to every single update they create and reconfigure it to something more reasonable every single time, that's exactly what happens. And I believe it's QA's job to come up with a better policy with better defaults and convince FESCo that it needs to be changed.

Let's use this ticket to track this issue and let's collect thoughts on what a better policy should look like either here on in the mailing lists.

This is a variation on #638 which has yielded this test case
https://fedoraproject.org/wiki/User:Sumantrom/Draft/preserve_home_data

Differences are in bold.

Step 0. Perform a Fedora 33 installation using Automatic partitioning (I used Workstation but any desktop should work because partitioning, subvolume layout, and bootloader stuff is all the same). Reboot and go through g-i-s.

1. Boot the second Fedora 33 installer, make sensible choices
2. In Installation Destination, click Custom under Storage Configuration
3. Locate the previous Fedora installation, click on it to reveal existing mount points for reuse
4. Select /boot/efi or BIOS Boot mount point, on the right hand side at the top, find ''Mount Point'' field, type in /boot/efi, click ''Update Settings'' button.
5. Select /boot mount point, on the right hand side at the top, find ''Mount Point'' field, type in /boot, do NOT check the ''Reformat'' box, click ''Update Settings'' button.
6. Select /home mount point, on the right hand side at the top, find ''Mount Point'' field, type in /home, click ''Update Settings'' button.
7. Click + button to create a new / mount point (this is required to be a new subvolume, reformat will be checked, cannot be uncheck but the underlying Btrfs is not reformatted, optionally change the name of the subvolume from root00 to rootkde or rootjam or whatever it is). Click ''Update Settings'' button.
8. Click Done
9. Perform the installation and reboot the machine
10. After logging in, check that your home directory or /home contains the files you created on your previous system (check their contents as well)
    ~~Note: If you don't delete the previous root subvolume, the installer will create a new subvolume root00 mounted at /. If you delete the previous root subvolume first, the installer will use the same root name for the new one. Both approaches are valid, you can use whichever approach you choose. Other than consuming space, there is no disadvantage to keeping the old root.~~ striking it, because we want to keep the old root

There are some armhfp and some aarch64 images marked as release blocking (note that armhfp XFCE will get removed and aarch64 Workstation added, the change is not live yet). But I haven't found a list of devices which are considered blocking. There is a list of supported devices. Do we block on all of them? Especially with aarch64 Workstation, which is going to be demanding, do we consider all aarch64 hardware from that list as supported and blocking that image, or just some? This needs to be clarified, so that we know which devices to use for testing which images, and when to block or not block. This clarified information will get linked from our test cases or release criteria.

Please communicate with the ARM team and ask them to provide documentation on release blocking devices <-> release blocking images mapping.

Note: This is not about Fedora IoT, that has specific release criteria. This is about general ARM32/64 composes.

@coremodule @sumantrom Do you want to take this?

Some past events in the testdays db have been polluted with spam from an attempted sql injection attack, e.g. https://testdays.fedoraproject.org/events/168 . This is unfortunate as the genuine results of the event are lost (I wanted to export them to the wiki). Can we retrieve the valid results for the event from a past db backup, or if not, at least just remove it?

Several release criteria are currently missing 'enforcing' release validation test cases:

• https://fedoraproject.org/wiki/Fedora_20_Alpha_Release_Criteria#Package_sets missing a test case for the KDE package set
• https://fedoraproject.org/wiki/Fedora_20_Alpha_Release_Criteria#Disk_selection
• https://fedoraproject.org/wiki/Fedora_20_Alpha_Release_Criteria#Required_applications missing a test case for terminal
• https://fedoraproject.org/wiki/Fedora_20_Alpha_Release_Criteria#Updates missing a test case for yum updates
• https://fedoraproject.org/wiki/Fedora_20_Beta_Release_Criteria#Scripted_installation associated test case does not entirely reflect criterion requirement
• https://fedoraproject.org/wiki/Fedora_20_Beta_Release_Criteria#Unattended_installation associated test case does not entirely reflect criterion requirement
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Media_consistency_verification test case does not confirm that verification mechanism works
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Live_image_persistent_overlays
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Network_attached_storage missing FC and FCoE test cases
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Installation_interfaces missing cmdline
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Bootloader_disk_selection
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Installer_translations
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Critical_path_translations
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Pre-release_notices
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Kickstarts
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Release_notes
• https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Release_identification

We should create, and add to the appropriate release validation template page, test cases to cover each of these criteria.

As the new Auth system is in place, unlike past, the members who want to join the FAS group now have to added by one of the person. In the past, we have seen people requesting for the QA group and it's outdated now.
There are two ways we can go from here:

1. we can have people who send intro emails to tell us their interest in joining the QA group and we can add them

2. We ask contributors to file ticket with sponsor tag on our fedora-qa pagure repo and post which we can add them up.

I understand we are closing in a release and this is probably not the time to think about this one. Hence we can revisit this topic during our next QA meeting. However, if anyone has thoughts and a spare bit of time, please weigh in!

There's a current proposal for specific release criteria for graphical package managers. Once that proposal is finalized, assuming we do actually adopt some new criteria, we should write some test cases to cover them. Once the test cases are written, we can automate them - I'll file another ticket for that.

Assigning to @kparal for now as he's drafting the criteria.

@ngompa proposed this [0] and I would love to see this added to the regular cadence as part of Test Days 2.0. Indeed a lot of our uses play games, watch movies on VLC , use VSCODE and many more such software. Having a community event around these and making sure all works well before taking the leap of faith (aka upgrading) sounds like a legit thing.

This also means we will need to add a few additional rows in the desktop release validation matrix.

Apps we think should benefit from testing:

Steam
Chrome
VS Code
VLC

[0] https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/thread/NKE52N6DRJ5GNBMKC52UDTQ3KAQKC3AG/

There various special considerations around Fedora IoT and CoreOS releases.

• IoT: IoT images are composed in a separate stream from the main "Fedora" compose - there are "Fedora IoT" composes. At each release point, we need to co-ordinate with the IoT team and nominate an image from the Fedora-IoT stream for release; we should make sure the image nominated for release has all (relevant, at least) fixes from the override repository, and passes all the IoT tests. Currently this is being done just in an ad hoc way, either in the Go/No-Go meeting if we remember or in Fedora Chat conversations; we should probably make this more formal and write it into the relevant policies (e.g. the Go/No-Go process documentation).

• CoreOS: similar to IoT, CoreOS has its own build/CI/release process, using streams. Similarly to IoT, we need to make sure that all the override repo fixes reach the CoreOS releases (this should always happen as the override repo packages are pushed stable soon after the candidate is signed off, but if we ever decrease the CoreOS stable release "lag time" this could be a consideration). Right now the CoreOS team more or less decides when to kick the stable stream over to the new release, and this happens a few weeks after release; this process is generally known to the relevant teams and doesn't cause problems, but it isn't really formally documented anywhere and we do not agree the schedule as part of the release process. Doing those things might be an improvement.

@sumantrom flagged this up as an issue and wants to work on it, I'm also interested in this.

We need to review and update the release criteria and test cases for the current implementation of the webUI for Workstation Change.

As of right now, the storage workflows are:

1. Let webUI completely wipe the selected disk(s) and auto-partition it/them
2. Use available free space for auto-partitioning without deleting anything
3. Assign mount points to existing partitions, optionally (except for /) reformatting them
4. Launch cockpit's storage module, with a bit of integration, to perform custom partitioning operations before proceeding to 2) or 3)

Broadly speaking, I think we need to review the 'guided partitioning' criteria and test cases to see exactly how to map the new flows onto them, and clone the 'custom partitioning' test case set again for cockpit-based partitioning, plus some criteria tweaks.

I am working through this at present. So far I've done minor and hopefully uncontroversial tweaks to the release criteria, and I've cloned the custom partitioning test case set in the install matrix template and implemented the three easiest test cases, with some new templates. The rest of the test case links are still to pages that don't exist, at present.

I think it used to work this way in the past, but now it doesn't, it seems. It looks reasonable to enable it. Otherwise people who follow just test list don't see the announcements. Thoughts?

I did this dumb draft a while back. We want to add something like this for real. We should cover systems we have access to which are used by a lot of folks, and especially ones which come preloaded with Fedora and/or have support from the vendor - like Framework, Slimbook, some Lenovo models.

We should also have a table for ARM hardware to try and make sure we actually do the ARM hardware testing at Beta. It should list at least Raspberry Pi (3 and 4), and any other ARM platforms we think are sufficiently testable and real-world interesting to merit planned hardware testing.

CC @pbrobinson @pwhalen @coremodule @lbrabec @ctsdownloads ... @mpearson er...anyone know someone at slimbook to CC?

I will do the work on this, just wanted folks in the loop, and to suggest specific models and things it would be good to test.

We recently approved new release criteria related to networking. At least some of those requirements probably need specific test cases, e.g. wifi and VPN stuff.

I'd really recommend writing a test suite before the code gets any more complex. You've already got quite the forest of conditionals in polish_data, for instance, and it's very easy for that to get thrown off. Ditto most of interpreter.py.

As part of this you may find it easier to refactor a bunch of those elif forests into separate functions / methods - then you can test each one separately.

It seems impossible to create milestones in the admin interface right now (in whatever's deployed to production). When you try to create a milestone, the form you're shown has no "release" field, as it used to. If you ignore that and try to create the milestone, you get an error "Failed to create model. 'MilestoneForm' object has no attribute 'release'".

We need this fixed before F39 Final is done or else we won't be able to track F40 blockers.

Related issue from infra : https://pagure.io/fedora-infrastructure/issue/10567

Error is :

Bad Request

Request Line is too large (4175 > 4094)

It show very long URL after I clicked login (I can see FAS Login then It shows error, open id works but qa.fpo doesn't login)

You can create a milestone that makes the app crash every time anybody tries to open the frontend.

It apparently happens when the milestone is not marked as final/beta or anything else. I'll investigate this later.

Traceback:

Apr 22 21:15:44 blockerbugs01.phx2.fedoraproject.org httpd[4589]: app          2019-04-22 21:15:44 ERROR   Exception on / [GET]
                                                                  Traceback (most recent call last):
                                                                    File "/usr/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
                                                                      response = self.full_dispatch_request()
                                                                    File "/usr/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
                                                                      rv = self.handle_user_exception(e)
                                                                    File "/usr/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
                                                                      reraise(exc_type, exc_value, tb)
                                                                    File "/usr/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
                                                                      rv = self.dispatch_request()
                                                                    File "/usr/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
                                                                      return self.view_functions[rule.endpoint](**req.view_args)
                                                                    File "/usr/lib/python2.7/site-packages/blockerbugs/controllers/main.py", line 173, in index
                                                                      return render_template('index.html', title='Fedora Blocker Bugs')
                                                                    File "/usr/lib/python2.7/site-packages/flask/templating.py", line 128, in render_template
                                                                      context, ctx.app)
                                                                    File "/usr/lib/python2.7/site-packages/flask/templating.py", line 110, in _render
                                                                      rv = template.render(context)
                                                                    File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 969, in render
                                                                      return self.environment.handle_exception(exc_info, True)
                                                                    File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 742, in handle_exception
                                                                      reraise(exc_type, exc_value, tb)
                                                                    File "/usr/lib/python2.7/site-packages/blockerbugs/templates/index.html", line 1, in top-level template code
                                                                      {% extends "base_nav.html" %}
                                                                    File "/usr/lib/python2.7/site-packages/blockerbugs/templates/base_nav.html", line 1, in top-level template code
                                                                      {% extends "layout.html" %} {% block info %} {% endblock %} {% macro navactive(name, subname='') -%} {%- if name == (request.path | getname) and subname == (request.path | getsubname)
                                                                    File "/usr/lib/python2.7/site-packages/blockerbugs/templates/layout.html", line 55, in top-level template code
                                                                      {% block navigation %}
                                                                    File "/usr/lib/python2.7/site-packages/blockerbugs/templates/base_nav.html", line 14, in block "navigation"
                                                                      <a class="nav-link" href="{{ url_for('main.display_buglist', num=ms.release.number, release_name=ms.version) }}"
                                                                    File "/usr/lib/python2.7/site-packages/flask/helpers.py", line 312, in url_for
                                                                      return appctx.app.handle_url_build_error(error, endpoint, values)
                                                                    File "/usr/lib/python2.7/site-packages/flask/app.py", line 1641, in handle_url_build_error
                                                                      reraise(exc_type, exc_value, tb)
                                                                    File "/usr/lib/python2.7/site-packages/flask/helpers.py", line 305, in url_for
                                                                      force_external=external)
                                                                    File "/usr/lib/python2.7/site-packages/werkzeug/routing.py", line 1620, in build
                                                                      raise BuildError(endpoint, values, method)
                                                                  BuildError: ('main.display_buglist', {'num': 31}, None)

Our infra SOP doesn't reflect that we're deployed to OpenShift now:
https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/blockerbugs/

@frantisekz Are you going to update that SOP? Or do something else? Thanks.

We are currently bundling Bootstrap 2 (because of admin UI) and we'll bundle also (Fedora) Bootstrap 4 once #66 gets merged.

We should convert Admin UI to use the same framework version as the rest of the app.

When the user sees data for an old invalid milestone, it is not immediately obvious. For example, look at data for F35 Final:
https://qa.fedoraproject.org/blockerbugs/milestone/35/final/buglist

It is not clear that it's F35. It is not clear that it's no longer updated.

Proposed solution:
If the displayed milestone is not active, show the milestone identification ("Fedora 35 Final") and some visible warning dialog that this milestone is no longer active and the displayed data are outdated. Make sure to display it in all related views (Info, Bug List, Updates).

I'd like to have a button in the admin interface to manually trigger blockerbugs sync. In some cases it can help to be able to refresh bugs when dealing with some issue, without actually having to connect to the server.

fedora-messaging publishers are supposed to provide a 'message schema' - https://fedora-messaging.readthedocs.io/en/stable/messages.html#schema . This will be used by the fedora-messaging-native rewrites of datagrepper, fmn etc. to provide human-readable representations of messages. We should provide one for openqa.* messages. The ci.* messages should be handled by https://pagure.io/fedora-qa/python-ci_messages .

It would be cool if we could test https://pagure.io/workstation-ostree-config pull requests. Then we'd find things like https://pagure.io/workstation-ostree-config/pull-request/330#comment-190406 before the PR gets merged and "pollutes" other tests.

The test code already checks out workstation-ostree-config, all it would need to do is use the pull request content instead of HEAD.

In the scheduler, we'd need to figure out how to 'listen out' for PRs (does pagure.io publish fedora-messaging messages?), have a path for scheduling them (we'd need a naming convention for the BUILD setting, and a setting that told the test exactly what PR to check out - those might be combined, even - and...I think that's it?), and a way to report the results back. I'm not sure exactly how pagure.io consumes results - if there's a way to get it to pay attention to resultsdb, or if we actually have to send it a result directly, or via webhook or whatever the heck mechanism these things use, I have never looked into it before.

I noticed recently that os-autoinst has had support for decompressing compressed disk images on-the-fly since 2018. Using this should allow us to POST .raw.xz disk images directly as HDD_X_URL rather than using HDD_X_DECOMPRESS_URL, which triggers decompression code in openQA, which we wrote (initially) and which predates the os-autoinst code.

Doing it in os-autoinst rather than openQA would make the scheduler code slightly simpler, and would also potentially allow us to drop the decompression code from openQA itself, which would be a benefit to upstream.

I'm already working on a branch that implements this; unfortunately we run into mysterious permission problems, even though I've set everything up now so the user which os-autoinst runs as should have permission to uncompress the files, and indeed it works fine in a test using runuser.

Currently, in the Assigned column, it seems that the repos are sorted alphabetically. However, that means that my currently assigned tickets with SPs are somewhere in the middle and I need to search for them:

Assigned
--------
repo1
- ticket 1
- ticket 2
repo2 (10 sp)
- ticket 1 (5 sp)
- ticket 2 (5 sp)
repo3
- ticket 1
- ticket 2

Instead, place repos which have any SPs on top (in alphabetical order), and only then place repos without any SPs (in alphabetical order), like this:

Assigned
--------
repo2 (10 sp)
- ticket 1 (5 sp)
- ticket 2 (5 sp)
repo1
- ticket 1
- ticket 2
repo3
- ticket 1
- ticket 2

This will help me find my currently "active" (assigned and rated -> working on them) tickets quickly and not miss them.

I think this change only makes sense to do in the Assigned column. I'd also limit it to only when a user is logged in.

Thanks.

At the moment there are so many redundant code in the files of the landingpage folder and use of class component where functional component is preferable. it will be performant to remove redundant code and make use of functional components when preferable.

I saw that at many places, className="col-md-11" is being used along with <Row />, I think we should move the same to use <Col/> components.
@lbrabec, What do you think about this?

Sometime yesterday, fedfind was updated not to include Fedora 32 in current Fedora releases. Everything seemed okay, the release disappeared from the frontend and sync commands continued to operate normally, without trying to include Fedora 32.

However, /api/v1/package_versions/<package> endpoint started to crash on KeyError: 'Fedora 32'. That stopped after I manually refreshed CACHE for 'fedora_releases' handle which returned even the Fedora 32 on CACHE.get('fedora_releases') in debug console before refreshing it manually.

I should have checked the timestamp in db before refreshing it, forgot to do that :(

app          2021-06-04 12:51:33 ERROR   Exception on /api/v1/package_versions/postfix [GET]
Traceback (most recent call last):
  File "/opt/app-root/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/app-root/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/app-root/lib/python3.6/site-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/opt/app-root/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/app-root/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/opt/app-root/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/app-root/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/app-root/src/oraculum/controllers/packager_dashboard.py", line 196, in route_package_versions
    return(jsonify(versions.get_package_versions(package)))
  File "/opt/app-root/src/oraculum/utils/versions.py", line 156, in get_package_versions
    if package in versions_cache[release_string]:
KeyError: 'Fedora 32'

We should add support for Fedora ELN Branch.

We'll need to adjust functions all over the place (generic dashboard helpers and modules parsing data from each source).

From a quick look, following parts would need an adjustment:

• dashboard_helpers: release_is_active(), release_from_dist(), release_from_nvr(), get_fedora_releases()
• health_check
• bugzilla
• pagure

I'll update the list and file a PR later. It's possible some parts would need an adjustment in the future (eg. bodhi and fedora health check currently don't care about ELN, but they might start doing so)

Infrastructure is backing up databases in 00:00 UTC + sleep $[ ( $RANDOM % 7200 ) + 1 ]s every day. We should pause scheduled sync jobs during that time frame.

Some users (ngompa, salimma) cause huge spikes in db load and ram consumption which cause trouble on the deployment side.

Find a way to lower resource consumption and db pressure or limit the features and data provided to users with number of packages >= some sensible value.

Exception in worker process
Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.11/site-packages/bugzilla/_session.py", line 108, in request
    response.raise_for_status()
  File "/opt/app-root/lib64/python3.11/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://bugzilla.redhat.com/rest/user?ids=1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.11/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
    worker.init_process()
  File "/opt/app-root/lib64/python3.11/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/opt/app-root/lib64/python3.11/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
                ^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
                    ^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/gunicorn/util.py", line 371, in import_app
    mod = importlib.import_module(module)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/app-root/src/oraculum/__init__.py", line 191, in <module>
    raise e
  File "/opt/app-root/src/oraculum/__init__.py", line 184, in <module>
    assert BUGZILLA.logged_in
           ^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/bugzilla/base.py", line 720, in logged_in
    raise e
  File "/opt/app-root/lib64/python3.11/site-packages/bugzilla/base.py", line 714, in logged_in
    self._backend.user_get({"ids": [1]})
  File "/opt/app-root/lib64/python3.11/site-packages/bugzilla/_backendrest.py", line 182, in user_get
    return self._get("/user", paramdict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/bugzilla/_backendrest.py", line 63, in _get
    return self._op("GET", *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/bugzilla/_backendrest.py", line 58, in _op
    response = self._bugzillasession.request(method, fullurl, data=data,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/bugzilla/_session.py", line 112, in request
    raise type(e)(message).with_traceback(sys.exc_info()[2])
  File "/opt/app-root/lib64/python3.11/site-packages/bugzilla/_session.py", line 108, in request
    response.raise_for_status()
  File "/opt/app-root/lib64/python3.11/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://bugzilla.redhat.com/rest/user?ids=1

I've worked on and deployed a new version of repochecker, which now provides a bit more data. In particular, the data now also includes all affected (co-)maintainers, and not only the "main admin".

Data entries now look like this:

  {
    "source": "lucene",
    "package": "lucene",
    "epoch": "0",
    "version": "8.8.2",
    "release": "1.fc35",
    "arch": "src",
    "admin": "orphan",
    "maintainers": [
      "kdaniel",
      "jerboaa",
      "rgrunber",
      "@eclipse-sig",
      "dbhole",
      "lef",
      "dchen",
      "java-sig",
      "orphan"
    ],
    "repo": "fedora-source",
    "repo_arch": "s390x",
    "broken": [
      "ivy-local"
    ],
    "since": "2021-09-09T17:06:03.439823324Z"
  },

The original "admin" field is still present with the same meaning (the "main admin" of the package) for backwards compatibility, but the "maintainers" field is new, and it contains all users and @-prefixed groups associated with a package.

Right now, only the "main admin" is shown FTI/FTBFS problems in the packager dashboard, using the new, more complete data should fix that.

Looking at packager_dashboard#127, I could also work on adding a field for the default bugzilla assignee(s) to the data, if that would help? cc @churchyard

We have (or had, till I temporarily removed them) a pair of tests install_blivet_no_swap and install_no_swap which test installing without a disk swap partition. Which until now was an unusual variant config we wanted to make sure was still allowed and kept working.

The new swap on ZRAM thing, though, changes this up a lot. Having no disk swap is now the default layout, so a test that assumes it's a variant configuration and explicitly tests it doesn't really make any sense any more.

We probably will want to replace it with a test that creates a disk swap partition, and tests booting with both swap-on-ZRAM and the disk swap partition active, and also booting with just the disk swap partition active but swap-on-ZRAM not active, and makes sure everything works as expected in both cases.

We'll also want to amend all other existing custom partitioning tests to not create a disk swap partition any more, I guess.

We might also want to have a test that starts from a default (no disk swap partition) install and boots with swap-on-ZRAM disabled to make sure not having swap at all still works.

We'll also want to change up the associated wiki test cases in the same way; I'll use this ticket to track that work too.

I'm planning to take this, but wanted to file an issue just to note what's going on and for the PR to point to when it's ready. As mentioned, for now I've removed these two tests entirely because they're not doing anything useful and they fail and it would be kinda weird to just "fix" them and let them run and not test anything we really need to test.

Rawhide has switched anaconda to Wayland, and as part of that, VNC is replaced with RDP. We need to adjust the tests to use RDP when testing F42+.

@adamwill, I have been instructed by @sdharane to start working on testing the subject.

I believe that we could re-use Gnome tests that we already have. We could switch on accessibility (various types of it) and just create another bunch of needles to match them with the changed graphics.

When an increased number of needles is a concern, we could theoretically run different Gnome tests with various accessibility settings, for instance we could run Calculator with increased fonts, and Text editor with different color combinations, etc. This would save some needle space.

What do you think about this?

DNF5 will be the new console package manager in Fedora 39. We might want to get ready for that by automating the test cases before it happens.

https://fedoraproject.org/wiki/QA:Testcase_rpm-ostree_upgrade is a blocking test for Fedora-IoT, and a good candidate for openQA automation. We should implement it.

Both custom partitioning paths (regular and blivet) have a "reset" button which should throw away all in-progress work and reset the UI to the actual state of the disks. This being broken was a late blocker for F32, so we should probably be testing it in openQA.

This is a request from the desktop team.

mcatanzaro: The gnome-remote-desktop developers are requesting a remote login test in OpenQA. Besides that it is currently broken by selinux, apparently the next problem is going to be that libadwaita applications are broken in remote sessions with some particular version of libadwaita. A "log in and start nautilus" would catch that.
mcatanzaro: Connect to Fedora host via RDP, log in, launch nautilus, good if nautilus window is shown
mcatanzaro: Yes, using Connections. That's the extent of my knowledge. I'll ask for more detail

When we developed the start_stop application tests we used different names for Gnome needles (apps_run_application) and KDE needles (application_runs).

We want these names be unified into one naming scheme that would be the same for all applications, no matter what DE they come from.

Currently, the menu_launch_type subroutine only starts an application. We want this extended so that it covers for:
* start the application
* check that the application has started
* the check should be skipped if requested
* the application should be maximized after start if requested

The initial GNOME 45 updates had a pretty bad bug - alt-tab didn't work correctly. The openQA tests didn't catch this, so we let it through.

This seems like something we should catch, really. I'm thinking of a test which opens several windows and tests that alt-tab and alt-(keyabovetab) work correctly. Also probably testing window switch via overview and a basic test that virtual desktops work correctly would be good.

The current podman tests in Fedora OpenQA are here:
https://openqa.fedoraproject.org/tests/2024118/modules/podman/steps/1/src

It would be nice to expand these. Discussion upstream is here with specific suggestions is here:
https://github.com/containers/podman/issues/19299

There's an ongoing proposal to add specific release criteria covering graphical package managers, and I've filed a ticket for creating manual test cases covering those criteria once they're finalized. Once the test cases exist, we should automate them in openQA.

The test should prove that the following features are supported.

• starts and stops
• displays applications
• displays installed applications
• displays updates
• installs an application
• removes an application
• displays repositories
• enables and disables repositories
• About screen can be displayed

I noticed looking at OpenQA that a significant number of our storage-based tests are in the "universal" flavor. This would be fine, except the "universal" flavor is Fedora Server, which is now the "special case" flavor by virtue of it being the only flavor using XFS+LVM instead of Btrfs.

Consequently, there seems to be some tests that would be applicable for both the Btrfs and XFS+LVM case, but are only being tested on the XFS+LVM path by virtue of inheriting from Fedora Server's default:

• install_shrink_ext4
• install_shrink_ntfs
• install_simple_encrypted
• install_simple_encrypted@uefi
• install_simple_free_space
• install_simple_free_space@uefi
• install_software_raid
• install_software_raid@uefi
• install_updates_img_local
• install_with_swap
• install_with_swap@uefi

I probably missed some more, but I think this shows the point that we should probably make sure our storage related tests are being done on both configurations.

In particular, we should make sure LUKS+Btrfs is tested, since there's a report that it doesn't work (RH#1876020).

@sgallagh noticed the EOL date in /etc/os-release in Fedora 41 is wrong (it's in May 2025, should be in November 2025).

We could add some kind of heuristic to catch this in the os_release.pm test. Perhaps we could do something like "when testing a compose whose release number is higher than current stable, the EOL date must be more than 8 months in the future", or something like that?

After clicking on of the bugs / updates / overrides / PRs / koschei / FTI / orphan / ABRT icons on the top left, below the search bar / options, it filters to show packages affected by the selected type.

In the past clicking again on the icon restored the view by removing the filter.

Example: if I click on the first icon, bugs, it will show all my packages that have bugs, but will remove the rest of the issues/info. In the past clicking again in the bugs icon would restore view by removing the bugs filter.

Changing sorting preference under "Filters and Options" has no effect and sorting is always based on "Package name". This was the case on the old pre-up prod branch and is the case on the current master/prod too.

When I go to my dashboard, I see 3 FTBFS: git-lfs, ocrmypdf, and tinygo and 2 FTI: ocrmypdf and tinygo.

However, I have been repeatedly updating these, so they most certainly do not FTBFS. On koschei, git-lfs was rebuilt today just fine, ocrmypdf was built yesterday by me, and both have been fine for months. tinygo at least started failing two weeks ago, but it's shown up there longer.

As for the FTI, the reasons given are highly outdated and not reproducible in mock.

When a package shows depends on orphaned packages, clicking the button shows the list of directly and remotely depending packages.

Trying to copy the name of any of those packages by clicking closes the view, making impossible to select the name.

FTI/FTBFS should link to Koschei to make it easy for the packager to fix.

Could the mass rebuild be please included in the Fedora release schedule displayed on the dashboard? Thanks.

I closed https://bugzilla.redhat.com/1905283 and it's still showing on my dashboard despite the sync at the bottom saying:

Last sync was on May 2 2021, 1:45:29 UTC (7 minutes ago).

Feature Request - Ability to hide bugs with tag needsinfo

I would like to be able to see only my packages, or only the packages I have access via groups.

Currently, the top-left corner shows something like this:

<user>: 👤 <counter1> 👥 <counter2> 👁 <counter3>

<counter1> shows my packages, but <counter2> shows all packages. I think that <counter2> should be current <counter2> - <counter1> instead, i.e. only the packages I have access via groups. Then it should be easy to make them clickable, so that the user can filter the view by those criteria.

• filter "Tracking" bugz
• filter "FutureFeature" bugz
• filter "Triaged" bugz
• filter bugz opened by release monitoring
• filter bugz by state

Please add EPEL{8,9}-next categories.

Packages with el8.next and el9.next dist-tags should be filtered too.

I'm getting this warning on my packager dashboard, for cachelib

created 9 days ago for Fedora 35, should have gone stable 21 hours ago

Per nirik, the update was submitted for stable 1.5 hour after the last daily push, so it is expected to not be pushed yet. Perhaps this check should be relaxed to, say, 24 hours + some buffer? maybe set it to 30 hours, and warn on "> (X-24) hours" e.g. if it's been 32 hours, say the update should have gone stable "more than 8 hours ago".

1. Go to Options
2. Disable Show outstanding ABRT reports
3. The ABRT icon in the top right "category" listing is greyed out
4. Click any other category icon than ABRT, e.g. bugs
5. Only bugziallas are displayed
6. Click the selected category (i.e. bugs in the example case) again
7. Everything is displayed, even ABRT reports

We use the python-maint user as a Bugzilla Assignee only, e.g. at https://src.fedoraproject.org/rpms/python3.10/

Problems:

• python-maint user is not a packager
• python-maint does not co-maintain the packages

What would it take to also collect the Bugzilla Assignees from src.fp.o, enable dashboard for them and add the packages they are asignees for to their dashboard?

Currently, the future of Fedora is shown, for the next release. But there is also future information that could be shown for current releases.

Namely, there's EOL for F32 right now, but also date after which updates cannot go stable would be nice as well.

According to https://src.fedoraproject.org/rpms/sssd @lslebodn co-maintains sssd.

sssd is visible at https://packager-dashboard.fedoraproject.org/orphan but not on https://packager-dashboard.fedoraproject.org/lslebodn

Is this a cache thing?

I think it might be good idea to offer an option to display versions of all packages you own for all releases (both stable and testing) in some sort of a table.

We already fetch version data for all the packages users own, I think we can expose it a bit more. (currently, you can see versions only for packages that have some action items on the frontend)

It'll be great if the Packager Dashboard supported the Notifications API.

Users could then allow receiving notifications from the Dashboard and get notifications when something changes on their dashboard, eg.

You packages have X new bugs, Y new pull requests, Z new updates. 

Some more interesting messages could be some/all of the following:

Some of your packages became directly/indirectly affected by an orphaned package. Some of your packages were marked as FTBFS/FTI. Blocker bug has been assigned to some of your packages.

Wdyt, I guess we can brainstorm some more messages here, @churchyard , @decathorpe ?

"failing to install for Fedora rawhide" could be replaced with: "python3-IPy fails to install in Fedora rawhide"

Note that actual package name (could be multiple).

It could also show the reason in the footnote:

Nothing provides: python(abi) = 3.8

testcase_stats doesn't show separate result series by result column in the original page, it smooshes all result columns for the same test instance together. This is IIRC a legacy of the original version which did fairly simple, scraper-based parsing. Now it's based on python-wikitcms there is (AFAIR) no technical reason we can't have it show a separate result series for each result column in the original page - it'll make it a bit harder to read at a glance, but this is very important for things like arch coverage.

We'd also need to tweak how the details pages are handled in this case, will have to think about that.

CC @kparal @ngompa