At the moment, we evaluate task-rpmdeplint only on x86_64 arch. That is of course sub-par, because armhfp is also a primary arch, and we should run at least that as well. With our current design, that would require us having armhfp virthost, so that we can spin up armhfp test client VMs on it. In case of task-rpmdeplint, there's also the option to evaluate armhfp on x86_64 clients, because rpmdeplint doesn't depend on the arch of the system. However, that would require non-trivial changes to our whole workflow (currently it is generic in the sense that running a task for $arch means it is executed on $arch client, and we would need to implement exceptions to this rule).

This is all related to just primary arches, though. The complaint in the linked ticket talks about s390x-only package. Even if we implemented the "run rpmdeplint for any arch on x86_64 client" approach mentioned above, running task-rpmdeplint for all alternative arches would probably be outside of our capabilities. The task is very time consuming because it requires downloading of all packages in the -pending koji tag.

A very nice solution here would be if koji started producing repo metadata for -pending tags. That would decrease the task complexity a lot, speed it up by several orders of magnitude, and there would be no problem running it for all arches including alternative ones.

As a general note, I was hoping Fedora gating would start small and focus on primary arches only. Why does greenwave block packages that are alternative arches only? Is that intentional? I'm not sure whether we can provide the results in a reliable fashion in the near future. We're already having too many issues with executing tasks just for x86_64.

Greenwave doesn't know anything about arches... I think we were working on the assumption that all arches are treated equally by the Taskotron tests.

It sounds like the rpmdeplint tests might be due for a rethink. There is no reason why they need to run on a freshly created VM of the same architecture as is being tested. They should all just run on one central place that can share the cached repodata. I also don't see why they would need to download all packages in -pending either? I think there must be some opportunities for simplification here...

Greenwave doesn't know anything about arches... I think we were working on the assumption that all arches are treated equally by the Taskotron tests.

Is that intentional, or simply hasn't been considered? If greenwave isn't aware of arches, I see the following problems:

• if the test passes for arch X and fails for arch Y, but X is reported an hour earlier than Y, greenwave will approve that update for the one hour duration
• if the failed result for Y doesn't get reported for some reason, greenwave approves the update completely
• if primary arch X passes but alt arch Y fails, the whole update is blocked (that might be OK, because it can be waived)

The assumption was definitely different on our side :) I assumed that greenwave was enabled just for primary arches. We can try to do adjustments from our side, but it's not going to be very soon, as I described. I'd prefer starting small and limiting gating to primary arches (or just x86_64) initially, because the more arches we execute on, the more time is spent on dealing with infra issues (tasks crashing or timing out, storage running out, network issues, resultsdb overloaded) instead of important bug fixes and new features. This is not just about rpmdeplint, task-abicheck is also a hot candidate for gating, and that one is extremely hardware demanding.

It sounds like the rpmdeplint tests might be due for a rethink. There is no reason why they need to run on a freshly created VM of the same architecture as is being tested.

I agree. The current workflow is done in such a way that it is universal, and not necessarily efficient in all cases. Any task can be run in there. Some particular tasks like rpmdeplint would benefit from a different approach, which would be more efficient. That will require non-trivial changes, though. Taskotron was designed and supposed to run dist-git tests, after all (which was recently moved to ci-pipeline).

They should all just run on one central place that can share the cached repodata.

I don't see how we could do that without rewriting the project from scratch. The goal was to get a completely clean environment for each task, and we implemented it like that.

I also don't see why they would need to download all packages in -pending either? I think there must be some opportunities for simplification here...

I'd be very happy to discuss it and find those. We evaluate whole updates-(testing-)pending as a set, instead as individual updates, because they can affect each other. So we process the whole set, and find those that would not be installable against the stable repos, using rpmdeplint. rpmdeplint requires repo metadata, and we don't have it for -pending tags. So we download all packages from it and create the repo metadata. Is there a logical error somewhere? Can we optimize this somehow (apart from asking Koji to create that metadata for us on the server side)?

With latest changes task-rpmdeplint is executed separately for x86_64 and armhfp packages. We still don't run it on everything supported by Fedora, but at least primary arches are covered. Closing this ticket.