Do we relegate because x86 hasn't had bugs like rhbz 2113005 since 2022-08-01?

I am doing a lot of the work on U-Boot for specific device enablement, but that isn't the architecture as a whole where there's a lot of devices that work OOTB without any Fedora shipped firmware. I think the two problems need to be disconnected.

@pbrobinson Sorry about my phrasing, I think I put that poorly. My concern is that you're overworked and under-assisted. Putting the responsibility of an entire blocking architecture on your sole shoulders seems unfair and unrealistic. I'm very much hoping that starting this discussion will lead to more people jumping in to help.

Putting the responsibility of an entire blocking architecture on

The problem is this ticket is considering blocking the entire architecture on a single package. I believe that is the wrong way to be looking at this.

For example I think we need to look at how QE tests since @pwhalen move on from dedicated Arm testing and QE took it over as part of their overall testing, we have comments like https://bugzilla.redhat.com/show_bug.cgi?id=2264415#c16 "almost exclusively use the 400" which means it had been broken for at least 3 weeks before it was reported.

I think before that point it wasn't totally clear that the testers were experiencing a different bug on the 400. In retrospect it's actually deducible from https://bugzilla.redhat.com/show_bug.cgi?id=2264415#c4 (on 20th Feb) but I guess nobody noticed the significance of the different results between 4 and 400 at the time.

But regardless, this is about a larger trend than just one bug; we're often in the position of waiting on an ARM fix and there is nobody who can do anything about it but you, which is an awkward position for everyone, I think. It's surely not fun for you that you wind up me with me and Aoife and anyone else who's taking an interest breathing down your neck for the fixes?

I am doing a lot of the work on U-Boot for specific device enablement, but that isn't the architecture as a whole where there's a lot of devices that work OOTB without any Fedora shipped firmware. I think the two problems need to be disconnected.

On this part, there were a couple of ideas floated in the meeting this ticket came out of, let's put them here:

• (adamw) but i guess we could also say 'no more hardware blocking but virt/cloud is blocking'
• (pwhalen) Perhaps non-blocking desktop images would be more appropriate than the arch completely?
• (pwhalen) Or SBC's, rather only blocking on enterprise.

There has been a combination of some problems with upstream U-Boot and another personal issue out of my control. Both of those issues are coming to an end.

Maybe we need to review how we deal with architecture blockers, sure, see my first comment above. I don't believe we should be throwing the baby out with the bath water.

Do we relegate because x86 hasn't had bugs like rhbz 2113005 since 2022-08-01?

I have considered proposing that we stop making Secure Boot release blocking over it, yes. The fact that this problem continues to exist in Fedora and we keep being forced to waive it again and again has been a point of frustration.

It's a separate conversation, but I am also not happy about the state of Fedora's secure boot support.

I am doing a lot of the work on U-Boot for specific device enablement, but that isn't the architecture as a whole where there's a lot of devices that work OOTB without any Fedora shipped firmware. I think the two problems need to be disconnected.

On this part, there were a couple of ideas floated in the meeting this ticket came out of, let's put them here:

• (adamw) but i guess we could also say 'no more hardware blocking but virt/cloud is blocking'
• (pwhalen) Perhaps non-blocking desktop images would be more appropriate than the arch completely?
• (pwhalen) Or SBC's, rather only blocking on enterprise.

This is basically backwards from how Fedora ARM gets used, so it wouldn't make sense to do that.

Fedora isn't always a tail wagged by its dog. If Fedora's own purpose in supporting aarch64 is not to be used on SBCs, then it's reasonable to consider not blocking on SBCs, even if there happen to be a lot of people using Fedora on SBCs. We're not a consumer product, those people aren't paying us money, so we don't necessarily have to decide their needs are our needs.

While that's true, "enterprise ARM" is generally inaccessible to pretty much everyone who uses Fedora, whereas SBCs are the opposite. I would prefer to just drop ARM entirely rather than flip our release-blocking status to things that practically no one has.

I should also note that we currently do not have release-blocking ARM for either Cloud or "enterprise ARM" platforms (both of which use UEFI). This situation has not changed in large part to the low penetration of both options in the Fedora community, despite being supported for almost ten years.

I should also note that we currently do not have release-blocking ARM for either Cloud or "enterprise ARM" platforms (both of which use UEFI). This situation has not changed in large part to the low penetration of both options in the Fedora community, despite being supported for almost ten years.

All Arm platforms we support use UEFI, they just use different implementations of it.

After listening to the discussion here, I think we do not want to touch the blocking status: the hard part is the support for boards like RPi, but that is also the part that is most useful, and the people working on this want to keep that support available.

But it also sounds like Peter Robinson and Paul Whalen have quite a lot work for (less than) two full time people. I think it would make sense to send a request for help to fedora-devel and ask some new people to get involved. The question is what could be done, so that it actually helps and does not interfere with work that is already being done.

So there's been a lot of changes in upstream U-Boot that broke some of Fedora's usecases which caused problems very late in the last cycle. I have been working with upstream to resolve those, that took longer than I expected due to some of my own personal circumstances. We're almost as resolution with the upstream issues and I don't foresee those reoccurring.

This is basically backwards from how Fedora ARM gets used, so it wouldn't make sense to do that.

There's a quite sizable number of people in Fedora that use aarch64 under virtualization, particularly users of recent Apple hardware. I'm not sure that's a sufficient population to justify blocking on, but it's worth pointing out.

Do we have any statistics on what the usage is in AWS (or other clouds) for aarch64?

But it also sounds like Peter Robinson and Paul Whalen have quite a lot work for (less than) two full time people.

Neither of us do this as part of our $dayjobs

The question is what could be done, so that it actually helps and does not interfere with work that is already being done.

Consistent and regular QE testing with timely bug reports so I don't get reports of regressions moments before we're due to freeze for a release.

There's also always need for HW enablement but that is a lot more focused skill set and I regularly work with HW vendors these days to enable their devices, and in some cases it's hard, eg LOTS of people want RPi5 support, but that needs kernel work upstream to get the new HW supported.

This is basically backwards from how Fedora ARM gets used, so it wouldn't make sense to do that.

There's a quite sizable number of people in Fedora that use aarch64 under virtualization, particularly users of recent Apple hardware. I'm not sure that's a sufficient population to justify blocking on, but it's worth pointing out.

I don't think that's significant enough yet. And of course, I should point out that Fedora Asahi Remix uses U-Boot, so it's emulated UEFI, just like SBCs. That doesn't count as a UEFI platform for the purposes of this discussion.

Do we have any statistics on what the usage is in AWS (or other clouds) for aarch64?

I believe @mattdm can get that information with his scripts.

I don't think that's significant enough yet. And of course, I should point out that Fedora Asahi

It's certainly growing based on antidotal feedback and data, but of course dropping it as a primary architecture will of course help drive that adoption :)

Remix uses U-Boot, so it's emulated UEFI, just like SBCs. That doesn't count as a UEFI platform for the purposes of this discussion.

It's an implementation of UEFI, just like there's proprietary, edk2 and other implementation so IMO it counts like all the others, but I think we can agree to disagree here.

Lots of Cloud, CoreOS, and Server:

So @mattdm was kind enough to give me some figures for the background unique check-in figures for that graph and they add up to 90785. I think over 90K daily users speaks for itself for leaving aarch64 as blocking.

So @mattdm was kind enough to give me some figures for the background unique check-in figures for that graph and they add up to 90785. I think over 90K daily users speaks for itself for leaving aarch64 as blocking.

OK, but which images should be blocking? Unfortunately for us, there's no easy way to know (with the exception of Cloud) whether those are cloud instances/virtualized or installed on bare-metal.

After switching Apple silicon systems to standard boot it will is possible to use the Fedora uboot-images-armv8. This will not work out for u-boot v2024.04. The current asahi patch stack has a little over 50 patches. I would guess 20 - 25 are essential or fix issues our user ran into. 17 patches are currently under discussion on the mailing. Aiming for v2024.07 to switch Fedora Asahi Remix to the Fedora uboot-images-armv8 looks like within reach.

We will need occasionally backport patches for new hardware enablement.

That should provide a little more resources for QA and bugfixes.

As we move toward that, it would make sense to add either a packager from the SIG or the SIG group to the package then. The current maintainer in the Asahi SIG for uboot-tools is @dcavalca, but it might make sense to add @marcan or I too.

OK, so not to drag this forever:
PROPOSAL: Aarch64 images remain blocking.

But even with the proposal above, I think we should try to push some more resources towards earlier testing and wider support for those images.

This was discussed during the FESCo meeting on 2024-03-18:

!agreed Aarch64 images remain blocking (+5, 1, -0)