It would be good to get something in place for 10.1 soon.
Although we branched for epel10.1, there is neither a directory named that, or anything in the mirror manager.
I keep getting pinged because internal RHEL 10.1 testing fails whenever it tries to test, or use anything epel.
https://bugzilla.redhat.com/show_bug.cgi?id=2353170

Some testing shows that dnf-automatic doesn't behave any different from regular dnf. A dependency problem will halt the process. This means that we definitely need to find a solution here.

I've also learned more details about mirrormanager that will influence this implementation. We set prefixes for repo names, which then get appended with a hyphen and the detected version. This means that the version must be at the end of the repo name, with no other characters after it. We do have some flexibility to use different prefixes for different regex matches. Currently we use one regex that matches both 10 and 10.0.

Here is the plan that I've come up with based on our current capabilities.

Step 1: Adjust EPEL 10.1 publishing path

We would change new-updates-sync to publish EPEL 10.1 to pub/epel/10.1. Future EPEL 10 releases would also be published to a path including their minor version. This frees up the epel-10 repo name to become a repo redirect.

Step 2: Use different repo mapping for minor verison repos

We would change scan-primary-mirror.toml to have separate EPEL mappings for with and without a minor version, so we can use different prefixes for each.

[[repository_mapping]]
regex="^pub/epel/\\d+/Everything/"
prefix="epel"
[[repository_mapping]]
regex="^pub/epel/\\d+\\.\\d+/Everything/"
prefix="epel-z"

This will give us repo names like epel-z-10.1.

Step 3: Create repo redirects and symlinks

We would create repo redirects in mirrormanager, one for the leading minor version (CentOS) and one for the trailing minor version (RHEL), that will be used in our metalinks.

• epel-10 → epel-z-10.1
• epel-z-10 → epel-z-10.0

We would create symlinks (which are ignored by mirrormanager) in the pub/epel directory that will be used in our commented out baseurls.

• 10 → 10.1
• 10z → 10.0

Step 4: Update metalinks in epel-release

We would update the repo arg in our epel-release metalinks to evaluate to epel-10 on CentOS and epel-z-10 on RHEL, corresponding to our repo redirects.

epel${releasever_minor:+-z}-$releasever

The commented out baseurls would also be updated to use the symlink names.

Mmmh I am not sure to follow everything, so most likely I am wrong. I find we have many redirects (more than necessary), and I don't understand the z naming convention (so clients using RHEL are unlikely to get it as well, and clients do care since they might mirror the EPEL repo instead of using the redhat ones directly, so they will have trouble understanding what z means).

In the final picture (so without caring about migration for existing CentOS 10/RHEL 10 beta), I would see something like:

• All repositories are published as pub/epel/10.x as described in your step 1.
• Create in pub/epel syminks for RHEL & CentOS (if the string rhel is too RedHat specific, maybe another term more neutral like just el would better suit folks from the Rocky/Alma/other EL rebuilds):
• centos-10 --> 10.1
• rhel-10 --> 10.0
• (and since Red Hat folks test internally RHEL 10.1 already, would it make sense to have some kind of rhel-next-10 --> 10.1 already ? To be discussed with your colleagues internally)
• The very same epel-package can check the presence of ${releasever_minor} to define a metalink name that will be either epel-centos-10 or epel-rhel-10.
• From metalink point of view, for me we shall have a single a single kind of redirect, that you just add the prefix "epel" everywhere. Something like:

[[repository_mapping]]
regex="^pub/epel/[^/]+/Everything/"
prefix="epel"

And then for backward compatibility with existing CentOS, we would create a symlink 10 --> centos-10

I'm seeing a problem with this whole thought process of making everything go to the next version.
That is RHEL development.

We (EPEL) branch everything when RHEL starts developing the next minor version. At that point redhat-release is updated to the next minor version, and everything starts being tested. That testing involves EPEL in several ways.
If we do the "next version" way of doing things, then all their tests and installs from EPEL will be getting things from the previous EPEL instead of the things built specifically for the latest version.
In other words, they will be running RHEL 10.1 but pointing to epel10.0

I find we have many redirects (more than necessary),

This plan uses exactly as many repo redirects as necessary, no more. Which ones would you eliminate?

and I don't understand the z naming convention (so clients using RHEL are unlikely to get it as well, and clients do care since they might mirror the EPEL repo instead of using the redhat ones directly, so they will have trouble understanding what z means).

It's somewhat arbitrary, but also ties into RHEL development terminology. Each minor version is a "z-stream". I'm open to alternative names, but what it comes down to is we need some string to inject based on the DNF variable releasever_minor to differentiate requests from CentOS and RHEL clients. The neat part is clients don't need to understand it, they just install epel-release and start installing packages like they always have.

Create in pub/epel syminks for RHEL & CentOS (if the string rhel is too RedHat specific, maybe another term more neutral like just el would better suit folks from the Rocky/Alma/other EL rebuilds):

The problem with that is "el" is short for Enterprise Linux, and CentOS is also Enterprise Linux.

(and since Red Hat folks test internally RHEL 10.1 already, would it make sense to have some kind of rhel-next-10 --> 10.1 already ? To be discussed with your colleagues internally)

We're definitely not going to re-use the "next" term so we can avoid confusion with EPEL Next.

The very same epel-package can check the presence of ${releasever_minor} to define a metalink name that will be either epel-centos-10 or epel-rhel-10.

This is what I wanted, and I don't see a way it is possible. DNF variables don't have ternary expressions to insert one string if a variable is set and a different string if it is unset. The only methods available are "default value" (insert param value if set, insert string if unset) or "alternative value" (insert string if set, insert nothing if unset). Maybe I'm just not creative enough, but after contemplating this for some time the only path forward I can see is two repo names, with the RHEL repo name containing an additional string that is gated on releasever_minor being set.

From metalink point of view, for me we shall have a single a single kind of redirect, that you just add the prefix "epel" everywhere.

This is what we have now, but in this proposed plan we would want to set a different prefix when the repo has a minor version. The mappings aren't redirects.

If we do the "next version" way of doing things, then all their tests and installs from EPEL will be getting things from the previous EPEL instead of the things built specifically for the latest version.
In other words, they will be running RHEL 10.1 but pointing to epel10.0

By default, yes. I don't think it's possible to have a metalink that works out of the box on all three of CentOS, RHEL current minor, and RHEL next minor. The only DNF variable difference is the value of releasever_minor, but we've already established that encoding the actual value of this in the metalink will cause upgrade problems.

My suggestion is we solve for the the two primary uses cases of CentOS and RHEL current minor, and instruct the exception case of RHEL next minor to explicitly set their releasever to the major.minor version they're using (e.g. --releasever 10.1). This is probably necessary anyways to properly integrate with layer product repos.

The only thing I'd like for us to add is equivalent symlink directories in https://kojipkgs.fedoraproject.org/compose/updates/

I mirror composes regularly and I'd like a static URL that points to the latest composes for RHEL and CentOS Stream targeted EPEL variants.

This will also significantly reduce the maintenance required for EPEL 10 mock configs (which has additional positive effects for Copr and Packit).

https://github.com/rpm-software-management/mock/issues/1427

We discussed this in today's EPEL Steering Committee meeting, and voted unanimously to move forward with this plan.

Steps 1 and 2 are set up in this pull request, but I believe since we just started the Fedora 42 Final Freeze we will need some freeze break votes as well before moving forward.

Some follow up steps after the main ones are in place:

• update epel-release with the new metalinks and baseurls
• update mock-core-configs for the same
• edit the EPEL 10.1 bodhi release to use version 10.1 (it's currently just 10)

Sorry, but I see 2 major issues with this workaround

1/ when 10z will be redirect to 10.1 ?

On RHEL 10.1 Release ?
Or when all the clones (Alma, Rocky...) will be ready ?

A delay of a few days to a few weeks may happen.

2/ what about user wanting 10.0

For various reasons, users may want to defer the upgrade to 10.1 and stay on 10.0 for some time

1/ The change will be made on RHEL minor releases, which won't change in regard to how it has always worked on previous EPEL releases. The experience of following the 10z will be the same as it has been with following the EPEL9 repo in this regard.

2/ There are many ways on how the user can manually achieve this, for example, by modifying the .repo file, but if dnf does detect the releasever_minor variable set, EPEL will work with that specific minor repo as how it is being distributed now.

EDIT: 2/ On this topic, there are also other easier ways on how a user could deal with this. For example, you can both call dnf with the --releasever 10.0 argument. You could also temporarily override the releasever variable by calling echo 10.0 > /etc/dnf/vars/releasever.

One hiccup we ran into is quick-fedora-mirror isn't able to switch from a 10 directory to a 10 symlink. To resolve this we need to temporarily remove that symlink, leave it in that state for a few days for mirrors to remove the directory cleanly, then add the symlink back. That shouldn't affect anyone using epel-release (which requests the epel-10 repo from mirrormanager, which resolves to the 10.1 repo). This is in place now, and I'll add the symlink back ~~next week~~ after giving mirrors 48 hours to sync up (about 22:00 UTC on 2025-04-09).

epel-release-10-5.el10_0 has been pushed to both the 10.0 and 10.1 repos. The metalinks and baseurls are working as planned.

CentOS 10 (releasever_minor not defined)

• epel-10 metalink:

root@c10:~# dnf -q --repo epel repolist -v | grep -e Repo-metalink -e Repo-baseurl
Repo-metalink      : https://mirrors.fedoraproject.org/metalink?repo=epel-10&arch=x86_64
Repo-baseurl       : http://mirror.nodesdirect.com/epel/10.1/Everything/x86_64/ (221 more)

• epel/10 baseurl:

root@c10:~# dnf -q --repo epel repolist -v | grep Repo-baseurl
Repo-baseurl       : https://mirror.fcix.net/epel/10/Everything/x86_64/

RHEL 10 (releasever_minor defined)

• epel-z-10 metalink:

root@rhel10.b:~# dnf -q --repo epel repolist -v | grep -e Repo-metalink -e Repo-baseurl
Repo-metalink      : https://mirrors.fedoraproject.org/metalink?repo=epel-z-10&arch=x86_64
Repo-baseurl       : https://nocix.mm.fcix.net/epel/10.0/Everything/x86_64/ (226 more)

• epel/10z baseurl:

root@rhel10.b:~# dnf -q --repo epel repolist -v | grep Repo-baseurl
Repo-baseurl       : https://mirror.fcix.net/epel/10z/Everything/x86_64/

Remaining work:

• update mock-core-configs
• adjust the branches page in the docs

The remaining work items I listed have been completed.

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/epel/steering/issues/324

Please continue any further discussion there.