Highlights from 2022-11-28 release

Updated client packages are shipped in Fedora 35, Fedora 36, Fedora 37, EPEL 7, EPEL 8, and EPEL 9 Bodhi updates.

Development moved to GitHub

The upstream development moved from Pagure back to GitHub:

The original Pagure issue tracker, and pull-requests for the repository are now read-only. We migrated the existing issues to GitHub and preserved matching IDs. However, all issues are opened by a bot account, and therefore you won’t receive any notifications for issues that you submitted to our Pagure issue tracker. Please find your issues and click the Subscribe button on GitHub.

The main motivation was that Pagure workers were very slow for the Copr project (e.g. rebasing and merging PRs took ages, and often failed). We also accumulated many issues over the years.

DistGit scheduler

Generally, packages are imported into Copr DistGit almost instantly. That’s why we made it this far a simple First-Come-First-Serve queue without any prioritization, and with very basic parallelization. This simple workflow worked from early days. But with increasing popularity of Copr service this started to not be enough. In some scenarios (multiple large uploads, mass-rebuilds without background priority, etc), Copr DistGit could get overloaded and block all users for many hours.

We now deployed the same dispatcher and standalone workers mechanism that we’ve been using for builders on the Copr backend side. This brings an import queue prioritization. So while even users with huge task chunks will be processed, other users (e.g. comming later) will get their tasks imported quickly at the same time.

Updated aarch64 builders to Graviton3 processors

Fedora Copr builders are hosted in AWS. Previously we were running them on a1.xlarge instances (4vcpu, 8GB RAM, Graviton processors) - now we are on c7g.xlarge. These aarch64 builders should be much faster.

Notable changes

  • The Task Queue numbers on the homepage are now cached for one minute, reducing the loading time by up to 0.5s

  • Each build now shows also a link to the import.log file. This log is helpful for debugging issues when SRPM package is built successfully but then it fails when importing to DistGit

  • Besides user groups from Fedora Accounts, we implemented support for groups from LDAP server. This option won’t be enabled for Fedora Copr instance.

  • Custom source builds now can install build-time dependencies from additional external repositories

  • After a decade of service, the copr-backend storage contained many backup, debug, and other miscellaneous untracked files. We added a script that helps with cleaning them up.

  • Packages are now signed with OpenPGP v4 signatures. We tested the signature compatibility on RHEL6+

Bugfixes

  • #2284 - Large SRPM uploads shouldn’t timeout anymore

  • #2303 - SRPM upload provided misleading information on expired authentication

  • #2232 - SRPM upload sometimes caused the WSGI daemon get stuck with 100% CPU usage. We believe the issue is fixed now