rpkg 1.67 Release Notes
Released on June 26, 2024
Python compatibility
rpkg works with Python 2.7, 3.6, 3.9 and 3.12
What’s new in rpkg 1.67
Not everything is actually new. The next description covers changes since last 1.66 release. Since then most of features/fixes were already released as patches.
Allow setting --force for sources command by env var
This is helpful for situations where the sources command is called by another subcommand like prep or srpm and changing the overall process is too difficult. The env variable is ‘RPKG_SOURCES_FORCE’.
mockbuild: use results/mock_* when results_dir=subdir
When results_dir=subdir is set in the configuration, most output and artifacts are stored under the results directory. Move the mock results dir from ‘results_$pkgname’ to avoid an extra, unnecessary top-level directory.
remote add command: create a non-anonymous remote
So far x-pkg remote add command has been creating only anonymous remote records. Now it creates remotes for the given user. And this is the default option.
To continue old way - creating anonymous remotes - use x-pkg remote add –a | –anonymous
mockbuild: new argument –extra-pkgs
Added the new argument –extra-pkgs to the mockbuild command. It installs additional packages to mock’s chroot.
In some cases is possible to use the current mechanism of extra args placed after – at the end of the command line instead of –extra-pkgs. Example:
fedpkg mockbuild -- --additional-package <pkg>
Argument(s) additional-package is passed to mock. And additional-package can’t be used together with –shell.
import: check specfile matches the repo name
Specfile in SRPM is checked whether has the same name as the dist-git repository name. It should prevent having multiple similar repositories accidentally created.
This behavior can be omitted by adding argument after import command: ‘–do-not-check-specfile-name’.
import: Undo rpmautospec processing
Till now srpm packages processed by rpmautospec were refused during import. Now such packages are processed:
Undo changes generated by rpmautospec. If there is something to undo, the specfile will be overwritten.
1. Removes everything between the following lines:
## START: Set by rpmautospec
## END: Set by rpmautospec
2. Replaces everything between the following lines with %autochangelog:
## START: Generated by rpmautospec
## END: Generated by rpmautospec
Both of the steps only happen once. If the specfile contains multiple such sections, only the first one is removed/replaced.
The saved spec file is not guaranteed to be bit-by-bit identical with the original spec file used as an input to rpmautospec. However, subsequent repeated conversions there and back should be quite stable.
More information: #527
Lookaside cache operations retries
Both “upload” and “download” network operations might fail and in this case, a retry mechanism was implemented. In case of failure, there is a delay and another attempt(s). Delays are increasing with every attempt.
The number of attempts for lookaside cache network operations is now configurable - there are new keys ‘lookaside_attempts’ and ‘lookaside_delay’ in the configuration. The former expresses a maximum number of attempts to try the operation. ‘0’ or ‘1’ is for a single try (no-retry). The latter means an initial delay between network operation attempts. Each attempt doubles the previous delay value. In seconds.
Support for checking exploded sources before push
pre-push-check now includes test whether source files listed in a specfile come from additional sources.
This functionality is relevant only for some x-pkg tools, others should not be affected.
Confituration itself belongs to fedpkg/rhpkg.
Split git credential data on first ‘=’ only
The value itself can contain a “=” character, but we don’t really care about that. We can treat the value of the key as opaque.
commit command fails on ‘containers’ namespace
Commit failed when ‘uses_rpmautospec’ tried to search for a specfile. There is no specfile in ‘containers’ namespace repository.
copr-build passes extra_args to copr-cli command
The right target for passing extra_args (arguments that are placed after ‘–’ on the command line) is the command copr-cli instead of rpmbuild command.
Do not require ‘sources’ file for all namespaces
Requirement for ‘sources’ file for all layouts except the RetiredLayout (and thus all namespaces) was too restrictive and unexpected.
Partially reverts the commit 1108810bdefd0d880517b274acd6a3bd0d4156e0.
Use release’s rpmdefines in unused sources check
Conditional Source: tags are problematic and, in fact, forbidden in at least Fedora. However, there are packages that conditionalize packages based on macros such as _%{rhel}_ or _%{fedora}_. x-pkg sources did not handle such packages correctly, because when the specfile was parsed to check for unused sources, values for those macros were not set. This was different from other commands which set such macros based on the value of –release parameter or Git branch name.
Improve support for conditional Source: tags by using the standard set of rpmdefines when the specfile is parsed in fedpkg sources.
Pre-push hook won’t check private branches
The pre-push hook script was failing the when user tried to push a private branch. It required using the –release argument with the pre-push-check command and passing additional arguments into the hook script. That was found unreliable and private branches won’t be checked.
Config file option to skip the hook script creation
A new option named “skip_hooks” can be added to the config file (into the main section). It accepts boolean values and when the option is present and set, it prevents creating the pre-push hook script during cloning a dist-git repository.
Allow empty commits when uses_rpmautospec
To avoid situations where the command commit by default does commit when no change in repo is present.
import_srpm: allow pre-generated srpms
When active, do not care specfile in the srpm is processed by rpmautospec. Can be activated only directly via pyrpkg ‘Commands’ object.
pre-push hook script contains a user’s config
When the clone command is called with an argument -C|–config <config_file>, this argument is placed to the generated pre-push script.
A HEAD query into a lookaside cache
A query about whether some file is present in the lookaside cache was under authentication and it prevented using command pre-push-check for those without the ‘packager’ permission.
Added another method (based on HTTP HEAD), that allows the same check without authentication.
pre-push-check have to use spectool with –define
To get all defined source files and patches from the specfile, the ‘spectool’ utility needs ‘–define’ argument(s) to set specific paths for the repository.
Update to spec file presence checking
Using a different approach to checking the layout. Older way prevented retire function working correctly. Layouts are detected at the beginning of the run and the result stays the same, unlike the direct checking files like dead.package in function is_retired.
More robust spec file presence checking
Some commands (verrel, sources, prep, import, …) need to check whether the dist-git repository is in the correct state. It means at least the presence of the specfile.
In the beginning, rpkg detects layouts. Layouts determine the file structure of the repository. For example, most commands can’t be executed for the RetiredLayout (there is no specfile).
When the repository directory exists, some layout can be always detected. Therefore ‘–path’ argument is now checked for a valid directory. The timeout change in the request fixes the new bandit’s finding.
Move warnings from missing Git repo to debug level and simplification
1) Warnings to debug level Currently, all use outside of a Git repository prints the following warnings:
> Failed to get repository name from Git url or pushurl
> Failed to get ns from Git url or pushurl
Since there are valid use cases for running outside of Git repository (such as working on a new package) and many operations work perfectly fine, and in case of errors, a more descriptive error is also printed, these notifications moved to debug level logging.
Avoid calling repo_name from load_nameverrel
Method load_repo_name() calls load_nameverrel(), which calls repo_name() to format its error message. That leads to calling load_repo_name() again, and so on to endless recursion. Prevent this situation by simplifying load_nameverrel() error messages.
Process source URLs with fragment in pre-push hook
Some download services do not have the actual filename in the URL. Packagers work around that by adding a fragment to the URL. This is then ignored by any server, but tricks RPM into getting the correct filename. Example:
Source0: https://crates.io/api/v1/crates/actix/0.13.0/download#/actix-0.13.0.crate
The filename is obviously ‘actix-0.13.0.crate’, but rpkg without this patch will come up with download.
Other small fixes
Show URL in upload/download to/from lookaside cache to provide more useful information,
Use python-requests-gssapi instead of python-requests-kerberos - unification with the Koji CLI.
man page generator: use $SOURCE_DATE_EPOCH (if specified) to have reproducible builds.
do not block tag –clog due autospec usage (the command actually works with rpmautospec)
Processing of another return message (“Required checksum is not present”) from lookaside cache.
Update docker image for Jenkinks tests. Regenerate the docker image: Fedora 37 –> Fedora 38.
copr-build: fixed srpm path when ‘results_dir=subdir’ is set in the configuration.
mockbuild: added –default-mock-resultdir argument not to use resultdir specified in x-pkg code but mockbuild’s default.
import: Restrict argcompletion to .src.rpm files.
import: Don’t delete changelog generated by rpmautospec convert
Fix flake8 complaints
Check remote file with correct hash - The configured hashtype doesn’t have to actually be used. There can be old repos that still use md5.
Ignore missing spec file in pre-push hook - For modules or containers there will be no spec file, and there is nothing to block the push on.
Add more information about pre-push hook - It’s not obvious to many users where the check is coming from, and they have the power to edit the script or delete it completely. Let’s try to improve that a bit.
Do not generate pre-push hook script in some cases - Some rpkg users don’t use fedpkg or similar tool for cloning and therefore there is no tool to execute the pre-push-check function. Do not generate the hook script in this case.
Update docker image for Jenkinks tests - Regenerate the docker image: Fedora 35 –> Fedora 37. Jenkinsfile as a pipeline script is unified with fedpkg.
container-build: update ‘–signing-intent’ help for OSBS 2 - OSBS 2 logs the reactor configuration in a slightly different way. Update the ‘–help’ text for ‘–signing-intent’ to point users at the new location.
Change Logs
Include URL in upload/download (walters)
Use python-requests-gssapi instead of -kerberos (lsedlar)
man page generator: use $SOURCE_DATE_EPOCH if specified (zbyszek)
do not block tag –clog due autospec usage (msuchy)
Processing of another return message from lookaside cache (onosek)
Allow setting –force for sources command by env var (lsedlar)
Update docker image for Jenkinks tests (onosek)
‘_set_token’ method moved to a shared place (onosek)
Fix copr-build command with results_dir=subdir option (otto.liljalaakso)
mockbuild: use results/mock_* when results_dir=subdir (tmz)
Add option to mockbuild use default resultdir of mock (v3) (sergio)
Restrict completion to .src.rpm files for import - #706 (orion)
remote add command: create a non-anonymous remote - #599 (onosek)
mockbuild: new argument –extra-pkgs - #498 (onosek)
*pkg import: check specfile matches the repo name - #529 (onosek)
Fixes syntax issues reported by flake8 (onosek)
Unittests for “Undo rpmautospec processing” (onosek)
*pkg import: Undo rpmautospec processing - #527 (miro)
*pkg import: Don’t delete changelog generated by rpmautospec convert (miro)
Make lookaside cache retries configurable (onosek)
Lookaside cache operations retries (onosek)
Prepare the lookaside cache code for retries (onosek)
Fix flake8 complaints (onosek)
Support for checking exploded sources before push (onosek)
Split git credential data on first = only - #694 (lsedlar)
commit command fails on ‘containers’ namespace (onosek)
copr-build passes extra_args to copr-cli command - #510 (onosek)
Do not require ‘sources’ file for all namespaces - #684 (onosek)
Use release’s rpmdefines in unused sources check - #671 (otto.liljalaakso)
Pre-push hook won’t check private branches - #683 (onosek)
Config file option to skip the hook script creation - #515 (onosek)
Allow empty commits when uses_rpmautospec - #677 (j1.kyjovsky)
Check remote file with correct hash (lsedlar)
Ignore missing spec file in pre-push hook (lsedlar)
import_srpm: allow pre-generated srpms - #655 (onosek)
Fix unittests for clone and pre-push hook script (onosek)
pre-push hook script contains a user’s config - #667 (onosek)
A HEAD query into a lookaside cache - #513 (onosek)
pre-push-check have to use spectool with –define - #672 (onosek)
Add more information about pre-push hook (lsedlar)
Update to spec file presence checking - #663 (onosek)
More robust spec file presence checking - #663 (onosek)
Do not generate pre-push hook script in some cases - #665 (onosek)
Avoid calling repo_name from load_nameverrel - #657 (otto.liljalaakso)
Move warnings from missing Git repo to debug level - #659 (otto.liljalaakso)
Update docker image for Jenkinks tests (onosek)
container-build: update –signing-intent help for OSBS 2 (kdreyer)
Process source URLs with fragment in pre-push hook (lsedlar)