Koji 1.36.0 Release notes

All changes can be found in the roadmap. Most important changes are listed here.

This is a fairly substantial release. Notable changes include:

Migrating from Koji 1.35.x

For details on migrating see Migrating to Koji 1.36

Security Fixes

None

System Changes

Support rpm v6 format

This adds initial support for v6 format rpms in Koji. The main issue was Koji’s past reliance on the sigmd5 field, which does not exist in v6. Koji will now store and report multiple digest values from the rpm header, not just sigmd5.

In light of these changes the Content Generator Metadata guidelines have been updated.

One notable omission – the oz/imagefactory feature (i.e. createImage) is not covered here (it will fail if an image includes v6 rpms).

Show more info on waitrepo task failures

This causes the client and waitrepo tasks to show a little more detail when repo requests fail.

Client specified signature keys

More complex signature headers cannot be described by a single key. RPM now supports cases like:

  • rpms signed with multiple keys

  • rpms signed with newer types of keys

  • rpms that include ima signing

Koji now takes the approach of treating the sigkey field as a descriptive label for the signature that can be specified at import or changed later. This gives us the ability to handle more complex signing without radically altering the api or file layout.

Policy enhancements

This extends the existing policy handling code adding support for:

  • negated tests – use ! before a test to negate it

  • break N action – break out of subrules without matching

  • stop action – stop the policy check

  • flag action – set a named flag (and continue) when a rule matches

  • flagged test – test for a named flag

  • multi match rules – option for policy hooks to check for multiple results

These changes are backwards compatible; no policies should require update. Admins may find that using these new options can help simplify complex logic in their existing policies.

The new support for multi-match is forward looking, and is not currently utilized in current policy checks.

Deprecate pathinfo.tmpdir

The pathinfo.tmpdir method in the koji library has never been used by Koji and is now deprecated.

Hub Changes

Disable url sessions by default

The DisableURLSessions setting on the hub now defaults to True. Koji clients before 1.31 will not be able to authenticate

If an admin wants to support auth for clients before 1.31, they can turn this back on.

Allow querying headers from signed copy

This adds a sigkey option to getRPMHeaders. If given, the corresponding signed copy will be queried instead of the primary copy. This allows querying signature headers that have changed in the signed copy (i.e. the signatures)

Adjust repo lag logic

This change adjusts the repo lag logic and obsoletes the RepoLagWindow setting.

The lag window was intended to avoid redundant regenerations for duplicate requests within a short span of time. Unfortunately, there are still a number of cases where nearby requests fall into separate windows. Depending on settings and repo regen time, this can even result in overlapping newRepo tasks for the same tag.

Recursive task policy data

When creating a new task, the data used for the policy tests includes values pulled from the task parameters. This is now handled recursively. That is, the parameters of any parent task also apply. In particular, this allows making channel policy rules for buildArch tasks based on the target value of the parent build task.

Capture more events in scheduler logs

The scheduler now includes more detail in the scheduler logs. This data will appear in the output of koji scheduler-logs

Enable parent tag as name, not ID only

Previously the setInheritanceData call required specifying parent tags by id. Now they can be specified by name as well.

Additional data for cg volume policy

This makes it possible to consider cg and draft status in the volume policy.

Channel defaults

This change allows policy to distinguish between a user channel override and a system provided default that is different from default (e.g. for maven and image builds).

There is a new action for the channel policy named default which indicates that the default channel should be used, regardless of what that default is.

Currently, this change must be enabled by setting DefaultChannelCompat = False. This default will change to True in a future release.

Add listBuilds query params for promotion data

This allows querying builds by promoter or promotion time

Less log noise for empty or invalid requests

Avoid noisy log output for invalid requests

Adjust upload offset and overwrite logic

This fix addresses a corner case that could cause failed uploads.

Accept both comma-separated and space-separated arches for external repos

This makes calls like addExternalRepoToTag and editTagExternalRepo a bit friendlier.

Match longest extension first for archivetype

This addresses a longstanding bug in get_archive_type. In cases of layered extensions, the code would match a shorter extension before a longer one. E.g. “.gz” would match before “.tar.gz”. We have avoided such overlapping archivetypes for this reason, however we can’t control what instances do.

Report tag_external_repos.arches in query_history

This fixes an oversight in query_history. The arches field of this table was omitted.

Allow setting ttl in protonmsg plugin

A new configuration option for the protonmsg plugin

Partial draft support for cg_import

This removes some obstacles for importing draft builds via the CG interface.

Generators can now set draft: true in the metadata to import draft builds, but this is currently only allowed for rpm builds. Admins can control such imports in the cg_import policy by using the is_draft test in a match rule.

Additionally, the new AllowDraftComponents hub option allows content generator imports to reference draft builds as components (e.g. a buildroot or image component for an imported build). We recommend that admins keep this option off (the default) unless they are sure that they need it.

Client Changes

Enable anonymous retries by default

The default value for the anon_retry setting is now True. This will cause the client to retry hub calls even when not authenticated.

Add override option to assign-task

This adds an --override option to the assign-task command. This option prevents the scheduler from reassigning the task.

Print build in import-cg

Print basic build info after import

Clean up cli for comps management

Adds missing commands for comps management and some new documentation for using these commands.

See also: Managing Comps Data

Enable specific number of builds for dist-repo tasks

The dist-repo command now accepts a --latest-n N argument to use the N latest builds per package instead of just the latest.

Support –wait-repo/–wait-build in wrapper-rpm command

Added --wait-repo/--wait-build options similar to those for the build command.

Download fallback to unsigned

Previously, if you passed --key to download-build and signed packages weren’t available, Koji would either skip the unsigned package or error out. This adds a modified behavior controlled by the new --fallback-unsigned arg. If this is passed with --key, unsigned copies will be downloaded for packages for which no signed copy can be found.

Web Changes

Port kojiweb to jinja2

The Koji web interface now uses Jinja2 templates instead of Cheetah templates.

Compatibility was a major focus in this port. The web output remains predominantly the same, and url compatibility is maintained.

Note: Cheetah3 is still required on builders for wrapper rpm templates

Add repo requests web ui

This makes repo requests more visible in the web interface.

  • new page for listing repo requests

  • new page for request details

  • new details on the repoinfo page

  • link from taginfo page to related requests

  • link from taskinfo page if it is for a request

Add weblink to NVR build for draft builds

This change provides a convenient link from the draft build page when it has been superceded by another build.

Add cg and draft filters for builds page

This change adds more filter options to the builds page in the web ui.

Show source files on srpm info page

When viewing a source rpm in the web ui, the source files are now listed.

Builder Changes

Ensure macros and env values passed to mock are strings

If an admin sets a macro value to a non-string value (which is easy to do by accident if the value is a number), the builder will now convert such values to a string.

Improve reliability of kojid service start

If chrony is enabled, wait for it to start before starting kojid.

Return BuildError when kickstart file does not exist

Clarify an error condition

Handle new mock option - bootstrap_image_ready

Admins can set mock.bootstrap_image_ready for a tag and this option will be added to the mock config.

See also: https://rpm-software-management.github.io/mock/Feature-container-for-bootstrap

Devel and testing

New dev tool: fakepolicy

This adds a new script under devtools that facilitates testing policy code against a remote instance. The script runs some heavily mocked hub code that calls out to a remote instance for actual data access.

The reason for this tool’s existence is that policy code is very hard to test without actual data, which is usually locked up inside a Koji instance. This script allows you to see what a given policy would do, without actually deploying it.

Add kojid –single mode

This is an option for debugging kojid. Using this option makes it feasible to use the python debugger on a task handler.

Unit test fixes

This release includes numerous fixes that could cause unit tests to fail in certain environments.

Bug fixes

Work around nfs glitch in ensuredir

In a multi-hub + nfs scenario, we’ve seen race situations where mkdir fails with EEXIST, but an immediate lstat does not find the directory. This change works around the unexpected kernel behavior.

Fixes for get_next_build

These fixes address cases where the automatic release incrementing mechanisms for image builds could fail.

Handle disabled hosts in scheduler

This fixes an issue with the how the scheduler handles tasks assigned to disabled builders.

Catch nvra duplicates when promoting

When promoting a build, we check for existing NVR conflicts and give a nice error. However, we did not do the same for the included NVRAs, which are also required to be unique. There are corner cases where a draft can have nvra overlap without nvr overlap. This is still an error case, but the error is now more informative.

Always prefer task arch in find_arch()

This addresses an issue where arch-independent tasks like rebuildSRPM and buildSRPMFromSCM could choose the wrong buildroot architecture.

Fix callnum lookup on hub

Since 1.31, Koji clients have used http headers to pass their session info. However, the hub has been misreading the callnum since that time.

When a draft build is promoted, Koji maintains a symlink at the original location. These fixes address cases where the hub did not create that link.

As part of this fix, the system now requires that each volume directory contain a toplink symlink that points to the main volume. We recommend using an absolute symlink. E.g.

$ file /mnt/koji/vol/vol3/toplink
/mnt/koji/vol/vol3/toplink: symbolic link to /mnt/koji

When adding a new volume, admins should create this symlink when they create the volume directory. If not, the hub will automatically create it using the configured topdir path and log a warning.

For existing volumes, admins must create this symlink for each preexisting volume during their migration to 1.36. Failure to do so can result in build promotion failures.

Fix listUsers type check

Support passing a list for userType

Ensure repos dir

Create the repos dir if it does not exist.

Write-signed-rpm: fix nvr fallback

This fixes a bug in the write-signed-rpm command where it would fail on valid NVR values.

Use label for waitrepo subtasks

This change prevents restarted tasks from spawning a redundant waitrepo subtask

Prefer getBuildLogs for downloading logs

Fixes a common failure case for log downloads

Windows build fixes

Various small fixes to the windows build daemon.

Handle x86_64_v2 arch for dist-repos

This fixes a KeyError when trying to make a distrepo for the arch “x86_64_v2”.

Other changes

Update Cheetah3 to CT3 PyPI repository

Note: While Koji now uses Jinja2 in the web ui, Cheetah3 is still required for wrapper rpm templates.

This simply updates our requirements to point to the new upstream.

koji-gc: Add ccache to koji-gc options

Allows specifying a Kerberos credential cache for koji-gc, similar to kojira and kojid.

Indexes for standard_buildroot

Adds database indexes for query performance

Schemas: wsl file suffix

The “new” WSL images need a WSL suffix. Currently this is managed on a per-instance basis but it’s a bit of work to sync across them.

Add archive types for erofs and squashfs

Add archive types produced by Kiwi

Drop custom threadlocal implementation

Koji previously used its own ThreadLocal class. Now it just uses threading.local.

Raise weight cap

This adjusts the cap on how much kojid can adjust the weight of a buildArch task.

Documentation updates

Various documentation changes