releng / failed-composes

#7791 Fedora-Rawhide-20250130.n.0 FINISHED_INCOMPLETE
Closed by jnsamyak. Opened by releng.

Closed
Reopen Issue

pungi.global.log

[ERROR   ] [FAIL] Ostree container (variant Kinoite, arch aarch64) failed, but going on anyway.
[ERROR   ] Runroot task failed: 128638079. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20250130.n.0/logs/aarch64/Kinoite/ostree-container-2/runroot.log for more details.

Variant: Kinoite, subvariant: task failed. Pinging maintainers: @siosm 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Games) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 128639883. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20250130.n.0/logs/x86_64/livemedia-Labs-Games.x86_64.log for more details.

Variant: Labs, subvariant: Games task failed. Pinging maintainers: @q5sys @ngompa 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Design_suite) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 128639875. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20250130.n.0/logs/x86_64/livemedia-Labs-Design_suite.x86_64.log for more details.

Variant: Labs, subvariant: Design_suite task failed. Pinging maintainers: @luya 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Security) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 128639887. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20250130.n.0/logs/x86_64/livemedia-Labs-Security.x86_64.log for more details.

Variant: Labs, subvariant: Security task failed. Pinging maintainers: @jsimon @q5sys @fab 

[ERROR   ] [FAIL] Live media (variant Spins, arch aarch64, subvariant i3) failed, but going on anyway.
[LIVE_MEDIA      ] [INFO    ] [DONE ] Live media: Fedora-i3-Live (arches: aarch64 x86_64, variant: Spins, subvariant: i3) (task id: 128639950)

[ERROR   ] [FAIL] Live media (variant Workstation, arch aarch64, subvariant Workstation) failed, but going on anyway.
[LIVE_MEDIA      ] [INFO    ] [DONE ] Live media: Fedora-Workstation-Live (arches: aarch64 ppc64le x86_64, variant: Workstation, subvariant: Workstation) (task id: 128639967)

Compose Total time: 1:58:19
Compose phase INIT time: 0:00:20.
Compose phase PKGSET time: 0:17:30.
Compose phase WEAVER: FAILED.
Compose phase BUILDINSTALL time: 0:39:46.
Compose phase GATHER time: 0:42:14.
Compose phase OSTREE time: 0:37:08.
Compose phase OSTREE_CONTAINER time: 0:31:14.
Compose phase OSTREE_INSTALLER time: 0:47:37.
Compose phase CREATEREPO time: 0:07:54.
Compose phase CREATEISO time: 0:06:18.
Compose phase REPOCLOSURE time: 0:01:47.
Compose phase IMAGE_BUILD time: 0:41:02.
Compose phase LIVE_MEDIA time: 0:35:49.
Compose phase OSBUILD time: 0:14:00.
Compose phase KIWIBUILD time: 0:45:01.
Compose phase IMAGE_CHECKSUM time: 0:03:48.
Compose phase TEST time: 0:00:01.

  • Kinoite - not sure what's going on, something to do with sysusers, maybe? "error: Installing packages: While applying overrides for pkg openvpn: Could not find group 'openvpn' in group file" . tagging @zbyszek and @siosm
  • Games - megaglest FTBFS, I've fixed it
  • Security - medusa FTBFS, I've fixed it
  • Others - good ol' https://bugzilla.redhat.com/show_bug.cgi?id=2247319 still
 
DEBUG util.py:461:  Running pre scripts...48 done
DEBUG util.py:461:  openvpn.prein: groupadd: /etc/group.4: lock file already used (nlink: 1)
DEBUG util.py:461:  openvpn.prein: groupadd: cannot lock /etc/group; try again later.
DEBUG util.py:461:  openvpn.prein: useradd: group 'openvpn' does not exist
DEBUG util.py:461:  NetworkManager-openvpn.prein: groupadd: lock /etc/group.lock already used by PID 4
DEBUG util.py:461:  NetworkManager-openvpn.prein: groupadd: cannot lock /etc/group; try again later.
DEBUG util.py:461:  NetworkManager-openvpn.prein: useradd: group 'nm-openvpn' does not exist
DEBUG util.py:461:  sddm.prein: groupadd: lock /etc/group.lock already used by PID 4
DEBUG util.py:461:  sddm.prein: groupadd: cannot lock /etc/group; try again later.
DEBUG util.py:461:  sddm.prein: useradd: group 'sddm' does not exist
DEBUG util.py:461:  colord.prein: groupadd: lock /etc/group.lock already used by PID 4
DEBUG util.py:461:  colord.prein: groupadd: cannot lock /etc/group; try again later.
DEBUG util.py:461:  colord.prein: useradd: group 'colord' does not exist
DEBUG util.py:461:  flatpak.prein: groupadd: lock /etc/group.lock already used by PID 4
DEBUG util.py:461:  flatpak.prein: groupadd: cannot lock /etc/group; try again later.
DEBUG util.py:461:  flatpak.prein: useradd: group 'flatpak' does not exist
DEBUG util.py:461:  samba-usershares.prein: groupadd: lock /etc/group.lock already used by PID 4
DEBUG util.py:461:  samba-usershares.prein: groupadd: cannot lock /etc/group; try again later.
DEBUG util.py:461:  dnsmasq.prein: groupadd: lock /etc/group.lock already used by PID 4
DEBUG util.py:461:  dnsmasq.prein: groupadd: cannot lock /etc/group; try again later.
DEBUG util.py:461:  dnsmasq.prein: useradd: group 'dnsmasq' does not exist
DEBUG util.py:461:  usbmuxd.prein: groupadd: lock /etc/group.lock already used by PID 4
DEBUG util.py:461:  usbmuxd.prein: groupadd: cannot lock /etc/group; try again later.
DEBUG util.py:461:  usbmuxd.prein: useradd: group 'usbmuxd' does not exist
DEBUG util.py:461:  plocate.prein: groupadd: lock /etc/group.lock already used by PID 4
DEBUG util.py:461:  plocate.prein: groupadd: cannot lock /etc/group; try again later.

I assume this is somehow related to the sysusers change, but I don't know how. rpm calls systemd-sysusers, and systemd-sysusers doesn't use the locking scheme that shadow-utils does, it just replaces the files via atomic rename. So I think the stale lock file must come from some earlier shadow-utils call. 🤷

Uh, so that's not your regular rpm transaction but an rpm-ostree compose, and I have no idea what it does or how it does it. Does it install through an rpm transaction with scripts disabled? I don't know. So this is just wild speculation, but if users + groups are now getting created before the "running pre scripts" stage through sysusers, maybe that's tripping up something in that specific environment.

shadow-utils leaving a stale lock behind does sound like a lurking shadow-utils issue first and foremost. Unless this stuff gets run in parallel or something. rpm doesn't, but I don't know about rpm-ostree compose.

Also, why did it start failing just now? Sysusers has been enabled for a week now.
Edited by pmatilai

FWIW, rpm-ostree had a major update just yesterday:
https://src.fedoraproject.org/rpms/rpm-ostree/c/2cc222975138a9ab3800c286ab3f8273b1b08c99?branch=rawhide

systemd-sysusers, and systemd-sysusers doesn't use the locking scheme that shadow-utils does

I was wrong here. sysusers uses /etc/.pwd.lock which is the standard. I think shadow-utils must use that too, though it may do some additional locking of its own. The message in the error log come from shadow-utils.

Metadata Update from @jnsamyak:
- Issue status updated to: Closed (was: Open)
Metadata
None

FINISHED_INCOMPLETE Fedora Rawhide

None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.