fedora-ci / simple-koji-ci

#2 Move creating the SRPM into mock instead of basic rpmbuild
Merged by pingou. Opened by pingou.
mock_build  into  master

Download 2.patch

Fixes https://pagure.io/fedora-ci/simple-koji-ci/issue/1

Signed-off-by: Pierre-Yves Chibon pingou@pingoured.fr

Note that:

new_sources = spec_sources(specfile, tmp)

should no longer be necessary when you --define "%_disable_source_fetch 0". But I would probably expect fedpkg sources (you can also use https://src.fedoraproject.org/rpms/rpkg-client) call there to fetch sources from Fedora DistGit instead of fetching Source0 from the URL in the spec file. Supposing these two things are the same, it's no problem, however. I guess this might be problem mainly for packages that do not contain URL in Source0 but only a filename (if there are any).

Downloading the Source0 from the spec file is required for PR that implement an update (and thus where the sources haven't been uploaded).

I'll test w/o the call to spectool to see :)

Downloading the Source0 from the spec file is required for PR that implement an update (and thus where the sources haven't been uploaded).

I am not sure why it is required. I think sources can be just downloaded from the project against which the PR is opened. Just an option but I would probably expect that. That's what the rpkg-client does (btw. it might become rpkg-util) when sources are being fetched on a checked out repo that is a fork (i.e. git url path is something like forks/user/rpms/pkg). I haven't tried fedpkg yet for this use-case.

I was thinking of the use-case where the sources aren't in dist-git, while I think you're speaking about the case where the user who opened the PR also uploaded the new-sources on dist-git, no?

I was thinking of the use-case where the sources aren't in dist-git, while I think you're speaking about the case where the user who opened the PR also uploaded the new-sources on dist-git, no?

Actually, I am speaking about a use-case where the sources are in dist-git and the PR just modifies specs or patches.

But actually, if you call fedpkg/rpkg sources and call mock --buildsrpmwith --define '%_disable_source_fetch 0' and networking enabled (after https://github.com/rpm-software-management/mock/commit/8fa475e3f398f110488cfd3c4eb4c4e051fc836f#diff-5116f9da9f94a81a3cc4881207a7203c), you cover both use-cases (the second one being when PR adds some new sources that are not yet in DistGit and references them through URL).

The third described use-case, that a user would upload new sources into DistGit as a part of a PR, is not covered by this but I don't think that is even supported right now (DistGit's upload script allows any-length namespaces if I remember correctly but fedpkg is missing support for that, rpkg-client would allow that after changing default configuration value for lookaside cache).

I changed the build line to be: mock --buildsrpm --define "%_disable_source_fetch 0" --spec /var/tmp/skc-G3ejJK/python-arrow.spec --sources /var/tmp/skc-G3ejJK --resultdir /var/tmp/skc-G3ejJK --old-chroot which succeeded (nspawn doesn't) but it still fails with in the build.log: File /builddir/build/SOURCES/arrow-0.10.0.tar.gz: No such file or directory so looks like %_disable_source_fetch 0 may not be doing what we thought :(

I changed the build line to be: mock --buildsrpm --define "%_disable_source_fetch 0" --spec /var/tmp/skc-G3ejJK/python-arrow.spec --sources /var/tmp/skc-G3ejJK --resultdir /var/tmp/skc-G3ejJK --old-chroot which succeeded (nspawn doesn't) but it still fails with in the build.log: File /builddir/build/SOURCES/arrow-0.10.0.tar.gz: No such file or directory so looks like %_disable_source_fetch 0 may not be doing what we thought :(

Well...

clime@coprbox ~ $ rpkg clone python-arrow
Cloning into 'python-arrow'...
remote: Counting objects: 127, done.
remote: Compressing objects: 100% (124/124), done.
remote: Total 127 (delta 48), reused 0 (delta 0)
Receiving objects: 100% (127/127), 17.76 KiB | 0 bytes/s, done.
Resolving deltas: 100% (48/48), done.
Checking connectivity... done.
clime@coprbox ~ $ cd pyt
python-arrow/                    python-distributed/              python-flask-whooshee/           python-rpm-macros-3-12.fc25.src/ 
clime@coprbox ~ $ cd python-arrow/
CHANGELOG.yml  python-arrow-remove-simplejson-test.patch  python-arrow.spec  sources
clime@coprbox ~/python-arrow $  mock --buildsrpm --define "%_disable_source_fetch 0" --spec python-arrow.spec --sources . --resultdir . --old-chroot
INFO: mock.py version 1.4.6 starting (python version = 3.5.3)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
INFO: Start(python-arrow.spec)  Config(fedora-25-x86_64)
Start: clean chroot
Finish: clean chroot
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
Start: unpacking root cache
Finish: unpacking root cache
INFO: enabled dnf cache
Start: cleaning dnf metadata
Finish: cleaning dnf metadata
INFO: enabled HW Info plugin
Mock Version: 1.4.6
INFO: Mock Version: 1.4.6
Start: dnf update
Dependencies resolved.
Nothing to do.
Complete!
Finish: dnf update
Finish: chroot init
Start: buildsrpm
Start: rpmbuild -bs
warning: Downloading https://pypi.io/packages/source/a/arrow/arrow-0.10.0.tar.gz to /builddir/build/SOURCES/arrow-0.10.0.tar.gz
Building target platforms: x86_64
Building for target x86_64
Wrote: /builddir/build/SRPMS/python-arrow-0.10.0-4.fc25.src.rpm
Finish: rpmbuild -bs
Finish: buildsrpm
INFO: Done(python-arrow.spec) Config(default) 0 minutes 7 seconds
INFO: Results and/or logs in: .
INFO: Cleaning up build root ('cleanup_on_success=True')
Start: clean chroot
Finish: clean chroot
Finish: run

Notice the line: 'warning: Downloading https://pypi.io/packages/source/a/arrow/arrow-0.10.0.tar.gz to /builddir/build/SOURCES/arrow-0.10.0.tar.gz'. That's produced by rpmbuild when downloading the sources when disable_source_fetch is zero.

I think this is some kind of misconfiguration or a bug when you tried it but I would need to see logs and configs.

Yes, this is odd, it works for me in a terminal but not in this code.

This is the output:

[simple_koji_ci.consumers INFO] Handling pagure msg u'2017-9cd98466-6962-458b-a072-7c11e5261097'
[simple_koji_ci.consumers INFO] Kicking a scratch build for python-arrow
[simple_koji_ci.buildsys INFO] Cloning 'https://src.stg.fedoraproject.org/rpms/python-arrow.git' to '/var/tmp/skc-UJd3MA'
[simple_koji_ci.buildsys INFO] Calling 'mock --init'
[simple_koji_ci.buildsys DEBUG] INFO: mock.py version 1.4.6 starting (python version = 3.5.4)...
Start: init plugins
INFO: selinux disabled
Finish: init plugins
Start: run
Start: clean chroot
Finish: clean chroot
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
Start: unpacking root cache
Finish: unpacking root cache
INFO: enabled dnf cache
Start: cleaning dnf metadata
Finish: cleaning dnf metadata
INFO: enabled HW Info plugin
Mock Version: 1.4.6
INFO: Mock Version: 1.4.6
Finish: chroot init
Finish: run
[simple_koji_ci.buildsys INFO] Calling u'mock --buildsrpm --define "%_disable_source_fetch 0" --spec /var/tmp/skc-UJd3MA/python-arrow.spec --sources /var/tmp/skc-UJd3MA --resultdir /var/tmp/skc-UJd3MA --old-chroot'
[simple_koji_ci.buildsys DEBUG] ['mock', '--buildsrpm', '--define', '"%_disable_source_fetch 0"', '--spec', u'/var/tmp/skc-UJd3MA/python-arrow.spec', '--sources', '/var/tmp/skc-UJd3MA', '--resultdir', '/var/tmp/skc-UJd3MA', '--old-chroot']
[simple_koji_ci.buildsys DEBUG] 1
[simple_koji_ci.buildsys DEBUG] INFO: mock.py version 1.4.6 starting (python version = 3.5.4)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
INFO: Start(/var/tmp/skc-UJd3MA/python-arrow.spec)  Config(fedora-25-x86_64)
Start: clean chroot
Finish: clean chroot
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
Start: unpacking root cache
Finish: unpacking root cache
INFO: enabled dnf cache
Start: cleaning dnf metadata
Finish: cleaning dnf metadata
INFO: enabled HW Info plugin
Mock Version: 1.4.6
INFO: Mock Version: 1.4.6
Start: dnf update
Finish: dnf update
Finish: chroot init
Start: buildsrpm
Start: rpmbuild -bs
Finish: rpmbuild -bs
Finish: buildsrpm
ERROR: Exception(/var/tmp/skc-UJd3MA/python-arrow.spec) Config(fedora-25-x86_64) 0 minutes 11 seconds
INFO: Results and/or logs in: /var/tmp/skc-UJd3MA
INFO: Cleaning up build root ('cleanup_on_failure=True')
Start: clean chroot
Finish: clean chroot
ERROR: Command failed: 
 # bash --login -c /usr/bin/rpmbuild -bs --target x86_64 --nodeps /builddir/build/SPECS/python-arrow.spec

5 new commits added

  • Drop requirement on sphinx
  • Checkout the pull-request's head so we actually test the PR
  • Do not download the sources manually
  • Run fedpkg source before trying to create the srpm
  • Fix building the srpm in mock, thanks @clime!

I think this is ready for review :)

1 new commit added

  • Project wide flake8 cleaning

LGTM.

Thanks! :)

Pull-Request has been merged by pingou
Metadata
None
No Tags
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.