#88 Use dist-git repo and Rawhide in GNU Hello tutorial, plus additional small updates
Closed by oturpe. Opened by oturpe.
fedora-docs/ oturpe/package-maintainer-docs tutorial-use-git  into  main

Download 88.patch

Main change

In practice, almost all packaging happens inside a dist-git repository.
Creating new specfiles can be done outside of one,
but it is also possible to use a dist-git compatible Git repository.

In order to stay close to actual workflows,
a dist-git-like repository is introduced to the packaging tutorial.

Since 'rawhide' is the default branch for dist-git repositories,
and normally all packaging work is first done in rawhide branch,
the tutorial is also switched to use it.
Also this moves the tutorial closer to actual workflows.
The downside is that then the tutorial package is built for Rawhide,
but installed in the user's system, which most probably is not Rawhide.
However, due to GNU Hello's few dependencies,
in practice this works well.
There is also a way to avoid this problem,
by introducing Toolbox and performing the test install there.

Additional small updates

  • Use 'fedpkg lint' in GNU Hello tutorial
  • Use xref to refer to other docs pages in GNU Hello tutorial
  • Fix grammar mistake
  • Add reading instructions to GNU Hello tutorial

One motivation for this work is FESCo's proposal to start recommended rpmautospec for all packages.
If that happens, this tutorial should also use rpmautospec,
which does not work without a Git repository.

Metadata Update from @gotmax23:
- Request assigned

I can review this later.

Thank you in advance!
I would also like to get rid of --release rawhide parts,
but this needs something to be done in either rpkg or fedpkg.
I have started the discussion: rpkg#635

Main change

In practice, almost all packaging happens inside a dist-git repository.
Creating new specfiles can be done outside of one,
but it is also possible to use a dist-git compatible Git repository.

I don't think we need to correlate those two things. Do some normal packaging with mock/rpmbuild and then import the package with fedpkg import. For one thing, doing these steps without fedpkg is perfectly feasible and works with or without a distgit repo. In fact, I think beginners should learn how to use the actual tools before using fedpkg which abstracts what's actually happening.

I think it's more confusing to start inside a git repository. If you do that, it's necessary to add new remotes and use git rebase just to import the package. I used this approach previously and stopped because of how painful it was. fedpkg clone newpackage and fedpkg import /path/to/srpm is much easier.

In order to stay close to actual workflows,
a dist-git-like repository is introduced to the packaging tutorial.

The workflow needn't be any different. Packagers can still use mock, rpmlint, and rpmbuild directly. The only new things are the git branches (git switch or fedpkg switch-branch), lookaside cache (fedpkg new-sources, fedpkg upload, fedpkg import), and koji builds (fedpkg build). I would venture that many packagers don't use fedpkg mockbuild or fedpkg lint at all.

One motivation for this work is [FESCo's proposal to start recommended rpmautospec for all packages][fesco].
If that happens, this tutorial should also use rpmautospec,
which does not work without a Git repository.

That's not entirely accurate. You can still build an rpmautospec package outside of a git repository. There are fallback values for %autorelease and %autochangelog that are perfectly workable for test builds. fedpkg srpm produces a source RPM that already has the rpmautospec information embedded into it and it thus can't be imported with fedpkg import. However, rpmbuild -bs does not do that.

I'm a bit conflicted about starting off new packagers with rpmautospec, as it can often cause more confusion for new packagers than it works, but if Fedora is moving in that direction, it's probably a good idea.


I apologize if this feedback isn't the most constructive. I can put together a PR implementing what I'm proposing if you'd like. /cc @decathorpe, as I kind of ended up resurrecting https://pagure.io/fedora-docs/package-maintainer-docs/issue/76 :).

I think it would be a good idea to split out the general cleanup commits into a separate PR so we can handle the git repo discussion separately.

I think you meant fedpkg --release rawhide lint.

This line really should be rpm -qpl /results_hello/2.10/1.fc{RAWHIDEVER}/hello-2.10-1.fc{RAWHIDEVER}.x86_64.rpm. repoquery is not the best tool for this job.

rpm -qpR ./results_hello/...

rpm -qp --provides ./results_hello/...

rebased onto 61cadb5363ef6f2ce54330dca2d009ebcabf3404

Thank you for the review.
I do not find it unconstructive.
it is possible to come up with many different packaging workflow,
and every packager has their own.
The difficult task for this tutorial, and Package Maintainer Docs in general,
is to present some sensible starting point so that newcomers can get started.
Discussing the alternatives is important, so a good approach can be found.

First, here is the separate pull request for the additional changes: #91

Second, regarding using dnf repoquery for tasks that rpm -qp could handle,
I have been considering changing that myself.
The original rationale for the current choice was to minimize the amount of different tools,
and dnf repoquery can cover both actual repo queries and local rpm file inspection.
But it is clumsy in many ways,
i.e. if you forget the -C (and sometimes even if you don't),
you get a nice 100 MB download of repodata that is not needed for anything,
tab completion is quirky and slow,
and, well, the subcommand is called "repoquery", but it does not query any repo in this case :laughing:
So also I think using rpm is better.
I would just make that rpm --query --package --list and so on,
as for tutorials, being explicit is good.
Packagers can learn about the short options somewhere else.

As for the main question of wrapping everything inside a Git repository,
it is driven by the choice of using fedpkg for everything.
It is a weakness of fedpkg that it expects to be run inside a repository.
It should allow getting rid of --release rawhide,
but at the moment it does not —
rpkg#635 would have to lead to some action before that.

Also, the Git repository it allows to run fedpkg new-sources --offline,
but that is not actually needed for anything here.

Thank you for explaining that rpmautospec works well enough even outside of git repository,
that removes the other driver for wrapping things in a repo.

So, at this moment it looks like the Git repository is not very useful here.
I'll retract that proposal.
Git will be involved in the future,
when we eventually get to a executable tutorial for a package update.

Do you agree that building for Rawhide initially is a good idea?
That is one part of this pull request, that could be pushed separately.
It makes the part about testing the resulting package trickier,
but on the other hand, that's what will be done in majority of cases.

Regarding if fedpkg is even the right choice for this kind of tutorial,
I am afraid it is a never ending discussion.
Anyhow, I will write more about that later, it is getting quite late here now.

For rpm --query vs. dnf repoquery: #92

Ok, regarding the workflow for this tutorial:
The main reason for working on this tutorial is that when I was starting out,
the available tutorials were not useful for me when starting out with Fedora packaging,
with no background in RPM above knowing how to run dnf install.
The old tutorials were in Quick Docs,
you can see me removing them here:

They are merged into this tutorial here.
As you can see, the other was using fedpkg, other rpmbuild
Neither was using mock in any form.
The choice of going with fedpkg was somewhat arbitrary,
though affected by the wiki page that is now page Package Maintenance Guide in this repository.

The main point was to gather all the pieces together into a tutorial that can actually be completed,
and is useful for actual Fedora packaging work.
These goals have been achieved in this tutorial.

The main weakness is that in reality, most work is done for existing packages,
and also new contributors would be well adviced to start out that way.
The tutorial should reflect that, starting out by doing fedpkg clone tutorial-package.
That requires having such package available, so we are not there yet.
Starting with an empty local directory has to do for now.

If you feel that also having a tutorial for a different workflow,
I am not opposed to also having such.
I agree that there is pedagogical value in going through different tools that fedpkg wraps.
Also, it seems that personal preference plays a big role in which workflows people use,
so having more than one option could help new contributors to choose what they like.
And finally, if you want to contribute something to these docs,
I am not in a position to prevent you from doing so :grinning:

In case there will be more than one workflow covered,
maybe the structure of the tutorial should that in a shared page,
the steps of producing an rpm package are described,
then there is a separate page for each workflow,
describing how those steps are performed in that workflow.

In general, I have a general mental idea how these docs should be,
and am workling, very slowly when time allows, to get them into that direction.
In case somebody has interest in contributing here,
I can put my ideas into writing,
so that they can be improved together,
so that there is a shared goal for all contributions.

It seems I forgot to send my comment that I wrote earlier this week :(. I'll respond to your second comment later.

Second, regarding using dnf repoquery for tasks that rpm -qp could handle,
I have been considering changing that myself.
The original rationale for the current choice was to minimize the amount of different tools,
and dnf repoquery can cover both actual repo queries and local rpm file inspection.
But it is clumsy in many ways,
i.e. if you forget the -C (and sometimes even if you don't),
you get a nice 100 MB download of repodata that is not needed for anything,
tab completion is quirky and slow,
and, well, the subcommand is called "repoquery", but it does not query any repo in this case :laughing:
So also I think using rpm is better.

Yes, rpm is the best tool for the job here.

I would just make that rpm --query --package --list and so on,
as for tutorials, being explicit is good.
Packagers can learn about the short options somewhere else.

I'd say that the short hand way is (much) more commonly used. The question is whether to teach the more verbose way or the way that packagers are most likely to see elsewhere. I didn't know that rpm --query even existed before now; I've always used -q. I feel like we're already having enough long theoretical discussions, so I won't push too hard to use the shorthand :).

As for the main question of wrapping everything inside a Git repository,
it is driven by the choice of using fedpkg for everything.
It is a weakness of fedpkg that it expects to be run inside a repository.
It should allow getting rid of --release rawhide,
but at the moment it does not —
rpkg#635 would have to lead to some action before that.

I don't think fedpkg works any differently in the local git repository that this guide proposes than it would without any git repository at all. I believe that it prints confusing warnings regardless. Thank you for opening the rpkg ticket! It would be great if fedpkg worked properly without a git repository.

Also, the Git repository it allows to run fedpkg new-sources --offline,
but that is not actually needed for anything hereusab, I don't think running that makes sense here. I think it detracts from the tutorial's focus.

Thank you for explaining that rpmautospec works well enough even outside of git repository,
that removes the other driver for wrapping things in a repo.

So, at this moment it looks like the Git repository is not very useful here.
I'll retract that proposal.

Yeah. Thank you for getting the discussion started, though :).

Git will be involved in the future,
when we eventually get to a executable tutorial for a package update.

That would be great! There's good docs about importing a package, but not a lot about what to do next.

Do you agree that building for Rawhide initially is a good idea?
That is one part of this pull request, that could be pushed separately.
It makes the part about testing the resulting package trickier,
but on the other hand, that's what will be done in majority of cases.

I'd recommend just using the current version that the packager is running so they can actually install the package afterwards. The build for rawhide first packaging workflow isn't relevant here. We just want to teach how to create a RPM.

rebased onto 5b0192db22b27944817a790230c8ff8897cbc273

rebased onto c35087be51c1e3f016f70d33390fc6f88f59a5ea

Ok, all the parts where a consensus was found have been moved to separate pull requests,
and have been or will be processed there.
Only Git and Rawhide remains here.
It looks like they should not be merged, at least not in this form,
so I close this pull request now.

Thank you for the comprehesive discussion,
many good points have been brought up here!

Pull-Request has been closed by oturpe

Metadata