Quick start

New project

Let’s say you have decided to create a new mind-blowing application and you would like to out-source the building process from your own machine or you would like to make the latest installable version of the app always available somewhere for you and your teammates to download and debug on it.

This is what rpkg can do for you among other things thanks to its rpkg build command that can send your sources for build in a public build system called Copr (stands for “Community projects”).

To make this command work, you need to write an rpm spec file template and place it along-side your sources. If you don’t know what spec file is or you know what it is but not exactly how to write one, you can optionally read some details here but you can also just continue reading on to get the quick start you came for.

rpkg enriches the basic rpm spec files with its own syntax. Such enriched spec file or “spec file template” is then used to generate the actual rpm spec file that is then used to build an installable rpm package but this whole process is already automated for you.

The long story short, below is an example spec file template called hello_rpkg.spec.rpkg with comments included that you can use as a starting point for your own Git project. {{{ ... }}} expressions (git_pack is the only one that has a “side effect”) are the above mentioned rpkg’s extensions to the basic rpm spec file syntax that make it possible to derive certain spec file parts dynamically from Git repository metadata.

hello_rpkg.spec.rpkg

# The following tag is to get correct syntax highlighting for this file in vim text editor
# vim: syntax=spec

# git_dir_name returns repository name derived from remote Git repository URL
Name:       {{{ git_dir_name }}}

# git_dir_version returns version based on commit and tag history of the Git project
Version:    {{{ git_dir_version }}}

# This can be useful later for adding downstream patches
Release:    1%{?dist}

# Basic description of the package
Summary:    Hello rpkg package.

# License. Hopefully free or at least open-source. We assume GPLv2+ here.
License:    GPLv2+

# Home page of the project. Can also point to the public Git repository page.
URL:        https://pagure.io/hello_rpkg

# Detailed information about the source Git repository and the source commit
# for the created rpm package
VCS:        {{{ git_dir_vcs }}}

# git_dir_pack macro places the repository content (the source files) into a tarball
# and returns its filename. The tarball will be used to build the rpm.
Source:     {{{ git_dir_pack }}}

# More detailed description of the package
%description
This is a hello world package.

# The following four sections already describe the rpm build process itself.
# prep will extract the tarball defined as Source above and descend into it.
%prep
{{{ git_dir_setup_macro }}}

# This will invoke `make` command in the directory with the extracted sources.
%build
make

# This will copy the files generated by the `make` command above into
# the installable rpm package.
%install
make install root=%{buildroot}

# This lists all the files that are included in the rpm package and that
# are going to be installed into target system where the rpm is installed.
%files
/usr/bin/hello_rpkg

# Finally, changes from the latest release of your application are generated from
# your project's Git history. It will be empty until you make first annotated Git tag.
%changelog
{{{ git_dir_changelog }}}

For this example spec file template to be any useful, you also need the Makefile so that the make commands in %build and %install sections work correctly and some application sources from which the hello_rpkg binary will be compiled. Let’s use the following two files for that purpose:

Makefile

CC=gcc
ARGS=-Wall -g

all: main.o
        ${CC} ${ARGS} -o main main.o

main.o: main.c
        ${CC} ${ARGS} -c main.c

install:
        mkdir -p $(root)/usr/bin
        cp main $(root)/usr/bin/hello_rpkg

clean:
        rm main *.o

main.c

#include <stdio.h>

int main()
{
    printf("%s", "Hello rpkg!\n");
    return 0;
}

If you have those three files (hello_rpkg.spec.rpkg, Makefile, main.c) placed in a plain directory, then you need to invoke the following commands in that directory to properly initialize the Git repository:

git init
git remote add origin https://pagure.io/hello_rpkg.git
git config --add branch.master.remote origin

But you can also just clone the repository at https://pagure.io/hello_rpkg.git where everything is already prepared for you ;).

git clone https://pagure.io/hello_rpkg.git
cd hello_rpkg

Either way, you are now ready to invoke your first rpkg command given that you have rpkg installed (see Installation).

$ rpkg local
git_dir_pack: packing directory /home/clime/hello_rpkg
git_dir_pack: Wrote: /tmp/rpkg/hello_rpkg-1-bo7pe2i1/hello_rpkg-0.0.wtree.zlr9pk.tar.gz
Wrote: /tmp/rpkg/hello_rpkg-1-bo7pe2i1/hello_rpkg.spec
... many lines later ...
Wrote: /tmp/rpkg/hello_rpkg-1-bo7pe2i1/x86_64/hello_rpkg-0.0.wtree.zlr9pk-1.fc28.x86_64.rpm
... some more no longer interesting lines ...

So this output file:

Wrote: /tmp/rpkg/hello_rpkg-1-bo7pe2i1/x86_64/hello_rpkg-0.0.wtree.zlr9pk-1.fc28.x86_64.rpm

is the installable rpm package we were looking for.

Note that the dynamic suffixes in the file path will be different for you and if you have cloned https://pagure.io/hello_rpkg.git and ran the rpkg local command on it, you won’t even see the .wtree.<suffix> indicating that the build was done from a dirty working Git tree.

If you see .wtree.<suffix> in the rpm filename, you can optionally get rid of it by committing your changes:

$ git add -A
$ git commit # you can also use rpkg commit

If you call rpkg local now, you will get a different filename without the .wtree.<suffix> part:

Wrote: /tmp/rpkg/hello_rpkg-2-j4eo3lk3/x86_64/hello_rpkg-0.0.git.1.3e85bc8-1.fc28.x86_64.rpm

That’s because package version generated by {{{ git_dir_version }}} rpkg macro in the spec file has changed to indicate that we are building from clean content of a git commit.

Anyway, if you want to finally install the resulting package to try your application out, you can do so with:

$ sudo rpm -i /tmp/rpkg/hello_rpkg-2-j4eo3lk3/x86_64/hello_rpkg-0.0.git.1.3e85bc8-1.fc28.x86_64.rpm

You can now launch the application binary being placed at /usr/bin/hello_rpkg.

$ hello_rpkg
Hello rpkg!

You can uninstall the package with:

$ sudo rpm -e hello_rpkg

To actually use the public Copr service for building, we will need to additionally install copr-cli package. On Fedora systems, that would be:

$ sudo dnf install copr-cli

Then you can follow this guide to create an account and the first project on Fedora Copr instance.

Afterwards, you can send your build from your project directly into Fedora Copr with:

$ rpkg build <your-project-name>

rpkg build is a very useful command to verify that something successfully builds and to share product of your work publicly in a very easy manner but there are other features that makes rpkg useful as well.

For example, you can use it’s tagging functionality to create an annotated Git tag on a commit, which can, for example, signify a release-ready state of your application:

$ rpkg tag

Such annotated tag (together with any new commits) can be then pushed into remote repository with:

$ rpkg push

You can also, for example, check if the generated spec file conform to certain rpm standards with:

$ rpkg lint

Or you can also use rpkg as a DistGit client, which will allow you to store large binary files (that are not suitable to be directly tracked by Git) only as links to an external storage from which they can be pulled on demand. See rpkg sources and rpkg upload commands for that in rpkg manual pages (man rpkg).

Existing project

For an already existing project, the situation is very similar to a new project. You still need a spec file template or at least a plain spec file (yes, rpkg can work with plain spec files as well) placed somewhere in your source tree. But additionally, if you already have some Git tags created in your project and you maintain a certain format in their names, you need to decide whether to keep that format or switch to the format that rpkg natively supports, which is:

<name>-<version>-<release>

Let’s illustrate the situation on https://pagure.io/hello_rpkg_existing.git repository where already some development has happened and where some tags already exists.

git clone https://pagure.io/hello_rpkg_existing.git
cd hello_rpkg_existing

If you now display the tags, you get:

$ git tag
0.1
0.2
0.3
0.4

If you look at https://pagure.io/hello_rpkg_existing/releases, you can see that each tag signifies an application release and the name of the tag is the version for each release.

The most recent tag 0.4 is attached to the latest commit:

$ git log -1
commit 437de4913bd38b37a0a462d65e5497d9c516df5d (HEAD -> master, tag: 0.4)
Author: clime <clime@redhat.com>
Date:   Tue Jul 31 13:04:41 2018 +0200

    fix project URL

which means the latest commit in the project corresponds to version 0.4 of your application and if you generate an rpm file from that particular commit, it should contain 0.4 in its name so that later anyone can immediatelly tell from what particular code that rpm was generated.

Yet, if you invoke rpkg -q verrel to find out what rpkg thinks the application version for the commit is (and what it will put into the rpm name as well), you get:

$ rpkg -q verrel
hello_rpkg_existing-0.0.git.11.437de49-1.fc28

rpkg thinks the current version is 0.0.git.11.437de49 (see git_version on how exactly the version string is constructed) and that’s because it wasn’t able to read the current version out from just 0.4, which is the most recent tag name. rpkg can only parse tag names of the form <name>-<version>-<release> as mentioned above, which would be hello_rpkg_existing-0.4-1 in this particular case. Tag names of any other format are simply ignored.

To make rpkg start recognizing the version correctly, you can do:

$ git tag -a hello_rpkg_existing-0.4-1

This will create a tag on the latest commit (next to the already existing tag 0.4) from which rpkg can parse out the current version. -a switch specifies that the created tag should be “annotated”, which means you will be able to attach a message to it, e.g. “Initial tag for rpkg”. If you call rpkg -q verrel now, you will finally get:

$ rpkg -q verrel
hello_rpkg_existing-0.4-1.fc28

Note that you can also create tags on past commits, e.g.:

$ git tag -a hello_rpkg_existing-0.3-1 38b4e21204042b1fae03b2ff96474d00614f1010

which is useful if the latest commit in your project is not in a release-ready-yet state.

So by creating the initial tag in the format recognized by rpkg, you can make rpkg immediatelly work for your project. You will be able to create future tags simply with rpkg tag command and rpkg will keep incrementing the version for you.

But you can also decide you don’t want to adopt the rpkg’s native tag format but at the same time, you would like to use rpkg tag command for tagging. Regrettably, this is not possible at the moment, although it could be implemented in a future release.

It shouldn’t matter that much, however, because instead of rpkg tag, you can just use git tag or git tag -a to create tags in your own format. Of course, there is still the problem with git_dir_version not being able to recognize that format but you can solve it by writing your own rpkg versioning macro. See User-defined macros and you might also want to look at Version and tag generation to get a bit of theory behind versioning.

Finally, you can pick a fully manual approach where instead of using git_dir_version in an input spec file template, you will be bumping (incrementing) version in the template manually with each new release. In other words, you will need to increment value of the Version: field, commit that change, and create a tag on that commit. This is how rpkg project also did its versioning in the early times.

Multi-package project

By a multi-package project we mean a project with multiple spec files. Typical setup is to have those spec files placed in separate directories together with the associated source files. You can find an example project here: https://pagure.io/hello_rpkg_multipackage.

If you clone the project:

$ git clone https://pagure.io/hello_rpkg_multipackage

and investigate the content:

$ cd hello_rpkg_multipackage
$ tree
.
├── package1
│   ├── hello_rpkg.spec.rpkg
│   ├── main.c
│   └── Makefile
├── package2
│   ├── hello_rpkg.spec.rpkg
│   ├── main.c
│   └── Makefile
└── README.md

you will find out that apart from the README.md file, there are two subdirectories package1 and package2, each one containing a spec file, thus each forming a subpackage.

In this example, package1 and package2 are just copies of each other, which won’t be typically the case, but even though they are just copies, each subpackage will display a little bit different behavior due to the fact that they are placed in different directories and git_dir_* rpkg macros are being used in each of the spec files.

While git_dir_* macros are also recommended to be used just for simple “flat” projects as shown in the New project section in this tutorial, their main use-case is exactly the multi-package projects. They take into account their placement inside the repository tree structure and use the parent directory names to form the final package name. Also, the source archive creation by git_dir_archive and git_dir_pack macros is limited in scope to the spec file’s directory. Let us show the behavior on concrete rpkg commands:

Switching to package1 subdirectory:

$ cd package1

Getting <name>-<version>-<release>:

$ rpkg -q verrel
hello_rpkg_multipackage-package1-0.0.git.1.a7d3874-1.fc28

Creating source rpm:

$ rpkg srpm
git_dir_pack: packing path /home/clime/hello_rpkg_multipackage/package1
git_dir_pack: Wrote: /tmp/rpkg/hello_rpkg-20-pb0m3ycv/hello_rpkg_multipackage-package1-0.0.git.1.a7d3874.tar.gz
Wrote: /tmp/rpkg/hello_rpkg-20-pb0m3ycv/hello_rpkg.spec
Wrote: /tmp/rpkg/hello_rpkg-20-pb0m3ycv/hello_rpkg_multipackage-package1-0.0.git.1.a7d3874-1.fc28.src.rpm

Tagging package1:

$ rpkg tag
...
Wrote: /tmp/rpkg/hello_rpkg-21-7ox8t_90/hello_rpkg.spec
Created tag: hello_rpkg_multipackage-package1-0.1-1
Wrote: /tmp/rpkg/hello_rpkg-21-7ox8t_90/hello_rpkg.spec

Getting <name>-<version>-<release> after tagging package1:

$ rpkg -q verrel
hello_rpkg_multipackage-package1-0.1-1.fc28

Switching to package2:

$ cd ../package2

Getting <name>-<version>-<release> for package2:

$ rpkg -q verrel
hello_rpkg_multipackage-package2-0.0.git.1.a7d3874-1.fc28

You can see how rpkg actions are scoped by the subpackages. We have tagged package1 with tag name hello_rpkg_multipackage-package1-0.1-1, which will influence names of any future generated rpms for package1 accordingly to the output of rpkg -q verrel. But when rpkg -q verrel is invoked again in the package2 subdirectory, it returns <name>-<version>-<release> completely unaffected by the tag created for package1.

To summarize, this is how you can maintain a multi-package project with rpkg - just by placing the spec files for each package into appropriately named directories and then let git_dir_* macros do their job when spec files are being renderred. Note that if you are unhappy with default way, the git_dir_* macros operate, you can tweak their behavior for each individual subpackage by passing parameters to them.

For example, let’s say that the resulting rpm names seem too long to you and you would like to shorten them by leaving out the word “multipackage”. Let’s show it on package1 only for simplicity. Open package1/hello_rpkg.spec.rpkg and replace the:

Name:       {{{ git_dir_name }}}

line with:

Name:       {{{ git_dir_name name=hello_rpkg-package1 append= }}}

Now if you save and close the file and invoke rpkg -q verrel, you will see that the generated name is different than before:

hello_rpkg-package1-0.0.git.1.a7d3874.wtree.zm2f6k-1.fc28

See Macro reference for more information about the macros and their parameters.

If you already maintain a multi-package project and you have your own tagging scheme, you can either adopt the rpkg’s native tagging scheme (<name>-<version>-<release>) by creating an initial tag manually by using git tag -a command for each subpackage or you can create your own variant of git_dir_version macro which is able to recognize your custom scheme. See Existing project section for more information on this topic.

Packed content

You can also use rpkg on a project content consisting just of spec file and tarballs, which are data the underlying rpmbuild command operates on. In this case, instead of using an archiving macro to generate an rpm source, you just use the tarball filename directly as a value of Source rpm field. It’s also good to combine this with storing the tarballs not in the Git repository directly but instead in a so-called dist-git lookaside cache that rpkg can work with (read more about dist-git).

Let’s see a concrete example of this:

$ rpkg clone prunerepo
$ cd prunerepo
$ ls
prunerepo.spec  sources

Here, we have cloned a package called prunerepo from https://src.fedoraproject.org, which is a dist-git instance rpkg operates on by default (you can change this by modifying /etc/rpkg.conf).

In the package repository, a plain spec file prunerepo.spec is present and the sources file which maintains links to the individual tarballs stored in the dist-git lookaside cache and which are also being referenced from the spec file:

$ cat sources
SHA512 (prunerepo-1.11.tar.gz) = 2df567872208e0181d0f85dff6a74248e516d95fe073322e7f849d568b988f2d330b942f72099d422e2cc48bfe46b5c4756d6201831bc9fa8287d665203092a5
$ grep '^Source' prunerepo.spec
Source0: %{name}-%{version}.tar.gz

Note

%{name}-%{version}.tar.gz is translated into final resulting filename prunerepo-1.11.tar.gz by rpm. Both %{name} and %{version} are rpm macros which reference values from Name: and Version: spec fields. These are not rpkg macros but rpm macros.

The source files are currently not present in our directory so we need to download them first before we will be able to do anything useful with them (e.g. build a source rpm).

$ rpkg sources
Downloading prunerepo-1.11.tar.gz
######################################################################## 100.0%
$ ls
prunerepo-1.11.tar.gz  prunerepo.spec  sources

Now we can use the downloaded source file prunerepo-1.11.tar.gz and, for example, build a source rpm:

$ rpkg srpm
Wrote: /tmp/rpkg/prunerepo-1-rlvpewp9/prunerepo.spec
Wrote: /tmp/rpkg/prunerepo-1-rlvpewp9/prunerepo-1.11-4.fc28.src.rpm

You can add new tarballs into dist-git lookaside cache by using rpkg upload command but in case of src.fedoraproject.org, the request would need to be authenticated with Fedora.

Plain directory content

In case you would like to maintain your project just locally in a plain directory (without Git), you can still use rpkg for building, linting, or verifying rpms. It is also possible to use rpkg for downloading and uploading large binary files from/to dist-git like this but you need to use rpkg --module-name parameter to specify the file location in the dist-git’s lookaside cache.