.. _quick_start_v3:

Quick start
===========

.. _new_project_v3:

New project
-----------

Let's say you have decided to create a new mind-blowing application and
you would like to out-source the building process from your own machine or
you would like to make the latest installable version of the app always available
somewhere for you and your teammates to download and debug on it.

This is what rpkg can do for you among other things thanks to its ``rpkg build``
command that can send your sources for build in a public build system called
Copr (stands for "Community projects").

To make this command work, you need to write an rpm spec file template and
place it along-side your sources. If you don't know what spec file is or you
know what it is but not exactly how to write one, you can optionally read some details
`here <https://rpm-packaging-guide.github.io/#what-is-a-spec-file>`_
but you can also just continue reading on to get the quick start you came for.

rpkg enriches the basic rpm spec files with its own syntax. Such enriched spec file
or "spec file template" is then used to generate the actual rpm spec file that
is then used to build an installable rpm package but this whole process is already
automated for you.

The long story short, below is an example spec file template called ``hello_rpkg.spec.rpkg``
with comments included that you can use as a starting point for your own Git project.
``{{{ ... }}}`` expressions (``git_dir_pack/git_dir_archive`` macros are the only one
that have a "side effect" of building a tarball) are the above mentioned rpkg's extensions
to the basic rpm spec file syntax that make it possible to derive certain spec file parts
dynamically from Git repository metadata.


``hello_rpkg.spec.rpkg``

.. code-block:: spec

    # The following tag is to get correct syntax highlighting for this file in vim text editor
    # vim: syntax=spec

    # git_dir_name returns repository name derived from remote Git repository URL
    Name:       {{{ git_dir_name }}}

    # git_dir_version returns version based on commit and tag history of the Git project
    Version:    {{{ git_dir_version }}}

    # This can be useful later for adding downstream patches
    Release:    1%{?dist}

    # Basic description of the package
    Summary:    Hello rpkg package.

    # License. We assume GPLv2+ here.
    License:    GPLv2+

    # Home page of the project. Can also point to the public Git repository page.
    URL:        https://pagure.io/hello_rpkg

    # Detailed information about the source Git repository and the source commit
    # for the created rpm package
    VCS:        {{{ git_dir_vcs }}}

    # git_dir_pack macro places the repository content (the source files) into a tarball
    # and returns its filename. The tarball will be used to build the rpm.
    Source:     {{{ git_dir_pack }}}

    # More detailed description of the package
    %description
    This is a hello world package.

    # The following four sections already describe the rpm build process itself.
    # prep will extract the tarball defined as Source above and descend into it.
    %prep
    {{{ git_dir_setup_macro }}}

    # This will invoke `make` command in the directory with the extracted sources.
    %build
    make

    # This will copy the files generated by the `make` command above into
    # the installable rpm package.
    %install
    make install root=%{buildroot}

    # This lists all the files that are included in the rpm package and that
    # are going to be installed into target system where the rpm is installed.
    %files
    /usr/bin/hello_rpkg

    # Finally, changes from the latest release of your application are generated from
    # your project's Git history. It will be empty until you make first annotated Git tag.
    %changelog
    {{{ git_dir_changelog }}}

For this example spec file template to be any useful, you also need the Makefile so that the
``make`` commands in ``%build`` and ``%install`` sections work correctly and some application
sources from which the ``hello_rpkg`` binary will be compiled. Let's use the following
two files for that purpose:

``Makefile``

.. code-block:: makefile

    CC=gcc
    ARGS=-Wall -g

    all: main.o
            ${CC} ${ARGS} -o main main.o

    main.o: main.c
            ${CC} ${ARGS} -c main.c

    install:
            mkdir -p $(root)/usr/bin
            cp main $(root)/usr/bin/hello_rpkg

    clean:
            rm main *.o

``main.c``

.. code-block:: c

    #include <stdio.h>

    int main()
    {
        printf("%s", "Hello rpkg!\n");
        return 0;
    }

If you have those three files (``hello_rpkg.spec.rpkg``, ``Makefile``, ``main.c``) placed
in a plain (i.e. not git-initialized) directory, then you need to invoke the following commands
in that directory to properly initialize the Git repository for further rpkg operations:

::

    git init
    git remote add origin https://pagure.io/hello_rpkg.git
    git add -A

But you can also just clone the repository at https://pagure.io/hello_rpkg.git where
everything is already prepared for you ;).

::

    git clone https://pagure.io/hello_rpkg.git
    cd hello_rpkg

Either way, you are now ready to invoke your first rpkg command given that you have
rpkg installed (see :ref:`installation_v3`).

.. code-block:: sh

    $ rpkg local
    git_dir_pack: packing path /home/clime/hello_rpkg_test
    git_dir_pack: Wrote: /tmp/rpkg/hello_rpkg-7-a5z13rvo/hello_rpkg-master-dirty.tar.gz
    Wrote: /tmp/rpkg/hello_rpkg-7-a5z13rvo/hello_rpkg.spec
    ... many lines later ...
    Wrote: /tmp/rpkg/hello_rpkg-7-a5z13rvo/hello_rpkg-0.0.dirty.02lx51-1.fc31.src.rpm
    Wrote: /tmp/rpkg/hello_rpkg-7-a5z13rvo/x86_64/hello_rpkg-debugsource-0.0.dirty.02lx51-1.fc31.x86_64.rpm
    Wrote: /tmp/rpkg/hello_rpkg-7-a5z13rvo/x86_64/hello_rpkg-0.0.dirty.02lx51-1.fc31.x86_64.rpm
    Wrote: /tmp/rpkg/hello_rpkg-7-a5z13rvo/x86_64/hello_rpkg-debuginfo-0.0.dirty.02lx51-1.fc31.x86_64.rpm
    ... some more no longer interesting lines ...

So this output file:

::

    Wrote: /tmp/rpkg/hello_rpkg-7-a5z13rvo/x86_64/hello_rpkg-0.0.dirty.02lx51-1.fc31.x86_64.rpm

is the installable rpm package we were looking for.

Note that the dynamic suffixes in the file path will be different for you and if you have cloned
https://pagure.io/hello_rpkg.git and ran the ``rpkg local`` command on it, you won't
even see the ``.dirty.<suffix>`` indicating that the build was done from a dirty working Git tree.

If you see ``.dirty.<suffix>`` in the rpm filename, you can optionally get rid of it by
committing your changes:

.. code-block:: sh

    $ git add -A
    $ git commit # you can also use rpkg commit

If you call ``rpkg local`` now, you will get a different filename without the ``.dirty.<suffix>`` part:

::

    Wrote: /tmp/rpkg/hello_rpkg-8-3ffakz0_/x86_64/hello_rpkg-0.0.git.1.028966b2-1.fc31.x86_64.rpm

That's because package version generated by ``{{{ git_dir_version }}}`` rpkg macro in the spec file
has changed to indicate that we are building from clean content of a git commit.

Anyway, if you want to finally install the resulting package to try your application out, you can do
so with:

.. code-block:: sh

    $ sudo rpm -i /tmp/rpkg/hello_rpkg-8-3ffakz0_/x86_64/hello_rpkg-0.0.git.1.028966b2-1.fc31.x86_64.rpm

You can now launch the application binary being placed at ``/usr/bin/hello_rpkg``.

.. code-block:: sh

    $ hello_rpkg
    Hello rpkg!

You can uninstall the package with:

.. code-block:: sh

    $ sudo rpm -e hello_rpkg

To actually use the public Copr service for building, we will need to additionally install `copr-cli`
package. On Fedora systems, that would be:

.. code-block:: sh

    $ sudo dnf install copr-cli

Then you can follow `this guide <https://docs.pagure.org/copr.copr/user_documentation.html#quick-start>`_
to create an account and the first project on Fedora Copr instance.

Afterwards, you can send your build from your project directly into Fedora Copr with:

.. code-block:: sh

    $ rpkg build <your-project-name>

``rpkg build`` is a very useful command to verify that something successfully builds and to share product
of your work publicly in a very easy manner but there are other features that makes ``rpkg`` useful as well.

.. note::

    `TIP:` You can specify rpkg.copr_project configuration variable to be able to type only `rpkg build`
    to build a project.

For example, you can use it's tagging functionality to create an annotated Git tag on a commit, which can,
for example, signify a release-ready state of your application:

::

    $ rpkg tag

Such annotated tag (together with any new commits) can be then pushed into remote repository with:

::

    $ rpkg push

You can also, for example, check if the generated spec file conform to certain rpm standards with:

::

    $ rpkg lint

Or you can also use `rpkg` as a `DistGit <https://github.com/release-engineering/dist-git>`_ client,
which will allow you to store large binary files (that are not suitable to be directly tracked by Git)
only as links to an external storage from which they can be pulled on demand. See ``rpkg sources`` and
``rpkg upload`` commands for that in rpkg manual pages (``man rpkg``).

.. _existing_project_v3:

Existing project
----------------

For an already existing project, the situation is very similar to a new project. You still need
a spec file template or at least a plain spec file (yes, rpkg can work with plain spec files as well)
placed somewhere in your source tree. But additionally, if you already have some Git tags created
in your project and you maintain a certain format in their names, you need to decide whether
to keep that format or switch to the format that rpkg natively supports, which is:

::

    <name>-<version>-<release>

Let's illustrate the situation on https://pagure.io/hello_rpkg_existing.git repository
where already some development has happened and where some tags already exists.

::

    git clone https://pagure.io/hello_rpkg_existing.git
    cd hello_rpkg_existing

If you now display the tags, you get:

::

     $ git tag
     0.1
     0.2
     0.3
     0.4

If you look at https://pagure.io/hello_rpkg_existing/releases, you can see that each tag
signifies an application release and the name of the tag is the version for each release.

The most recent tag ``0.4`` is attached to the latest commit:

::

    $ git log -1
    commit 437de4913bd38b37a0a462d65e5497d9c516df5d (HEAD -> master, tag: 0.4)
    Author: clime <clime@redhat.com>
    Date:   Tue Jul 31 13:04:41 2018 +0200

        fix project URL

which means the latest commit in the project corresponds to version ``0.4`` of your application
and if you generate an rpm file from that particular commit, it should contain ``0.4`` in its
name so that later anyone can immediatelly tell from what particular code that rpm was generated.

Yet, if you invoke ``rpkg nvr`` to find out what rpkg thinks the application version for
the commit is (and what it will put into the rpm name as well), you get:

::

    $ rpkg nvr
    hello_rpkg_existing-0.0.git.11.437de491-1.fc31

rpkg thinks the current version is ``0.0.git.11.437de491`` (see :ref:`git_version <git_version_v3>` on
how exactly the version string is constructed) and that's because it wasn't able to read the current version
out from just ``0.4``, which is the most recent tag name. rpkg can only parse tag names of the form
``<name>-<version>-<release>`` as mentioned above, which would be ``hello_rpkg_existing-0.4-1``
in this particular case. Tag names of any other format are simply ignored.

To make rpkg start recognizing the version correctly, you can do:

::

    $ git tag -a hello_rpkg_existing-0.4-1

This will create a tag on the latest commit (next to the already existing tag ``0.4``) from which
rpkg can parse out the current version. ``-a`` switch specifies that the created tag should be "annotated",
which means you will be able to attach a message to it, e.g. "Initial tag for rpkg". If you call
``rpkg nvr`` now, you will finally get:

::

     $ rpkg nvr
     hello_rpkg_existing-0.4-1.fc31

Note that you can also create tags on past commits, e.g.:

::

    $ git tag -a hello_rpkg_existing-0.3-1 38b4e21204042b1fae03b2ff96474d00614f1010

which is useful if the latest commit in your project is not in a release-ready-yet state.

So by creating the initial tag in the format recognized by rpkg, you can make rpkg
immediatelly work for your project. You will be able to create future tags simply with
``rpkg tag`` command and rpkg will keep incrementing the version for you.

But you can also decide you don't want to adopt the rpkg's native tag format
but at the same time, you would like to use ``rpkg tag`` command for tagging. Regrettably,
this is not possible at the moment, although it could be implemented in a future
release.

It shouldn't matter that much, however, because instead of ``rpkg tag``, you can just
use ``git tag`` or ``git tag -a`` to create tags in your own format. Of course,
there is still the problem with ``git_dir_version`` not being able to recognize
that format but you can solve it by writing your own rpkg versioning macro.
See :ref:`user_defined_macros_v3` and you might also want to look at
:ref:`version_and_tag_generation_v3` to get a bit of theory behind versioning.

Finally, you can pick a fully manual approach where instead of using ``git_dir_version``
in an input spec file template, you will be bumping (incrementing) version in the
template manually with each new release. In other words, you will need to increment
value of the ``Version:`` field, commit that change, and create a tag on that commit.
This is how rpkg project also did its versioning in the early times.

.. _multipackage_project_v3:

Multi-package project
---------------------

By a multi-package project we mean a project with multiple spec files. Typical setup
is to have those spec files placed in separate directories together with the associated
source files. You can find an example project here: https://pagure.io/hello_rpkg_multipackage.

If you clone the project:

::

    $ git clone https://pagure.io/hello_rpkg_multipackage

and investigate the content:

::

    $ cd hello_rpkg_multipackage
    $ tree
    .
    ├── package1
    │   ├── hello_rpkg.spec.rpkg
    │   ├── main.c
    │   └── Makefile
    ├── package2
    │   ├── hello_rpkg.spec.rpkg
    │   ├── main.c
    │   └── Makefile
    └── README.md

you will find out that apart from the ``README.md`` file, there are two subdirectories
``package1`` and ``package2``, each one containing a spec file, thus each
forming a subpackage.

In this example, ``package1`` and ``package2`` are just copies of each other, which won't
be typically the case, but even though they are just copies, each subpackage will display
a little bit different behavior due to the fact that they are placed in different directories
and ``git_dir_*`` rpkg macros are being used in each of the spec files.

While ``git_dir_*`` macros are also recommended to be used just for simple "flat" projects as
shown in the *New project* section in this tutorial, their main use-case is exactly the multi-package
projects. They take into account their placement inside the repository tree structure and
use the parent directory names to form the final package name. Also, the source archive
creation by ``git_dir_archive`` and ``git_dir_pack`` macros is limited in scope to the
spec file's directory. Let us show the behavior on concrete rpkg commands:

Switching to ``package1`` subdirectory:

::

    $ cd package1

Getting ``<name>-<version>-<release>``:

::

    $ rpkg nvr
    hello_rpkg_multipackage-package1-0.0.git.1.a7d38743-1.fc31

Creating source rpm:

::

    $ rpkg srpm
    git_dir_pack: archiving /home/clime/hello_rpkg_multipackage/package1:
    commit a7d38743b24d1a79bcd0c513d628732890d18162 (HEAD -> master, origin/master, origin/HEAD)
    Author: clime <clime@redhat.com>
    Date:   Wed Aug 1 09:17:53 2018 +0200

        Initial commit
    git_dir_pack: Wrote: /tmp/rpkg/hello_rpkg-9-qzsyjd8s/hello_rpkg_multipackage-package1-a7d38743.tar.gz
    Wrote: /tmp/rpkg/hello_rpkg-9-qzsyjd8s/hello_rpkg.spec
    Wrote: /tmp/rpkg/hello_rpkg-9-qzsyjd8s/hello_rpkg_multipackage-package1-0.0.git.1.a7d38743-1.fc31.src.rpm

Tagging ``package1``:

::

    $ rpkg tag
    ...
    Created tag hello_rpkg_multipackage-package1-0.1-1

Getting ``<name>-<version>-<release>`` after tagging ``package1``:

::

    $ rpkg nvr
    hello_rpkg_multipackage-package1-0.1-1.fc31

Switching to ``package2``:

::

    $ cd ../package2

Getting ``<name>-<version>-<release>`` for ``package2``:

::

    $ rpkg nvr
    hello_rpkg_multipackage-package2-0.0.git.1.a7d38743-1.fc31

You can see how rpkg actions are scoped by the subpackages. We have tagged ``package1``
with tag name ``hello_rpkg_multipackage-package1-0.1-1``, which will influence names of any future
generated rpms for ``package1`` accordingly to the output of ``rpkg nvr``. But when
``rpkg nvr`` is invoked again in the ``package2`` subdirectory, it returns ``<name>-<version>-<release>``
completely unaffected by the tag created for ``package1``.

To summarize, this is how you can maintain a multi-package project with rpkg - just by placing
the spec files for each package into appropriately named directories and then let ``git_dir_*``
macros do their job when spec files are being renderred. Note that if you are unhappy with
default way, the ``git_dir_*`` macros operate, you can tweak their behavior for each individual
subpackage by passing parameters to them.

For example, let's say that the resulting rpm names seem too long to you and you would like to shorten
them by leaving out the word "multipackage". Let's show it on ``package1`` only for simplicity. Open
``package1/hello_rpkg.spec.rpkg`` and replace the:

.. code-block:: spec

    Name:       {{{ git_dir_name }}}

line with:

.. code-block:: spec

    Name:       {{{ git_dir_name name=hello_rpkg-package1 append= }}}

Now if you save and close the file and invoke ``rpkg nvr``, you will see that the generated name
is different than before:

::

    hello_rpkg-package1-0.0.git.1.a7d38743.dirty.02ly99-1.fc31

See :ref:`macro_reference_v3` for more information about the macros and their parameters.

If you already maintain a multi-package project and you have your own tagging scheme, you can either
adopt the rpkg's native tagging scheme (``<name>-<version>-<release>``) by creating an initial tag manually
by using ``git tag -a`` command for each subpackage or you can create your own variant of ``git_dir_version``
macro which is able to recognize your custom scheme. See :ref:`existing_project_v3` section for more information
on this topic.

.. _packed_content_v3:

Packed content
--------------

You can also use ``rpkg`` on a project content consisting just of spec file and tarballs, which
are data the underlying ``rpmbuild`` command operates on. In this case, instead of using an
archiving macro to generate an rpm source, you just use the tarball filename directly
as a value of ``Source`` rpm field. It's also good to combine this with storing the tarballs
not in the Git repository directly but instead in a so-called dist-git lookaside cache  that rpkg
can work with (read more about `dist-git <https://github.com/release-engineering/dist-git>`_).

Let's see a concrete example of this:

::

    $ rpkg clone prunerepo
    $ cd prunerepo
    $ ls
    prunerepo.spec  sources

Here, we have cloned a package called ``prunerepo`` from https://src.fedoraproject.org, which
is a dist-git instance rpkg operates on by default (you can change this by modifying ``/etc/rpkg.conf``).

In the package repository, a plain spec file ``prunerepo.spec`` is present and the ``sources``
file which maintains links to the individual tarballs stored in the dist-git lookaside cache
and which are also being referenced from the spec file:

::

    $ cat sources
    SHA512 (prunerepo-1.20.tar.gz) = f467f0bf70197a5caacf6245a598ea1153d0f34ac75fa81d7c0db597d90244a78e78bce35453e35833a2b69f5efa8aa089a53e30094dbb3191f9e65283286f63
    $ grep '^Source' prunerepo.spec
    Source0: %{name}-%{version}.tar.gz

.. note::

    ``%{name}-%{version}.tar.gz`` is translated into final resulting filename ``prunerepo-1.20.tar.gz`` by rpm.
    Both ``%{name}`` and ``%{version}`` are rpm macros which reference values from ``Name:`` and ``Version:`` spec
    fields. These are not rpkg macros but rpm macros.

The source files are currently not present in our directory so we need to download them
first before we will be able to do anything useful with them (e.g. build a source rpm).

::

    $ rpkg sources
    Downloading prunerepo-1.20.tar.gz from rpms/prunerepo at clime@pkgs.fedoraproject.org:
    ######################################################################## 100.0%
    $ ls
    prunerepo-1.20.tar.gz  prunerepo.spec  sources

Now we can use the downloaded source file ``prunerepo-1.20.tar.gz`` and, for example, build a source rpm:

::

    $ rpkg srpm
    Wrote: /tmp/rpkg/prunerepo-1-_aja0zfg/prunerepo.spec
    setting SOURCE_DATE_EPOCH=1619481600
    Wrote: /tmp/rpkg/prunerepo-1-_aja0zfg/prunerepo-1.20-1.fc31.src.rpm

You can add new tarballs into dist-git lookaside cache by using ``rpkg upload`` command but
in case of ``src.fedoraproject.org``, the request would need to be authenticated with Fedora
(e.g. by Kerberos with ``kinit``).

.. _plain_directory_content_v3:

Plain directory content
-----------------------

In case you would like to maintain your project just locally in a plain directory (without Git),
you can still use ``rpkg`` for building, linting, or verifying rpms. It is also possible to use rpkg
for downloading and uploading large binary files from/to dist-git like this but you need to use
``rpkg --repo-path`` parameter to specify the file location in the dist-git's lookaside cache.