Sorry for the hard-to-read patch. I copied the lack of line-breaks from the rest of the file, but can wrap and repush if desired.

aka I should BR /usr/bin/grep over %{_bindir}/grep?

This isn't a trivial change; it should have a ticket for discussion.

It is generally preferred that you use semantic line breaks when adding text even thought the rest of the document hasn't been converted to use them. We will have to fix them all at some point.

@churchyard: I believe that's the point, since flatpak leverages redefinition of %_prefix and such to redirect things.

I guess I would be OK with this, but note that I see at least 265 packages which would need fixing to comply. (grep -li 'buildrequires:.*%.*bindir' * over all rawhide specs).

I do wonder if it's insufficient, though. You basically can't reference any path macro outside of a context where it refers to the output of the package itself. Fortunately things like %__cp use absolute paths so they won't break (though we discourage their use for other reasons).

@churchyard: aka I should BR /usr/bin/grep over %{_bindir}/grep?

Ah, the wording here is was confusing (now tweaked). I meant this mostly to refer to running %{_bindir}/grep instead of just running grep. I didn't really think about the BuildRequires case, since I hadn't hit such a problem in going through 100-200 packages needed for Flatpaks. But yes, for the BuildRequires case, you should either BR /usr/bin/grep or the package name (grep, in this case).

@tibbs This isn't a trivial change; it should have a ticket for discussion.

OK, filed an issue #848

It is generally preferred that you use semantic line breaks when adding text even thought the rest of the document hasn't been converted to use them. We will have to fix them all at some point.

Ah, I see that in the README.md. Repushed that way.

I guess I would be OK with this, but note that I see at least 265 packages which would need fixing to comply. (grep -li 'buildrequires:.%.bindir' * over all rawhide specs).

265/(number of fedora packages) is something like a 2% rate, so it's actually not a very common problem (and isn't even when you go past BuildRequires to use in %build, etc.) I mostly want the assumption documented so that as we go through and make things work for Flatpak we have something to point packagers to as necessary. I'm not sure it's worth spending a lot of effort fixing Fedora packages that aren't relevant to the Flatpak effort.

I do wonder if it's insufficient, though. You basically can't reference any path macro outside of a context where it refers to the output of the package itself. Fortunately things like %__cp use absolute paths so they won't break (though we discourage their use for other reasons).

Yes, you can have other problems outside of BuildRequires - and that's what I was thinking about. So that makes it harder to automatically find issues. It's hard for a script to know if a reference in %build is to an installation location (the common case) or to a file in a build dependency (the rare and problematical case.)

I think the text is unnecessarily complicated and maybe it can be approached differently?

Aka (this is an outline, not actual text proposal):

Only use %{_prefix} (and derived macros, such as %{_bindir} etc.) when specifying output of your package (e.g. when installing files into specific locations or when listing files in the %files section), never use it when using stuff needed for your package (e.g. don't call or BuildRequire %{_bindir}/foo, just use foo or hardcode /usr/bin/foo). Rule of thumb: If you redefine %{_prefix}, your package should not fail to build due to broken dependencies or commands. Flatpak rationale here.

What if you need to build two packages into a flatpack and the later one build-requires the former one? E.g. first package installs foo into %{_bindir}/foo (i.e. /flatpack/usr/bin/foo). If the second spec file calls /usr/bin/foo it will use wrong (or a non-existing) foo. If the second package calls foo, again it will use wrong or non-existing foo. (Or does flatpack build root add /flatpack/usr/bin to PATH?) Obviously the second spec file should call %{_bindir}/foo but exactly that is being forbidden.

Do we use flatpaks to build flatpaks? I don't know the exact process.

I meant one flatpack containing both the packages. (But I also don't know anything about building flatpacks.)

Although having multiple flatpacks is also another case when the approach breaks:

Let's say I'm building a flatpack with the later package only. The flatpack will use the former package from a system. I change the second spec file according the proposed change and my flatpack becomes building successfully.

Then someone else will create an independent flatpack that will contain both the packages. He will hit the issue I described. So he fixes the second spec file to use %{_bindir}/foo again (effectively reverts the change). No his flapack builds fine, but the first flatpack is broken again.

I worry the proposed change cannot cater both cases.

Working fix would be to come with a foo-specific macro that would define the desired path and both the packages would use it. Then only the flatpacks that needs foo inside would redefine the macro (as they do with %{_prefix}).

One could say that that leads to proliferating macros. That's true. It's because the number of possible slices between a base system and a flatpak is exponential to a number of the packages. In my opinion this problem is isomorphic to problem of rings or software collections. That's one of the reasons wile software collection were never allowed in Fedora. Modules solved it by not changing %{_prefix} and maintaining build order. However it seems that flatpacks are trying the hard way of software collections again.

What if you need to build two packages into a flatpack and the later one build-requires the former one? E.g. first package installs foo into %{_bindir}/foo (i.e. /flatpack/usr/bin/foo). If the second spec file calls /usr/bin/foo it will use wrong (or a non-existing) foo.

If you have something that is just a build requirement (a compiler or documentation generation tool, for example), then it doesn't need to be rebuilt with prefix=/app. But if a source package both provides build requirements and also a runtime requirements then it needs to be rebuilt. Common examples include:

• -devel packages
• Language interpreters/runtimes

Because of this, hard-coding an absolute path is, yes, doesn't always work. So it's the least preferred alternative. Better options:

• Letting the shell search $PATH
• Using RPM macros provided by the dependency
• Using pkg-config
• Build requiring the providing package rather than an executable (grep rather than /usr/bin/grep)

If the second package calls foo, again it will use wrong or non-existing foo. (Or does flatpack build root add /flatpack/usr/bin to PATH?) Obviously the second spec file should call %{_bindir}/foo but exactly that is being forbidden.

The Flatpak buildroot does add its bindir (/app/bin) to the PATH, so if you just call 'foo', that works in either case.

In complicated situations %if 0%{?flatpak}` conditionalization may be needed, but there isn't a lot of that. https://src.fedoraproject.org/rpms/perl/pull-request/1 is probably the most complicated situation I've hit yet - not only is a language runtime rebuilt, but the core packages BuildRequire and Require each other.

I don't think it makes sense to try to get into detail about what you could do in complicated situations in the packaging guidelines yet, since we don't have enough examples to know what's general, and what's just a one-off fix for a particular situation. (All fixes I've needed so far are found at: https://fedoraproject.org/wiki/Flatpak:Fixes) The information I'd like to convey to packagers at this point is:

• Path macros define installation locations
• Build dependencies you are depending on may have been built with different paths
• Try to avoid ~~hard-coding~~absolute paths. If you do need to ~~hardcode~~absolute paths, it's better to use the standard system paths. [updated to clarify what I meant]

Then someone else will create an independent flatpack that will contain both the packages. He will hit the issue I described. So he fixes the second spec file to use %{_bindir}/foo again (effectively reverts the change). No his flapack builds fine, but the first flatpack is broken again.

More likely, at this point, then the second Flatpak maintainer and the RPM maintainer, say "wait a moment, this doesn't work, we need to figure something else out".

(Generally, packages either need to be rebuilt with prefix=/app for Flatpaks or not. gcc doesn't need to be rebuilt, libvpx-devel does. But there are exceptions - e.g. going back to perl - it could either be a dependency of automake or what the application is written in. )

I worry the proposed change cannot cater both cases.

Yes - if you need to use an absolute path, then there can be issues.

Working fix would be to come with a foo-specific macro that would define the desired path and both the packages would use it. Then only the flatpacks that needs foo inside would redefine the macro (as they do with %{_prefix}).

Yes, this is one possible approach, and in many cases, these macros already exist.

One could say that that leads to proliferating macros. That's true. It's because the number of possible slices between a base system and a flatpak is exponential to a number of the packages.

That does not imply that the number of macros needed is exponential! If it's necessary to abstract paths, a single macro per package works.

In my opinion this problem is isomorphic to problem of rings or software collections. That's one of the reasons wile software collection were never allowed in Fedora. Modules solved it by not changing %{_prefix} and maintaining build order. However it seems that flatpacks are trying the hard way of software collections again.

Because Flatpaks build on a) modules and b) containers, things are much simpler than with software collections. Modules: you don't need to BuildRequire a "Flatpak version" of package X, you simply depend on package X, and if the Flatpak version should be used, that's what's in your build root. Containers: there is an isolated namespace at runtime with only the libraries and executables that you depend upon.

Sorry for long dealy.

I opened RPM upstream ticket which describes proper solution to this problem from my POV: https://github.com/rpm-software-management/rpm/issues/721

Let's see what RPM folks will say about that.

We discussed this at this weeks meeting (https://meetbot-raw.fedoraproject.org/fedora-meeting-1/2019-11-21/fpc.2019-11-21-17.00.txt):

• #847 PR: Clarify the use of path macros wrt build dependencies
  (geppetto, 17:20:34)
• LINK: https://pagure.io/packaging-committee/pull-request/847
  (geppetto, 17:20:43)
• Rpm upstream probably fine with ignatenkobrain patch, need some time
  to do one though. (geppetto, 17:24:34)

This PR is years old and has conflicts. The upstream RFE for solving this a different way got a bit of discussion but then didn't really go anywhere (though using %_root_bindir` was eventually preferred.

I guess that means the original change might still be needed, but it's tough to know after all of this time. Closing; feel free to reopen or just post a new PR without conflicts if we do need to consider this again.