A quick check of some jquery (as an example) js files in html docs present on my system indicate this might be possible:

diff ./alsa-lib-devel/doxygen/html/jquery.js ./mpg123-devel/html/jquery.js

No differences found, so it definitely seems like static content, and not generated per run. So even though this file is a combination of multiple javascript files from doxygen, it is consistent between runs.

Similarly, the sphinx docs use a consistent jquery for example:

diff ./python3-netaddr/html/_static/jquery.js ./python3-pyocr/html/_static/jquery.js

Relevant previous discussion:

https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.org/thread/LLUAURXZVADATHK65HBPPBHKF4EM4UC3/

In my opinion, sphinx-generated HTML documentation bundles too much and should not be packaged.

For what it's worth, I agree. Building documentation is a massive headache to get right, for very small benefit. It's not discoverable, and it would often be better to link upstream docs (if available).

I appreciate packaged documentation. The upstream HTML documentation may be for a different version that what Fedora packages, it many send analytics to Google, and it definitely isn't available offline.

Also, is the "bundling" that goes on in Sphinx documentation that different from Go or Rust packages? The sphinx themes can be considered a devel package whose code is embedded in each package.

The upstream HTML documentation may be for a different version that what Fedora packages,

Upstream documentation sites typically have a toggle to select the matching version of documentation.

it many send analytics to Google,

People that are worried about this probably already use browser extensions to block it. It's not Fedora's responsibility to solve this problem.

and it definitely isn't available offline.

This is a benefit, but how often in practice does it benefit Fedora users? I would wager virtually never. Air-gapped deployments are most common with the government, and they tend to use RHEL, not Fedora. Even if some number of people find the offline docs valuable, we have to compare it holistically with the burden of getting the packaging correct.

Also, is the "bundling" that goes on in Sphinx documentation that different from Go or Rust packages?

It's bundling either way, and we have bundling guidelines. But as detailed in the previously linked thread, the guidelines are very rarely followed for documentation with bundled javascript.

It is very common for Python "library" packages to build (and ship) the Sphinx documentation. However, I believe that they are used as a dependency of some tool or app in the majority of our cases. Their consumers might be interested in the documentation of the tool or app, but unlikely for the library. Whoever needs to read documentation of the library is likely a developer and I'd very much like to steer Python developers away from using RPM-packaged libraries for development.

The upstream HTML documentation may be for a different version that what Fedora packages,

Upstream documentation sites typically have a toggle to select the matching version of documentation.

Some larger projects do, but many of the documentation sites that I use do not.

it many send analytics to Google,

People that are worried about this probably already use browser extensions to block it. It's not Fedora's responsibility to solve this problem.

Whether or not it's Fedora's responsibility to solve this (it probably isn't), this is still a benefit of offline documentation in my book.

and it definitely isn't available offline.

This is a benefit, but how often in practice does it benefit Fedora users? I would wager virtually never. Air-gapped deployments are most common with the government, and they tend to use RHEL, not Fedora. Even if some number of people find the offline docs valuable, we have to compare it holistically with the burden of getting the packaging correct.

Offline docs are useful while traveling or on a bad internet connection.

Also, is the "bundling" that goes on in Sphinx documentation that different from Go or Rust packages?

It's bundling either way, and we have bundling guidelines. But as detailed in the previously linked thread, the guidelines are very rarely followed for documentation with bundled javascript.

What exactly is needed on a -doc subpackage basis to comply with the guidelines here? The FPC hasn't spelled that out. Are bundled() Provides needed when if the assets are not included in the package's Sources? I was under the impression that they're not.

Some larger projects do, but many of the documentation sites that I use do not.

Which ones? By far the most common one I see in Fedora packages is sphinx, which does have this feature. A percentage breakdown of what is currently being used would be useful here.

Offline docs are useful while traveling or on a bad internet connection.

Yes they're useful, but so is cloning the documentation sources or downloading the docs website as offline web pages. There are multiple options.

What exactly is needed on a -doc subpackage basis to comply with the guidelines here? The FPC hasn't spelled that out. Are bundled() Provides needed when if the assets are not included in the package's Sources? I was under the impression that they're not.

The same as what is required for any bundled library. Either debundle it or properly mark all the bundled provides. The packaging guidelines don't make an exception for bundling that happens during the build process as opposed to being part of the source tarball.

Downloaded 15619 spec files from https://src.fedoraproject.org/rpms/
Of these about
- 591 have BuildRequires on Doxygen
- 465 have indirect BuildRequires on Rubygem RDoc through Rubygems-devel
- 220 have BuildRequires on Sphinx
- 13 have BuildRequires on Rubygem RDoc
- 5 have BuildRequires on Rubygem Yard
Have not checked for other documentation generators

Note that querying spec files from build dependencies does not provide complete information because packages may generate the dependency via https://fedoraproject.org/wiki/Changes/DynamicBuildRequires

$ repoquery -q --repo=rawhide{,-source} --whatrequires python3-sphinx | grep src$ | wc -l
744

We discussed this at this week's FPC meeting. We did not reach agreement on how strongly the guidelines should be (e.g. MAY, SHOULD, etc.) for packaging documentation. But there were a few observations that can be implemented to improve this pull request.

• Packaging documentation is beneficial to some degree.
• Those packages are not excluded from the bundling guidelines. At a minimum, this PR should call that out and link to the bundling guidelines.
• A sphinx theme that doesn't bundle anything (as suggested by @churchyard here) would be beneficial. This does not yet exist and would need someone to implement it. Ideally a similar solution could be done with the other documentation generators used in Fedora.
• Another possible solution that would help with getting the bundling correct would be an RPM provides generator that detects bundled javascript in /usr/share/doc/. This also does not yet exist and would need someone to implement it.

While writing this, I noticed something else. Generally when we have a separate page in the packaging guildelines, most of the related content is moved there and the section on the main page is a link to the separate page. We already have a documentation section and a manpages section. Does it make more sense to add this new content to those sections, or move most of the content to the new page with links from the main page?

It is good to have guidelines for different documentation systems together and then linked to the languages they are most often associated with. This would give better consistency in policies. It seems fine to move it into a section (documentation seems most appropriate, if it will not become too long), though maybe roadmap should be developed first?

Most of these which use HTML will require bundled javascript if the aim is to closely match what is provided online by the packages. With an entirely new theme, there is a long term maintenance burden and possibly strong coupling to upstream.

When the documentation generator can produce man pages or similar, packagers should do this when the generated man pages are reasonable (upstream may not test this, especially if they primarily focus on producing HTML documentation). If packagers wish to provide HTML documentation, this must be in a separate sub-package.

Another possible solution that would help with getting the bundling correct would be an RPM provides generator that detects bundled javascript in /usr/share/doc/. This also does not yet exist and would need someone to implement it.

That is at present not technically possible, see https://github.com/rpm-software-management/rpm/issues/1297

It seems @jjames has spent some time determining the licenses of sphinx and doxygen and themes: https://www.jamezone.org/pleasure/software/Fedora/license/#doc

Also, it would be possible to create a bundled Provides generator if we put html documentation in a different location. This could be done with symlinks (from another directory to /usr/share/doc), with a %ghosted copy, a helper macro, or something similar. We could also patch rpm to stop ignoring files in /usr/share/doc, but that might have unwanted side effects for other packages.

I was directed here from a package review. Guess I missed being mentioned in the previous comment. I did mention that page of license information on fedora-devel-list, sometime last summer I think. If it is useful, I am happy to donate it to the Fedora project.

I am a little unclear on how to proceed with labeling the files that are injected by documentation builders. Independent items, such as jquery, are easy. In a package that generates documentation with doxygen, for example, how do I mark that? Adding Provides: bundled(doxygen) is wrong. The doxygen documentation-generating system is most certainly not bundled in the package. It's more like Provides: bundled(data-files-from-doxygen). I am tempted to suggest creating subpackages, doxygen-data, python-sphinx-data, etc. to hold the files that are included in generated documentation; then we could say our doc packages bundle those. But that will inevitably lead to some wise guy deciding to avoid license hassles by not including those files, but symlinking to them instead. And that will work great, right up until the next update of the documentation generator that removes, renames, or incompatibly changes some of those files.

Would it be a bad thing to list bundled Provides for independent items, such as jquery, then say, "The documentation subpackage has files with this list of licenses" and leave it at that, without labeling every single file with its package of origin? Or to say, "If you want to know the origins of these documentation files, look at this web page that lists each file, its origin, and its license?"