From cd171814d9684699ff44b90b67450752a5b9f537 Mon Sep 17 00:00:00 2001 From: Miro Hrončok Date: Aug 15 2025 12:13:36 +0000 Subject: [PATCH 1/2] No longer allow the 201x Python guidelines See https://fedoraproject.org/wiki/Changes/DeprecateSetuppyMacros --- diff --git a/guidelines/modules/ROOT/pages/Python.adoc b/guidelines/modules/ROOT/pages/Python.adoc index fe651a7..1fc5936 100644 --- a/guidelines/modules/ROOT/pages/Python.adoc +++ b/guidelines/modules/ROOT/pages/Python.adoc @@ -33,18 +33,16 @@ This version of Python Packaging Guidelines is in effect since 2021 and represents a major rewrite and paradigm shift. Not all packages are updated to reflect this. -Older guidelines are still being kept up to date, -and existing packages *MAY* use them instead of this document: +Older guidelines are kept as a historical reference, +but new packages *MUST NOT* use them. +Existing packages *MUST* migrate to the current Python guidelines: * xref:Python_201x.adoc[“201x-era” Python packaging guidelines] -(Packages using these usually use the +(Packages using these usually use the deprecated <> or <> or call `+setup.py install+`.) -* xref:Python_Appendix.adoc#_python_2_packages[Python 2 appendix] -Note that Python 2 packages require a FESCo exception. - //// NOTE: A summary of changes from the older guidelines is available at https://hackmd.io/@python-maint/rJmQQc4DP diff --git a/guidelines/modules/ROOT/pages/Python_201x.adoc b/guidelines/modules/ROOT/pages/Python_201x.adoc index 65b9e41..b694172 100644 --- a/guidelines/modules/ROOT/pages/Python_201x.adoc +++ b/guidelines/modules/ROOT/pages/Python_201x.adoc @@ -3,9 +3,9 @@ [IMPORTANT] ==== -These guidelines are being replaced by xref:Python.adoc[a newer version]. +These guidelines were replaced by xref:Python.adoc[a newer version]. -You are free to use either version of the Python Packaging Guidelines. +They exist as a historical reference for packages that don't follow the new guidelines yet. ==== == Python Version Support From a93c54cf4a9bb4b2f50673e6832ab542439c5ecd Mon Sep 17 00:00:00 2001 From: Miro Hrončok Date: Aug 15 2025 12:13:36 +0000 Subject: [PATCH 2/2] Remove outdated Python 2 section from Python Appendix --- diff --git a/guidelines/modules/ROOT/pages/Python_Appendix.adoc b/guidelines/modules/ROOT/pages/Python_Appendix.adoc index ac1b8e3..01d4631 100644 --- a/guidelines/modules/ROOT/pages/Python_Appendix.adoc +++ b/guidelines/modules/ROOT/pages/Python_Appendix.adoc @@ -3,345 +3,6 @@ Here are some additional Python-related guidelines, moved here in order to keep the main page manageable. -== Python 2 packages - -If your package needs to build for Python 2 only, -follow the xref:Python_201x.adoc[201x-era Python guidelines], -except substitute 3 for 2. -For example instead of `+%{python3_sitearch}+`, -use `+%{python2_sitearch}+`. - -Special emphasis on the rule that packages MUST NOT use `+/usr/bin/python+`. - -=== Naming - -The source package for a Python library MUST be named with the python- prefix except for a specific case bellow. A built package must include the Python major version in the name, using the python2- prefix. This is accomplished by adding a subpackage. - -If you need to package the python2 module as a separate source package (usually because upstream has dropped Python 2 support in newer versions but still supports the last version that works with Python 2), you MUST name the source package using the python2- prefix instead of creating a subpackage. - -=== Common spec files for python3 and python2 - -It is possible to create both python3 and python2 subpackage from one common spec file. -The following is a very simple spec file for a module building for both python2 -and python3. It builds both versions in the same directory; this is possible -because the build products for different versions of Python usually do not conflict. - -There are cases where it is not possible to build in a single directory. Most -commonly this happens when the sources are modified during the build process to -convert them from python2 to python3 using the the `+2to3+` tool (see bellow). - -As you can see in the `+%install+` section below, the order in which -you do the python2 versus python3 install can sometimes matter. You need to be -aware of when the install is writing to the same file in both packages (in this -example, a script in `+%{_bindir}+`) and make sure that you're getting -the version you expect. - -.... -%global srcname example - -Name: python-%{srcname} -Version: 1.2.3 -Release: 1%{?dist} -Summary: An example python module - -License: MIT -URL: https://pypi.python.org/pypi/%{srcname} -Source: %pypi_source - -BuildArch: noarch - -%global _description %{expand: -A python module which provides a convenient example.} - -%description %_description - -%package -n python2-%{srcname} -Summary: %{summary} -BuildRequires: python2-devel - -%description -n python2-%{srcname} %_description - - -%package -n python3-%{srcname} -Summary: %{summary} -BuildRequires: python3-devel - -%description -n python3-%{srcname} %_description - - -%prep -%autosetup -n %{srcname}-%{version} - -%build -%py2_build -%py3_build - -%install -# Must do the python2 install first because the scripts in /usr/bin are -# overwritten with every setup.py install, and in general we want the -# python3 version to be the default. -%py2_install -%py3_install - -%check -%{python2} setup.py test -%{python3} setup.py test - -# Note that there is no %%files section for the unversioned python module if we are building for several python runtimes -%files -n python2-%{srcname} -%license COPYING -%doc README.rst -%{python2_sitelib}/%{srcname}/ -%{python2_sitelib}/%{srcname}-*.egg-info/ - -%files -n python3-%{srcname} -%license COPYING -%doc README.rst -%{python3_sitelib}/%{srcname}/ -%{python3_sitelib}/%{srcname}-*.egg-info/ -%{_bindir}/sample-exec - -%changelog -.... - -=== Using separate build directories - -Sometimes is it impossible to build both versions from the same source directory. Most often this happens when sources are "translated" to python3 in the source directory and made incompatible with python2 in the process. This used to be fairly common, but is fortunately much rarer now. Some things to look for are: - -* Sources are not Python 3 compatible (`+print+` without parentheses is used, old module names like `+ConfigParser+` are imported), -* `+six+` module is not used, -* `+2to3+` is run in `+setup.py+` without creating a separate build directory. - -Our method in building from the same code to make the two separate modules is to keep each build as independent as possible. To do that, we copy the source tree to python3 so that the python 2 sources are entirely independent from the python 3 sources. - -Some things to watch out for: - -* Make sure that you are copying the correct code. The example is copying the code from within the top directory of the untarred source. If the `+%prep+` has changed directory you will need to change back to the tarball location. -* Patching the source code is done before copying to `+python3+`. Since you have both a python2 and a python3 directory you might be tempted to patch each one separately. *Resist!* Upstream for your package has chosen to distribute a single source tree that builds for both python2 and python3. For your patches to https://docs.fedoraproject.org/en-US/package-maintainers/Staying_Close_to_Upstream_Projects/[get into upstream], you need to write patches that work with both as well. - -`+rpmbuild+` resets the directory at the end of each phase, so you don't need to restore the directory at the end of `+%prep+`. - -.... -%prep -%autosetup -N -c -mv %{srcname}-%{version} python2 -pushd python2 -%autopatch -p1 - -find -name '*.txt' | xargs chmod -x -# copy common doc files to top dir -cp -pr docs psfl.txt zpl.txt ../ -popd - -cp -a python2 python3 -find python3 -name '*.py' | xargs sed -i '1s|^#!.*|#!%{python3}|' -find python2 -name '*.py' | xargs sed -i '1s|^#!.*|#!%{python2}|' - -%build -pushd python2 -%py2_build -popd - -pushd python3 -%py3_build -popd - -%install -# Must do the python2 install first because the scripts in /usr/bin are -# overwritten with every setup.py install, and in general we want the -# python3 version to be the default. -pushd python2 -%py2_install -popd - -pushd python3 -%py3_install -popd - -%check -pushd python2 -%{python2} setup.py test -popd - -pushd python3 -%{python3} setup.py test -popd -.... - -You'll notice that the `+%build+`, `+%install+`, and `+%check+` sections again follow a pattern similar to the previous example. They switch to the python2 directory and do the normal steps for building the python2 module, and then switch to the python3 directory and run the same steps for python3. The usage of `+pushd/popd+` commands will ensure that the directories are logged. - -=== Conditionalizing the Python 2 parts - -If the maintainer wishes to support a similar specfile across several Fedora releases, -sometimes the Python 2 packages are only appropriate for the older ones. -Conditionalizing the Python 2 parts of the specfile (if applicable) -SHOULD be done with the following bcond: - -.... -%bcond_with python2 # to disable python2 by default, or: -%bcond_without python2 # to enable python2 by default - -%if %{with python2} -... -%endif -.... - -See the following example for reference: - -.... -%global srcname example - -# Disable python2 by default -%bcond_with python2 - -Name: python-%{srcname} -Version: 1.2.3 -Release: 1%{?dist} -Summary: An example python module - -License: MIT -URL: https://pypi.python.org/pypi/%{srcname} -Source: %pypi_source - -BuildArch: noarch - -%global _description %{expand: -A python module which provides a convenient example.} - -%description %_description - -%if %{with python2} -%package -n python2-%{srcname} -Summary: %{summary} -BuildRequires: python2-devel - -%description -n python2-%{srcname} %_description -%endif - - -%package -n python3-%{srcname} -Summary: %{summary} -BuildRequires: python3-devel - -%description -n python3-%{srcname} %_description - - -%prep -%autosetup -n %{srcname}-%{version} - -%build -%if %{with python2} -%py2_build -%endif -%py3_build - -%install -# Must do the python2 install first because the scripts in /usr/bin are -# overwritten with every setup.py install, and in general we want the -# python3 version to be the default. -%if %{with python2} -%py2_install -%endif -%py3_install - -%check -%if %{with python2} -%{python2} setup.py test -%endif -%{python3} setup.py test - -%if %{with python2} -%files -n python2-%{srcname} -%license COPYING -%doc README.rst -%{python2_sitelib}/%{srcname}/ -%{python2_sitelib}/%{srcname}-*.egg-info/ -%endif - -%files -n python3-%{srcname} -%license COPYING -%doc README.rst -%{python3_sitelib}/%{srcname}/ -%{python3_sitelib}/%{srcname}-*.egg-info/ -%{_bindir}/sample-exec - -%changelog -.... - -=== Avoiding collisions between the python 2 and python 3 stacks - -The python 2 and python 3 stacks are intended to be fully-installable in parallel. When generalizing the package for both python 2 and python 3, it is important to ensure that two different built packages do not attempt to place different payloads into the same path. - -==== Executables in `+/usr/bin+` - -Many existing python packages install executables into `+/usr/bin+`. - -For example if we have a `+console_scripts+` in a `+setup.py+` shared between -python 2 and python 3 builds: these will spit out files in `+/usr/bin/+`, -and these will collide. - -For example `+python-coverage+` has a `+setup.py+` that contains: - -.... - entry_points = { - 'console_scripts': [ - 'coverage = coverage:main', - ] - }, -.... - -which thus generates a `+/usr/bin/coverage+` executable (this is a python -script that runs another python script whilst generating code-coverage -information on the latter). - -Similarly for the 'scripts' clause; see e.g. `+python-pygments+`: -`+Pygments-1.1.1/setup.py+` has: - -.... - scripts = ['pygmentize'], -.... - -which generates a `+/usr/bin/pygmentize+` (this is a python script that leverages the pygments syntax-highlighting module, giving a simple command-line interface for generating syntax-highlighted files) - -If the executables provide the same functionality independent of whether they are run on top of Python 2 or Python 3, then only the Python 3 version of the executable SHOULD be packaged, the python2 version MUST NOT be packaged. - -Examples of this: - -* `+/usr/bin/pygmentize+` ought to generate the same output regardless of whether it's implemented via Python 2 or Python 3, so only one version needs to be shipped. - -If the executables provide different functionality for Python 2 and Python 3, then Python 2 version MAY be packaged. - -Examples of this: - -* `+/usr/bin/coverage+` runs a python script, augmenting the interpreter with code-coverage information. Given that the interpreter itself is the thing being worked with, it's reasonable to package both versions of the executable. -* `+/usr/bin/bpython+` augments the interpreter with a "curses" interface. Again, it's reasonable to package both versions of this. -* `+/usr/bin/easy_install+` installs a module into one of the Python runtimes: we need a version for each runtime. - -==== Naming - -Many executables already contain a "-MAJOR.MINOR" suffix, for example `+/usr/bin/easy_install-3.4+`. These obviously can be used as-is, as they won't conflict. - -For other executables, the general rule is: - -* If only one executable is to be shipped, then it owns its own slot and should use /usr/bin/python3. -* If executables are to be shipped for both python 2 and python 3: -** Both python 2 and python 3 variants MUST provide symlinks with a '-X' and '-X.Y' suffix (python runtime major version, or python runtime major.minor version), unless upstream already provides appropriately versioned executables without the dash. -** The unversioned executable MUST be the python3 version. -** For example, the python3 version of "twisted" MUST ship executables `+/usr/bin/twistd-3+` and `+/usr/bin/twistd-3.7+` (assuming python3 is currently version 3.7) and `+/usr/bin/twistd+`; while the python2 version MUST provide `+/usr/bin/twistd-2+` and `+/usr/bin/twistd-2.7+`. -** For compatibility packages, the Python version is appended *after* the specific package version, for example `+/usr/bin/coverage-v1.2-3+` and `+/usr/bin/coverage-v1.2-3.4+` for python3-coverage1.2 compat package. - -=== Running 2to3 from the spec file - -Sometimes, upstream hasn't integrated running 2to3 on the code into their build scripts but they support making a python3 module from it if you manually run 2to3 on the source. This is the case when it's documented on the upstream's website, in a file in the tarball, or even when email with the module's author has instructions for building a python3 module from the python2 source and the authors are willing to support the result. In these cases it's usually just a matter of the upstream not having written the build script that can turn the python2 source into python3. When this happens you can run `+2to3+` from the spec file. Once you have it working, you can also help upstream integrate it into their build scripts which will benefit everyone in the long term. - -You should usually follow upstream's directions on how to run `+2to3+` and build the python3 module in these cases but there's a few things you should check to make sure upstream is doing it correctly. - -* Since the code is being built from a unified source, you need to copy the code to a new directory before invoking 2to3 just like the link:#Building_more_than_once[ building more than once] method. -* If the `+2to3+` program is invoked instead of using the `+lib2to3+` library functions, make sure it's invoked with `+--write --nobackups+`. `+--write+` is needed to make `+2to3+` actually change the files. `+--nobackups+` avoids leaving `+foo.py.bak+` files in the module directories that then make it into the final package payload. -* Be sure to run 2to3 on the correct directory. When you run `+2to3+` you need to run it on the whole tree. A common mistake here for distutils packages has been to run it on the directory below `+setup.py+`, missing the `+setup.py+` file itself. This leads to errors when `+python3+` tries to execute `+setup.py+` -* If you need to run `+2to3+` to fix code, use `+2to3+` or `+/usr/bin/2to3+`. At the moment, this program is coming from the `+python-tools+` rpm. Using `+2to3+` means that you'll be using a name that is supported upstream and across distros rather than `+/usr/bin/python3-2to3+` which we have renamed in Fedora to avoid filesystem conflicts. This also makes it easier for us to test and eventually change from using the python2 `+2to3+` to the python3 `+2to3+`. We just need to change the python3 package to provide the `+/usr/bin/2to3+` program instead of python and all of our python packages will start using that version instead. -* If `+2to3+` runs into a problem, please https://bugzilla.redhat.com/enter_bug.cgi?component=python&product=Fedora[file a Fedora bug]. Please try to isolate a minimal test case that reproduces the problem when doing so. - [#manual-bytecompilation] == Manual byte compilation