https://pagure.io/389-ds-base/pull-request/50085

Using scripts for cli tools sadly doesn't work, as it supports only bin directory, but we're installing under sbin

Commit c4a2eb4a fixes this issue

Hi, this causes a regression:

{william@leap 23:00} ~/development/389ds/ds I0> find /usr | grep -i dscreate
/usr/lib/python3.6/site-packages/lib389-1.4.0.1-py3.6.egg/sbin/dscreate
/usr/lib/python3.6/site-packages/lib389-1.4.0.1-py3.6.egg/share/man/man8/dscreate.8

Doing a make && make install, the sbin tools install into the egg, not into the sbin paths. It's likely this may also break rpm packaging too. So I think this needs to be done differently to avoid this problem ... :(

IMO, ds* tools should not be installed from lib389 in the first place. They should be part of 389-ds-base package, not python3-lib389.

An admin may want to install the admin tools on a machine (IE their workstation, or an admin tool container) without installing all of the 389-ds package. So being in python3-lib389 is very reasonable, and intentional. If they were not in python3-lib389 they would be in a 389-ds-cli which deps on python3-lib389.

Either way, that comment does not address the fact that make && make install is now unable to access the cli after this change, and it will probably break our rpms. So either we need to revert this setup.py change, or the cli tools have to be installed as part of autotools instead of setup.py which I don't really want to do given that's not "pythonic".

https://pagure.io/389-ds-base/pull-request/50444

An admin may want to install the admin tools on a machine (IE their workstation, or an admin tool container) without installing all of the 389-ds package. So being in python3-lib389 is very reasonable, and intentional. If they were not in python3-lib389 they would be in a 389-ds-cli which deps on python3-lib389.

lib389 is a library (it's in its name!). Shipping administration tools as part of library doesn't make sense to me. I think the split to 389-ds-cli is a very reasonable option.

Either way, that comment does not address the fact that make && make install is now unable to access the cli after this change, and it will probably break our rpms.

Key word is "probably". It didn't break Fedora rpms:

[root@server-f30 ds]# rpm -q 389-ds-base 
389-ds-base-1.4.1.3-1.fc30.x86_64
[root@server-f30 ds]# rpm -ql python3-lib389 | grep dsc
/usr/sbin/dsconf
/usr/sbin/dscontainer
/usr/sbin/dscreate
/usr/sbin/dsctl
/usr/share/man/man8/dsconf.8.gz
/usr/share/man/man8/dscreate.8.gz
/usr/share/man/man8/dsctl.8.gz

So either we need to revert this setup.py change, or the cli tools have to be installed as part of autotools instead of setup.py which I don't really want to do given that's not "pythonic".

They can be installed as part of spec file (this is how it's done currently):
https://src.fedoraproject.org/rpms/389-ds-base/blob/master/f/389-ds-base.spec#_750

Again, IMO, the split is the most sensible option: have another python package with the cli tools that will use python3-lib389 as a dependency. Having lib389 installed in venv is a step in the right direction, since the administrator can install latest and the greatest ds* cli tools and lib389 in venv without root access to the system (as it was before).

Okay, so I can do the suse work to make it 389-ds-cli on my side - but here the thing is prefix builds still break with this change .... So those are used by myself and some others so we need to do something, not just virtualenv (because I don't use them).

Before we start doing changes, let's step back for a moment and discuss what exactly are the requirements. Perhaps we can come up with a solution that works for everyone.

I see the following, please correct me if I miss something:

• lib389 should be installable in venv without root. This is useful for testing, having lib389 installed separately from the system one, ability to install from pypi, etc.
• lib389 should be installable in prefix build and have a working ds* cli tools.
• ds* cli tools should be installable without the main server as they can be used by the administrator from their workstation.
  • note: dscreate and dsctl are useless without the server on the same host, dsconf and dsidm can be used from another host.

lib389 should be installable in venv without root. This is useful for testing, having lib389 installed separately from the system one, ability to install from pypi, etc.
lib389 should be installable in prefix build and have a working ds* cli tools.

So these two are the concern of this ticket. By adding venv support, prefix installs are broken. IMO prefix builds matter more as it's how development occurs. So my suggestion:

• We add the revert patch to allow developers to work again.
• To support venv, we need to look at how setup.py in the rpm works, and extract that "magic" and make it part of the lib389-install/lib389 targets in Makefile.am
• Then we can re-do the change from /usr/sbin/ -> sbin/ in setup.py to support venvs and prefix.

ds* cli tools should be installable without the main server as they can be used by the administrator from their workstation.
note: dscreate and dsctl are useless without the server on the same host, dsconf and dsidm can be used from another host.

Yep, I'm aware. dscontainer also is useless unless it's on the host. Saying this, this is really a packaging problem for downstream, and I'm confused why it was brought up in this ticket when the problem here is about broken prefix builds and potential to break downstream, rather than saying the package names for downstream seem incorrect .....

So I think this is up to me for SUSE to do, and @mreynolds for RH/Fedora to do if we consider it worthwhile for those distros/platforms.

Lets focus on the prefix build issue.

Here are the suse build macros, I'll try and see if these work for this.

{william@leap 17:50} ~/development/389ds I0> rpm --eval '%python_build'
python_flavor=`[ -f _current_flavor ] && cat _current_flavor || true`
if [ -z "$python_flavor" ]; then python_flavor="tmp"; fi
if [ "$python_flavor" != "python2" ]; then
    if [ -d build ]; then mv build _build.$python_flavor; fi
    if [ -d _build.python2 ]; then mv _build.python2 build; fi
fi
echo python2 > _current_flavor
/usr/bin/python2 setup.py  build \
    --executable="/usr/bin/python2 -s"
python_flavor=`[ -f _current_flavor ] && cat _current_flavor || true`
if [ -z "$python_flavor" ]; then python_flavor="tmp"; fi
if [ "$python_flavor" != "python3" ]; then
    if [ -d build ]; then mv build _build.$python_flavor; fi
    if [ -d _build.python3 ]; then mv _build.python3 build; fi
fi
echo python3 > _current_flavor
/usr/bin/python3 setup.py  build \
    --executable="/usr/bin/python3 -s"
{william@leap 17:50} ~/development/389ds I0> rpm --eval '%python_install'
python_flavor=`[ -f _current_flavor ] && cat _current_flavor || true`
if [ -z "$python_flavor" ]; then python_flavor="tmp"; fi
if [ "$python_flavor" != "python2" ]; then
    if [ -d build ]; then mv build _build.$python_flavor; fi
    if [ -d _build.python2 ]; then mv _build.python2 build; fi
fi
echo python2 > _current_flavor
/usr/bin/python2 setup.py  install \
    -O1 --skip-build --force --root /home/william/rpmbuild/BUILDROOT/%{NAME}-%{VERSION}-%{RELEASE}.x86_64 --prefix /usr
python_flavor=`[ -f _current_flavor ] && cat _current_flavor || true`
if [ -z "$python_flavor" ]; then python_flavor="tmp"; fi
if [ "$python_flavor" != "python3" ]; then
    if [ -d build ]; then mv build _build.$python_flavor; fi
    if [ -d _build.python3 ]; then mv _build.python3 build; fi
fi
echo python3 > _current_flavor
/usr/bin/python3 setup.py  install \
    -O1 --skip-build --force --root /home/william/rpmbuild/BUILDROOT/%{NAME}-%{VERSION}-%{RELEASE}.x86_64 --prefix /usr

And here are the fedora ones

sh-4.4# cat /usr/lib/rpm/macros.d/macros.python3
%__python3 /usr/bin/python3
%python3_sitelib %(%{__python3} -Ic "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
%python3_sitearch %(%{__python3} -Ic "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")
%python3_version %(%{__python3} -Ic "import sys; sys.stdout.write(sys.version[:3])")
%python3_version_nodots %(%{__python3} -Ic "import sys; sys.stdout.write(sys.version[:3].replace('.',''))")
%python3_platform %(%{__python3} -Ic "import sysconfig; print(sysconfig.get_platform())")
%py3dir %{_builddir}/python3-%{name}-%{version}-%{release}
%py3_shbang_opts -s
# Use the slashes after expand so that the command starts on the same line as
# the macro
%py3_build() %{expand:\\\
  CFLAGS="${CFLAGS:-${RPM_OPT_FLAGS}}" LDFLAGS="${LDFLAGS:-${RPM_LD_FLAGS}}"\\\
  %{__python3} %{py_setup} %{?py_setup_args} build --executable="%{__python3} %{py3_shbang_opts}" %{?*}
  sleep 1
}
%py3_build_egg() %{expand:\\\
  CFLAGS="${CFLAGS:-${RPM_OPT_FLAGS}}" LDFLAGS="${LDFLAGS:-${RPM_LD_FLAGS}}"\\\
  %{__python3} %{py_setup} %{?py_setup_args} bdist_egg %{?*}
  sleep 1
}
%py3_build_wheel() %{expand:\\\
  CFLAGS="${CFLAGS:-${RPM_OPT_FLAGS}}" LDFLAGS="${LDFLAGS:-${RPM_LD_FLAGS}}"\\\
  %{__python3} %{py_setup} %{?py_setup_args} bdist_wheel %{?*}
  sleep 1
}
%py3_install() %{expand:\\\
  CFLAGS="${CFLAGS:-${RPM_OPT_FLAGS}}" LDFLAGS="${LDFLAGS:-${RPM_LD_FLAGS}}"\\\
  %{__python3} %{py_setup} %{?py_setup_args} install -O1 --skip-build --root %{buildroot} %{?*}
}
%py3_install_egg() %{expand:\\\
  mkdir -p %{buildroot}%{python3_sitelib}
  easy_install-%{python3_version} -m --prefix %{buildroot}%{_prefix} -Z dist/*-py%{python3_version}.egg %{?*}
}
%py3_install_wheel() %{expand:\\\
  pip%{python3_version} install -I dist/%{1} --root %{buildroot} --strip-file-prefix %{buildroot} --no-deps
}

After some testing I can not get ds* to install in /usr/sbin with --root or --prefix or --install-data, so I stand by my comment that we should revert this, and if venvs are needed, they should be done in a way that doesn't break prefix builds.

Obligatory xkcd comic: https://xkcd.com/1172/
Let's revert this change to unblock you. Once I have a proper solution, I'll CC you in PR to review it properly (you did review the last time, but apparently it was not a concern back then).

Well, I didn't realise it would cause this issue .... :( :( there are plenty of times we commit code and things go wrong, and we fix it later. It's the nature of the business I think ...

So I look forward to the next update where we manage to have both working!

Commit 09ba2514 relates to this ticket

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/3096

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.