#2452 Fedora 24: PKI upgrade breaks existing installations of FreeIPA
Closed: Fixed Opened by abbra.

Ticket https://fedorahosted.org/pki/ticket/2373 took care of changing pki-core code to use new location for the jaxrs-api package. However, the problem is that existing deployments will have a jaxrs-api.jar symlink broken after resteasy upgrade. Thus, there is a need to handle upgrade of the configuration in /var/lib/pki/pki-tomcat/common/lib/ to make sure all JARs are correctly found.

This is pretty serious bug as all updated Fedora 24 (and Fedora 25) systems with FreeIPA will break.

A solution is to fix /var/lib/pki/pki-tomcat/common/lib/jaxrs-api.jar symlink to point to /usr/share/java/jboss-jaxrs-2.0-api.jar


This should have been fixed in ticket #2403. Are you using PKI 10.3.5?

There's an upgrade script that fixes the links in existing instances by converting the /var/lib/pki/pki-tomcat/common folder into a link to the /usr/share/pki/server/common folder which contains the correct links to the JAR files.

Could you attach the following file:

  • /var/log/pki/pki-server-upgrade-10.3.5.log

Could you also post the output of this command?

$ ls -la /var/lib/pki/pki-tomcat

Thanks.

Here is the content of pki-server-upgrade-10.3.5.log:

Upgrading PKI server configuration at Sat Sep  3 22:26:25 CEST 2016.
Upgrading from version 10.3.0 to 10.3.1:
1. Enable Tomcat ALLOW_ENCODED_SLASH parameter
2. Add authz realm constraint and default to registry
Upgrading from version 10.3.1 to 10.3.2:
No upgrade scriptlets.
Tracker has been set to version 10.3.2.
Upgrading from version 10.3.2 to 10.3.3:
No upgrade scriptlets.
Tracker has been set to version 10.3.3.
Upgrading from version 10.3.3 to 10.3.4:
1. Fix JAVA_HOME path
Upgrading from version 10.3.4 to 10.3.5:
No upgrade scriptlets.
Tracker has been set to version 10.3.5.
Upgrading from version 10.3.5 to 10.3.5:
Traceback (most recent call last):
  File "/sbin/pki-server-upgrade", line 196, in <module>
    main(sys.argv)
  File "/sbin/pki-server-upgrade", line 189, in main
    upgrader.upgrade()
  File "/usr/lib/python2.7/site-packages/pki/upgrade.py", line 666, in upgrade
    self.upgrade_version(version)
  File "/usr/lib/python2.7/site-packages/pki/upgrade.py", line 604, in upgrade_version
    scriptlets = self.scriptlets(version)
  File "/usr/lib/python2.7/site-packages/pki/upgrade.py", line 551, in scriptlets
    exec(bytecode, variables)  # pylint: disable=W0122
  File "/usr/share/pki/server/upgrade/10.3.5/02-FixSELinuxContexts", line 22, in <module>
    import selinux
  File "/usr/lib64/python2.7/site-packages/selinux/__init__.py", line 28, in <module>
    _selinux = swig_import_helper()
  File "/usr/lib64/python2.7/site-packages/selinux/__init__.py", line 24, in swig_import_helper
    _mod = imp.load_module('_selinux', fp, pathname, description)
ImportError: /usr/lib64/python2.7/site-packages/selinux/_selinux.so: undefined symbol: rpm_execcon

and here is the listing:

# ls -la /var/lib/pki/pki-tomcat/
total 76
drwxrwx---. 8 pkiuser pkiuser 4096 Sep  4 00:18 .
drwxr-xr-x. 3 root    root    4096 Aug 31 01:38 ..
lrwxrwxrwx. 1 pkiuser pkiuser   25 Aug  9  2013 alias -> /etc/pki/pki-tomcat/alias
lrwxrwxrwx. 1 pkiuser pkiuser   21 Aug  9  2013 bin -> /usr/share/tomcat/bin
drwxrwx---. 5 pkiuser pkiuser 4096 Jun 24  2015 ca
lrwxrwxrwx. 1 pkiuser pkiuser   28 Sep  4 00:18 common -> /usr/share/pki/server/common
lrwxrwxrwx. 1 pkiuser pkiuser   19 Aug  9  2013 conf -> /etc/pki/pki-tomcat
drwxrwx---. 3 pkiuser pkiuser 4096 Aug 20  2015 kra
drwxrwx---. 2 pkiuser pkiuser 4096 Sep  4 00:18 lib
lrwxrwxrwx. 1 pkiuser pkiuser   23 Aug  9  2013 logs -> /var/log/pki/pki-tomcat
lrwxrwxrwx. 1 root    root      21 Aug  9  2013 pki-tomcat -> /usr/sbin/tomcat-sysd
drwxrwx---. 2 pkiuser pkiuser 4096 Aug  9  2013 temp
drwxrwx---. 2 pkiuser pkiuser 4096 Jun 24  2015 webapps
drwxrwx---. 3 pkiuser pkiuser 4096 Aug  9  2013 work

I ran pki-server-upgrade now and it didn't produce any errors, but I did fix the symlink myself when the previous failure had happened -- it was part of upgrade from F23 to F24 first and then F24 to F25 alpha.

I suspect that when system dist upgrade happens, not everything is in place until the upgrade is completed. At the very least, pki-server-upgrade needs to catch import errors and gracefully handle them.

# pki-server-upgrade -v
Upgrading from version 10.3.5 to 10.3.5:
1. Fix server library (Yes/No) [Y]: Y
Saving /var/lib/pki/pki-tomcat/ca/conf/CS.cfg
Saving /var/lib/pki/pki-tomcat/kra/conf/CS.cfg
Saving /etc/pki/pki-tomcat/tomcat.conf
2. Fix SELinux contexts (Yes/No) [Y]: Y
Saving /var/lib/pki/pki-tomcat/ca/conf/CS.cfg
Saving /var/lib/pki/pki-tomcat/kra/conf/CS.cfg
Saving /etc/pki/pki-tomcat/tomcat.conf
3. Fix deployment descriptor (Yes/No) [Y]: Y
Saving /var/lib/pki/pki-tomcat/ca/conf/CS.cfg
Saving /var/lib/pki/pki-tomcat/kra/conf/CS.cfg
Saving /etc/pki/pki-tomcat/tomcat.conf
Upgrade complete.

Based on the upgrade log the problem actually happens in FixSELinuxContexts upgrade script which was created for ticket #2421. The script was executed before the RESTEasy upgrade script, and for some reason during the import selinux command failed, so the rest of the upgrade did not execute.

It's unclear why the import selinux command fails during RPM upgrade but not when it's run separately after upgrade. For now the FixSELinuxContexts has been removed temporarily in the following commits:

  • 76b3ae5062aef22eece89117a28bd9b86ddef92d
  • 238d14bb8790037c8d1ca6d9123362ba3bb9fbf1

Please retest with PKI 10.3.5-5 or later when it becomes available on F24. This build removes the failing upgrade script as mentioned above. Note that we may restore the upgrade script in a later build once we figure out the cause of problem.

I was actually unable to reproduce the problem on my system. What versions of the following packages were installed on your system before and after the upgrade?

  • libselinux-*
  • policycoreutils-*

I assume it has happened on either f23 -> f24 upgrade because it was following:

    Upgraded   pki-base-10.2.6-20.fc23.noarch                                      @updates-testing/23
    Upgrade             10.3.5-4.fc24.noarch                                       @updates-testing/24

The next one (f24 -> f25) was merely 10.3.5-4.f24 -> 10.3.5-4.f25 upgrade. So, f23 -> f24 upgrade for selinux packages was

    Upgraded   libselinux-2.4-4.fc23.x86_64                                        @@commandline/23
    Upgrade               2.5-9.fc24.x86_64                                        @updates/24
...
    Upgraded   policycoreutils-2.4-21.fc23.x86_64                                  @updates/23
    Upgrade                    2.5-13.fc24.x86_64                                  @updates/24

https://bodhi.fedoraproject.org/updates/FEDORA-2016-994f943797

Metadata Update from @abbra:
- Issue set to the milestone: 10.3.6

Dogtag PKI is moving from Pagure issues to GitHub issues. This means that existing or new
issues will be reported and tracked through Dogtag PKI's GitHub Issue tracker.

This issue has been cloned to GitHub and is available here:
https://github.com/dogtagpki/pki/issues/2572

If you want to receive further updates on the issue, please navigate to the
GitHub issue and click on Subscribe button.

Thank you for understanding, and we apologize for any inconvenience.

Metadata