#8867 [Tracker] ipa-replica-install fails in ca-range-request call (rawhide): NoClassDefFoundError
Closed: fixed by frenaud. Opened by frenaud.

The nightly tests are failing in rawhide when they install a replica, see PR #936 with for instance dns_locations: logs and report.

The test fails when installing the second replica:

cls = <class 'ipatests.test_integration.test_dns_locations.TestDNSLocations'>
mh = <pytest_multihost.plugin.MultihostFixture object at 0x7f50edc69eb0>
    @classmethod
    def install(cls, mh):
        cls.domain = DNSName(cls.master.domain.name).make_absolute()
        tasks.install_master(cls.master, setup_dns=True)
        tasks.install_replica(cls.master, cls.replicas[0], setup_dns=True,
                              setup_ca=False)
>       tasks.install_replica(cls.master, cls.replicas[1], setup_dns=True,
                              setup_ca=True)
test_integration/test_dns_locations.py:117: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pytest_ipa/integration/tasks.py:513: in install_replica
    result = replica.run_command(args, raiseonerr=raiseonerr,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <ipatests.pytest_ipa.integration.host.Host replica0.ipa.test (replica)>
argv = ['ipa-replica-install', '--admin-password', 'Secret.123', '--password', 'Secret.123', '-U', ...]
set_env = True, stdin_text = None, log_stdout = True, raiseonerr = True
cwd = None, bg = False, encoding = 'utf-8', ok_returncode = 0
    def run_command(self, argv, set_env=True, stdin_text=None,
                    log_stdout=True, raiseonerr=True,
                    cwd=None, bg=False, encoding='utf-8', ok_returncode=0):
        """Wrapper around run_command to log stderr on raiseonerr=True
        :param ok_returncode: return code considered to be correct,
                              you can pass an integer or sequence of integers
        """
        result = super().run_command(
            argv, set_env=set_env, stdin_text=stdin_text,
            log_stdout=log_stdout, raiseonerr=False, cwd=cwd, bg=bg,
            encoding=encoding
        )
        # in FIPS mode SSH may print noise to stderr, remove the string
        # "FIPS mode initialized" + optional newline.
        result.stderr_bytes = FIPS_NOISE_RE.sub(b'', result.stderr_bytes)
        try:
            result_ok = result.returncode in ok_returncode
        except TypeError:
            result_ok = result.returncode == ok_returncode
        if not result_ok and raiseonerr:
            result.log.error('stderr: %s', result.stderr_text)
>           raise subprocess.CalledProcessError(
                result.returncode, argv,
                result.stdout_text, result.stderr_text
            )
E           subprocess.CalledProcessError: Command '['ipa-replica-install', '--admin-password', 'Secret.123', '--password', 'Secret.123', '-U', '--setup-ca', '--setup-dns', '--forwarder', '192.168.122.1', '--ip-address', '192.168.122.40', '--realm', 'IPA.TEST', '--domain', 'ipa.test', '--dirsrv-config-file', '/ipatests/ipatests_dse.ldif']' returned non-zero exit status 1.
pytest_ipa/integration/host.py:202: CalledProcessError
 -----------------------------Captured stderr setup------------------------------ 
ipa: WARNING: ipa: ERROR: DNS zone with name "122.168.192.in-addr.arpa." already exists
ipa: ERROR: stderr: Lookup failed: Preferred host replica0.ipa.test does not provide DNS.
Reverse DNS resolution of address 192.168.122.12 (master.ipa.test) failed. Clients may not function properly. Please check your DNS setup. (Note that this check queries IPA DNS directly and ignores /etc/hosts.)
Replica DNS records could not be added on master: Insufficient access: Insufficient 'add' privilege to add the entry 'idnsname=replica0,idnsname=ipa.test.,cn=dns,dc=ipa,dc=test'.
Custodia uses 'master.ipa.test' as master peer.
Failed to configure CA instance
See the installation logs and the following files/directories for more information:
  /var/log/pki/pki-tomcat
CA configuration failed.
The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

ipa-replica-install log shows the failure happens in pkispawn:

2021-05-29T23:13:48Z DEBUG args=['/usr/sbin/pkispawn', '-s', 'CA', '-f', '/tmp/tmpm5k8uzmb', '--debug']
2021-05-29T23:14:12Z DEBUG Process finished, return code=1
[...]
DEBUG: Command: pki -d /etc/pki/pki-tomcat/alias -f /etc/pki/pki-tomcat/password.conf -U https://master.ipa.test:443 ca-range-request request --install-token /tmp/tmpwrwtph41/install-token --output-format json --debug
INFO: Connecting to https://master.ipa.test:443
java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
    at org.apache.http.impl.client.CloseableHttpClient.<init>(CloseableHttpClient.java:59)
    at org.apache.http.impl.client.AbstractHttpClient.<init>(AbstractHttpClient.java:232)
    at org.apache.http.impl.client.DefaultHttpClient.<init>(DefaultHttpClient.java:148)
    at com.netscape.certsrv.client.PKIConnection.<init>(PKIConnection.java:82)
    at com.netscape.certsrv.client.PKIClient.<init>(PKIClient.java:76)
    at com.netscape.certsrv.client.PKIClient.<init>(PKIClient.java:65)
    at com.netscape.cmstools.cli.MainCLI.getClient(MainCLI.java:588)
    at org.dogtagpki.cli.CLI.getClient(CLI.java:207)
    at com.netscape.cmstools.ca.CACLI.getSubsystemClient(CACLI.java:65)
    at com.netscape.cmstools.range.RangeRequestCLI.execute(RangeRequestCLI.java:77)
    at org.dogtagpki.cli.CommandCLI.execute(CommandCLI.java:57)
    at org.dogtagpki.cli.CLI.execute(CLI.java:357)
    at org.dogtagpki.cli.CLI.execute(CLI.java:357)
    at com.netscape.cmstools.cli.SubsystemCLI.execute(SubsystemCLI.java:76)
    at org.dogtagpki.cli.CLI.execute(CLI.java:357)
    at com.netscape.cmstools.cli.MainCLI.execute(MainCLI.java:660)
    at com.netscape.cmstools.cli.MainCLI.main(MainCLI.java:696)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
    ... 17 more

Installed pki version: pki-server-10.11.0-0.2.alpha1.fc35.noarch


Companion issue https://github.com/dogtagpki/pki/issues/3547 on pki side

This issue is also tracked in Fedora bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1965721

Tested with latest rawhide release of pki: pki-server-11.0.0-0.1.alpha1.fc35.noarch in PR 994, and the tests are green. Issue can be closed.

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata