#9383 Random nightly test failure in test_acme.py::TestACMEPrune::test_prune_cert_manual
Closed: fixed by frenaud. Opened by frenaud.

Issue

The test test_acme.py::TestACMEPrune::test_prune_cert_manual is unstable and randomly fails. See for instance PR #2689 with the following logs and report:

self = <ipatests.test_integration.test_acme.TestACMEPrune object at 0x7fa8f3491250>
issue_and_expire_acme_cert = <function issue_and_expire_acme_cert.<locals>._issue_and_expire_acme_cert at 0x7fa8f3455c60>
    def test_prune_cert_manual(self, issue_and_expire_acme_cert):
        """Test to prune expired certificate by manual run"""
        if (tasks.get_pki_version(self.master)
           < tasks.parse_version('11.3.0')):
            raise pytest.skip("Certificate pruning is not available")
        issue_and_expire_acme_cert(
            self.master, self.clients[0], self.acme_server)
        # check that the certificate issued for the client
>       result = self.master.run_command(
            ['ipa', 'cert-find', '--subject', self.clients[0].hostname]
        )
test_integration/test_acme.py:825: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <ipatests.pytest_ipa.integration.host.Host master.ipa.test (master)>
argv = ['ipa', 'cert-find', '--subject', 'client0.ipa.test'], set_env = True
stdin_text = None, log_stdout = True, raiseonerr = True, cwd = None, bg = False
encoding = 'utf-8', ok_returncode = 0
    def run_command(self, argv, set_env=True, stdin_text=None,
                    log_stdout=True, raiseonerr=True,
                    cwd=None, bg=False, encoding='utf-8', ok_returncode=0):
        """Wrapper around run_command to log stderr on raiseonerr=True
        :param ok_returncode: return code considered to be correct,
                              you can pass an integer or sequence of integers
        """
        result = super().run_command(
            argv, set_env=set_env, stdin_text=stdin_text,
            log_stdout=log_stdout, raiseonerr=False, cwd=cwd, bg=bg,
            encoding=encoding
        )
        # in FIPS mode SSH may print noise to stderr, remove the string
        # "FIPS mode initialized" + optional newline.
        result.stderr_bytes = FIPS_NOISE_RE.sub(b'', result.stderr_bytes)
        try:
            result_ok = result.returncode in ok_returncode
        except TypeError:
            result_ok = result.returncode == ok_returncode
        if not result_ok and raiseonerr:
            result.log.error('stderr: %s', result.stderr_text)
>           raise subprocess.CalledProcessError(
                result.returncode, argv,
                result.stdout_text, result.stderr_text
            )
E           subprocess.CalledProcessError: Command '['ipa', 'cert-find', '--subject', 'client0.ipa.test']' returned non-zero exit status 1.
pytest_ipa/integration/host.py:202: CalledProcessError
 ------------------------------Captured stderr call------------------------------ 
ipa: ERROR: stderr: ipa: ERROR: did not receive Kerberos credentials

Test scenario:
- prepare for ACME (enable ACME, register with certbot on IPA client)
- request a cert using ACME
- move date in the future on master and client, so that the ACME cert is expired
- reset the admin password (because of the new date, the admin password is expired)
- call ipa cert-find on the master to check the content of the cert
- launch the pruning

The test often fails in the call to ipa cert-find because the kerberos ticket is expired. The admin password is reset on the client but not on the master, meaning that the cache on the master contains an expired ticket.

The test should reset the admin password on the master if it needs to perform ipa calls on the master later on.


Metadata Update from @frenaud:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/6848

master:

  • 359e1a3d95a90d720db9872ce78c5c4401b40798 ACME tests: fix issue_and_expire_acme_cert method

ipa-4-10:

  • a6f485fcade619980f6538edadf115dca69e1314 ACME tests: fix issue_and_expire_acme_cert method

ipa-4-9:

  • 7a94acca6a9efb546f1cf59f63fcb89f98944ea5 ACME tests: fix issue_and_expire_acme_cert method

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata