#9849 Random test failure in test_otp
Closed: fixed by frenaud. Opened by frenaud.

Issue

The test test_integration/test_otp.py is randomly failing in
- test_2fa_only_with_password
- test_2fa_with_otp_password

This happens when the test is launched with a IPATEST_YAML_CONFIG file that specifies different names for the hostname and the external_hostname, for instance:

ad_admin_name: Administrator
ad_admin_password: Secret123
admin_id: admin
admin_name: admin
admin_password: Secret.123
admin_pw: Secret123
dirman_dn: cn=Directory Manager
dirman_id: '"cn=Directory Manager"'
dirman_password: Secret.123
dirman_pw: Secret123
dns_forwarder: 10.11.5.19
domains:
- hosts:
  - external_hostname: ip-10-0-197-20.xxx.redhat.com
    ip: 10.0.197.20
    name: runner.testrelm.test
    role: test
  - external_hostname: ip-10-0-197-227.xxx.redhat.com
    ip: 10.0.197.227
    name: master.testrelm.test
    role: master
  name: testrelm.test
  type: IPA
ssh_key_filename: ~/.ssh/id_rsa

The tests executed downstream use such a configuration and randomly fail.

Example of run in https://$ARTIFACTS_SERVER/idm-ci/freeipa/Nightly-Tier1/RHEL9.7/2025-09-01_16-00/tier-1/upstream-otp/1/report.html

self = <ipatests.test_integration.test_otp.TestOTPToken object at 0x7f3033318b20>
    def test_2fa_only_with_password(self):
        """Test ssh with 2FA only with the password(first factor) when
        user-auth-type is opt and password.
        Test for :  https://github.com/SSSD/sssd/pull/7500
        Add the IPA user and user-auth-type set to opt and password.
        Authenticate the user only with password, just press enter
        at `Second factor`
        """
        master = self.master
        USER3 = 'sshuser3'
        sssd_conf_backup = tasks.FileBackup(master, paths.SSSD_CONF)
        first_prompt = 'Enter first factor:'
        second_prompt = 'Enter second factor:'
        add_contents = textwrap.dedent('''
            [prompting/2fa/sshd]
            single_prompt = False
            first_prompt = {0}
            second_prompt = {1}
            ''').format(first_prompt, second_prompt)
        set_sssd_conf(master, add_contents)
        tasks.create_active_user(master, USER3, PASSWORD)
        tasks.kinit_admin(master)
        master.run_command(['ipa', 'user-mod', USER3, '--user-auth-type=otp',
                            '--user-auth-type=password'])
        try:
            otpuid, totp = add_otptoken(master, USER3, otptype='totp')
            master.run_command(['ipa', 'otptoken-show', otpuid])
            totp.generate(int(time.time())).decode('ascii')
            otpvalue = "\n"
            tasks.clear_sssd_cache(self.master)
            github_ticket = "https://github.com/SSSD/sssd/pull/7500"
            sssd_version = tasks.get_sssd_version(master)
            rhel_fail = (
                osinfo.id == 'rhel'
                and sssd_version < tasks.parse_version("2.9.5")
            )
            fedora_fail = (
                osinfo.id == 'fedora'
                and sssd_version == tasks.parse_version("2.9.5")
            )
            with xfail_context(rhel_fail or fedora_fail, reason=github_ticket):
                result = ssh_2fa_with_cmd(master,
                                          self.master.external_hostname,
                                          USER3, PASSWORD, otpvalue=otpvalue,
                                          command="klist")
                print(result.stdout_text)
>               assert ('Authentication successful') in result.stdout_text
E               AssertionError: assert 'Authentication successful' in 'spawn ssh -o NumberOfPasswordPrompts=1 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -l sshuser3 ip-10-... UNKNOWN port 65535\r\r\n\nexpect result: Unexpected end of file\nexpect exit code: 202\nexpect spawn exit code: 255\n'
E                +  where 'spawn ssh -o NumberOfPasswordPrompts=1 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -l sshuser3 ip-10-... UNKNOWN port 65535\r\r\n\nexpect result: Unexpected end of file\nexpect exit code: 202\nexpect spawn exit code: 255\n' = <pytest_multihost.transport.SSHCommand object at 0x7f30344503a0>.stdout_text
USER3      = 'sshuser3'
add_contents = '\n[prompting/2fa/sshd]\nsingle_prompt = False\nfirst_prompt = Enter first factor:\nsecond_prompt = Enter second factor:\n'
fedora_fail = False
first_prompt = 'Enter first factor:'
github_ticket = 'https://github.com/SSSD/sssd/pull/7500'
master     = <ipatests.pytest_ipa.integration.host.Host master.testrelm.test (master)>
otpuid     = '487c8d49-17ff-4df1-a14f-0adb9c8f4cb2'
otpvalue   = '\n'
result     = <pytest_multihost.transport.SSHCommand object at 0x7f30344503a0>
rhel_fail  = False
second_prompt = 'Enter second factor:'
self       = <ipatests.test_integration.test_otp.TestOTPToken object at 0x7f3033318b20>
sssd_conf_backup = <ipatests.pytest_ipa.integration.tasks.FileBackup object at 0x7f30333184c0>
sssd_version = <Version('2.9.7')>
totp       = <cryptography.hazmat.primitives.twofactor.totp.TOTP object at 0x7f303444f7c0>
test_integration/test_otp.py:511: AssertionError
 ------------------------------Captured stdout call------------------------------ 
spawn ssh -o NumberOfPasswordPrompts=1 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -l sshuser3 ip-10-0-197-227.xxx.redhat.com klist
sss_ssh_knownhostsproxy: Could not resolve hostname ip-10-0-197-227.xxx.redhat.com
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535
expect result: Unexpected end of file
expect exit code: 202
expect spawn exit code: 255

Note the error: Could not resolve hostname
I suspect that the problem happens because the test is using the external_hostname instead of the hostname value:
https://github.com/freeipa/freeipa/blob/master/ipatests/test_integration/test_otp.py#L554-L557
and
https://github.com/freeipa/freeipa/blob/master/ipatests/test_integration/test_otp.py#L506-L509


Metadata Update from @frenaud:
- Issue assigned to frenaud

Metadata Update from @frenaud:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/7922

master:

  • b0e4cdbf9dcaf8d46002f7b89a714b561ab97e03 ipatests: fix test_otp

ipa-4-12:

  • 9b631f80720fe1f2492d1a30bb1c2410af5eb587 ipatests: fix test_otp

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata