#9404 Nightly test failure in test_integration/test_backup_and_restore.py::TestBackupAndRestoreWithReplica::test_full_backup_and_restore_with_replica
Closed: fixed by frenaud. Opened by amore.

The nightly test test_backup_and_restore.py::TestBackupAndRestoreWithReplica::test_full_backup_and_restore_with_replica is failing with the nightly build of 389ds. and pki.

Test scenario: execute
See PR with the following report
PR2744 report

PR2777 report


Metadata Update from @amore:
- Issue tagged with: test-failure, tests

Test scenario:
install server with CA +DNS, install replica
add a service TEST/master_fqdn on the server
add user test1_master on the server
add user test1_replica on the replica
perform a backup on the server with ipa-backup -v --disable-role-check
delete test1_master on the server
delete test1_replica on the replica
add test2_master on the master
add test2_replica on the replica
simulate server crash: ipactl stop; ipa-server-install --uninstall -U --ignore-topology-disconnect --ignore-last-of-role
restore server with ipa-restore -U path_to_backup
add a user test3_master on the server
add a user test3_replica on the replica
check replication status

The replication failed to resume: Unable to acquire replica: permission denied. The bind dn does not have permission to supply replication updates to the replica. Will retry later. (permission denied)

I have a wild guess, but the uninstall code recently changed and now restarts the services => does not really simulate a server crash because the server and replica can still communicate. Maybe that's the reason why the bind DN for the server gets deleted during server uninstallation and prevents the replication from resuming. Something to be investigated...

Suspicion confirmed: from the logs of PR32774:

ipa-server-install --uninstall is called at 2023-06-26 06:29:11,779 and ends 06:30:46,527:

2023-06-26 06:29:11,779    DEBUG  [ipatests.pytest_ipa.integration.host.Host.master.IPAOpenSSHTransport] RUN ['ipa-server-install', '--uninstall', '-U', '--ignore-topology-disconnect', '--ignore-last-of-role']
2023-06-26 06:29:11,779    DEBUG  [ipatests.pytest_ipa.integration.host.Host.master.cmd97] RUN ['ipa-server-install', '--uninstall', '-U', '--ignore-topology-disconnect', '--ignore-last-of-role']
...
2023-06-26 06:30:46,527    DEBUG  [ipatests.pytest_ipa.integration.host.Host.master.cmd97] Exit code: 0

The audit log on the replica shows the deletion of krbprincipalname=ldap/master.ipa.test@IPA.TEST during the uninstallation:

time: 20230626062953
dn: krbprincipalname=ldap/master.ipa.test@IPA.TEST,cn=services,cn=accounts,dc=ipa,dc=test
result: 0
changetype: modify
delete: managedBy
managedBy: fqdn=master.ipa.test,cn=computers,cn=accounts,dc=ipa,dc=test
-
replace: modifiersName
modifiersName: cn=referential integrity postoperation,cn=plugins,cn=config
-
replace: modifyTimestamp
modifyTimestamp: 20230626062953Z
-
replace: entryusn
entryusn: 259
-

Metadata Update from @frenaud:
- Issue assigned to frenaud

Metadata Update from @frenaud:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/6901

master:

  • 8de6405b1130a9b21bae87689a18439059515399 tests: fix backup-restore scenario with replica

ipa-4-10:

  • 5e291da42898cc646f699c21a44b03b833d346e8 tests: fix backup-restore scenario with replica

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata