#7796 ipa-replica-install fails migrating CentOS 6 to 7
Closed: fixed Opened by wia.

Issue

Trying to migrate a production CentOS 6 IPA setup to CentOS 7. Both are fully updated CentOS 6.10 (ipa-server-3.0.0-51) and CentOS 7.6 (ipa-server-4.6.4-10) as of Dec 4 2018.

Following:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/linux_domain_identity_authentication_and_policy_guide/migrate-6-to-7

Running copy-schema-to-ca.py on centos6 created the replica info file without any issues. Copied file to centos7 and then ran:

[root@centos7]$ ipa-replica-install /var/lib/ipa/replica-info-centos7.gpg --setup-ca --ip-address 192.168.1.1 --setup-dns --no-forwarders

Script dies with the below error. End of ipareplica-install.log attched.

Steps to Reproduce

This setup has been in production for years. I'm assuming fresh installs would not produce this error.

Actual behavior

Install script dies with:

[27/41]: ignore time skew for initial replication
[28/41]: setting up initial replication
[error] DatabaseError: Server is unwilling to perform: modification of attribute nsds5replicabinddngroupcheckinterval is not allowed in replica entry
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

ipapython.admintool: ERROR Server is unwilling to perform: modification of attribute nsds5replicabinddngroupcheckinterval is not allowed in replica entry
ipapython.admintool: ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

Expected behavior

Install script to finish with a new CentOS 7 replica.

Version/Release/Distribution

CentOS 6.10:
ipa-server-3.0.0-51.el6.centos.x86_64
ipa-client-3.0.0-51.el6.centos.x86_64
389-ds-base-1.2.11.15-97.el6_10.x86_64
pki-ca-9.0.3-53.el6.noarch
krb5-server-1.10.3-65.el6.x86_64
CentOS 7.6:
ipa-server-4.6.4-10.el7.centos.x86_64
ipa-client-4.6.4-10.el7.centos.x86_64
389-ds-base-1.3.8.4-18.el7_6.x86_64
pki-ca-10.5.9-6.el7.noarch
krb5-server-1.15.1-34.el7.x86_64

Additional info:

Posted this issue to freeipa-users@lists.fedorahosted.org and was asked to submit a ticket:

indeed the error happens in a code path that wasn't fixed. Could you
open a new pagure ticket (https://pagure.io/freeipa/new_issue)? Please
attach the end of the logs, after the line
[28/41]: setting up initial replication
(you can replace your domain name with XX).

thanks,
flo


End of ipareplica-install.log:

2018-12-04T19:57:51Z DEBUG [28/41]: setting up initial replication
2018-12-04T19:57:51Z DEBUG retrieving schema for SchemaCache url=ldapi://%2fvar%2frun%2fslapd-XX-NET.socket conn=
2018-12-04T19:57:51Z DEBUG Destroyed connection context.ldap2_140277831984464
2018-12-04T19:57:51Z DEBUG Starting external process
2018-12-04T19:57:51Z DEBUG args=/bin/systemctl --system daemon-reload
2018-12-04T19:57:51Z DEBUG Process finished, return code=0
2018-12-04T19:57:51Z DEBUG stdout=
2018-12-04T19:57:51Z DEBUG stderr=
2018-12-04T19:57:51Z DEBUG Starting external process
2018-12-04T19:57:51Z DEBUG args=/bin/systemctl restart dirsrv@XX-NET.service
2018-12-04T19:57:55Z DEBUG Process finished, return code=0
2018-12-04T19:57:55Z DEBUG stdout=
2018-12-04T19:57:55Z DEBUG stderr=
2018-12-04T19:57:55Z DEBUG Restart of dirsrv@XX-NET.service complete
2018-12-04T19:57:55Z DEBUG Created connection context.ldap2_140277831984464
2018-12-04T19:57:56Z DEBUG Fetching nsDS5ReplicaId from master [attempt 1/5]
2018-12-04T19:57:56Z DEBUG retrieving schema for SchemaCache url=ldap://centos6.XX.net:389 conn=
2018-12-04T19:57:56Z DEBUG Successfully updated nsDS5ReplicaId.
2018-12-04T19:57:56Z DEBUG Add or update replica config cn=replica,cn=dc\=XX\,dc\=net,cn=mapping tree,cn=config
2018-12-04T19:57:56Z DEBUG Added replica config cn=replica,cn=dc\=XX\,dc\=net,cn=mapping tree,cn=config
2018-12-04T19:57:56Z DEBUG Add or update replica config cn=replica,cn=dc\=XX\,dc\=net,cn=mapping tree,cn=config
2018-12-04T19:57:56Z DEBUG Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 570, in start_creation
run_step(full_msg, method)
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 560, in run_step
method()
File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 456, in setup_replica
cacert=self.ca_file
File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 1787, in setup_promote_replication
self.basic_replication_setup(r_conn, r_id, self.repl_man_dn, None)
File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 1119, in basic_replication_setup
self.replica_config(conn, replica_id, repldn)
File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 534, in replica_config
conn.update_entry(entry)
File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1618, in update_entry
self.conn.modify_s(str(entry.dn), modlist)
File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit

self.gen.throw(type, value, traceback)
File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1069, in error_handler
raise errors.DatabaseError(desc=desc, info=info)
DatabaseError: Server is unwilling to perform: modification of attribute nsds5replicabinddngroupcheckinterval is not allowed in replica entry

2018-12-04T19:57:56Z DEBUG [error] DatabaseError: Server is unwilling to perform: modification of attribute nsds5replicabinddngroupcheckinterval is not allowed in replica entry
2018-12-04T19:57:56Z DEBUG Destroyed connection context.ldap2_140277821365840
2018-12-04T19:57:56Z DEBUG File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 178, in execute
return_value = self.run()
File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 319, in run
return cfgr.run()
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 364, in run
return self.execute()
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 389, in execute
for rval in self._executor():
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 434, in runner
exc_handler(exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 463, in _handle_execute_exception
self._handle_exception(exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 453, in _handle_exception
six.reraise(exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 424, in __runner
step()
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 421, in
step = lambda: next(self.__gen)
File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
six.reraise(
exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
value = gen.send(prev_value)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 658, in _configure
next(executor)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 434, in __runner
exc_handler(exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 463, in _handle_execute_exception
self._handle_exception(exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 521, in _handle_exception
self.__parent._handle_exception(exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 453, in _handle_exception
six.reraise(exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 518, in _handle_exception
super(ComponentBase, self)._handle_exception(exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 453, in _handle_exception
six.reraise(
exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 424, in __runner
step()
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 421, in
step = lambda: next(self.__gen)
File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
six.reraise(*exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
value = gen.send(prev_value)
File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 65, in _install
for unused in self._installer(self.parent):
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/__init
.py", line 622, in main
replica_install(self)
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 406, in decorated
func(installer)
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 1431, in install
fstore=fstore)
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 113, in install_replica_ds
setup_pkinit=not options.no_pkinit,
File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 419, in create_replica
self.start_creation(runtime=30)
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 570, in start_creation
run_step(full_msg, method)
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 560, in run_step
method()
File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 456, in setup_replica
cacert=self.ca_file
File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 1787, in setup_promote_replication
self.basic_replication_setup(r_conn, r_id, self.repl_man_dn, None)
File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 1119, in basic_replication_setup
self.replica_config(conn, replica_id, repldn)
File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 534, in replica_config
conn.update_entry(entry)
File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1618, in update_entry
self.conn.modify_s(str(entry.dn), modlist)
File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit

self.gen.throw(type, value, traceback)
File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1069, in error_handler
raise errors.DatabaseError(desc=desc, info=info)

2018-12-04T19:57:56Z DEBUG The ipa-replica-install command failed, exception: DatabaseError: Server is unwilling to perform: modification of attribute nsds5replicabinddngroupcheckinterval is not allowed in replica entry
2018-12-04T19:57:56Z ERROR Server is unwilling to perform: modification of attribute nsds5replicabinddngroupcheckinterval is not allowed in replica entry
2018-12-04T19:57:56Z ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

For full reference:
e-mail thread
and thread2

The fix for issue 7617 introduced a regression that was fixed with commit 02f4a7a DS replication settings: fix regression with <3.3 master
The commit did not fix all the code paths where nsds5ReplicaReleaseTimeout, nsds5ReplicaBackoffMax and nsDS5ReplicaBindDnGroupCheckInterval are modified. Please note that adding a replication entry with these attributes does not throw any error even with older 389-ds version, only modifications are failing.
The culprit line can easily be found in the above trace, in method ReplicationManager.replica_config():

        for key, value in REPLICA_CREATION_SETTINGS.items():
            entry[key] = value
        try:
            conn.update_entry(entry)

The attributes can be added only if the vendor_version of the 389-ds server >= (1, 3, 0).

As the issue is present on ipa-4-6 branch, setting milestone to 4.6.5

Metadata Update from @frenaud:
- Issue set to the milestone: FreeIPA 4.6.5

Workaround:

  • on the 6.x master, stop IPA with #ipactl stop
  • open /etc/dirsrv/slapd-DOMxxx/dse.ldif and replace or add the following attributes to the entry cn=replica,cn=dc=...,cn=mapping tree,cn=config:
nsds5ReplicaReleaseTimeout: 20
nsds5ReplicaBackoffMax: 3
nsDS5ReplicaBindDnGroupCheckInterval: 2
  • If the replica will be installed with a CA instance, also edit /etc/dirsrv/slapd-PKI-IPA/dse.ldif, replace or add the same attributes on the entry cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
  • restart IPA on the 6.x master
  • install the 7.6 replica with ipa-replica-install /path/to/replica-file
    Once the installation is successful, , stop ipa, edit the dse.ldif (the 2 files) on the 6.x master and modify the attribute values to:
nsds5ReplicaReleaseTimeout: 60
nsds5ReplicaBackoffMax: 300
nsDS5ReplicaBindDnGroupCheckInterval: 60

and finally restart IPA on the 6.x master.

Metadata Update from @frenaud:
- Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1658701

Issue linked to Bugzilla: Bug 1658701

Metadata Update from @frenaud:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/2682
- Custom field rhbz reset (from https://bugzilla.redhat.com/show_bug.cgi?id=1658701)

master:

  • faa122a8b8fc9a6850811ed1466166a7bff13be3 replication: check remote ds version before editing attributes

Automatic backports have failed:

Closing pull request 2682
Aplying to ipa-4-7: replication: check remote ds version before editing attributes
Failed to apply patches onto origin/ipa-4-7. Manual backport is needed.
Cleaning up
Aplying to ipa-4-6: replication: check remote ds version before editing attributes
Failed to apply patches onto origin/ipa-4-6. Manual backport is needed.
Cleaning up

ipa-4-7:

  • 690a8103e0ea6ed5f3f33c47a18910ad468b0291 replication: check remote ds version before editing attributes

ipa-4-6:

  • 5639974c6c4f56e4c2122176ee0abcf97d56648f replication: check remote ds version before editing attributes

The fix will be available in 4.7.3 and next release of 4.6 branch.

Metadata Update from @cheimes:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata Update from @frenaud:
- Issue assigned to frenaud

Metadata