#7188 Issues after promoting one CA-less IPA server to CA-full
Closed: fixed by ftweedal. Opened by orion.

We have 5 ipa servers installed CA-less. Stating to move to CA-full so I ran ipa-ca-install on one.

Then when using the IPA web interface on one of the other IPA servers when accessing the certificate pages I would get:

[Tue Oct 10 09:37:37.554766 2017] [:error] [pid 29547] ipa: ERROR: non-public: AttributeError: ra
[Tue Oct 10 09:37:37.554799 2017] [:error] [pid 29547] Traceback (most recent call last):
[Tue Oct 10 09:37:37.554804 2017] [:error] [pid 29547]   File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 367, in wsgi_execute
[Tue Oct 10 09:37:37.554808 2017] [:error] [pid 29547]     result = command(*args, **options)
[Tue Oct 10 09:37:37.554812 2017] [:error] [pid 29547]   File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 447, in __call__
[Tue Oct 10 09:37:37.554816 2017] [:error] [pid 29547]     return self.__do_call(*args, **options)
[Tue Oct 10 09:37:37.554820 2017] [:error] [pid 29547]   File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 475, in __do_call
[Tue Oct 10 09:37:37.554823 2017] [:error] [pid 29547]     ret = self.run(*args, **options)
[Tue Oct 10 09:37:37.554827 2017] [:error] [pid 29547]   File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 797, in run
[Tue Oct 10 09:37:37.554830 2017] [:error] [pid 29547]     return self.execute(*args, **options)
[Tue Oct 10 09:37:37.554834 2017] [:error] [pid 29547]   File "/usr/lib/python2.7/site-packages/ipaserver/plugins/cert.py", line 1515, in execute
[Tue Oct 10 09:37:37.554838 2017] [:error] [pid 29547]     **options)
[Tue Oct 10 09:37:37.554841 2017] [:error] [pid 29547]   File "/usr/lib/python2.7/site-packages/ipaserver/plugins/cert.py", line 1370, in _ca_search
[Tue Oct 10 09:37:37.554844 2017] [:error] [pid 29547]     ra = self.api.Backend.ra
[Tue Oct 10 09:37:37.554848 2017] [:error] [pid 29547]   File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 335, in __getattr__
[Tue Oct 10 09:37:37.554851 2017] [:error] [pid 29547]     raise AttributeError(key)
[Tue Oct 10 09:37:37.554854 2017] [:error] [pid 29547] AttributeError: ra
[Tue Oct 10 09:37:37.555112 2017] [:error] [pid 29547] ipa: INFO: [jsonserver_session] admin@NWRA.COM: cert_find(None, version=u'2.228'): InternalError

Changed /etc/ipa/default.conf on the CA-less server from:

enable_ra = False
ra_plugin = none

to

enable_ra = True
ra_plugin = dogtag
dogtag_version = 10

And that resolved that error. But now I get ' cannot connect to 'https://.nwra.com:443/ca/rest/account/login': [Errno 2] No such file or directory'

No messages on the remote CA server httpd logs. This was due to missing /var/lib/ipa/ra-agent.{key,pem} files. Copied them from CA server, but then the CA server does not recognize the RA certificate from the CA-less server. Running ipa-certupdate on the CA server cleared that up.

See also #6577 for related issues with certs.


Did the ipa-ca-install program complete without error?

What versions of IPA are the masters in your topology? What version master did
you run ipa-ca-install on?

Fraser, in this case he had multiple servers installed in a CAless environment. He added an IPA CA on one and had asked on irc whether he should install a CA on all of them and I told him that wasn't strictly required (though > 1 CA was recommended). Unfortunately this transition exposed an issue with the IPA handling where a CA isn't locally configured.

IPA knows which masters has a CA and will forward certificate requests. Unfortunately the non-CA masters still basically thought they were CAless installs so weren't configured to use dogtag as a CA backend.

I'm not entirely sure how we can address this other than to somehow detect this situation (e.g. a CA in cn=masters) and fail in as graceful a way as possible and point to some script to run or docs to read.

And this actually uncovered another bug as well. The Errno 2 was due to the missing RA agent certs in /var/lib/ipa. We should check for existence of those in as low a level function call as possible and raise proper exceptions (I have a PR for a similar issue related to the CA certificate).

Metadata Update from @ftweedal:
- Issue assigned to ftweedal

According to the nightly runs, the scenario for CA-less to CA-full replica migration is succeeding; http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/d1461ad0-b161-11e8-9411-fa163e2ec43b/ (a recent run). I wonder if this ticket can actually be closed.

@ftweedal Do you agree to close this ticket?

@pcech I'll need to find some time to carry out the scenario as exactly as described by @orion . It may be that it has been resolved but I'd like to be able to clarify how and when.

So, please leave open for now.

Pull request [wip]: https://github.com/freeipa/freeipa/pull/4861

Ready for review: https://github.com/freeipa/freeipa/pull/4861

Metadata Update from @ftweedal:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/4861

master:

  • 2fcc260cae4bd1186a7312c61a63144e8d656cd0 cainstance.update_ipa_conf: allow specifying ca_host
  • a1b3b34b906808c2deb69a3838edd0cf4739b467 cainstance: extract function import_ra_key
  • 53d472b490ac7a14fc78516b448d4aa312b79b7f (HEAD) certupdate: update config after deployment becomes CA-ful

Metadata Update from @ftweedal:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata