#7534 Investigate failures to restore 389-ds attriubtes on upgrade failure
Closed: fixed by rcritten. Opened by rcritten.

Issue

ipa-server-upgrade disables the 389-ds listeners so the server is "quiet" during upgrade. Twice this week I've run into users who had to manually restore the values after a failed upgrade.

This should be bullet-proof and is something we've looked at before but it is still happening. I have a memory that German or Thorsten mentioned that this is common as well.

It very well may be a regression with the switch to the new installer framework.

mail thread https://lists.fedoraproject.org/archives/list/freeipa-users@lists.fedorahosted.org/thread/7YNTBRSRJKU2IXQH5CEVZQPDEKOCKHDV/

I don't know how to reproduce this.


This is because there is no call to __restore_config() upon exception.

There used to be a monster try/except around the entire upgrade process where no matter what the listeners were restored. In the switch to the admintool framework this was lost and while exceptions are caught there is no call to restore the configuration upon failure.

The double underscores are not at all needed as this class will/should never be subclassed anyway.

At least __restore_config() needs to be promoted to a public function and called during all except handlers (if not moving it to a finally with some care that it does nothing if the values have already been restored).

Metadata Update from @rcritten:
- Issue priority set to: critical
- Issue set to the milestone: FreeIPA 4.7

Metadata Update from @rcritten:
- Issue assigned to rcritten

Ah ok, I missed the run_after_failure bit in ipaserver/install/service.py so it looks like it should be restoring things properly (but doesn't)

Metadata Update from @frenaud:
- Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1569011

Issue linked to Bugzilla: Bug 1569011

The only way I've been able to reproduce this is with keyboard interrupt ^C. If one spams ^C then it can break out of all exception handlers.

Metadata Update from @rcritten:
- Issue set to the milestone: FreeIPA 4.7.1 (was: FreeIPA 4.7)

FreeIPA 4.7 has been released, moving to FreeIPA 4.7.1 milestone

Metadata Update from @rcritten:
- Issue set to the milestone: FreeIPA 4.7.2 (was: FreeIPA 4.7.1)

FreeIPA 4.7.1 has been released, moving to FreeIPA 4.7.2 milestone

https://github.com/freeipa/freeipa/pull/5207

Metadata Update from @rcritten:
- Issue set to the milestone: None (was: FreeIPA 4.7.2)

master:

  • 2f8eb73f588c96b391be57020ecf0b247c646d19 Use a state to determine if a 389-ds upgrade is in progress

ipa-4-8:

  • ae4e549f7911e0e66e5b3c1d2e35db604235324e Use a state to determine if a 389-ds upgrade is in progress

Metadata Update from @rcritten:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata Update from @rcritten:
- Custom field affects_doc adjusted to on
- Custom field knownissue adjusted to on
- Issue status updated to: Open (was: Closed)

There is still some exposure to killing. Someone reproduced this by killing the process in the parser.parse() call within __restore_config (line 230) so the values were restored from the backup but the new dse.ldif never written.

I'm not sure this can ever be 100% bullet-proof since it can be externally killed but if rather than calling restore_state() on the values in __restore_config we use get_state() which will leave them in the state file. Then the last step is to pop upgrade-in-progress and then the rest.

If the values have been restored and the new ldif written and copied then it's only upgrade-in-progress that really matters. The rest will be overwritten.

https://github.com/freeipa/freeipa/pull/5312

master:

  • ba9b9502e4f4e99d0f11a8cdc40e8e57cb774848 Move where the restore state is marked during IPA server upgrade

ipa-4-9:

  • 20055ddaf169787c041f0baf0bd0cdca1f5fe7b5 Move where the restore state is marked during IPA server upgrade

ipa-4-8:

  • 9ecd7c2319d9341edf271d2fd101f974413218f4 Move where the restore state is marked during IPA server upgrade

Metadata Update from @rcritten:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata