#50765 Issue 50747 - Port readnsstate to dsctl
Closed by spichugi. Opened by mreynolds.
mreynolds/389-ds-base issue50747  into  master

Download 50765.patch

Description:

Port the legacy tool readnsstate to dsctl, and add a healthcheck for local and remote offset that are close to triggering replication time skew errors

relates: https://pagure.io/389-ds-base/issue/50747

This server will generate new_csn = now + local_offset + remote_offset
This server is fine, but others may refuse updates from this server if new_csn - now > 24h.

So I think messages should say that replication from this server is potentially at risk. Potentially, because if csngen on others servers have the same offset, it will be fine.

Also, I am not sure it is useful to separate the test of local/remote offset rather than testing local+remote. At the end of the day, replication is potentially at risk since local_offset+remote_offset > 24h.

Also I am not sure what we can say about the system time (behind/ahead) of the server itself. I would rather say something like that the mechanism to handle the time skew may reach a limit.

This server will generate new_csn = now + local_offset + remote_offset
This server is fine, but others may refuse updates from this server if new_csn - now > 24h.
So I think messages should say that replication from this server is potentially at risk. Potentially, because if csngen on others servers have the same offset, it will be fine.

Well we can not look at the other server's nsState in healthcheck. So I was looking at csngen_adjust_time(), and if the offset is greater than CSN_MAX_TIME_ADJUST we return an error CSN_LIMIT_EXCEEDED.

Should I be checking the offset differential differently?

Or, what can I learn from nsState that is worth reporting in the health check in regards to clock skew?

Also, I am not sure it is useful to separate the test of local/remote offset rather than testing local+remote. At the end of the day, replication is potentially at risk since local_offset+remote_offset > 24h.

Well I thought it would be useful to know if the local server's clock is off, or a remote server clock is off. We want the customer to be able to fix what is broken, so I was trying to point them in the right direction.

Also I am not sure what we can say about the system time (behind/ahead) of the server itself. I would rather say something like that the mechanism to handle the time skew may reach a limit.

Which messages are you referring to?

IMHO if nsstate reveal a high remote and local offset, it worth that the healthcheck tool report there is a potential risk. I agree the 6h=low, 12h=medium and 24h=high as 24h offset may break replication. But to break it depends on the csngen.offsets off the consumers that we do not know, and the ignore-time-skew config that we do not know either.

investigation that admin could take is to monitor the time-skew and if it continue to increase, he should identify why (is a server system time jumping, is nscd making frequent change..). A protective action is to configure ignore-time-skew=no on the topology if the risk become high even if the risk is potential.

I was thinking at messages DSSKEWLE00[1-3] that start with 'MSG has a system time that is over xxx hours behind this server...'. I think that healthcheck may detect offsets even if system time look in sync (because time was set forward/backward for a short period of time). So the message should just say there is possible time skew limit without reference to the current system time.

1 new commit added

  • Revise lint messages per Thierry's requests

@tbordaz I revised the lint messages. I'm not sure it's exactly what you want. So please review it again, and I can make further changes. Thanks!

@mreynolds the messages looks perfect to me !

The only remaining concern is regarding test of local and remote offsets in _lint_nsstate.
IMHO it is useless to separate them in your test. We should rather have only one value tested: local_offset+remote_offset.

rebased onto 30cf0c462dfabdf0a36d5c2cbbba23c4f1747f63

@mreynolds the messages looks perfect to me !
The only remaining concern is regarding test of local and remote offsets in _lint_nsstate.
IMHO it is useless to separate them in your test. We should rather have only one value tested: local_offset+remote_offset.

Done! Added a time_skew field to the get-nstate output as well. Please review...

Thanks @mreynolds . The patch looks good to me. ACK

rebased onto 7301d4356b19b55de892970a6311b901900e6af8

Pull-Request has been merged by mreynolds

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This pull request has been cloned to Github as issue and is available here:
- https://github.com/389ds/389-ds-base/issues/3820

If you want to continue to work on the PR, please navigate to the github issue,
download the patch from the attachments and file a new pull request.

Thank you for understanding. We apologize for all inconvenience.

Pull-Request has been closed by spichugi

Metadata