389-ds-base

#47912 Proper handling of "No original_tombstone for changenumber" errors
Closed: wontfix Opened by amessina.

Closed: wontfix
Reopen Issue

Using two fully updated FreeIPA F20 masters with freeipa-server-3.3.5-1.fc20.x86_64 and 389-ds-base-1.3.2.23-1.fc20.x86_64, I've noticed that I'm getting a lot of the following errors in the 389 DS error log, especially at server startup.

"No original_tombstone for changenumber=511851,cn=changelog!!"

Most often, the same changenumber repeats over and over, but there are some other changenumbers listed as well. The changenumbers are different on each master. Sometimes the logging loops repeatedly for about 10 minutes or so, running the CPU to 100%

My concern is I'm preparing my thought process about the upcoming upgrade from F20 to F21 and it looks like I may need to create a new FreeIPA master and "migrate" the Dogtag stuff, etc. rather than a simple "yum distro-sync" on each master. Yuck!

What is the proper way to correct for these apparent errors and get these masters working with each other in a clean manner?

Ludwig Krispenz suggested that I open a ticket here.

[28/Sep/2014:09:46:17 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522680,cn=changelog!!
[28/Sep/2014:09:46:17 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522681,cn=changelog!!
[28/Sep/2014:09:46:18 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522689,cn=changelog!!
[28/Sep/2014:09:46:19 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522691,cn=changelog!!
[28/Sep/2014:09:46:19 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522693,cn=changelog!!
[28/Sep/2014:09:46:20 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522694,cn=changelog!!
[28/Sep/2014:09:46:20 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522695,cn=changelog!!
[28/Sep/2014:09:46:20 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522697,cn=changelog!!
[28/Sep/2014:09:46:21 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522698,cn=changelog!!
[28/Sep/2014:09:46:21 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522699,cn=changelog!!
[28/Sep/2014:09:46:21 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522702,cn=changelog!!
[28/Sep/2014:09:46:22 -0500] ldbm_back_delete - conn=0 op=0 [retry: 2] No original_tombstone for changenumber=522702,cn=changelog!!
[28/Sep/2014:09:46:22 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522703,cn=changelog!!
[28/Sep/2014:09:46:22 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522704,cn=changelog!!
[28/Sep/2014:09:46:28 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522754,cn=changelog!!
[28/Sep/2014:09:46:28 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522755,cn=changelog!!
[28/Sep/2014:09:46:29 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=522758,cn=changelog!!
[28/Sep/2014:09:50:22 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=525583,cn=changelog!!
[28/Sep/2014:09:50:22 -0500] ldbm_back_delete - conn=0 op=0 [retry: 2] No original_tombstone for changenumber=525583,cn=changelog!!
[28/Sep/2014:09:50:24 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=525601,cn=changelog!!
[28/Sep/2014:09:51:40 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=526738,cn=changelog!!
[28/Sep/2014:09:52:22 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527371,cn=changelog!!
[28/Sep/2014:09:52:23 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527381,cn=changelog!!
[28/Sep/2014:09:52:25 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527390,cn=changelog!!
[28/Sep/2014:09:52:27 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527411,cn=changelog!!
[28/Sep/2014:09:52:29 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527415,cn=changelog!!
[28/Sep/2014:09:52:30 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527416,cn=changelog!!
[28/Sep/2014:09:52:30 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527417,cn=changelog!!
[28/Sep/2014:09:52:31 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527420,cn=changelog!!
[28/Sep/2014:09:52:32 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527428,cn=changelog!!
[28/Sep/2014:09:52:33 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527439,cn=changelog!!
[28/Sep/2014:09:52:35 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527467,cn=changelog!!
[28/Sep/2014:09:52:44 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527596,cn=changelog!!
[28/Sep/2014:09:52:49 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527667,cn=changelog!!
[28/Sep/2014:09:52:51 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527681,cn=changelog!!
[28/Sep/2014:09:53:05 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527894,cn=changelog!!
[28/Sep/2014:09:53:10 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527951,cn=changelog!!
[28/Sep/2014:09:53:10 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527958,cn=changelog!!
[28/Sep/2014:09:53:14 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=527992,cn=changelog!!
[28/Sep/2014:09:53:17 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=528013,cn=changelog!!
[28/Sep/2014:09:53:18 -0500] ldbm_back_delete - conn=0 op=0 [retry: 1] No original_tombstone for changenumber=528021,cn=changelog!!

Adding my comment from the mailing list:

I think the message was introduced when tombstone handling was partly rewritten to fix some bugs and improve entry cache management.
It is logged if a delete transaction has to be retried and no tombstone entry is found. In the normal case this should not happen, but looks like this is in the retro changelog during changelog trimming and for the changelog a delete does not create a tombstone.
In my opinion there should be an additional check before logging this message.

Reviewed by Ludwig (Thank you!!)

Pushed to master:
3104346..36381c1 master -> master
commit 36381c120773872d3d4d2cb2417f155e6ac790a6

Pushed to 389-ds-base-1.3.3:
e7a26dd..7df1729 389-ds-base-1.3.3 -> 389-ds-base-1.3.3
commit 7df17293d7dff918e05bc9376064290db61d1f70

added RH
bz 1235847 - handling of "No original_tombstone for changenumber" errors
for a RHEL 7.1 report

Metadata Update from @nhosoi:
- Issue assigned to nhosoi
- Issue set to the milestone: 1.3.3 - 10/31 (October)

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/1243

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @spichugi:
- Issue close_status updated to: wontfix (was: Fixed)
Metadata
None
None
None
Normal
ack
Community
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.