389 would not start up. after some debugging, we determined we accidentally had the Retro changelog plugin enabled. There were several hundred thousand entries in the DB. We don't know ultimately what was the problem with Retro but we disabled it and then 389 would start up. We have also deleted the cn=changelog db so as to ensure this problem wouldn't happen again - this means we lost the evidence that caused the problem - sorry!
Here is the debugging info we captured at time of failure.
uname -a Linux ldap-master-01.andrew.cmu.edu 2.6.32-358.18.1.el6.x86_64 #1 SMP Fri Aug 2 17:04:38 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux
389-Directory/1.2.11.15 B2013.238.2155 starting up
yum list | grep 389 389-admin.x86_64 1.1.29-1.el6 @epel-x86_64-server-6 389-admin-console.noarch 1.1.8-1.el6 @epel-x86_64-server-6 389-admin-console-doc.noarch 1.1.8-1.el6 @epel-x86_64-server-6 389-adminutil.x86_64 1.1.15-1.el6 installed 389-console.noarch 1.1.7-3.el5 installed 389-ds.noarch 1.2.2-1.el6 @epel-x86_64-server-6 389-ds-base.x86_64 1.2.11.15-22.el6_4 @rhel-x86_64-server-6 389-ds-base-libs.x86_64 1.2.11.15-22.el6_4 @rhel-x86_64-server-6 389-ds-console.noarch 1.2.6-1.el6 @epel-x86_64-server-6 389-ds-console-doc.noarch 1.2.6-1.el6 @epel-x86_64-server-6 389-dsgw.x86_64 1.1.10-1.el6 @epel-x86_64-server-6 389-admin.i686 1.1.29-1.el6 epel-x86_64-server-6 389-adminutil.i686 1.1.15-1.el6 epel-x86_64-server-6 389-adminutil-devel.i686 1.1.15-1.el6 epel-x86_64-server-6 389-adminutil-devel.x86_64 1.1.15-1.el6 epel-x86_64-server-6 389-ds-base-debuginfo.x86_64 1.2.10.26-1.el6_3 389_rhel6_x86_64 389-ds-base-devel.i686 1.2.11.15-22.el6_4 rhel-x86_64-server-optional-6 389-ds-base-devel.x86_64 1.2.11.15-22.el6_4 rhel-x86_64-server-optional-6 389-ds-base-libs.i686 1.2.11.15-22.el6_4 rhel-x86_64-server-6
open("/var/run/dirsrv/slapd-cmu.pid", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 58 fstat(58, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa004a0d000 write(58, "31972\n", 6) = 6 close(58) = 0 munmap(0x7fa004a0d000, 4096) = 0 chmod("/var/run/dirsrv/slapd-cmu.pid", 0644) = 0 poll([{fd=56, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}, {fd=-1}], 4, 250 +++ killed by SIGSEGV +++
errors log shows nothing other than the usual start-up messages. ns-slapd seems to die fairly quickly after start-up.
we are able to get a backtrace via gdb
(gdb) bt
from /usr/lib64/dirsrv/libslapd.so.0
from /usr/lib64/dirsrv/plugins/libretrocl-plugin.so
This report filed at the suggestion of Rich Megginson.
I cannot reproduce - steps 1) use latest 1.2.11 branch 2) setup instance 3) stop dirsrv - edit dse.ldif to enable Retro Changelog - start dirsrv 4) add many entries - verify entries are in changelog 5) restart dirsrv
No problems. I even ran dirsrv with valgrind - no reported errors.
Looking at the stack trace - slapi_entry_attr_find is called from trim_changelog() and from handle_cnum_entry() - in both cases, the entry and type and entry->e_attrs arguments are all checked for NULL before dereferencing, and the type field is always a constant, so never NULL.
In the absence of a reproducer or more detailed stack trace, I'm going to close this ticket. Please reopen or file a new ticket if this happens again.
Metadata Update from @rmeggins: - Issue set to the milestone: N/A
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This issue has been cloned to Github and is available here: - https://github.com/389ds/389-ds-base/issues/874
If you want to receive further updates on the issue, please navigate to the github issue and click on subscribe button.
subscribe
Thank you for understanding. We apologize for all inconvenience.
Metadata Update from @spichugi: - Issue close_status updated to: wontfix (was: Invalid)