#627 ns-slapd crashes sporadically with segmentation fault in libslapd.so
Closed: wontfix Opened by nkinder.

Description of problem:

ns-slapd is occasionally crashing while using schema-reload.pl. Crashes seems
to be more frequent when there is more data/more schema files/more load in the
system. Copy paste from /var/log/messages:

Mar 11 11:03:21 sprintlab070vm1 kernel: ns-slapd[62980]: segfault at bc ip
0000003be1cabfd4 sp 00007fc522bfa400 error 4 in
libslapd.so.0.0.0[3be1c00000+f6000]
Mar 11 12:11:03 sprintlab070vm1 kernel: ns-slapd[52594] general protection
ip:3be1cabfd4 sp:7f3b0f5f6ba0 error:0 in libslapd.so.0.0.0[3be1c00000+f6000]
Mar 11 12:26:49 sprintlab070vm1 kernel: ns-slapd[53038] general protection
ip:3be1cabfd4 sp:7f78e1660460 error:0 in libslapd.so.0.0.0[3be1c00000+f6000]

Version-Release number of selected component (if applicable):

389-ds-base-1.2.11.15-12.el6_4.x86_64

How reproducible:

Occasionally

Steps to Reproduce:
1. Run the following script:

!/bin/bash

DM="cn=manager"
DM_PASSWD="Manager"
INSTANCE="myinstancee"

while true
do
echo "triggering schema reload"
entry=/usr/lib64/dirsrv/slapd-${INSTANCE}/schema-reload.pl -D "${DM}" -w ${DM_PASSWD} |grep "adding new entry"|cut -c 18-|sed s/\"//g

echo "reload triggered, task entry: ${entry}"
while true
do
e=/usr/lib64/mozldap/ldapsearch -1 -D "${DM}" -w ${DM_PASSWD} -b "${entry}" "(objectclass=*)" nstaskstatus
ec=$?
if echo $e|grep "Schema reload task finished"
then
break
fi

echo "$ec waiting for task ${entry} to finish."
if [ "$ec" -ne "0" ]
then
  exit
fi
sleep 1

done
echo "schema reloaded"
echo ""
done

Actual results:
ns-slapd throws segfault from time to time

Expected results:
no segfault should happen

Additional info:

Attached abrt outcome of one crash.
backtrace follows:

0 PL_HashTableRawLookup_const (ht=0x1043b10, keyHash=1630,

key=0x7fa7b6bfa4b0) at ldap/servers/slapd/slapd_plhash.c:73

1 0x00007fa7d4848037 in PL_HashTableLookup_const (ht=0x1043b10,

key=0x7fa7b6bfa4b0) at ldap/servers/slapd/slapd_plhash.c:91

2 0x00007fa7d47e0620 in attr_syntax_get_by_name_locking_optional

(name=0x7fa7b6bfa4b0 "cn", use_lock=1) at ldap/servers/slapd/attrsyntax.c:321

3 0x00007fa7d47df520 in slapi_attr_init_locking_optional (a=0x7fa7ac003510,

type=0x7fa770317df0 "cn", use_lock=1) at ldap/servers/slapd/attr.c:252

4 0x00007fa7d47df6f7 in slapi_attr_dup (attr=0x7fa7702b2520) at

ldap/servers/slapd/attr.c:328

5 0x00007fa7d47f3731 in slapi_entry_dup (e=0x7fa7702ef080) at

ldap/servers/slapd/entry.c:2107

6 0x00007fa7d47ee533 in dse_write_entry (data=,

arg=0x7fa7b6bfa970 "\360$") at ldap/servers/slapd/dse.c:1087

7 0x00007fa7d485b6fe in avl_inapply (root=,

fn=0x7fa7d47ee4e0 , arg=0x7fa7b6bfa970 "\360$", stopflag=-2)
at ldap/libraries/libavl/avl.c:514

8 0x00007fa7d485b6ef in avl_inapply (root=,

fn=0x7fa7d47ee4e0 , arg=0x7fa7b6bfa970 "\360$", stopflag=-2)
at ldap/libraries/libavl/avl.c:510

9 0x00007fa7d485b6ef in avl_inapply (root=,

fn=0x7fa7d47ee4e0 , arg=0x7fa7b6bfa970 "\360$", stopflag=-2)
at ldap/libraries/libavl/avl.c:510

10 0x00007fa7d485b6ef in avl_inapply (root=,

fn=0x7fa7d47ee4e0 , arg=0x7fa7b6bfa970 "\360$", stopflag=-2)
at ldap/libraries/libavl/avl.c:510

11 0x00007fa7d47ed8aa in dse_write_file_nolock (pdse=0x1105450) at

ldap/servers/slapd/dse.c:1023

12 0x00007fa7d47ee4dd in dse_delete_entry (pb=0x7fa7ac005080) at

ldap/servers/slapd/dse.c:1462

13 dse_delete (pb=0x7fa7ac005080) at ldap/servers/slapd/dse.c:2322

14 0x00007fa7d47e8123 in op_shared_delete (pb=0x7fa7ac005080) at

ldap/servers/slapd/delete.c:364

15 0x00007fa7d47e8272 in delete_internal_pb (pb=0x7fa7ac005080) at

ldap/servers/slapd/delete.c:242

16 0x00007fa7d484d214 in destroy_task (when=1363267827, arg=<value optimized

out>) at ldap/servers/slapd/task.c:636

17 0x00007fa7d47f9a97 in eq_call_all (arg=) at

ldap/servers/slapd/eventq.c:312

18 eq_loop (arg=) at ldap/servers/slapd/eventq.c:359

19 0x0000003469e29a73 in _pt_root (arg=0x1454820) at

../../../mozilla/nsprpub/pr/src/pthreads/ptthread.c:167

20 0x0000003466607851 in start_thread (arg=0x7fa7b6bfd700) at

pthread_create.c:301

21 0x00000034662e890d in nfsservctl () at

../sysdeps/unix/syscall-template.S:82

22 0x0000000000000000 in ?? ()


Bug Description: Schema reload task (schema-reload.pl) was not
thread safe.

Fix Description: Attribute Syntax is stored in the hash and
retrieved based upon the attribute syntax. When Schema reload
task is invoked, the attribute syntax objects were completely
replaced ignoring the lock protection. This patch protects
the attribute syntax replacement (attr_syntax_delete_all_for_
schemareload) with the write lock. Also, attribute syntax
object maintains the reference count. The schema reload
respects the reference count instead of blindly deleting them.

Reviewed by Rich (Thank you!!)

Pushed to master: commit 81b997480956b2b6fa3a5d0e8d6abf5113a06400
Pushed to 389-ds-base-1.3.0 branch: commit a7ae8de6e50ae209737f8927cf3a63e875620911
Pushed to 389-ds-base-1.2.11 branch: commit c82987275bec0e1eb1a6c9b853fcaf389a300e71

Metadata Update from @nhosoi:
- Issue assigned to nhosoi
- Issue set to the milestone: 1.2.11.20

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/627

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @spichugi:
- Issue close_status updated to: wontfix (was: Fixed)

Metadata