freeipa

#9253 At random intervals Kerberos segfaults during ldap_get_entry_controls
Closed: duplicate by abbra. Opened by demigod.

Closed: duplicate
Reopen Issue

Issue

At random intervals, the KDC segfaults at ldap_get_entry_controls.

coredumpctl info 736854

       PID: 736854 (krb5kdc)
       UID: 0 (root)
       GID: 0 (root)
    Signal: 6 (ABRT)
 Timestamp: Tue 2022-09-27 05:29:18 CDT (4 days ago)

Command Line: /usr/sbin/krb5kdc -P /var/run/krb5kdc.pid -w 4
Executable: /usr/sbin/krb5kdc
Control Group: /system.slice/krb5kdc.service
Unit: krb5kdc.service
Slice: system.slice
Boot ID: d8b022d9fb8d40beba050e782fb6058b
Machine ID: d1a418c31e1e4a8d9feca7f4210978d0
Hostname: XXXX
Storage: /var/lib/systemd/coredump/core.krb5kdc.0.d8b022d9fb8d40beba050e782fb6058b.736854.1664274558000000.lz4 (inaccessible)
Message: Process 736854 (krb5kdc) of user 0 dumped core.

            Stack trace of thread 736854:
            #0  0x00007f96e8d6437f raise (libc.so.6)
            #1  0x00007f96e8d4edb5 abort (libc.so.6)
            #2  0x00007f96e8d4ec89 __assert_fail_base.cold.0 (libc.so.6)
            #3  0x00007f96e8d5ca76 __assert_fail (libc.so.6)
            #4  0x00007f96da538518 ldap_get_entry_controls (libldap_r-2.4.so.2)
            #5  0x00007f96db4a8bf7 ipadb_ldap_deref_results (ipadb.so)
            #6  0x00007f96db4ae38b ipadb_get_pac (ipadb.so)
            #7  0x00007f96db4b0784 ipadb_sign_authdata (ipadb.so)
            #8  0x00005610b4e56e71 handle_authdata (krb5kdc)
            #9  0x00005610b4e48084 finish_process_as_req (krb5kdc)
            #10 0x00005610b4e51556 finish_check_padata (krb5kdc)
            #11 0x00005610b4e53e74 enc_ts_verify (krb5kdc)
            #12 0x00005610b4e5204f next_padata (krb5kdc)
            #13 0x00005610b4e48f77 process_as_req (krb5kdc)
            #14 0x00005610b4e475b8 dispatch (krb5kdc)
            #15 0x00005610b4e5cf6d process_tcp_connection_read (krb5kdc)
            #16 0x00007f96e90f55ec verto_fire (libverto.so.1)
            #17 0x00007f96d2544ff1 event_process_active_single_queue (libevent-2.1.so.6)
            #18 0x00007f96d2545787 event_base_loop (libevent-2.1.so.6)
            #19 0x00005610b4e462e9 main (krb5kdc)
            #20 0x00007f96e8d50493 __libc_start_main (libc.so.6)
            #21 0x00005610b4e4667e _start (krb5kdc)

Additional info:

The following is found in /var/log/krb5kdc.log at the time of the crash
Sep 25 15:56:45 XXXX krb5kdc709128: worker 709130 exited with status 134

A snip from the full stacktrace shows the segfault "error: Cannot access memory at address".

4 0x00007f96da538518 in ldap_get_entry_controls (ld=ld@entry=0x0, entry=entry@entry=0x5610b6fc57c0, sctrls=sctrls@entry=0x7ffc3830e330) at getentry.c:89

    rc = <optimized out>
    be = {ber_opts = {lbo_valid = -16064, lbo_options = 56171, lbo_debug = 3}, ber_tag = 0, ber_len = 5, ber_usertag = 0, ber_buf = 0x5610b6ee1e34 "",
      ber_ptr = 0x3b9b563cdd950c00 <error: Cannot access memory at address 0x3b9b563cdd950c00>, ber_end = 0x5610b6ef5890 "\247~\002\f", ber_sos_ptr = 0x7ffc3830e450 "", ber_rwptr = 0x0,            
      ber_memctx = 0x7f96db4a7ea1 <ipadb_need_retry+49>}
    __PRETTY_FUNCTION__ = "ldap_get_entry_controls"

Steps to Reproduce

Unknown. Random crashes of KDC a few times a week.

Version/Release/Distribution

ipa-server-4.9.3-1.module_el8.5.0+836+f9e1ecc9.x86_64
ipa-client-4.9.3-1.module_el8.5.0+836+f9e1ecc9.x86_64
389-ds-base-1.4.3.23-7.module_el8.5.0+889+90e0384f.x86_64
pki-ca-10.11.0-1.module_el8.5.0+876+d4bb8aa6.noarch
krb5-server-1.18.2-13.el8.x86_64

cat /etc/redhat-release
CentOS Stream release 8

uname -r
4.18.0-331.el8.x86_64

Full stacktrace:

0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50

    set = {__val = {0, 94629793542944, 2, 140286128448587, 94630941917184, 94629793542944, 94629793542944, 94629793542944, 94629793542944, 94629793543024, 94629793543044, 94629793542944, 94629793543044, 0, 0, 0}}
    pid = <optimized out>
    tid = <optimized out>
    ret = <optimized out>

1 0x00007f96e8d4edb5 in __GI_abort () at abort.c:79

    save_stage = 1
    act = {__sigaction_handler = {sa_handler = 0x5610b73ab320, sa_sigaction = 0x5610b73ab320}, sa_mask = {__val = {0, 140286131882944, 140286129526304, 0, 0, 0, 140721251213800, 21474836480, 140721251213648, 140286129582768,
          140286129567384, 0, 4295121489043459072, 140286129552301, 0, 140286129567384}}, sa_flags = -631936836, sa_restorer = 0x7f96da559228}
    sigs = {__val = {32, 0 <repeats 15 times>}}

2 0x00007f96e8d4ec89 in __assert_fail_base (fmt=0x7f96e8eb7698 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x7f96da559228 "ld != NULL", file=0x7f96da5568bc "getentry.c", line=89, function=)

at assert.c:92
    str = 0x5610b73ab320 "\020\204\030\267\020V"
    total = 4096

3 0x00007f96e8d5ca76 in __GI___assert_fail (assertion=assertion@entry=0x7f96da559228 "ld != NULL", file=file@entry=0x7f96da5568bc "getentry.c", line=line@entry=89,

function=function@entry=0x7f96da5568e0 <__PRETTY_FUNCTION__.8960> "ldap_get_entry_controls") at assert.c:101

No locals.

4 0x00007f96da538518 in ldap_get_entry_controls (ld=ld@entry=0x0, entry=entry@entry=0x5610b6fc57c0, sctrls=sctrls@entry=0x7ffc3830e330) at getentry.c:89

    rc = <optimized out>
    be = {ber_opts = {lbo_valid = -16064, lbo_options = 56171, lbo_debug = 3}, ber_tag = 0, ber_len = 5, ber_usertag = 0, ber_buf = 0x5610b6ee1e34 "",
      ber_ptr = 0x3b9b563cdd950c00 <error: Cannot access memory at address 0x3b9b563cdd950c00>, ber_end = 0x5610b6ef5890 "\247~\002\f", ber_sos_ptr = 0x7ffc3830e450 "", ber_rwptr = 0x0,          
      ber_memctx = 0x7f96db4a7ea1 <ipadb_need_retry+49>}
    __PRETTY_FUNCTION__ = "ldap_get_entry_controls"

5 0x00007f96db4a8bf7 in ipadb_ldap_deref_results (lcontext=0x0, le=le@entry=0x5610b6fc57c0, results=results@entry=0x7ffc3830e3e0) at ipa_kdb_common.c:572

    ctrls = 0x0
    derefctrl = 0x0
    ret = <optimized out>

6 0x00007f96db4ae38b in ipadb_fill_info3 (info3=0x5610b6f374a0, authtime=1664274523, memctx=0x5610b7403bb0, flags=112, lentry=0x5610b6fc57c0, ipactx=0x5610b6ef5890) at ipa_kdb_mspac.c:669

    deref_results = 0x0
    ret = <optimized out>
    is_host = <optimized out>
    is_service = <optimized out>
    is_ipauser = <optimized out>
    sid = {sid_rev_num = 16 '\020', num_auths = -27 '\345', id_auth = "08\374\177\000", sub_auths = {3717532672, 1000035900, 20, 0, 3932682600, 32662, 0, 0, 0, 0, 0, 0, 0, 0, 0}}
    intres = -491473159
    timeres = 1652291979
    objectclasses = 0x5610b6fcddd0
    c = <optimized out>
    is_idobject = <optimized out>
    prigid = 515
    strres = 0x5610b721fdf0 "\020n\373\266\020V"
    is_user = <optimized out>
    princ = 0x5610b6f48b70
    deref_results = <optimized out>
    sid = <optimized out>
    prigid = <optimized out>
    timeres = <optimized out>
    strres = <optimized out>
    intres = <optimized out>
    ret = <optimized out>
    objectclasses = <optimized out>                                                                                                                                                                                     [60/192534]
    c = <optimized out>
    is_host = <optimized out>
    is_user = <optimized out>
    is_service = <optimized out>
    is_ipauser = <optimized out>
    is_idobject = <optimized out>
    princ = <optimized out>
    sep = <optimized out>
    is_master = <optimized out>
    dres = <optimized out>
    dval = <optimized out>
    gsid = <optimized out>
    trid = <optimized out>
    tgid = <optimized out>
    s = <optimized out>
    count = <optimized out>

7 ipadb_get_pac (kcontext=kcontext@entry=0x5610b6fb8cc0, client=0x5610b70590d0, flags=112, authtime=1664274523, pac=0x7ffc3830e630) at ipa_kdb_mspac.c:868

    tmpctx = 0x5610b7403bb0
    ied = <optimized out>
    ipactx = 0x5610b6ef5890
    results = 0x5610b6fc57c0
    lentry = 0x5610b6fc57c0
    pac_data = {data = 0x5610b6f48b70 "\200\265\f\267\020V", length = 140721251215184}
    data = {magic = -1223920800, length = 22032, data = 0x7f96ea93b330 ""}
    pac_info = {logon_info = {info = 0x5610b6f374a0}, credential_info = {version = 3069408416, encryption_type = 22032, encrypted_data = {data = 0x0, length = 0}}, srv_cksum = {type = 3069408416, signature = {data = 0x0,

--Type for more, q to quit, c to continue without paging--c
length = 0}}, kdc_cksum = {type = 3069408416, signature = {data = 0x0, length = 0}}, logon_name = {logon_time = 94629788873888, size = 0, account_name = 0x0}, constrained_delegation = {info = 0x5610b6f374a0}, upn_dns_
info = {upn_name_size = 29856, upn_name = 0x0, dns_domain_name_size = 0, dns_domain_name = 0x0, flags = 0}, unknown = {remaining = {data = 0x5610b6f374a0 "\200\377\303\352[\322\330\001\377\377\377\377\377\377\377\177\377\377\377\37
7\377\377\377\177\200\267\372\341e\330\001\200\267\372\341e\330\001\377\377\377\377\377\377\377\177", length = 0}}}
kerr =
ndr_err =
pac_upn = {logon_info = {info = 0x0}, credential_info = {version = 0, encryption_type = 0, encrypted_data = {data = 0x3b9b563cdd950c00 , length = 109044118360876996
98}}, srv_cksum = {type = 0, signature = {data = 0x3b9b563cdd950c00 , length = 10904411836087699698}}, kdc_cksum = {type = 0, signature = {data = 0x3b9b563cdd950c00 , length = 10904411836087699698}}, logon_name = {logon_time = 0, size = 3072, account_name = 0x975440ca769288f2 }, const
rained_delegation = {info = 0x0}, upn_dns_info = {upn_name_size = 0, upn_name = 0x3b9b563cdd950c00 , dns_domain_name_size = 35058, dns_domain_name = 0xa , flags = 3069479792}, unknown = {remaining = {data = 0x0, length = 4295121489043459072}}}
principal = 0x0

8 0x00007f96db4b0784 in ipadb_sign_authdata (context=0x5610b6fb8cc0, flags=112, client_princ=, client=0x5610b70590d0, server=0x5610b7151eb0, krbtgt=0x5610b7151eb0, client_key=0x5610b6f45f70, server_key=0x5610b6f45f

58, krbtgt_key=0x5610b6f45f40, session_key=0x5610b6f46070, authtime=1664274523, tgt_auth_data=0x0, signed_auth_data=0x7ffc3830e890) at ipa_kdb_mspac.c:2285
ks_client_princ = 0x5610b6f45e40
pac_auth_data = 0x0
authdata = {0x0, 0x0}
ad = {magic = -1226383344, ad_type = 22032, length = 64, contents = 0xfffffffffffffea8 }
is_as_req = 1
kerr =
pac = 0x0
pac_data = {magic = 16, length = 0, data = 0x6e00000002 }
ipactx =
with_pac = true
with_pad = false
make_ad =
result = -344
client_entry = 0x5610b70590d0
is_equal =
force_reinit_mspac =

9 0x00005610b4e56e71 in fetch_kdb_authdata (req=0x5610b701dae0, req=0x5610b701dae0, auth_indicators=0x5610b6f460d0, enc_tkt_reply=0x5610b6f45e68, enc_tkt_req=0x0, ad_info=0x0, altcprinc=0x0, local_tgt_key=0x5610b6f45f40, header_k

ey=0x0, server_key=0x5610b6f45f58, client_key=0x5610b6f45f70, local_tgt=0x5610b7151eb0, header_server=0x0, server=0x5610b7151eb0, client=0x5610b70590d0, flags=112, context=0x5610b6fb8cc0) at kdc_authdata.c:366
ret =
tgt_authdata =
db_authdata = 0x0
tgs_req =
actual_client =
ret =
tgt_authdata =
db_authdata =
tgs_req =
actual_client =
PRETTY_FUNCTION = "fetch_kdb_authdata"

10 handle_authdata (context=0x5610b6fb8cc0, flags=112, client=0x5610b70590d0, server=0x5610b7151eb0, subject_server=subject_server@entry=0x0, local_tgt=0x5610b7151eb0, local_tgt_key=0x5610b6f45f40, client_key=0x5610b6f45f70,

server_key=0x5610b6f45f58, subject_key=0x0, req_pkt=0x5610b6ef30b0, req=0x5610b701dae0, altcprinc=0x0, ad_info=0x0, enc_tkt_req=0x0, auth_indicators=0x5610b6f460d0, enc_tkt_reply=0x5610b6f45e68) at kdc_authdata.c:854
    h = <optimized out>
    ret = <optimized out>
    i = <optimized out>

11 0x00005610b4e48084 in finish_process_as_req (state=0x5610b6f45e30, errcode=) at do_as_req.c:284

    server_key = 0x5610b6fb2780
    as_encrypting_key = 0x0
    response = 0x0
    emsg = 0x0
    did_log = 0
    oldrespond = 0x5610b4e472a0 <finish_dispatch_cache>
    oldarg = 0x5610b6ee0720
    kdc_active_realm = 0x5610b6ef5b50
    au_state = 0x5610b7037d00
    __PRETTY_FUNCTION__ = "finish_process_as_req"

12 0x00005610b4e51556 in finish_check_padata (state=0x5610b7537730, code=) at kdc_preauth.c:1197

    respond = 0x5610b4e48440 <finish_preauth>
    arg = 0x5610b6f45e30

13 0x00005610b4e53e74 in enc_ts_verify (context=0x5610b6fb8cc0, req_pkt=, request=, enc_tkt_reply=0x5610b6f45e68, pa=, cb=, rock=0x5610b6f45fb8, moddata=0x0,

respond=0x5610b4e520d0 <finish_verify_padata>, arg=0x5610b7537730) at kdc_preauth_encts.c:118
    pa_enc = 0x5610b717fb20
    retval = 0
    scratch = {magic = -384887408, length = 66,
      data = 0x5610b6fcde30 "0@\240\003\002\001\022\242\071\004\067\020ϒ_d\342\223q,L*D\371\a\226Z\365]\316\347\003\260\341\244i¨Q\324q\203\220\\Q\033ڿp\215\244\031qME\031\nM\372\071A\327(B\367G"}
    enc_ts_data = {magic = -1225028416, length = 27, data = 0x0}
    enc_data = 0x5610b70591b0
    key = {magic = -1760647421, enctype = 18, length = 0, contents = 0x0}
    client_key = 0x5610b7058d20
    start = 1

14 0x00005610b4e5204f in next_padata (state=) at kdc_preauth.c:1308

    __PRETTY_FUNCTION__ = "next_padata"

15 0x00005610b4e48f77 in process_as_req (request=, req_pkt=req_pkt@entry=0x5610b6ef30b0, local_addr=local_addr@entry=0x5610b6ef3098, remote_addr=remote_addr@entry=0x5610b704f650, kdc_active_realm=0x5610b6ef5b50,

--Type for more, q to quit, c to continue without paging--
vctx=vctx@entry=0x5610b6f14f30, respond=0x5610b4e472a0 , arg=0x5610b6ee0720) at do_as_req.c:797
errcode =
s_flags =
encoded_req_body = {magic = 421, length = 156, data = 0x5610b776e701 "0\201\231\240\a\003\005"}
useenctype =
state = 0x5610b6f45e30
au_state = 0x5610b7037d00

16 0x00005610b4e475b8 in dispatch (cb=0x5610b5065140 , local_addr=local_addr@entry=0x5610b6ef3098, remote_addr=remote_addr@entry=0x5610b704f650, pkt=pkt@entry=0x5610b6ef30b0, is_tcp=is_tcp@entry=1,

vctx=vctx@entry=0x5610b6f14f30, respond=0x5610b4e5a850 <process_tcp_response>, arg=0x5610b6ef3000) at dispatch.c:201
    retval = 0
    req = 0x5610b701dae0
    response = 0x0
    state = 0x5610b6ee0720
    handle = <optimized out>
    kdc_err_context = 0x5610b6fb8cc0

17 0x00005610b4e5cf6d in process_tcp_connection_read (ctx=0x5610b6f14f30, ev=) at net-server.c:1359

    local_saddrlen = 16
    state = 0x5610b6ef3000
    conn = 0x5610b704f560
    nread = <optimized out>
    len = 425

18 0x00007f96e90f55ec in verto_fire () from /lib64/libverto.so.1

No symbol table info available.

19 0x00007f96d2544ff1 in event_process_active_single_queue () from /lib64/libevent-2.1.so.6

No symbol table info available.

20 0x00007f96d2545787 in event_base_loop () from /lib64/libevent-2.1.so.6

No symbol table info available.

21 0x00005610b4e462e9 in main (argc=5, argv=0x7ffc3830f0f8) at main.c:1064

    retval = 0
    kcontext = 0x5610b6e6f610
    realm = <optimized out>
    ctx = 0x5610b6f14f30
    tcp_listen_backlog = 5
    i = <optimized out>

This is duplicate of https://pagure.io/freeipa/issue/9251
I am closing this one to keep discussion in one place.

Metadata Update from @abbra:
- Issue close_status updated to: duplicate
- Issue status updated to: Closed (was: Open)

I have another core dump. Where would you like that sent?

On Sat, Oct 1, 2022, 11:13 PM Alexander Bokovoy pagure@pagure.io wrote:

abbra added a new comment to an issue you are following:
This is duplicate of https://pagure.io/freeipa/issue/9251 I am closing this one to keep discussion in one place.

To reply, visit the link below or just reply to this email
https://pagure.io/freeipa/issue/9253

please send it abokovoy@ redhat.com.
Metadata
None
None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.