389-ds-base

#51262 performance search rate: nagle triggers high rate of setsocketopt
Closed: wontfix by spichugi. Opened by tbordaz.

Closed: wontfix
Reopen Issue

Issue Description

nagle is enabled by default. To prevent returned search entries to be stuck, before dispatching a search, the nagle is overwritten by TCP_CORK. It can give a network benefit in term of transmitted overhead, but the cost is a bug use of systemcall setsocketopt.

Switching on/off nagle I did not see any throughput change but a benefit in terms of syscall

tracing: perf trace -t <worker> -s
NAGLE: off
 ns-slapd (108026), 1527476 events, 100.0%
   syscall            calls    total       min       avg       max      stddev
                               (msec)    (msec)    (msec)    (msec)        (%)
   --------------- -------- --------- --------- --------- ---------     ------
   futex             328122  3059.967     0.000     0.009   360.607     19.06%
   poll              124478   854.697     0.002     0.007   100.142     20.29%
   sendto            124149   785.173     0.002     0.006     0.041      0.11%
   write             123627   483.753     0.002     0.004     0.080      0.10%
   recvfrom           62295   245.209     0.002     0.004     0.040      0.12%
   madvise                9     0.211     0.016     0.023     0.042     12.46%
   sched_yield           34     0.146     0.004     0.004     0.005      1.32%
NAGLE: off
 ns-slapd (108026), 1784094 events, 100.0%
   syscall            calls    total       min       avg       max      stddev
                               (msec)    (msec)    (msec)    (msec)        (%)
   --------------- -------- --------- --------- --------- ---------     ------
   futex             328704  3054.546     0.000     0.009   346.687     17.23%
   setsockopt        124702   656.760     0.001     0.005     0.034      0.14%
   poll              125437   655.709     0.002     0.005   100.141     15.28%
   sendto            124501   464.307     0.001     0.004     0.039      0.14%
   write             125026   462.438     0.001     0.004     0.031      0.10%
   recvfrom           62799   248.383     0.002     0.004     0.030      0.11%
   madvise                3     0.066     0.014     0.022     0.034     26.97%
   sched_yield           10     0.043     0.004     0.004     0.004      1.69%

Package Version and Platform

Since 1.2.x

Steps to reproduce

  1. run searchrate with nsslapd-nagle:on / off and monitor the syscall of a worker

Actual results

by default high rate of setsockopt

Expected results

by default low rate of setsockopt

Thanks for this @tbordaz. I did a bit of research to refresh my memory, and this article came up: https://baus.net/on-tcp_cork/

It appears that when nagle's algorithm is enabled, you need to send a TCP_CORK to signal the end of the write buffer so that a partial frame can be sent without delaying forever.

More over, I think that it needs to be understood how nagles algorithm may affect client latency here. For a stream focused application like LDAP, and for the type of data we send, it would seem that nagle's algorithm is not effective for us. Nagle exists to limit sending many partial frames, rather that filling a buffer to allow writing up to MSS. But when we send data, we generally have complete messages to write, rather than say telnet or ssh sending single characters at a time.

So I'd be happy to disable nagle by default here as I do not see how it benefits a stream oriented protocol like LDAP.

The article in question does also point out that a potentially good alternative is to use writev() instead of nagle+ tcp_cork, so I think this could be a future improvement too.

Metadata Update from @firstyear:
- Custom field origin adjusted to None
- Custom field reviewstatus adjusted to None

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/4315

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @spichugi:
- Issue close_status updated to: wontfix
- Issue status updated to: Closed (was: Open)
Metadata
None
None
None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.