Description: This patch uses "local thread storage" to keep track of the connection and op id numbers for internal operations. This makes it easier to foloow the access logs and know what connection is actually doing what.
It also maintains its own operation id count for each internal operation. This adds complexity once internal ops start nesting (plugins calling plugins). To handle this two LTS variables were added to keep track of the nested level or count, and one to know once a series of nested operations are finally unnested/complete. This is needed to maintain the correct internal operation id counter correct and in sequence.
https://pagure.io/389-ds-base/issue/49029
Reviewed by: ?
Not sure to understand, why we need this second increase of count_ptr. Will it jump the internal op counter ?
Not sure if was discussed elsewhere. I have a slight preference to have both connid/op_id either in '( )' or without. Here we will have connid in parentheses and op_id not
I had another suggestion, was just testing it, will update the ticket soon
I'm fine changing the log format - that easy :-) @lkrispen has some ideas he is working on too
I tried to explain this in the comments, but it is confusing. So once we get nested (plugins calling plugin that all do logging) the internal op id basically gets out of sync and we have to keep it inline so the logging is correct. Once we are out of the nested calls, and back to the first internal op that started the whole nested loop, the op id is behind by 1. So in that particular case we have to increase it a second time for the next internal op - this is actually what caused me so much pain to fix and why I needed the SLAPI_TD_OP_NESTED thread data variable.
I hope that helps explain it, but feel free to ask more questions.
@mreynolds thanks for the explanation. I understand the difficulty with nested operation. I agree with your patch and wait for format agreement.
Just a remark would it help/simplify the nested problem to log the nested level as well. for example
conn=Internal(3) op=66(1) lvl=2 MOD dn="dc=example,dc=com" conn=Internal(3) op=1(1) lvl=3 SRCH base="cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config" scope=0 filter="objectclass=*" attrs="nsslapd-backend nsslapd-distribution-plugin conn=Internal(3) op=1(1) lvl=3 RESULT err=0 tag=48 nentries=1 etime=0.0000165786 conn=Internal(3) op=66(1) lvl=2 RESULT err=0 tag=48 nentries=0 etime=0.0014946418 conn=Internal(3) op=67(1) lvl=2 SRCH base="cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config" scope=0 filter="objectclass=*" attrs="nsslapd-backend nsslapd-distribution-plugin conn=Internal(3) op=67(1) lvl=2 RESULT err=0 tag=48 nentries=1 etime=0.0000165786
conn=Internal(3) op=1(1) lvl=3 RESULT err=0 tag=48 nentries=1 etime=0.0000165786
but you then no longer see that it is part of op=66, the lvl alone is, in my opinion not so helpful.
I think it is already a huge improvement, and we can improve further eg if we will amke logconv deal with it
rebased onto 9bc85d80b360360803f9ac22c0f0e3fad231ea7d
rebased onto c1f217edeb72ebb2640b7fa01a6aa75a8e0bf04b
logconv.pl has been amended, please review final patch. Thanks!
rebased onto 90bf17995505dc1534827e50bda77358c0d5be28
Pull-Request has been merged by mreynolds
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This pull request has been cloned to Github as issue and is available here: - https://github.com/389ds/389-ds-base/issues/2954
If you want to continue to work on the PR, please navigate to the github issue, download the patch from the attachments and file a new pull request.
Thank you for understanding. We apologize for all inconvenience.
Pull-Request has been closed by spichugi
Description: This patch uses "local thread storage" to keep track of
the connection and op id numbers for internal operations.
This makes it easier to foloow the access logs and know
what connection is actually doing what.
https://pagure.io/389-ds-base/issue/49029
Reviewed by: ?