#8032 API call group-find with allfields very slow
Closed: insufficientinfo by fcami. Opened by aethylred.

Request for enhancement

As an administrator using a script to check all group memberships against our Customer Management System I want to list all groups with all fields and store it in memory.

As an IPA user running ipa group-find --all to i want to get information about all the groups.

Issue

The command & API call works as expected but is very slow, doing a group-find API call takes minutes to respond with the group list. We have about 2,700 groups, and some of these are nested groups, and some of the nested groups have large numbers of member groups. Some nested groups also have nested groups as members.

I can't give too much information about the specific information about our group structures and content for privacy and confidentially reasons.

The response time ranges from about 1:30 to nearly 6 minutes!

My script logs the duration of the API response:

1 2019-08-07T11:13:46.538610+12:00 localhost testscript 2040 - - Retrieved 2670 IPA groups from ipa.example.com in 0:01:37.341919

Is this a performance issue, or do my IPA servers need more resources (currently, VMware VMs with 4vCPU and 16GB memory)

From the command line we see similar:

$ ipa group-find --all
... actual output omitted ...
-------------------------------
Number of entries returned 2670
-------------------------------
real    1m38.033s
user    0m2.187s
sys     0m0.242s

Steps to Reproduce

  1. Populate your IPA directory with many nested groups with nested groups
  2. Call the group-find API request with all fields (command line --all equivalent)
  3. Wait

Actual behavior

It takes a long time to respond

Expected behavior

I expect the response to be quicker

Version/Release/Distribution

  $ rpm -q freeipa-server freeipa-client ipa-server ipa-client 389-ds-base pki-ca krb5-serverpackage freeipa-server is not installed
package freeipa-client is not installed
ipa-server-4.6.4-10.el7_6.3.x86_64
ipa-client-4.6.4-10.el7_6.3.x86_64
389-ds-base-1.3.8.4-23.el7_6.x86_64
pki-ca-10.5.9-13.el7_6.noarch
krb5-server-1.15.1-37.el7_6.x86_64

Additional info:

IPA set up in a 3 host replication running on RHEL7.6


Oops, sorry, the VMs are 8 vCPU and 32GB memory.

Can you check the 389-ds logs to see if any of the related searches are unindexed

nested groups require a lot of dereferencing, both for the nesting lookups and also to convert the DN into the object name, but 6 minutes sounds a bit excessive.

Can you provide some rough details on what large number of members means and how deeply nested the groups are?

Have you done any research into which particular group seems to be take the most time? Perhaps we can try to use that as a starting point for determining the bottleneck.

Metadata Update from @pcech:
- Issue tagged with: Falcon

Closing as we don't have the required information ( requested above by @rcritten ) to reproduce.

Metadata Update from @fcami:
- Issue close_status updated to: insufficientinfo
- Issue status updated to: Closed (was: Open)

Metadata