freeipa

#8493 Synchronize index LDIF and index update files
Closed: fixed by cheimes. Opened by cheimes.

Closed: fixed
Reopen Issue

Request for enhancement

FreeIPA has two files that deal with database indices

  • install/share/indices.ldif contains initial index definitions. The file is used very early in server and replica installer, shortly after DS instance is created
  • install/updates/20-indices.update contains index updates. The file is used at the very end of of server and replica installation process as well as by the server updater.

Any index that is defined only in 20-indices.update or modified in 20-indices.update impacts installation process in two ways

  • Since the update indexes are created very late, a missing index may negatively impact any LDAP query that would benefit from the index otherwise.
  • 389-DS has to perform a full DB read to create the new indexes or update modified indexes. This can take a long time.

I assume that it would be more efficient and faster to keep the files in sync so that 20-indices.update does not create any index tasks for ipa-server-install and ipa-replica-install. This may reduce installation time of a large replica, too.

https://pagure.io/freeipa/issue/8491

Perhaps it makes sense to get rid of indices.ldif and define the indices just in the update file? Update LDIF files works as soon as root autobind is configured. PR https://github.com/freeipa/freeipa/pull/5102 adds a convenient wrapper to apply update LDIF files.

Script to dump all indices in ULDIF format:

#!/usr/bin/python3
import operator
from ipapython.dn import DN
from ipalib import api
INDEX_DN = DN("cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config")
INDEX_FILTER = "(objectClass=nsIndex)"
DEFAULT_ATTRS = ["cn", "objectClass", "nsSystemIndex"]
ADD_ATTRS = ["nsIndexType" "nsMatchingRule"]
def main():
    api.bootstrap(in_server=True)
    api.finalize()
    ldap2 = api.Backend.ldap2
    ldap2.connect()
    entries, truncated = ldap2.find_entries(
        filter=INDEX_FILTER, base_dn=INDEX_DN, attrs_list=["*"]
    )
    for entry in sorted(entries, key=operator.attrgetter("dn")):
        if entry.single_value['nsSystemIndex'] == 'true':
            continue
        print(f"dn: {entry.dn}")
        for name in DEFAULT_ATTRS:
            for value in sorted(entry.get(name, ())):
                print(f"default: {name}: {value}")
        for name in ADD_ATTRS:
            for value in sorted(entry.get(name, ())):
                print(f"only: {name}: {value}")
        if entry.single_value["cn"] == "ipaAnchorUUID":
            print("# see https://pagure.io/freeipa/issue/6975")
            print("remove: cn: ipaOriginalUid")
        print()
if __name__ == "__main__":
    main()
Edited by cheimes

Initial LDIF + update LDIF define a total of 62 unique indices. None of the indices is flagged as system index.

$ grep -h 'cn=index' install/updates/20-indices.update install/updates/20-winsync_index.update install/updates/20-idoverride_index.update install/share/indices.ldif | sort | uniq 
dn: cn=accessRuleType,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=altSecurityIdentities,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=automountkey,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=automountMapName,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=carLicense,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=description,cn=index,cn=userroot,cn=ldbm database,cn=plugins,cn=config
dn: cn=displayname,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=fqdn,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=gidnumber,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=hostCategory,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=idnsName,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaallowedtarget,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaAnchorUUID,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaassignedidview,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaCertmapData,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaConfigString,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaEnabledFlag,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaKrbAuthzData,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipakrbprincipalalias,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipalocation,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaMemberCa,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaMemberCertProfile,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaNTSecurityIdentifier,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaNTTrustPartner,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipaOriginalUid,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipasudorunas,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipasudorunasgroup,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipatokenradiusconfiglink,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipauniqueid,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ipServicePort,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=krbCanonicalName,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=krbPasswordExpiration,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=krbPrincipalName,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=l,cn=index,cn=userroot,cn=ldbm database,cn=plugins,cn=config
dn: cn=macAddress,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=managedby,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=manager,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=memberallowcmd,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=member,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=memberdenycmd,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=memberHost,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=memberManager,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=memberof,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=memberservice,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=memberuid,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=memberUser,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=nsHardwarePlatform,cn=index,cn=userroot,cn=ldbm database,cn=plugins,cn=config
dn: cn=nsHostLocation,cn=index,cn=userroot,cn=ldbm database,cn=plugins,cn=config
dn: cn=nsOsVersion,cn=index,cn=userroot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ntUniqueId,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ntUserDomainId,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=ou,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=owner,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=secretary,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=seealso,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=serverhostname,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=sourcehost,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=title,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=uid,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=uidnumber,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=uniquemember,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
dn: cn=userCertificate,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config

Metadata Update from @cheimes:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/5157

master:

  • e46c3792f3419f807864adc1cb36887fe6c9e36c Use single update LDIF for indices
  • 9f0ec27e9f13ed40b8e58162d99bf9b0e8b4afd5 Add more indices

ipa-4-8:

  • 1d7b108bf9e3810079464171deef21659b6a4aee Add ldap_update() helper to service class
  • d74bf64701b3c6489af90f0878255950dc335337 Simplify LDAPUpdater
  • ff3bc4edb6e3f963b43d8db8f4e94ac54a415ff4 Use single update LDIF for indices
  • b4b834f8512ada4d85d52f11ccc247203ffc41ac Add more indices

Metadata Update from @cheimes:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata Update from @abbra:
- Custom field changelog adjusted to Configuration of LDAP indices was moved into a single place. New indices were added to attributes related to trusted domains operations. Performance improvement is expected for Kerberos service tickets requested by users from trusted Active Directory domains.

Indices backported to ipa-4-6:
ipa-4-6:

  • 2c802ab10479342e6d0e7e6d1589f0d31a608c1b Add more indices
Metadata
None

performance

None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.