bind-dyndb-ldap design overview of LDAP synchronization

bind-dyndb-ldap version 11 is doing run-time synchronization of resource records, zones, and configuration between BIND and LDAP server. Some of the changes done by the plugin are not directly supported by BIND interfaces used so care needs to be taken to ensure proper locking and synchronization of events inside BIND.

To understand this area it is necessary to understand SyncRepl protocol, BIND internals, and the locking layer the plugin adds on top of all this.

Preliminary reading

• BIND9/Design/SyncRepl
• BIND9/Design/Overview
• BIND9/Design/LockingOverview

Synchronization direction

• Resource records are synchronized bi-directionally (e.g. when processing DNS updates).
• Zones and configuration only uni-directionally from LDAP into BIND.

Synchronization states

SyncRepl protocol itself supports two different modes: refreshOnly and refreshAndPersist. The refreshAndPersist mode is further divided to refresh and persist stages:

• During refresh stage all present and/or deleted entries are sent from LDAP server to the plugin.
• In persist stage, LDAP entries which were changed are incrementally sent to the plugin.

Because of interdependence between LDAP entries representing DNS data, the plugin needs to load configuration and all the data first. DNS zone loading is done only after initial synchronization is finished.

The plugin internally tracks state of synchronization in struct sync_ctx, which uses enum sync_state to as synchronization state machine.

The plugin version 11 is using refreshOnly mode for configuration synchronization during startup. When configuration is synchronized, the plugin uses refreshAndPersist mode to pull all data from LDAP and to keep them synchronized on the fly.

ldap_syncrepl_watcher() thread works like this:

1. Start in state configinit:

• Start SyncRepl session in refreshOnly mode to read configuration objects
• (object classes idnsConfigObject and idnsServerConfigObject)
• For each received LDAP entry:
  • Execute callback ldap_sync_search_entry() and transform the entry to an BIND event.
  • These events are processed asynchronously to ldap_syncrepl_watcher() thread.
  • Store list of tasks used to process the events in struct sync_ctx.

2. When all entries were received, execute callback ldap_sync_search_result() and enter state configbarrier:

• Send LDAPDB_EVENT_SYNCREPL_BARRIER event to each task which participates in event processing from the previous step.
• Wait until all LDAPDB_EVENT_SYNCREPL_BARRIER events are processed.
  • Tasks are executing events in the order events where sent. This implies that when all barrier events are processed, all the events generated before the barrier were processed as well.

3. When all barrier events were processed, enter state datainit:

• Bump LDAP entry generation ID (see metaLDAP chapter below)
• Start SyncRepl session in refreshAndPersisnt mode to read all DNS objects from LDAP
• For each received LDAP entry use the same process as in state configinit.
• During asynchronous processing of data events skip following operations:
  • zone loading (do not call dns_zone_load()): the zone data are not yet completely available so zone loading would fail anyway
  • DNS journal generation: DNS journal needs to be deleted after plugin restart because we do not implement logic to detect what changed when the plugin was not running
  • SOA serial auto-incrementation for records outside of zone apex: it does not make sense to increment SOA serial for each entry we read from LDAP

4. When all entries from refresh stage were received, execute callback ldap_sync_intermediate() and enter state databarrier:

• Send LDAPDB_EVENT_SYNCREPL_BARRIER event to each task which participates in event processing from the previous step. Same as in step configbarrier.
• Remove all entries which exist in metaLDAP but do not exist in LDAP anymore. (See chapter metaLDAP below.)
  • Shoudn’t it be the other way around, i.e. remove entries first and **then* execute the barrier logic?*

5. When all barrier events were processed, enter state finished:

• Publish and activate DNS zones and load them.
• Configure zone forwarding.
• At this point the plugin enters persist stage of SyncRepl session. The plugin receives change notification about each modified LADP entry. The DNS journal is updated for each modification and SOA serial is incremented.

LDAP metadata (metaLDAP)

Real implementation of synchronization machinery in the plugin needs to store metadata about LDAP entries. In certain cases the SyncRepl protocol can use LDAP entry UUID RFC4530 instead of sending full LDAP entry or its DN. In such cases the plugin has to map LDAP entry UUID to an internal object which needs to be modified (e.g. deleted). See Design/MetaDB for full metaLDAP design.

In plugin version 11, the SyncRepl callback ldap_sync_search_entry() is responsible for keeping internal metadata database. It keeps mapping from LDAP entry UUID to:

• generation ID (monotonic integer)
• object class
• zone and record DNS names (if applicable to particular object class)

When entering datainit state, the plugin increments the generation ID (this includes reconnections to LDAP from the same plugin instance). For all LDAP entries processed by the plugin, the generation ID is stored in the metaLDAP database.

At the end of refresh stage, the ldap_sync_intermediate() callback walks through all entries in metaLDAP and triggers object removal for all entries which do have generation ID smaller than current generation ID. Such objects were loaded sometime in the past but are not present in LDAP anymore.