bind-dyndb-ldap design overview

bind-dyndb-ldap is LDAP back-end plugin for BIND that provides an LDAP database and run-time reconfiguration.

The plugin integrates very deeply with BIND. The implementation is based on library which is dynamically loaded into main BIND process during startup.

After initialization, BIND is calling the plugin APIs and the plugin is calling BIND APIs as necessary. Some actions require multiple nested calls between BIND and the plugin using different APIs.

Preliminary reading

The plugin acts as bridge between two worlds: DNS implemented by BIND vs. LDAP implemented by OpenLDAP libraries. For understanding how the plugin works it is crucial to first understand BIND internals and how to use LDAP using OpenLDAP libraries.

• The tight integration with BIND requires understanding of BIND internals. Some of these are covered in directory doc/design in BIND source tree.
  • Even if you decide to skip everything else, read at least doc/design/tasks. BIND has own implementation of ‘tasks’ which is heavily used everywhere.
  • Description of zones and databases is going to be very important as well.
• The plugin reads data from LDAP server using so-called SyncRepl protocol specified by RFC 4533. Understanding this is crucial.
  • SyncRepl interface in OpenLDAP libraries is described in manual page ldap_sync.

BIND interaction overview

During BIND startup, BIND parses named.conf and loads library containing the plugin. The very first interaction with BIND looks like this:

Startup

1. BIND calls dyndb_init() in the plugin library. This creates new internal ldap_instance_t of LDAP database.
2. The plugin calls dns_db_register() in BIND to add name of the LDAP instance into list of registered databases.
3. The plugin starts ldap_syncrepl_watcher() thread. There is a gotcha: This is an ordinary pthread and not BIND task!
   • Keep in mind that the plug is in the same address space as BIND - i.e. inside BIND process.

SyncRepl initialization

Following operations are done by the syncrepl thread in cycle:

Beware: Operations done by ldap_syncrepl_watcher() thread in the plugin and BIND are running in parallel and need to be carefully synchronized. Be very careful! ldap_syncrepl_watcher() thread is totally separate and does not respect BIND internal threading/task model.

4. The syncrepl thread calls ldap_sync_init() to start LDAP SyncRepl sessions (using OpenLDAP library).
5. OpenLDAP library asynchronously calls callbacks ldap_sync_search_entry, ldap_sync_search_reference, ldap_sync_intermediate, ldap_sync_search_result in the plugin code. From BIND’s point of view these calls are completely asynchronous.
6. ldap_sync_search_entry callback is called for each LDAP entry.
   • In our example the entry we are processing right now is DNS master zone (idnsZone entry). The callback transforms LDAP entry into BIND internal event structure. The event is sent to event queue to one of BIND internal tasks. From this point the event follows BIND synchronization rules.

Initial data processing

The event queue is managed by BIND. Each event is processed by task determined by event sender. Different tasks are running in parallel, each task processing its event queue. At this point BIND synchronization primitives and task model is in play again. An example of event processing follows:

7. Event generated from LDAP master zone entry calls several internal functions in the plugin:

• update_zone() to process the event
  • ldap_parse_master_zoneentry() to process DNS master zone update
    • create_zone() to create a new DNS zone in BIND
  • zone_sync_apex() to store data from LDAP into internal database used by the plugin

At this point the plugin is creating new DNS zone inside BIND. To do this, the plugin is calling many dns_zone*() functions from BIND API. The new zone is associated with LDAP plugin using dns_zone_setdbtype() function. (This would not be possible without calling dns_db_register() function during plugin initialization.)

Please note that at this point the new DNS zone is not associated with an BIND database yet. The zone data are stored internally by the plugin (see plugin function zr_add_zone()) but not yet exposed to BIND. The association is done later in a separate event.

8. When all LDAP entries from LDAP are read, the SyncRepl protocol indicates end of refresh stage by Sync Info Message with option refreshDone = TRUE. Based on this message, OpenLDAP libraries call ldap_sync_intermediate() callback in syncrepl thread.

Zone publication

9. ldap_sync_intermediate() in the plugin calls sync_barrier_wait() to finalize event processing from refresh stage. When the synchronization is done, chain of calls following from sync_finishev_create() function calls plugin internal function activate_zones().
10. Plugin internal function activate_zones() uses BIND APIs dns_view_*() and dns_zone_setview() to publish the DNS zone over DNS protocol. After publication, the plugin calls dns_zone_*() APIs to load zone data into BIND. BIND function dns_zone_load() internally calls BIND function dns_db_create() which in turn calls plugin function ldapdb_associate(). This way the plugin-internal zone database gets exposed to BIND and zone data become accessible over DNS.

Normal DNS operation

At this point DNS zone is fully functional and DNS operations can be done on it.

11. DNS operations (DNS query, update, zone transfer etc.) are received by BIND and go through multiple layers of indirection. When an request is received, BIND uses following APIs:
    • view dns_view_*()
    • zone table dns_zt_*(): determine what zone is affected by the operation. View and zone table APIs return DNS zone structure.
    • zone dns_zone_*(): DNS zone structure internally contains metadata and DNS database structure.
    • database dns_db_*(): Database API is used to manipulate the data in the zone database.
12. The database structure works as layer of indirection for dns_db_*() APIs. DB calls are translated to function calls according to dns_dbmethods_t ldapdb_methods structure in tree/src/ldap_driver.c. For example, dns_db_addrdataset() API function is redirected to plugin internal function addrdataset().
13. The plugin internal function addrdataset() modifies in-memory database and also uses OpenLDAP libraries to make the same modification in LDAP.

LDAP update processing

Keep in mind that modifications to LDAP are watched by the plugin using SyncRepl persist stage. Even changes done by the plugin itself generate change notifications which will be received by the plugin again with some delay!

14. OpenLDAP libraries translate LDAP change notification into ldap_sync_search_entry() callback. This in turn generates event and triggers usual event processing described above. Keep in mind that these LDAP events are generated asynchronously so there is risk of race conditions with other events (e.g. DNS/timed/administrative operations).
15. The generated event might call BIND APIs as necessary. Keep in mind that some of the events might be actually no-op because the change was already done in reaction to DNS operation and the generated LDAP event records the same change. For example:
    • DNS update adds A record with value 192.0.2.1
    • The change is written to in-memory DNS database right away.
    • At the same time the change will be written to LDAP.
    • After some delay, the LDAP server will generate change notification which will trigger entry processing again. This time there will be no changes to the entry because everything was already processed.
    • This assumes that there were no other changes to the entry in meantime. We will get race condition if there were some other changes.

Shutdown

16. On shutdown, BIND calls dyndb_destroy() function in the plugin. It stops the syncrepl thread, de-registers plugin database type from the database list, and destroys ldap_instance_t and other structures.
    • Keep in mind that calling dyndb_destroy() function does not affect pending events, i.e. already sent events waiting in the event queue of various BIND tasks.
    • These pending events might be processed eventually. In general this leads to problems when an event references already-freed structure. Proper reference counting would solve this.