How to setup refresh-only replicated OpenLDAP server using syncreplica

Linux

How to setup refresh-only replicated OpenLDAP server using syncreplica

By novicejava1 0
openldap_syncreplication_setup

Here in this article we will try to setup a replicated openldap directory service using the LDAP Sync-based replication, called syncrepl. We are going to implement this service using a very basic producer consumer model configuration.

Test Environment

Master LDAP server – ubscratch.stack.com
Replica LDAP server – ubreplica.stack.com

Producer – Consumer Architecture

If you are interested in watching the video. Here is the YouTube video on the same step by step procedure outlined below.

Procedure

Step1: Download and Extract OpenLDAP on Master server

As a first we are going to download and extract the OpenLDAP source package on master server as shown below.

ubadmin@ubscratch:~$ sudo apt-get install make gcc
ubadmin@ubscratch:~$ wget https://www.openldap.org/software/download/OpenLDAP/openldap-release/openldap-2.6.7.tgz
ubadmin@ubscratch:~$ tar -xzvf openldap-2.6.7.tgz

Step2: Build and Install OpenLDAP on Master server

Once the OpenLDAP source package is extracted it can be compiled and installed using the below procedure. Please note as we want to implement the producer-consumer model. We will need to update the configuration script to enable “–enable-syncprov” overlay feature as shown below.

ubadmin@ubscratch:~/openldap-2.6.7$ ./configure --enable-syncprov
ubadmin@ubscratch:~/openldap-2.6.7$ make depend
ubadmin@ubscratch:~/openldap-2.6.7$ make
ubadmin@ubscratch:~/openldap-2.6.7$ sudo make install

Step3: Update slapd.conf on Master server

Here we are going to take the default slapd configuration file available at “/usr/local/etc/openldap/slapd.conf.default” and update it as shown below in the file “slapd_syncreplica.conf”.

The functionality of the master server is implemented in an overlay called syncprov (which is short for Synchronization Provider). Getting SyncRepl running requires configuration on both the master and the shadow server. The configuration directives for both are added to the backend sections of the slapd.conf files

The entryCSN attribute, and the entryUUID attribute are the two important attributes which are frequently accessed by SyncRepl process for equality check so we need to index them.
The entryCSN attribute is used to store a Change Sequence Number (CSN) in each record and entryUUID attribute contains a (universally) unique identifier for that entry.

Next we need to load and configure the syncprov overlay. The two configuration directives generally used by this overlay are as detailed below.

  • syncprov-checkpoint – After a write operation has succeeded, write the contextCSN to the underlying database if write operations or more than time have passed since the last checkpoint. Checkpointing is disabled by default
  • syncprov-sessionlog – Configures an in-memory session log for recording information about write operations made on the database
ubadmin@ubscratch:~/openldap-2.6.7$ cat ~/testldapconfig/syncreplica/slapd_syncreplica.conf
#
# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#
include		/usr/local/etc/openldap/schema/core.schema
include 	/usr/local/etc/openldap/schema/cosine.schema
include 	/usr/local/etc/openldap/schema/inetorgperson.schema

# Define global ACLs to disable default read access.

# Do not enable referrals until AFTER you have a working directory
# service AND an understanding of referrals.
#referral	ldap://root.openldap.org

pidfile		/usr/local/var/run/slapd.pid
argsfile	/usr/local/var/run/slapd.args

# Load dynamic backend modules:
modulepath	/usr/local/libexec/openldap
moduleload	back_mdb.la
# moduleload	back_ldap.la

# Sample security restrictions
#	Require integrity protection (prevent hijacking)
#	Require 112-bit (3DES or better) encryption for updates
#	Require 63-bit encryption for simple bind
# security ssf=1 update_ssf=112 simple_bind=64

# Sample access control policy:
#	Root DSE: allow anyone to read it
#	Subschema (sub)entry DSE: allow anyone to read it
#	Other DSEs:
#		Allow self write access
#		Allow authenticated users read access
#		Allow anonymous users to authenticate
#	Directives needed to implement policy:
# access to dn.base="" by * read
# access to dn.base="cn=Subschema" by * read
# access to *
#	by self write
#	by users read
#	by anonymous auth
#
# if no access controls are present, the default policy
# allows anyone and everyone to read anything but restricts
# updates to rootdn.  (e.g., "access to * by * read")
#
# rootdn can always read and write EVERYTHING!

#######################################################################
# config database definitions
#######################################################################
database config
# Uncomment the rootpw line to allow binding as the cn=config
# rootdn so that temporary modifications to the configuration can be made
# while slapd is running. They will not persist across a restart.
# rootpw secret

#######################################################################
# MDB database definitions
#######################################################################

database	mdb
maxsize		1073741824
suffix		"dc=stack,dc=com"
rootdn		"cn=Manager,dc=stack,dc=com"
# Cleartext passwords, especially for the rootdn, should
# be avoid.  See slappasswd(8) and slapd.conf(5) for details.
# Use of strong authentication encouraged.
rootpw		secret
# The database directory MUST exist prior to running slapd AND 
# should only be accessible by the slapd and slap tools.
# Mode 700 recommended.
directory	/usr/local/var/openldap-data-com
# Indices to maintain
index	objectClass	eq
index 	cn 		eq,sub,pres,approx
index 	uid 		eq,sub,pres
index 	sn 		eq,sub,approx
index 	member 		eq
index 	entryCSN,entryUUID eq
access to * by *

overlay syncprov
syncprov-checkpoint 50 10
syncprov-sessionlog 100

#######################################################################
# monitor database definitions
#######################################################################
database monitor

Ensure the configuration file can be parsed using slaptest

ubadmin@ubscratch:~/openldap-2.6.7$ sudo slaptest -f ~/testldapconfig/syncreplica/slapd_syncreplica.conf 
config file testing succeeded

NOTE: Above test works only if the data file already exists

Step4: Import configuration database on Master server

Now its time to load our configuration file into the slapd.d configuration database location.

ubadmin@ubscratch:~/openldap-2.6.7$ sudo /usr/local/sbin/slaptest -v -n 0 -F /usr/local/etc/slapd.d -f ~/testldapconfig/syncreplica/slapd_syncreplica.conf

Step5: Import LDIF data on Master server

Here in this step we are going to load the ldap directory tree data. Here are the details of the “data.ldif” file. Here is this ldif file we will be creating a system account with uid “syncrepl” which we will further use in the Replica server configuration to connect to the Master server.

ubadmin@ubscratch:~$ cat ~/testldapconfig/syncreplica/data.ldif
# This is the root of the directory tree
dn: dc=stack,dc=com
description: stack.com, your trusted non-existent corporation.
dc: stack
o: stack.com
objectClass: top
objectClass: dcObject
objectClass: organization

# Subtree for users
dn: ou=Users,dc=stack,dc=com
ou: Users
description: stack.com Users
objectClass: organizationalUnit

# Subtree for groups
dn: ou=Groups,dc=stack,dc=com
ou: Groups
description: stack.com Groups
objectClass: organizationalUnit

# Subtree for system accounts
dn: ou=System,dc=stack,dc=com
ou: System
description: Special accounts used by software applications
objectClass: organizationalUnit

# Barbara Jensen
dn: uid=barbara,ou=Users,dc=stack,dc=com
ou: Users
uid: Barbara
sn: Jensen
cn: Barbara Jensen
givenName: Barbara
displayName: Barbara Jensen
mail: barbara@stack.com
userPassword: barbara@1234
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson

# Matt Butcher
dn: uid=matt,ou=Users,dc=stack,dc=com
ou: Users
# Name info:
uid: Matt
cn: Matt Butcher
sn: Butcher
givenName: Matt
givenName: Matthew
displayName: Matt Butcher
# Work Info:
title: Systems Integrator
description: Systems Integration and IT for Example.Com
employeeType: Employee
departmentNumber: 001
employeeNumber: 001-08-98
mail: mbutcher@stack.com
mail: matt@stack.com
roomNumber: 301
telephoneNumber: +1 555 555 4321
mobile: +1 555 555 6789
st: Illinois
l: Chicago
street: 1234 Cicero Ave.
# Home Info:
homePhone: +1 555 555 9876
homePostalAddress: 1234 home street $ Chicago, IL $ 60699-1234
# Misc:
userPassword: matt@1234
preferredLanguage: en-us,en-gb
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson

# LDAP Admin Group:
dn: cn=LDAP Admins,ou=Groups,dc=stack,dc=com
cn: LDAP Admins
ou: Groups
description: Users who are LDAP administrators
uniqueMember: uid=barbara,dc=stack,dc=com
uniqueMember: uid=matt,dc=stack,dc=com
uniqueMember: uid=syncrepl,ou=System,dc=stack,dc=com
objectClass: groupOfUniqueNames

# Special Account for Authentication:
dn: uid=authenticate,ou=System,dc=stack,dc=com
uid: authenticate
ou: System
description: Special account for authenticating users
userPassword: secret
objectClass: account
objectClass: simpleSecurityObject

# Special Account for Authentication:
dn: uid=syncrepl,ou=System,dc=stack,dc=com
uid: syncrepl
ou: System
description: Special account for syncrepl
userPassword: secret
objectClass: account
objectClass: simpleSecurityObject

Validate the data.ldif file using dry run mode with “-u” and “-c” options

ubadmin@ubscratch:~/testldapconfig$ sudo /usr/local/sbin/slapadd -v -u -c -F /usr/local/etc/slapd.d -l ~/testldapconfig/syncreplica/data.ldif 
added: "dc=stack,dc=com" (00000000)
added: "ou=Users,dc=stack,dc=com" (00000000)
added: "ou=Groups,dc=stack,dc=com" (00000000)
added: "ou=System,dc=stack,dc=com" (00000000)
added: "uid=barbara,ou=Users,dc=stack,dc=com" (00000000)
added: "uid=matt,ou=Users,dc=stack,dc=com" (00000000)
added: "cn=LDAP Admins,ou=Groups,dc=stack,dc=com" (00000000)
added: "uid=authenticate,ou=System,dc=stack,dc=com" (00000000)
added: "uid=syncrepl,ou=System,dc=stack,dc=com" (00000000)
Closing DB...

Load the data.ldif directory tree data.

ubadmin@ubscratch:~/testldapconfig$ sudo /usr/local/sbin/slapadd -v -F /usr/local/etc/slapd.d -l ~/testldapconfig/syncreplica/data.ldif

Step5: Start LDAP service on Master Server

Now, we can start the LDAP service on master server as shown below.

ubadmin@ubreplica:~/testldapconfig/syncreplica$ sudo /usr/local/libexec/slapd -F /usr/local/etc/slapd.d

Step6: Validate LDAP service

Validate that you are able to fetch the entries with LDAP directory tree as shown below.

ubadmin@ubscratch:~/testldapconfig/syncreplica$ ldapsearch -x -D "uid=matt,ou=Users,dc=stack,dc=com" -b 'dc=stack,dc=com' '(uid=matt)' -W

With this our Master configuration is completed and the server is up and running.

Step7: Download and Extract OpenLDAP on Replica server

Here let’s download and extract the OpenLDAP package on the Replica server as shown below.

ubadmin@ubreplica:~$ sudo apt-get install make gcc
ubadmin@ubreplica:~$ wget https://www.openldap.org/software/download/OpenLDAP/openldap-release/openldap-2.6.7.tgz
ubadmin@ubreplica:~$ tar -xzvf openldap-2.6.7.tgz

Step8: Build and Install OpenLDAP on Replica server

Once extracted, we can configure, build and install the OpenLDAP package as shown below.

ubadmin@ubreplica:~/openldap-2.6.7$ ./configure
ubadmin@ubreplica:~/openldap-2.6.7$ make depend
ubadmin@ubreplica:~/openldap-2.6.7$ make
ubadmin@ubreplica:~/openldap-2.6.7$ sudo make install

Step9: Update slapd.conf file on Replica server

Here we are going to take the default slapd configuration file available at “/usr/local/etc/openldap/slapd.conf.default” and update it as shown below in the file “slapd_syncreplica.conf”.

  • All of the schemas that the master uses must be loaded on the shadow server too
  • The suffix should be the same on master and replica server
  • No need to configure rootDN password as the data will be replicated from the master to replica server
  • Ensure that objectclass, entryCSN, and entryUUID are all indexed, since those are important for SLAPD’s performance.

The Replica server acts like a special sort of LDAP client. It binds to the master server and performs LDAP operations—usually the special LDAP synchronization operation.

Configuration on Replica server has to do with providing information about how the Replica server should bind to the master and how it should perform searches.
The majority of the configuration work for implementing a shadow server is done with one slapd.conf directive: syncrepl.

  • rid – identifies the current syncrepl directive within the replication consumer site. It is a non-negative integer having no more than three decimal digits
  • provider – specifies the replication provider site containing the master content as an LDAP URI. If is not given, the standard LDAP port number (389 or 636) is used
  • type – specifies the type of replication to configure
  • interval – specifies the regular interval at which the Replica server need to pull data from Master server
  • searchbase – this indicates what the base DN for the synchronization request will be
  • binddn – used to bind to the directory server
  • credentials – credentials for the binddn user

The “updateref” is used to point the client to the correct LDAP server for any modify operations that are requested on the Replica server which is not supported.

Please go through the follownig man pages for detailed information.

ubadmin@ubreplica:~$ cat ~/testldapconfig/syncreplica/slapd_syncreplica_slave.conf 
#
# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#
include		/usr/local/etc/openldap/schema/core.schema
include 	/usr/local/etc/openldap/schema/cosine.schema
include 	/usr/local/etc/openldap/schema/inetorgperson.schema

# Define global ACLs to disable default read access.

# Do not enable referrals until AFTER you have a working directory
# service AND an understanding of referrals.
#referral	ldap://root.openldap.org

pidfile		/usr/local/var/run/slapd.pid
argsfile	/usr/local/var/run/slapd.args

# Load dynamic backend modules:
modulepath	/usr/local/libexec/openldap
moduleload	back_mdb.la
# moduleload	back_ldap.la

# Sample security restrictions
#	Require integrity protection (prevent hijacking)
#	Require 112-bit (3DES or better) encryption for updates
#	Require 63-bit encryption for simple bind
# security ssf=1 update_ssf=112 simple_bind=64

# Sample access control policy:
#	Root DSE: allow anyone to read it
#	Subschema (sub)entry DSE: allow anyone to read it
#	Other DSEs:
#		Allow self write access
#		Allow authenticated users read access
#		Allow anonymous users to authenticate
#	Directives needed to implement policy:
# access to dn.base="" by * read
# access to dn.base="cn=Subschema" by * read
# access to *
#	by self write
#	by users read
#	by anonymous auth
#
# if no access controls are present, the default policy
# allows anyone and everyone to read anything but restricts
# updates to rootdn.  (e.g., "access to * by * read")
#
# rootdn can always read and write EVERYTHING!

#######################################################################
# config database definitions
#######################################################################
database config
# Uncomment the rootpw line to allow binding as the cn=config
# rootdn so that temporary modifications to the configuration can be made
# while slapd is running. They will not persist across a restart.
# rootpw secret

#######################################################################
# MDB database definitions
#######################################################################

database	mdb
maxsize		1073741824
suffix		"dc=stack,dc=com"
rootdn		"cn=Manager,dc=stack,dc=com"
# Cleartext passwords, especially for the rootdn, should
# be avoid.  See slappasswd(8) and slapd.conf(5) for details.
# Use of strong authentication encouraged.
#rootpw		secret
# The database directory MUST exist prior to running slapd AND 
# should only be accessible by the slapd and slap tools.
# Mode 700 recommended.
directory	/usr/local/var/openldap-data-com
# Indices to maintain
index	objectClass	eq
index 	cn 		eq,sub,pres,approx
index 	uid 		eq,sub,pres
index 	sn 		eq,sub,approx
index 	member 		eq
index 	entryCSN,entryUUID eq


syncrepl rid=001
  provider=ldap://ubscratch.stack.com
  type=refreshOnly
  interval=00:00:05:00
  searchbase="dc=stack,dc=com"
  binddn="uid=syncrepl,ou=system,dc=stack,dc=com"
  credentials=secret

updateref ldap://ubscratch.stack.com

#######################################################################
# monitor database definitions
#######################################################################
database monitor

Ensure that the following configuration database directory “/usr/local/etc/slapd.d” and MDB database directory “/usr/local/var/openldap-data-com” exists and have the required permissions

ubadmin@ubreplica:~/testldapconfig/syncreplica$ sudo mkdir -p /usr/local/etc/slapd.d
ubadmin@ubreplica:~/testldapconfig/syncreplica$ sudo mkdir -p /usr/local/var/openldap-data-com
ubadmin@ubreplica:~/testldapconfig/syncreplica$ sudo chmod 755 /usr/local/var/openldap-data-com

Step10: Import configuration database on Replica server

Now we can import the configuration file into the slapd.d configuration directory as shown below.

ubadmin@ubreplica:~/testldapconfig/syncreplica$ sudo /usr/local/sbin/slaptest -v -n 0 -F /usr/local/etc/slapd.d -f ~/testldapconfig/syncreplica/slapd_syncreplica_slave.conf
[sudo] password for ubadmin: 
syncrepl rid=001 searchbase="dc=stack,dc=com": no retry defined, using default
config file testing succeeded

Step11: Start LDAP service on Replica server

Once the configuration is loaded. We can start the LDAP service on Replica server.

ubadmin@ubreplica:~/testldapconfig/syncreplica$ sudo /usr/local/libexec/slapd -F /usr/local/etc/slapd.d

Step12: Validate Synchronized data directory

As you can see the below data directory locations on both the master and replica server. The replica server data directory is updated with the master server copy.

ubadmin@ubscratch:~$ ls -ltr /usr/local/var/openldap-data-com/
total 148
-rw------- 1 root root 147456 May  5 18:34 data.mdb
-rw------- 1 root root   8192 May  6 09:23 lock.mdb
ubadmin@ubreplica:~$ ls -ltr /usr/local/var/openldap-data-com/
total 148
-rw------- 1 root root 147456 May  5 18:14 data.mdb
-rw------- 1 root root   8192 May  6 09:23 lock.mdb

Step13: Modify LDAP data

Here we will first try to send a modify request to the Replica server. This is the modify.ldif file that we are using for this operation.

ubadmin@ubreplica:~$ cat ~/testldapconfig/syncreplica/modify.ldif
dn: uid=matt,ou=Users,dc=stack,dc=com
changetype: modify
replace: description
description: testing modify against shadow.

As replica server is not supposed to update any data in the directory tree it provides a referral information with the correct LDAP server (ie. Master server) which will be able to carry out the modify operation as shown below.

ubadmin@ubreplica:~$ ldapmodify -x -D 'uid=matt,ou=Users,dc=stack,dc=com' -W -f ~/testldapconfig/syncreplica/modify.ldif
Enter LDAP Password: 
modifying entry "uid=matt,ou=Users,dc=stack,dc=com"
ldap_modify: Referral (10)
	referrals:
		ldap://ubscratch.stack.com/uid=matt,ou=Users,dc=stack,dc=com

Now we can try the same modify operation on the Master server by connecting to it as shown below. The modify operation is completed succesfully on the Master server.

ubadmin@ubreplica:~$ ldapmodify -H ldap://ubscratch.stack.com -x -D 'uid=matt,ou=Users,dc=stack,dc=com' -W -f ~/testldapconfig/syncreplica/modify.ldif
Enter LDAP Password: 
modifying entry "uid=matt,ou=Users,dc=stack,dc=com"

Step14: Validate the modified entry

Verify the entry that we have modified using the ldapsearch as shown below on Master server.

ubadmin@ubscratch:~$ ldapsearch -x -b "dc=stack,dc=com" -D "uid=matt,ou=Users,dc=stack,dc=com" -W '(uid=matt)' description
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base <dc=stack,dc=com> with scope subtree
# filter: (uid=matt)
# requesting: description 
#

# matt, Users, stack.com
dn: uid=matt,ou=Users,dc=stack,dc=com
description: testing modify against shadow.

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

After a few seconds as defined in the interval directive, the Replica server pulls the updated LDAP directory tree copy from the Master server and you should be able to do an ldapsearch on the Replica server and it should show you the same data as on the Master server.

ubadmin@ubreplica:~$ ldapsearch -x -b "dc=stack,dc=com" -D "uid=matt,ou=Users,dc=stack,dc=com" -W '(uid=matt)' description
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base <dc=stack,dc=com> with scope subtree
# filter: (uid=matt)
# requesting: description 
#

# matt, Users, stack.com
dn: uid=matt,ou=Users,dc=stack,dc=com
description: testing modify against shadow.

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Hope you enjoyed reading this article. Thank you..