How to secure OpenSearch Cluster using OpenLDAP Identity Provider

opensearch

How to secure OpenSearch Cluster using OpenLDAP Identity Provider

By novicejava1 0
opensearch_with_ldap_auth_demo

Here in this article we will try to secure OpenSearch Cluster using OpenLDAP based authentication and authorization.

Test Environment

  • Fedora Server 41
  • OpenSearch v3.3.0
  • OpenLDAP v2

If you are interested in watching the video. Here is the YouTube video on the same step by step procedure outlined below.

Procedure

Step1: Ensure OpenSearch Cluster and OpenLDAP running

As a first step we will try to setup OpenSearch cluster and OpenLDAP service using docker compose file. Here are the details below.

Ensure that we have the OpenSearch “admin” user password initialized in the environment file before starting up the OpenSearch cluster.

admin@linuxser:~/opensearch_with_openldap$ cat .env 
OPENSEARCH_INITIAL_ADMIN_PASSWORD=Se@rch@2025

Below is the custom LDAP directory structure LDIF file that we will be using to setup LDAP domain, groups and users for this demo. As you can see there are two types of users.

  • “osdev1” and “osdev2” part of “Developers” group
  • “osadmin1” user part of “Administrator” group
admin@linuxser:~/opensearch_with_openldap$ cat ldifs/custom.ldif 
dn: dc=stack,dc=com
objectClass: top
objectClass: domain
dc: stack

dn: ou=groups,dc=stack,dc=com
objectClass: organizationalUnit
objectClass: top
ou: groups

dn: ou=users,dc=stack,dc=com
objectClass: organizationalUnit
objectClass: top
ou: users

dn: cn=osdev1,ou=users,dc=stack,dc=com
objectClass: person
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: top
cn: osdev1
sn: osdev1
givenName: osdev1
mail: osdev1@stack.com
uid: 1001
userPassword: osdev1

dn: cn=osdev2,ou=users,dc=stack,dc=com
objectClass: person
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: top
cn: osdev2
sn: osdev2
givenName: osdev2
mail: osdev2@stack.com
uid: 1002
userPassword: osdev2

dn: cn=osadmin1,ou=users,dc=stack,dc=com
objectClass: person
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: top
cn: osadmin1
sn: osadmin1
givenName: osadmin1
mail: osadmin1@stack.com
uid: 1003
userPassword: osadmin1

dn: cn=Administrator,ou=groups,dc=stack,dc=com
objectClass: groupOfNames
objectClass: top
cn: Administrator
member: cn=osadmin1,ou=users,dc=stack,dc=com

dn: cn=Developers,ou=groups,dc=stack,dc=com
objectClass: groupOfNames
objectClass: top
cn: Developers
member: cn=osdev1,ou=users,dc=stack,dc=com
member: cn=osdev2,ou=users,dc=stack,dc=com

Here is the docker compose file to setup OpenSearch Cluster along with OpenLDAP service.

admin@linuxser:~/opensearch_with_openldap$ cat docker-compose.yml 
services:
  opensearch-node1: # This is also the hostname of the container within the Docker network (i.e. https://opensearch-node1/)
    image: opensearchproject/opensearch:3.3.0
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster # Name the cluster
      - node.name=opensearch-node1 # Name the node that will run in this container
      - discovery.seed_hosts=opensearch-node1,opensearch-node2 # Nodes to look for when discovering the cluster
      - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 # Nodes eligibile to serve as cluster manager
      - bootstrap.memory_lock=true # Disable JVM heap memory swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms3072m -Xmx3072m" # Set min and max JVM heap sizes to at least 50% of system RAM
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD} # Sets the demo admin user password when using demo configuration (for OpenSearch 2.12 and later)
    ulimits:
      memlock:
        soft: -1 # Set memlock to unlimited (no soft or hard limit)
        hard: -1
      nofile:
        soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
        hard: 65536
    volumes:
      - opensearch-data1:/usr/share/opensearch/data # Creates volume called opensearch-data1 and mounts it to the container
    ports:
      - 9200:9200 # REST API
      - 9600:9600 # Performance Analyzer
    networks:
      - opensearch-net # All of the containers will join the same Docker bridge network
  opensearch-node2:
    image: opensearchproject/opensearch:3.3.0 # This should be the same image used for opensearch-node1 to avoid issues
    container_name: opensearch-node2
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node2
      - discovery.seed_hosts=opensearch-node1,opensearch-node2
      - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms3072m -Xmx3072m"
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - opensearch-data2:/usr/share/opensearch/data
    networks:
      - opensearch-net
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:3.3.0 # Make sure the version of opensearch-dashboards matches the version of opensearch installed on other nodes
    container_name: opensearch-dashboards
    ports:
      - 5601:5601 # Map host port 5601 to container port 5601
    expose:
      - "5601" # Expose port 5601 for web access to OpenSearch Dashboards
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]' # Define the OpenSearch nodes that OpenSearch Dashboards will query
    networks:
      - opensearch-net

  openldap:
    image: bitnami/openldap:2
    container_name: openldap
    ports:
      - '1389:1389'
      - '1636:1636'
    environment:
      - LDAP_ADMIN_USERNAME=admin
      - LDAP_ADMIN_PASSWORD=admin@1234
      #- LDAP_USERS=devuser1,devuser2
      #- LDAP_PASSWORDS=devuser1,devuser2
      - LDAP_ROOT=dc=stack,dc=com
      - LDAP_ADMIN_DN=cn=admin,dc=stack,dc=com
      - LDAP_CUSTOM_LDIF_DIR=/ldifs         # Configuring LDAP to use custom LDIF file
    networks:
      - opensearch-net
    volumes:
      - 'openldap-data:/bitnami/openldap'
      - './ldifs:/ldifs'                    # Custom LDIF file directory loaded from host to container

volumes:
  opensearch-data1:
  opensearch-data2:
  openldap-data:

networks:
  opensearch-net:

Let’s now instantiate the OpenSearch cluster services.

admin@linuxser:~/opensearch_with_openldap$ docker compose -f opensearch/docker-compose.yml up -d

If you face any issue in pulling the OpenLDAP docker image you can build your own docker image for OpenLDAP using the below steps and try starting up the service.

admin@linuxser:~/openldapbuild$ git clone https://github.com/bitnami/containers.git
admin@linuxser:~/openldapbuild$ cd containers/bitnami/openldap/2.6/debian-12
admin@linuxser:~/openldapbuild/containers/bitnami/openldap/2.6/debian-12$ docker build -t bitnami/openldap:2 .

Validate your OpenSearch cluster is up and running.

URL: http://linuxser.stack.com:5601/

Once the OpenLDAP services is running we can validate LDAP DIT tree structure using the below command.

admin@linuxser:~/opensearch_with_openldap$ ldapsearch -H ldap://linuxser.stack.com:1389 -x -b 'dc=stack,dc=com' -D 'cn=admin,dc=stack,dc=com' '(objectClass=*)' -W

NOTE: Install openldap-clients package for ldap client tools installation

Step2: Configure OpenSearch Cluster Security config with OpenLDAP

Here in this step we will configure the OpenSearch cluster to leverage Chained authentication feature to enabled authentication and authorization using both “basicauth” and “ldap” auth.

For internal users such as “admin” and “kibanaserver” we will leverage the “basicauth” with internal user database for authc and authz. For all other users, authc and authz will be carried out using the “ldap” identity provider.

admin@linuxser:~/opensearch_with_openldap$ cat config.yml
---
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: intern
      ldap:
        description: "Authenticate via LDAP or Active Directory"
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          # LDAP authentication backend (authenticate users against a LDAP or Active Directory)
          type: ldap
          config:
            # enable ldaps
            enable_ssl: false
            # enable start tls, enable_ssl should be false
            enable_start_tls: false
            # send client certificate
            enable_ssl_client_auth: false
            # verify ldap hostname
            verify_hostnames: true
            hosts:
            - linuxser.stack.com:1389
            bind_dn: cn=admin,dc=stack,dc=com
            password: admin@1234
            userbase: 'ou=users,dc=stack,dc=com'
            # Filter to search for users (currently in the whole subtree beneath userbase)
            # {0} is substituted with the username
            usersearch: '(cn={0})'
            # Use this attribute from the user as username (if not set then DN is used)
            username_attribute: cn
    authz:
      roles_from_myldap:
        description: "Authorize using LDAP"
        http_enabled: true
        transport_enabled: true
        authorization_backend:
          type: ldap
          config:
            enable_ssl: false
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: true
            hosts:
            - linuxser.stack.com:1389
            bind_dn: cn=admin,dc=stack,dc=com
            password: admin@1234
            userbase: ou=users,dc=stack,dc=com
            usersearch: (cn={0})
            username_attribute: cn
            skip_users:
              - admin
              - kibanaserver
            rolebase: ou=groups,dc=stack,dc=com
            rolesearch: (member={0})
            userroleattribute: null
            userrolename: disabled
            rolename: cn
            resolve_nested_roles: false

Step3: Update Docker compose file

Let’s now update the docker compose file to volume mount the OpenSearch security “config.yml” file to each cluster node as shown below.

admin@linuxser:~/opensearch_with_openldap$ cat docker-compose.yml 
services:
  opensearch-node1: # This is also the hostname of the container within the Docker network (i.e. https://opensearch-node1/)
    image: opensearchproject/opensearch:3.3.0
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster # Name the cluster
      - node.name=opensearch-node1 # Name the node that will run in this container
      - discovery.seed_hosts=opensearch-node1,opensearch-node2 # Nodes to look for when discovering the cluster
      - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 # Nodes eligibile to serve as cluster manager
      - bootstrap.memory_lock=true # Disable JVM heap memory swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms3072m -Xmx3072m" # Set min and max JVM heap sizes to at least 50% of system RAM
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD} # Sets the demo admin user password when using demo configuration (for OpenSearch 2.12 and later)
    ulimits:
      memlock:
        soft: -1 # Set memlock to unlimited (no soft or hard limit)
        hard: -1
      nofile:
        soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
        hard: 65536
    volumes:
      - opensearch-data1:/usr/share/opensearch/data # Creates volume called opensearch-data1 and mounts it to the container
      - ./config.yml:/usr/share/opensearch/config/opensearch-security/config.yml
    ports:
      - 9200:9200 # REST API
      - 9600:9600 # Performance Analyzer
    networks:
      - opensearch-net # All of the containers will join the same Docker bridge network
  opensearch-node2:
    image: opensearchproject/opensearch:3.3.0 # This should be the same image used for opensearch-node1 to avoid issues
    container_name: opensearch-node2
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node2
      - discovery.seed_hosts=opensearch-node1,opensearch-node2
      - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms3072m -Xmx3072m"
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - opensearch-data2:/usr/share/opensearch/data
      - ./config.yml:/usr/share/opensearch/config/opensearch-security/config.yml
    networks:
      - opensearch-net
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:3.3.0 # Make sure the version of opensearch-dashboards matches the version of opensearch installed on other nodes
    container_name: opensearch-dashboards
    ports:
      - 5601:5601 # Map host port 5601 to container port 5601
    expose:
      - "5601" # Expose port 5601 for web access to OpenSearch Dashboards
    #volumes:
    #  - ./opensearch_dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]' # Define the OpenSearch nodes that OpenSearch Dashboards will query
    networks:
      - opensearch-net

  openldap:
    image: bitnami/openldap:2
    container_name: openldap
    ports:
      - '1389:1389'
      - '1636:1636'
    environment:
      - LDAP_ADMIN_USERNAME=admin
      - LDAP_ADMIN_PASSWORD=admin@1234
      #- LDAP_USERS=devuser1,devuser2
      #- LDAP_PASSWORDS=devuser1,devuser2
      - LDAP_ROOT=dc=stack,dc=com
      - LDAP_ADMIN_DN=cn=admin,dc=stack,dc=com
      - LDAP_CUSTOM_LDIF_DIR=/ldifs
      #- LDAP_DEBUG_LEVEL=-1
    networks:
      - opensearch-net
    volumes:
      - 'openldap-data:/bitnami/openldap'
      - './ldifs:/ldifs'

volumes:
  opensearch-data1:
  opensearch-data2:
  openldap-data:

networks:
  opensearch-net:

Once we have the config file and docker compose file ready we can restart our OpenSearch cluster for the changes to take effect.

admin@linuxser:~/opensearch_with_openldap$ docker compose down opensearch-node1 -v
admin@linuxser:~/opensearch_with_openldap$ docker compose down opensearch-node2 -v
admin@linuxser:~/opensearch_with_openldap$ docker compose down opensearch-dashboards -v

admin@linuxser:~/opensearch_with_openldap$ docker compose up -d opensearch-node1
admin@linuxser:~/opensearch_with_openldap$ docker compose up -d opensearch-node2
admin@linuxser:~/opensearch_with_openldap$ docker compose up -d opensearch-dashboards

Once the cluster is up and running, we can verify the security config that is currently enabled within the cluster as shown below.

### Get cluster security config
GET https://linuxser.stack.com:9200/_plugins/_security/api/securityconfig?pretty
Authorization: Basic admin:Se@rch@2025
Accept: application/json
Content-Type: application/json

Step4: Create Role

Here we will create two roles “Developers” and “Administrator” with a set of permissions to manager the OpenSearch cluster.

  • cluster_composite_ops: cluster_composite_ops is a default role in OpenSearch that grants permissions to perform actions related to cluster-level composite operations, such as managing data in multiple indices at once. This can include tasks like bulk indexing or large-scale data manipulation.
  • cluster_monitor: cluster_monitor is a feature within OpenSearch used to continuously check the health and performance of a cluster and take action when certain conditions are met.
PUT https://linuxser.stack.com:9200/_plugins/_security/api/roles/Developers
Authorization: Basic admin:Se@rch@2025
Accept: application/json
Content-Type: application/x-ndjson

{
  "cluster_permissions": [
    "cluster_composite_ops"
  ],
  "index_permissions": [{
    "index_patterns": [
      "*"
    ],
    "allowed_actions": [
      "read",
      "search"
    ]
  }]
}

PUT https://linuxser.stack.com:9200/_plugins/_security/api/roles/Administrator
Authorization: Basic admin:Se@rch@2025
Accept: application/json
Content-Type: application/x-ndjson

{
  "cluster_permissions": [
    "cluster_composite_ops",
    "cluster_monitor"
  ],
  "index_permissions": [{
    "index_patterns": [
      "*"
    ],
    "allowed_actions": [
      "read",
      "search"
    ]
  }]
}

Step5: Create Role Mapping

Here in this step we will map the roles that were created in our previous step to backend ldap roles “Developers” and “Administrator” as shown below.

### Create a role mapping
PUT https://linuxser.stack.com:9200/_plugins/_security/api/rolesmapping/Developers
Authorization: Basic admin:Se@rch@2025
Accept: application/json
Content-Type: application/json

{
  "backend_roles" : [ "Developers" ]
}

### Create a role mapping
PUT https://linuxser.stack.com:9200/_plugins/_security/api/rolesmapping/Administrator
Authorization: Basic admin:Se@rch@2025
Accept: application/json
Content-Type: application/json

{
  "backend_roles" : [ "Administrator" ]
}

Step6: Validate access using LDAP users

Now its time to access our OpenSearch cluster service using the LDAP users as shown below.

Search all documents within the indicies

The following request succeeds with “osdev1” user as per the permissions granted.

GET https://linuxser.stack.com:9200/_search
Authorization: Basic osdev1:osdev1
Accept: application/json
Content-Type: application/json

{
  "query": {
    "match_all": {}
  }
}

Get OpenSearch cluster state

The following request fails as “osdev1” user does not have “cluster_monitor” permissions as per the “Developers” role which is assigned.

GET https://linuxser.stack.com:9200/_cluster/state
Authorization: Basic osdev1:osdev1
Accept: application/json
Content-Type: application/json

Search and Get OpenSearch cluster state

The same two requests will succeed for user “osadmin1” as the assigned role “Administrator” has the required permissions to search and monitor the cluster status.

GET https://linuxser.stack.com:9200/_search
Authorization: Basic osadmin1:osadmin1
Accept: application/json
Content-Type: application/json

{
  "query": {
    "match_all": {}
  }
}

GET https://linuxser.stack.com:9200/_cluster/state
Authorization: Basic osadmin1:osadmin1
Accept: application/json
Content-Type: application/json

You can also access the OpenSearch Dashboard with LDAP users which would leverage “ldap” identity provider.

Here are the screenshots below for “osdev1” and “osadmin1” users.

For “admin” user accessing the OpenSearch Dashboard it will use the “basicauth” for authentication and authorization.

Hope you enjoyed reading this article. Thank you..