How to use Index Templates in Elasticsearch

Here in this article we will see how we can make use of Index Templates and Component Templates to prepare our Index Template and apply this template for documents that are getting indexed.

Table of Contents

Test Environment

Fedora 37 workstation
Docker
Docker Compose

What is Index Template

Whenever we create an index either manually or while indexing a document it actually makes use template settings which are configured prior to index creation. So a Index Template actually consists of settings, mappings and alias.

There are two types of index templates.

Templates	Description
Index Templates	An index template consist of settings, mappings and alias. It can also be a collection Component Templates
Component Templates	These are nothing but resuable blocks that configure settings, mappings and alias. These templates can be used to create an Index Template

If you are interested in watching the video. Here is the YouTube video on the same step by step procedure outlined below.

https://youtu.be/Y8oHEHLzFik

Procedure

Step1: Create a settings Component Template

As a first step let us build a Component Template named “generic_settings” which only consist settings related to index as shown below.

Request

PUT /_component_template/generic_settings
{
  "template": {
    "settings": {
      "index.number_of_shards": 3,
      "index.number_of_replicas": 3
    }
  }
}

Step2: Create a mappings Component Template

As a second step let us build a Component Template named “prizes_mapping” which only consist of mapping of fields to their respective types as shown below.

Request

PUT /_component_template/prizes_mapping
{
  "template": {
    "mappings": {
      "properties": {
        "category": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "laureates": {
          "properties": {
            "firstname": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "id": {
              "type": "integer",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "motivation": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "share": {
              "type": "integer",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "surname": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "overallMotivation": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "year": {
          "type": "date",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    }
}
}

Step3: Simulate an Index Template with Component Templates

In this step we are reusing the component templates to create our index template which will be applied to any index with patther “prizes*”. We have also set the priority of the index template to 500 so that it takes highest precedence if more than one index templates match for a particular index.

Before creating the index template, we can execute the below simulate request to see how our index template looks once its actually created.

Request

POST _index_template/_simulate
{
  "index_patterns": ["prizes*"],
  "priority": 500,
  "composed_of": ["generic_settings", "prizes_mapping"], 
  "version": 1,
  "_meta": {
    "description": "prizes custom index template"
  }
}

Response

{
  "template": {
    "settings": {
      "index": {
        "number_of_shards": "3",
        "number_of_replicas": "3",
        "routing": {
          "allocation": {
            "include": {
              "_tier_preference": "data_content"
            }
          }
        }
      }
    },
    "mappings": {
      "properties": {
        "category": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "laureates": {
          "properties": {
            "firstname": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "id": {
              "type": "integer",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "motivation": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "share": {
              "type": "integer",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "surname": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "overallMotivation": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "year": {
          "type": "date",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    },
    "aliases": {}
  },
  "overlapping": []
}

Step4: Create an Index Template with Component Templates

Here we are now going to create the “prizes_template” with component templates included as shown below.

Request

PUT _index_template/prizes_template
{
  "index_patterns": ["prizes*"],
  "priority": 500,
  "composed_of": ["generic_settings", "prizes_mapping"], 
  "version": 1,
  "_meta": {
    "description": "prizes custom index template"
  }
}

Step5: Index JSON dataset using Index Template

Now we are going to run our script again with a new index named “prizes_with_index_template” to create an index which maps to “prizes_template” template. Refer to “How to index json dataset in Elasticsearch” for the script details.

[admin@fedser elastic-kibana]$ python indexDocuments.py prizes_with_index_template

Step6: Validate the Indexed Documents

Once our indexing is done we can navigate to Stack Management – Data View and Create our Data View based on the new index source that got generated. Here as you can see there is a prizes_with_index_template index created with each document indexed within that index as per the mapping definition that was provided.

Step6: Validate Index Template used by Index

We can validate that the index template that was used to create index by getting the details of the index as shown below.

Request

GET prizes_with_index_template

Response

{
  "prizes_with_index_template": {
    "aliases": {},
    "mappings": {
      "properties": {
        "category": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "laureates": {
          "properties": {
            "firstname": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "id": {
              "type": "integer",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "motivation": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "share": {
              "type": "integer",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "surname": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "overallMotivation": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "year": {
          "type": "date",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    },
    "settings": {
      "index": {
        "routing": {
          "allocation": {
            "include": {
              "_tier_preference": "data_content"
            }
          }
        },
        "number_of_shards": "3",
        "provided_name": "prizes_with_index_template",
        "creation_date": "1674412720387",
        "number_of_replicas": "3",
        "uuid": "401jk7n_SbWkgUtrma1ivA",
        "version": {
          "created": "8050399"
        }
      }
    }
  }
}

Hope you enjoyed reading this article. Thank you..

How to use Index Templates in Elasticsearch