How to use Apache APISIX Observability prometheus plugin for API metrics

Here in this article we will see how we can capture the metrics related to our API using the Prometheus plugin. We are going to setup our APISIX service and enable Prometheus plugin at the Route layer to expose the metrics related to the API application. These metrics can further be scraped by the Prometheus server for aggregating on the data.

Table of Contents

Test Environment

Fedora 37 workstation
Docker
Docker Compose
curl

If you are interested in watching the video. Here is the YouTube video on the same step by step procedure outlined below.

Procedure

Step1: Ensure APISIX services running

As a pre-requisite step, we need to ensure that the APISIX and related services are up and running. Please follow “How to use Opensource Apache APISIX as an API Gateway” to install and configure APISIX service using Docker Compose file and ensure that its up and running.

Step2: Launch Microservices Flask Applications

Here in this step we are going to install python flask package and create a flask based microservices application. This application will be serving on port 2121 with context /postjson. It’s a very basic application which accepts a JSON request body and returns the same as response in JSON format.

Ensure flask python package is installed

[admin@fedser apisix]$ mkdir flask; cd flask
[admin@fedser flask]$ pip install Flask

Create a flask based application which accepts JSON payload data

[admin@fedser flask]$ cat postjson.py 
from flask import Flask, request

app = Flask(__name__)

@app.route('/postjson', methods=['POST'])
def process_json():
    content_type = request.headers.get('Content-Type')
    if (content_type == 'application/json'):
        json = request.json
        return json
    else:
        return 'Content-Type not supported!'

Launch flask application

[admin@fedser flask]$ flask --app postjson run --host=0.0.0.0 --port=2121 &

Create a json payload

[admin@fedser flask]$ cat postdata.json 
{
"name": {
	"firstname": "Alice",
  	"middlename": "Wonder",
	"lastname": "land"
},
"age": 20,
"gender": "Male"
}

Verify flask application using the postdata.json payload

[admin@fedser flask]$ curl -X POST -H "Content-type: application/json" 'http://192.168.29.117:2121/postjson' -d @postdata.json
192.168.29.117 - - [04/Mar/2023 05:38:47] "POST /postjson HTTP/1.1" 200 -
{"age":20,"gender":"Male","name":{"firstname":"Alice","lastname":"land","middlename":"Wonder"}}

Step3: Create upstream for backend flask application

Upstream is the service to forward your requests to. They can be configured to a Route or abstracted out to an Upstream object.

[admin@fedser flask]$ curl "http://192.168.29.117:9180/apisix/admin/upstreams/1" -H "X-API-KEY: edd1c9f034335f136f87ad84b625c8f1" -X PUT -d '
{
  "type": "roundrobin",
  "nodes": {
    "192.168.29.117:2121": 1
  }
}'
{"key":"\/apisix\/upstreams\/1","value":{"type":"roundrobin","pass_host":"pass","id":"1","scheme":"http","update_time":1678620188,"hash_on":"vars","nodes":{"192.168.29.117:2121":1},"create_time":1678620188}}

Step4: Create route for backend flask application

Routes specify how requests to APISIX are forwarded to the Upstream. They match a client’s request based on defined rules and loads and executes the configured Plugins.
Here in this route as you can see we have enabled the Prometheus plugin and set the attribute prefer_name to true so that the metrics data will be capture with the route name instead of route id in case the route name is specified in the route definition.

[admin@fedser flask]$ curl "http://192.168.29.117:9180/apisix/admin/routes/1" -H "X-API-KEY: edd1c9f034335f136f87ad84b625c8f1" -X PUT -d '
{
  "methods": ["POST"],
  "host": "example.com",
  "uri": "/postjson",
  "name": "route-postjson",
  "plugins": {
    "prometheus": {
        "prefer_name": true
    }
  },
  "upstream_id": "1"
}'
{"key":"\/apisix\/routes\/1","value":{"methods":["POST"],"name":"route-postjson","uri":"\/postjson","priority":0,"status":1,"id":"1","create_time":1678643085,"upstream_id":"1","update_time":1678643085,"plugins":{"prometheus":{"prefer_name":true}},"host":"example.com"}}

Step5: Test the Services

We can validate that we are able to reach the backend application through the API gateway as shown below.

[admin@fedser flask]$ curl -X POST -H "Content-type: application/json" -H "Host: example.com" 'http://192.168.29.117:9080/postjson' -d @postdata.json
172.20.0.3 - - [04/Mar/2023 06:07:24] "POST /postjson HTTP/1.1" 200 -
{"age":20,"gender":"Male","name":{"firstname":"Alice","lastname":"land","middlename":"Wonder"}}

Step6: Fetch the API metrics

By default APISIX metrics are captured at the follwing URI “/apisix/prometheus/metrics”. We can send the following curl request to get the metrics data as shown below.

[admin@fedser flask]$ curl -i http://192.168.29.117:9091/apisix/prometheus/metrics
HTTP/1.1 200 OK
Server: openresty
Date: Sun, 12 Mar 2023 17:45:20 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive

# HELP apisix_bandwidth Total bandwidth in bytes consumed per service in APISIX
# TYPE apisix_bandwidth counter
apisix_bandwidth{type="egress",route="route-postjson",service="",consumer="",node="192.168.29.117"} 750
apisix_bandwidth{type="ingress",route="route-postjson",service="",consumer="",node="192.168.29.117"} 729
# HELP apisix_etcd_modify_indexes Etcd modify index for APISIX keys
# TYPE apisix_etcd_modify_indexes gauge
apisix_etcd_modify_indexes{key="consumers"} 0
apisix_etcd_modify_indexes{key="global_rules"} 0
apisix_etcd_modify_indexes{key="max_modify_index"} 17
apisix_etcd_modify_indexes{key="prev_index"} 14
apisix_etcd_modify_indexes{key="protos"} 0
apisix_etcd_modify_indexes{key="routes"} 17

Hope you enjoyed reading this article. Thank you..

How to use Apache APISIX Observability prometheus plugin for API metrics