Glen Mazza's Weblog

https://glenmazza.net/blog/date/20230520 Saturday May 20, 2023

Using Micrometer to send custom metrics to Datadog

This entry provides a simple example of using Spring Shell (within Spring Boot) and Micrometer to send custom metrics to Datadog. For a fuller example with Docker and the Datadog Agent, I recommend Datadog Learning Center's free Datadog 101: Developer course. This course will also provide you a free two-week training Datadog account which you can use to receive custom metrics for this example, useful if you don't care to test against your company account. Note custom metrics ordinarily carry billing costs, requiring a paid Datadog account.

Spring Boot already provides numerous web metrics in several areas that can be sent to Datadog without explicit need to capture them. The jvm.* properties, for example, are readable in Datadog's Metrics Explorer, filtering by statistic:value for this example in the "from" field. For custom metrics, we'll have the Spring Shell app commands modify a Timer and a Gauge.

  1. Create the Spring Shell application. From Spring Initializr, choose a Java JAR app with Spring Shell and Datadog as dependencies. For some reason I needed to choose the Spring Boot 2.7.x series for an app to download. Prior to running the demo in your IDE (I use IntelliJ), the management.metrics.export.datadog.apiKey=... value needs to be added to the main/resources/application.properties file. Your API key can be determined by logging into Datadog, and from the bottom of the left-side menu, click on your name, then Organization Settings, then Access, the API Keys.

  2. Create the shell commands to add to the timer and gauge values:

    package com.example.demo;
    
    import io.micrometer.core.instrument.MeterRegistry;
    import io.micrometer.core.instrument.Tags;
    import io.micrometer.core.instrument.Timer;
    import org.springframework.shell.standard.ShellComponent;
    import org.springframework.shell.standard.ShellMethod;
    
    import java.util.ArrayList;
    import java.util.List;
    import java.util.concurrent.TimeUnit;
    
    @ShellComponent
    public class MyCommands {
    
        private final Timer timer;
    
        private final List<Integer> integerList;
    
        public MyCommands(MeterRegistry registry) {
            timer = registry.timer("demoapp.timer", Tags.empty());
            integerList = registry.gauge("demoapp.listsize", Tags.empty(), new ArrayList<>(), List::size);
        }
    
        @ShellMethod("Note a timer event of given duration in seconds")
        public String timer(int seconds) {
            timer.record(seconds, TimeUnit.SECONDS);
            return "Timer event noted";
        }
        @ShellMethod("Add an element to list")
        public String listAdd() {
            integerList.add(10);
            return "List has " + integerList.size() + " elements";
        }
    
        @ShellMethod("Remove an element from list (if possible)")
        public String listRemove() {
            if (integerList.size() > 0) {
                integerList.remove(integerList.size() - 1);
            }
            return "List has " + integerList.size() + " elements";
        }
    }
    

    For the above we're keeping a gauge on the size of the list, and for the timer, we provide the number of seconds that an arbitrary event took.

  3. Run the application and enter several timer #secs, list-add, and list-remove commands. Run -> Run Application from the IntelliJ menu should work fine to enter the commands in the IDE's Console view. To keep the connection with Datadog, keep the command-line app running, even if you're not entering commands. After 2 or 3 minutes, check Datadog's Metrics Explorer to confirm that the demoapp.timer and demoapp.listsize metrics are getting received:

    timercount

    A dashboard can be created to show both properties at once (with separate average time and counts given for the Timer):

    dashboard

Resources

Posted by Glen Mazza in Programming at 03:00AM May 20, 2023 | Tags:  datadog | Comments[0]


Calendar
« May 2023 »
Sun Mon Tue Wed Thu Fri Sat
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Today
About Me
Java Software Engineer
TightBlog project maintainer
Arlington, Virginia USA
glen.mazza at pm dot me
GitHub profile for Glen Mazza at Stack Overflow, Q&A for professional and enthusiast programmers
Blog Search


Blog article index
Navigation
About Blog
Blog software: TightBlog 4.0.0
Application Server: Tomcat
Database: MySQL
Hosted on: Linode
SSL Certificate: Let's Encrypt
Installation Instructions