Datadog

How to send metrics and logs to Datadog.

Key takeaways

Datadog application Online application used to monitor metrics, traces, and logs.
DataDogCounters Component used to send metrics to the Datadog application.
DataDogLogger Component used to send logs to the Datadog application.

Introduction

In this tutorial, you will learn how to send different metrics and logs to Datadog. For this, we will first create a class with a method that, once executed, sends metrics to Datadog via the DataDogCounters component. Then, we will modify this class and use it to send log information to Datadog via the DataDogLogger class.

DataDogCounters

The DataDogCounters class is used to send metrics to the Datadog application. This component inherits its methods from the CachedCounters class, which, in turn, implements the ICounters interface. It communicates with Datadog via the DataDogMetricsClient, which is a REST component.

Pre-requisites

In order to use this component, we must first import it. To do this, we can use the following command:

import { DataDogCounters } from 'pip-services-datadog-nodex';

using PipServices3.DataDog.Count;

import (
    dcount "github.com/pip-services3-gox/pip-services3-datadog-gox/count"
)
Not available 
from pip_services3_datadog.count import DataDogCounters
Not available

Creating a component with metrics

Our next step is to create a component that contains some metrics. For this, we create a class that contains a method that prints a greeting message. Within this class, we create two counters: one that increments by one each time the method is executed and one that measures execution time. The following code shows how this class is constructed:

class MyComponentA {
    private consoleLog = true;

    protected counters: DataDogCounters;

    public constructor(counters: DataDogCounters) {
        this.counters = counters;

        if (this.consoleLog)
            console.log('MyComponentA has been created.'); 
    }

    public myMethod(): void {
        this.counters.increment('mycomponent.mymethod.calls', 1);
        let timing = this.counters.beginTiming('mycomponent.mymethod.exec_time');

        try {
            if (this.consoleLog) {
                console.log('Hola amigo');
                console.log('Hola amigoBonjour mon ami');
            }
        } finally {
            timing.endTiming();
        }

        this.counters.dump();
    }
}

    public class MyComponentA
    {
        private bool consoleLog = true;

        protected DataDogCounters counters;

        public MyComponentA(DataDogCounters counters)
        {
            this.counters = counters;
            if (consoleLog)
                Console.WriteLine("MyComponentA has been created.");
        }

        public void MyMethod()
        {
            counters.Increment("mycomponent.mymethod.calls", 1);
            var timing = counters.BeginTiming("mycomponent.mymethod.exec_time");

            try
            {
                if (consoleLog)
                {
                    Console.WriteLine("Hola amigo");
                    Console.WriteLine("Hola amigoBonjour mon ami");
                }
            } finally
            {
                timing.EndTiming();
            }

            counters.Dump();
        }
    }

import (
	"fmt"

	conf "github.com/pip-services3-gox/pip-services3-commons-gox/config"
	dcount "github.com/pip-services3-gox/pip-services3-datadog-gox/count"
)

type MyComponentA struct {
	consoleLog bool
	counters   *dcount.DataDogCounters
}

func NewMyComponentA(counters *dcount.DataDogCounters) *MyComponentA {
	c := &MyComponentA{
		consoleLog: true,
		counters:   counters,
	}

	if c.consoleLog {
		fmt.Println("MyComponentA has been created.")
	}
	return c
}

func (c *MyComponentA) MyMethod(ctx context.Context) {
	c.counters.Increment(ctx, "mycomponent.mymethod.calls", 1)
	timing := c.counters.BeginTiming(ctx, "mycomponent.mymethod.exec_time")

	defer timing.EndTiming(ctx)

	if c.consoleLog {
		fmt.Println("Hola amigo")
		fmt.Println("Hola amigoBonjour mon ami")
	}

	c.counters.Dump(ctx)
}
Not available 
class MyComponentA:
    
    _console_log = True
    
    def __init__(self, counters: DataDogCounters):
        self.counters = counters

        if self._console_log:
            print("MyComponentA has been created.")

    def mymethod(self):
        self.counters.increment("mycomponent.mymethod.calls", 1)
        timing = self.counters.begin_timing("mycomponent.mymethod.exec_time")
        try:
            if self._console_log:
                print("Hola amigo")
                print("Bonjour mon ami")
        finally:
            timing.end_timing()
        self.counters.dump()
Not available

Creating a DataDogCounters object

Now, we create a DataDogCounters object and connect it to the Datadog application. This object only requires the API key as a configuration parameter. The code below shows how to do it.

import { ConfigParams } from 'pip-services3-commons-nodex';

let counters = new DataDogCounters();
counters.configure(ConfigParams.fromTuples(
    "credential.access_key", "e1be2e70036b8f05f2777f5f038fc222"
));

await counters.open(null);

using PipServices3.Commons.Config;

var counters = new DataDogCounters();
counters.Configure(ConfigParams.FromTuples(
    "credential.access_key", "e1be2e70036b8f05f2777f5f038fc222"
));

counters.OpenAsync(null).Wait();



import (
	conf "github.com/pip-services3-gox/pip-services3-commons-gox/config"
)

counters := dcount.NewDataDogCounters()
counters.Configure(context.Background(), conf.NewConfigParamsFromTuples(
	"credential.access_key", "e1be2e70036b8f05f2777f5f038fc222",
))

_ = counters.Open(context.Background(), "123")
Not available 
from pip_services3_commons.config import ConfigParams

counters = DataDogCounters()
counters.configure(ConfigParams.from_tuples(
   "credential.access_key", "e1be2e70036b8f05f2777f5f038fc222"
))

counters.open('123')
Not available

Generating values for the metrics

Once we have our components ready, we can generate some values for our metrics by running the component’s method. In the example below, we create an instance of the object and run “mymethod” five times.

let mycomponent = new MyComponentA(counters);

for (let i = 0; i < 5; i++)
    mycomponent.myMethod();

var mycomponent = new MyComponentA(counters);

for (var i = 0; i < 5; i++)
    mycomponent.MyMethod();

mycomponent := NewMyComponentA(counters)

for i := 1; i < 5; i++ {
	mycomponent.MyMethod(context.Background())
}
Not available 
mycomponent = MyComponentA(counters)
for i in range(5):
    mycomponent.mymethod()
Not available

After running it, we obtain the following output:

figure 1

Our metrics in Datadog

In order to verify that our program sent the credentials to Datadog, we can go to its site and we will see that the metrics appear on the list in the Metrics Explorer section.

figure 2

Final code

We can now improve our component by adding some interfaces and functionality commonly associated with Pip.Services. The code below shows the result.

import { ConfigParams, IConfigurable, IOpenable } from 'pip-services3-commons-nodex';


class MyComponentA implements IConfigurable, IOpenable {
    private consoleLog = true;

    protected counters: DataDogCounters;

    public constructor(counters: DataDogCounters) {
        this.counters = counters;

        if (this.consoleLog)
            console.log('MyComponentA has been created.'); 
    }

    public configure(config: ConfigParams): void {
        this.counters.configure(config);
    }

    public getCounters(): DataDogCounters {
        return this.counters;
    }

    public isOpen(): boolean {
        return this.counters.isOpen()
    }

    public async open(correlationId: string): Promise<void> {
        this.counters.open(correlationId);
    }

    public async close(correlationId: string): Promise<void> {
        this.counters.close(correlationId);
    }

    public myMethod(): void {
        this.counters.increment('mycomponent.mymethod.calls', 1);
        let timing = this.counters.beginTiming('mycomponent.mymethod.exec_time');

        try {
            if (this.consoleLog) {
                console.log('Hola amigo');
                console.log('Hola amigoBonjour mon ami');
            }
        } finally {
            timing.endTiming();
        }

        this.counters.dump();
    }
}


using System.Threading.Tasks;
using PipServices3.Commons.Config;
using PipServices3.Commons.Run;


    public class MyComponentA: IConfigurable, IOpenable
    {
        private bool consoleLog = true;

        protected DataDogCounters counters;

        public MyComponentA(DataDogCounters counters)
        {
            this.counters = counters;
            if (consoleLog)
                Console.WriteLine("MyComponentA has been created.");
        }

        public void Configure(ConfigParams config)
        {
            counters.Configure(config);
        }

        public DataDogCounters GetCounters()
        {
            return counters;
        }

        public bool IsOpen()
        {
            return counters.IsOpen();
        }

        public async Task OpenAsync(string correlationId)
        {
            await counters.OpenAsync(correlationId);
        }

        public async Task CloseAsync(string correlationId)
        {
            await counters.CloseAsync(correlationId);
        }

        public void MyMethod()
        {
            counters.Increment("mycomponent.mymethod.calls", 1);
            var timing = counters.BeginTiming("mycomponent.mymethod.exec_time");

            try
            {
                if (consoleLog)
                {
                    Console.WriteLine("Hola amigo");
                    Console.WriteLine("Hola amigoBonjour mon ami");
                }
            } finally
            {
                timing.EndTiming();
            }

            counters.Dump();
        }
    }


type MyComponentA struct {
	consoleLog bool
	counters   *dcount.DataDogCounters
}

func NewMyComponentA(counters *dcount.DataDogCounters) *MyComponentA {
	c := &MyComponentA{
		consoleLog: true,
		counters:   counters,
	}

	if c.consoleLog {
		fmt.Println("MyComponentA has been created.")
	}
	return c
}

func (c *MyComponentA) Configure(ctx context.Context, config *conf.ConfigParams) {
	c.counters.Configure(ctx, config)
}

func (c *MyComponentA) GetCounters() *dcount.DataDogCounters {
	return c.counters
}

func (c *MyComponentA) IsOpen() bool {
	return c.counters.IsOpen()
}

func (c *MyComponentA) Open(ctx context.Context, correlationId string) error {
	return c.counters.Open(ctx, correlationId)
}

func (c *MyComponentA) Close(ctx context.Context, correlationId string) error {
	return c.counters.Close(ctx, correlationId)
}

func (c *MyComponentA) MyMethod(ctx context.Context) {
	c.counters.Increment(ctx, "mycomponent.mymethod.calls", 1)
	timing := c.counters.BeginTiming(ctx, "mycomponent.mymethod.exec_time")

	defer timing.EndTiming(ctx)

	if c.consoleLog {
		fmt.Println("Hola amigo")
		fmt.Println("Hola amigoBonjour mon ami")
	}

	c.counters.Dump(ctx)
}
Not available 
import time

from typing import Optional

from pip_services3_commons.config import IConfigurable, ConfigParams
from pip_services3_commons.run import IOpenable
from pip_services3_datadog.count import DataDogCounters


class MyComponentA(IConfigurable, IOpenable):
    _console_log = True

    def __init__(self):
        self._counters = DataDogCounters()

        if self._console_log:
            print("MyComponentA has been created.")

    def configure(self, config: ConfigParams):
        self._counters.configure(config)

    def get_counters(self) -> DataDogCounters:
        return self._counters

    def is_open(self) -> bool:
        return self._counters.is_open()

    def open(self, correlation_id: Optional[str]):
        self._counters.open(correlation_id)

    def close(self, correlation_id: Optional[str]):
        self._counters.close(correlation_id)

    def my_method(self):
        self._counters.increment("mycomponent.mymethod.calls", 1)
        timing = self._counters.begin_timing("mycomponent.mymethod.exec_time")
        try:
            if self._console_log:
                print("Hola amigo")
                print("Bonjour mon ami")
        finally:
            timing.end_timing()
        self._counters.dump()
Not available

DataDogLogger

The DataDogLogger class is used to capture log messages and send them to the Datadog application. This component extends the CachedCounters class and uses the REST client DataDogLogClient to send the logs to Datadog.

Pre-requisites

To use this component, we need to import it first. The following command shows how to do this:

import { DataDogLogger } from 'pip-services-datadog-nodex';

using PipServices3.DataDog.Log;

import (
    dlog "github.com/pip-services3-go/pip-services3-datadog-go/log"
)
Not available 
from pip_services3_datadog.log import DataDogLogger
Not available

Creating a component with logging

Next, we modify our previous component by changing the print statements to log statements of type DataDogLogger. The following code shows how to do this:

import { DataDogLogger } from 'pip-services-datadog-nodex';

class MyComponentA {
    private dataDogLog = true;

    protected logger: DataDogLogger;

    public constructor(logger: DataDogLogger) {
        this.logger = logger; 
        if (this.dataDogLog) {
            this.logger.info("123", "MyComponentA has been created.");
        }
    }

    public myMethod(): void {
        try {
            if (this.dataDogLog) {
                console.log("Hola amigo");
                console.log("Bonjour mon ami");
                this.logger.info("123", "Greetings created.");
            }
        } finally {
            this.logger.info("123", "Finally reached.");
        }
    }
}

using PipServices3.DataDog.Log;

    public class MyComponentA
    {
        private bool dataDogLog = true;

        protected DataDogLogger logger;

        public MyComponentA(DataDogLogger logger)
        {
            this.logger = logger;
            if (dataDogLog)
                logger.Info("123", "MyComponentA has been created.");
        }

        public void MyMethod()
        {
            try
            {
                if (dataDogLog)
                {
                    Console.WriteLine("Hola amigo");
                    Console.WriteLine("Bonjour mon ami");
                    logger.Info("123", "Greetings created.");
                }
            } finally
            {
                logger.Info("123", "Finally reached.");
            }
        }
    }

import (
    dlog "github.com/pip-services3-gox/pip-services3-datadog-gox/log"
)

func NewMyComponentA(logger *dlog.DataDogLogger) *MyComponentA {
	c := &MyComponentA{
		consoleLog: true,
		logger:     logger,
	}

	if c.consoleLog {
		logger.Info(context.Background(), "123", "MyComponentA has been created.")
	}
	return c
}

func (c *MyComponentA) MyMethod(ctx context.Context) {
	defer c.logger.Info(ctx, "123", "Finally reached.")

	if c.consoleLog {
		fmt.Println("Hola amigo")
		fmt.Println("Hola amigoBonjour mon ami")
		c.logger.Info(ctx, "123", "Greetings created.")
	}
}
Not available 
from pip_services3_datadog.log import DataDogLogger

class MyComponentA:
    _Datadog_log = True
    
    def __init__(self, logger: DataDogLogger):
        self._logger = logger

        if self._Datadog_log:
            logger.info("123" , "MyComponentA has been created.")

    def mymethod(self):

        try:
            if self._Datadog_log:
                print("Hola amigo")
                print("Bonjour mon ami")
                logger.info("123" , "Greetings created.")
        finally:
                logger.info("123" , "Finally reached.")
Not available

Creating a DataDogLogger object

Once we have our component ready, we create an instance of DataDogLogger and configure it with our access key. The following code shows how to do this.

import { ConfigParams } from 'pip-services3-commons-nodex';

let logger = new DataDogLogger();
logger.configure(ConfigParams.fromTuples(
    "credential.access_key", "e1be2e70036b8f05f2777f5f038fc222"
));

await logger.open(null);

using PipServices3.Commons.Config;

 var logger = new DataDogLogger();
logger.Configure(ConfigParams.FromTuples(
    "credential.access_key", "e1be2e70036b8f05f2777f5f038fc222"
));

logger.OpenAsync(null).Wait();

import (
    dlog "github.com/pip-services3-gox/pip-services3-datadog-gox/log"
)

logger := dlog.NewDataDogLogger()
logger.Configure(context.Background(), conf.NewConfigParamsFromTuples(
	"credential.access_key", "e1be2e70036b8f05f2777f5f038fc222",
))
_ = logger.Open(context.Background(), "123")
Not available 
from pip_services3_commons.config import ConfigParams

logger = DataDogLogger()
logger.configure(ConfigParams.from_tuples(
   "credential.access_key", "e1be2e70036b8f05f2777f5f038fc222"
))
Not available

Generating logs

To generate logs and send them to the Datadog application, we need to create an instance of our component and run “mymethod”. The following code shows how to do it by using a loop whereby we run the method five times.

let mycomponent = new MyComponentA(logger);

for (let i = 0; i < 5; i++)
    mycomponent.myMethod();

var mycomponent = new MyComponentA(logger);

for (var i = 0; i < 5; i++)
    mycomponent.MyMethod();


mycomponent := NewMyComponentA(logger)

for i := 1; i < 5; i++ {
	mycomponent.MyMethod(context.Background())
}
Not available 
mycomponent = MyComponentA(logger)
for i in range(5):
    mycomponent.mymethod()
Not available

After running this code, we obtain the following output:

figure 3

Our logs in Datadog

After running our program, we can see the logs on the Datadog application screen. The following figure shows these results:

figure 3

Final code

Now we can re-arrange our code to create a component according to Pip.Services practices. For this, we add the IConfigurable and IOpenable interfaces, and the configure, open and close methods. We also define the DataDogLogger component within the class, and use the configure, open and close methods to perform these operations on our DataDogLogger component. The following code shows how this can be done:

import { DataDogLogger } from 'pip-services-datadog-nodex';
import { ConfigParams, IConfigurable, IOpenable } from 'pip-services3-commons-nodex';

class MyComponentA implements IConfigurable, IOpenable {
    private dataDogLog = true;

    protected logger: DataDogLogger;

    public constructor(logger: DataDogLogger) {
        this.logger = logger; 
        if (this.dataDogLog) {
            this.logger.info("123", "MyComponentA has been created.");
        }
    }
    public configure(config: ConfigParams): void {
        this.logger.configure(config);
    }

    public isOpen(): boolean {
        return this.logger.isOpen();
    }

    public async open(correlationId: string): Promise<void> {
        this.logger.open(correlationId);
    }

    public async close(correlationId: string): Promise<void> {
        this.logger.close(correlationId);
    }

    public myMethod(): void {
        try {
            if (this.dataDogLog) {
                console.log("Hola amigo");
                console.log("Bonjour mon ami");
                this.logger.info("123", "Greetings created.");
            }
        } finally {
            this.logger.info("123", "Finally reached.");
        }
    }
}

using System;
using System.Threading.Tasks;
using PipServices3.Commons.Config;
using PipServices3.Commons.Run;
using PipServices3.DataDog.Log;

    public class MyComponentA: IConfigurable, IOpenable
    {
        private bool dataDogLog = true;

        protected DataDogLogger logger;

        public MyComponentA(DataDogLogger logger)
        {
            this.logger = logger;
            if (dataDogLog)
                logger.Info("123", "MyComponentA has been created.");
        }

        public void Configure(ConfigParams config)
        {
            logger.Configure(config);
        }

        public DataDogLogger GetCounters()
        {
            return logger;
        }

        public bool IsOpen()
        {
            return logger.IsOpen();
        }

        public async Task OpenAsync(string correlationId)
        {
            await logger.OpenAsync(correlationId);
        }

        public async Task CloseAsync(string correlationId)
        {
            await logger.CloseAsync(correlationId);
        }

        public void MyMethod()
        {
            try
            {
                if (dataDogLog)
                {
                    Console.WriteLine("Hola amigo");
                    Console.WriteLine("Bonjour mon ami");
                    logger.Info("123", "Greetings created.");
                }
            } finally
            {
                logger.Info("123", "Finally reached.");
            }
        }
    }

import (
	"fmt"

	conf "github.com/pip-services3-gox/pip-services3-commons-gox/config"
	dlog "github.com/pip-services3-gox/pip-services3-datadog-gox/log"
)

type MyComponentA struct {
	consoleLog bool
	logger     *dlog.DataDogLogger
}

func NewMyComponentA(logger *dlog.DataDogLogger) *MyComponentA {
	c := &MyComponentA{
		consoleLog: true,
		logger:     logger,
	}

	if c.consoleLog {
		logger.Info(context.Background(), "123", "MyComponentA has been created.")
	}
	return c
}

func (c *MyComponentA) Configure(ctx context.Context, config *conf.ConfigParams) {
	c.logger.Configure(ctx, config)
}

func (c *MyComponentA) GetCounters() *dlog.DataDogLogger {
	return c.logger
}

func (c *MyComponentA) IsOpen() bool {
	return c.logger.IsOpen()
}

func (c *MyComponentA) Open(ctx context.Context, correlationId string) error {
	return c.logger.Open(ctx, correlationId)
}

func (c *MyComponentA) Close(ctx context.Context, correlationId string) error {
	return c.logger.Close(ctx, correlationId)
}

func (c *MyComponentA) MyMethod(ctx context.Context) {
	defer c.logger.Info(ctx, "123", "Finally reached.")

	if c.consoleLog {
		fmt.Println("Hola amigo")
		fmt.Println("Hola amigoBonjour mon ami")
		c.logger.Info(ctx, "123", "Greetings created.")
	}
}
Not available 
from pip_services3_datadog.log import DataDogLogger
from pip_services3_commons.run import IOpenable
from pip_services3_commons.config import IConfigurable, ConfigParams

from typing import Optional

class MyComponentA(IConfigurable, IOpenable):

    _Datadog_log = True
    
    def __init__(self):
        self._logger = DataDogLogger()

        if self._Datadog_log:
            self._logger.info("123" , "MyComponentA has been created.")

    def configure(self, config: ConfigParams):
        self._logger.configure(config)
        
    def open(self, correlation_id: Optional[str]):
        self._logger.open(correlation_id)
       
    def close(self, correlation_id: Optional[str]):
        self._logger.close(correlation_id)
        
    def mymethod(self):

        try:
            if self._Datadog_log:
                print("Hola amigo")
                print("Bonjour mon ami")
                self._logger.info("123" , "Greetings created.")
        finally:
                self._logger.info("123" , "Finally reached.")
Not available

Wrapping up

In this tutorial, you have learned how to send metrics and logs to Datadog. First, we saw how to work with the DataDogCounters component. By using this class, we constructed two different counters: one that measures the number of times a method is executed, and another that measures execution time. Then, we executed this class and saw that the values were received by the Datadog application.

Next, we learned about the DataDogLogger class, which we used to send log information to the Datadog application by modifying our previous example. After executing the example’s method, we verified that the logs had been received by the Datadog application.

Finally, for both cases, we presented a final version of the code with some improvements that considered common aspects of Pip.Services programming.