Elasticsearch

How to send logs to Elasticsearch.

Key takeaways

Elasticsearch	Search application. In this tutorial, it used to store log messages from different components. Pip.Services supports versions 6 and 7.
ElasticSearchLogger	Component used to send log messages to Elasticsearch.
configure()	Method used to set the values of configuration parameters.
open()	Method used to connect to Elasticsearch.
close()	Method used to close the logger and free used resources.
fatal()	Method used to log a fatal level or level 1 message.
error()	Method used to log an error level or level 2 message.
warn()	Method used to log a warning level or level 3 message.
info()	Method used to log an info level or level 4 message.
debug()	Method used to log a debug level or level 5 message.
trace()	Method used to log a trace level or level 6 message.

Introduction

In this tutorial you will learn how to work with the ElasticSearchLogger component, which is used to send logs to Elasticsearch. First, we will see the necessary pre-requisites. Then, we will continue by creating a custom component with an example method, running it and generating the logs. After this, we will verify that the messages reached Elasticsearch by using the Elasticvue online tool. Finally, we will improve our custom component by including several Pip.Services best practices.

ElasticSearchLogger

This component, which is a subclass of the CachedLogger component, allows us to send log messages to Elasticsearch. In order to learn how to use it, in the following sections, we will first see how to create a custom component that uses it to send log messages to Elasticsearch. Then, we will learn how to verify that the log messages have arrived. And finally, we will improve our component by adding some interfaces and methods.

Pre-requisites

In order to use this component, we must first import it. The following command shows how to do this:

import { ElasticSearchLogger } from 'pip-services-elasticsearch-nodex';

using PipServices3.ElasticSearch.Log;

import ellog "github.com/pip-services3-gox/pip-services3-elasticsearch-gox/log"

import 'package:pip_services3_elasticsearch/pip_services3_elasticsearch.dart';

from pip_services3_elasticsearch.log import ElasticSearchLogger

Not available

Creating a component with logging

Now, we can create a custom component that sends log messages. As we want to send those messages to Elasticsearch, we define our logger of type ElasticSearchLogger. We also create a method called “mymethod” with the aim of calling it and sending log messages. All log messages are considered level 4 or Info level. The code below shows how this class looks like.

export class MyComponentA {
    private _logger: ILogger;
    private _console_log = true;

    public constructor(logger: ElasticSearchLogger) {
        this._logger = logger;

        this._logger.info("123", "MyComponentA has been created.");
    }

    public myMethod(): void {
        try {
            if (this._console_log) {
                console.log("Hola amigo");
                console.log("Bonjour mon ami");
            }
            this._logger.info("123", "Greetings created.")
        } finally {
            this._logger.info("123", "Finally reached.")
        }
    }
}

class MyComponentA
{
    private ILogger _logger;
    private bool _console_log = true;

    public MyComponentA(ElasticSearchLogger logger)
    {
        this._logger = logger;

        this._logger.Info("123", "MyComponentA has been created.");
    }

    public void myMethod()
    {
        try
        {
            if (this._console_log)
            {
                Console.WriteLine("Hola amigo");
                Console.WriteLine("Bonjour mon ami");
            }
            this._logger.Info("123", "Greetings created.");
    }
        finally
        {
            this._logger.Info("123", "Finally reached.");
      }
    }
}

import (
	"context"
	"fmt"

	clog "github.com/pip-services3-gox/pip-services3-components-gox/log"
	ellog "github.com/pip-services3-gox/pip-services3-elasticsearch-gox/log"
)

type MyComponentA struct {
	_logger      clog.ILogger
	_console_log bool
}

func NewMyComponentA(logger *ellog.ElasticSearchLogger) *MyComponentA {
	c := &MyComponentA{}
	c._logger = logger

	c._logger.Info(context.Background(), "123", "MyComponentA has been created.")
	return c
}

func (c *MyComponentA) MyMethod() {
	if c._console_log {
		fmt.Println("Hola amigo")
		fmt.Println("Bonjour mon ami")
	}
	c._logger.Info(context.Background(), "123", "Greetings created.")
}

class MyComponentA {
  ILogger _logger;
  bool _console_log = true;

     MyComponentA(ElasticSearchLogger logger): _logger = logger {
        _logger.info('123', 'MyComponentA has been created.');
    }

    void myMethod() {
        try {
            if (_console_log) {
                print('Hola amigo');
                print('Bonjour mon ami');
            }
            _logger.info('123', 'Greetings created.')
        } finally {
            _logger.info('123', 'Finally reached.')
        }
    }
}


class MyComponentA:

    _console_log = True
    
    def __init__(self, logger: ElasticSearchLogger):
        self._logger = logger

        self._logger.info("123" , "MyComponentA has been created.")

    def mymethod(self):

        try:
            if self._console_log:
                print("Hola amigo")
                print("Bonjour mon ami")
            self._logger.info("123" , "Greetings created.")
        finally:
                self._logger.info("123" , "Finally reached.")

Not available

Creating a ElasticSearchLogger component

Once we have our custom component, we create an instance of the ElasticSearchLogger, and configure and connect it to Elasticsearch via the open() method. In our example, we have Elasticsearch running on our machine, and we connect to it via the port 9200. The open() method requires a correlation_id, which we define as “123”.

let logger = new ElasticSearchLogger();

logger.configure(ConfigParams.fromTuples(
    "connection.protocol", "http",
    "connection.host", "localhost",
    "connection.port", 9200
));

logger.open("123")

var logger = new ElasticSearchLogger();

logger.Configure(ConfigParams.FromTuples(
    "connection.protocol", "http",
    "connection.host", "localhost",
    "connection.port", 9200
));

logger.OpenAsync("123");

logger := ellog.NewElasticSearchLogger()
logger.Configure(context.Background(), cconf.NewConfigParamsFromTuples(
	"connection.protocol", "http",
	"connection.host", "localhost",
	"connection.port", 9200,
))
err := logger.Open(context.Background(), "123")
if err != nil {
	panic(err)
}

var logger = ElasticSearchLogger();

logger.configure(ConfigParams.fromTuples(
    ['connection.protocol', 'http',
    'connection.host', 'localhost',
    'connection.port', 9200]
));

logger.open('123');

logger = ElasticSearchLogger()
logger.configure(ConfigParams.from_tuples(
    "connection.protocol", "http",
    "connection.host", "localhost",
    "connection.port", 9200
))
logger.open("123")

Not available

Generating logs

Now that our connection is ready, we create an instance of our custom component and run “mymethod”. The following code shows how to do this.

let mycomponent = new MyComponentA(logger);
for (let i =0; i < 10; i++)
    mycomponent.myMethod();

var mycomponent = new MyComponentA(logger);
for (var i = 0; i < 10; i++)
    mycomponent.myMethod();

mycomponent := NewMyComponentA(logger)
for i := 0; i < 10; i++ {
	mycomponent.MyMethod()
}

var mycomponent = MyComponentA(logger);
for (var i =0; i < 10; i++) {
  mycomponent.myMethod();
}

mycomponent = MyComponentA(logger)
for i in range(10):
    mycomponent.mymethod()

Not available

And, after running the above code, we obtain the following output.

Our logs in Elasticsearch

We can check that Elasticsearch received our message by using a tool such as Elasticvue. This online application provides us we a UI where the received messages are displayed. The figure below shows the messages obtained after running the code in the previous section.

Final code

Our last step is to modify our custom component. We want it to follow the Pip.Services best practices. Thus, we add the IConfigurable and IOpenable interfaces, and the configure(), open() and close() methods. Then, we use these methods to configure, open and close our logger. The code below shows how to do this.

import { ElasticSearchLogger } from 'pip-services-elasticsearch-nodex';
import { ConfigParams, IConfigurable, IOpenable } from 'pip-services3-commons-nodex';

export class MyComponentA implements IConfigurable, IOpenable {
    private _logger: ElasticSearchLogger;
    private _console_log = true;

    public constructor(logger: ElasticSearchLogger) {
        this._logger = logger;

        this._logger.info("123", "MyComponentA has been created.");
    }

    public configure(config: ConfigParams): void {
        this._logger.configure(config);
    }

    public isOpen(): boolean {
        return this._logger.isOpen();
    }

    public async open(correlationId: string): Promise<void> {
        await this._logger.open(correlationId);
    }

    public async close(correlationId: string): Promise<void> {
        await this._logger.close(correlationId);
    }

    public myMethod(): void {
        try {
            if (this._console_log) {
                console.log("Hola amigo");
                console.log("Bonjour mon ami");
            }
            this._logger.info("123", "Greetings created.")
        } finally {
            this._logger.info("123", "Finally reached.")
        }
    }
}

using System;
using PipServices3.Commons.Config;
using PipServices3.ElasticSearch.Log;
using PipServices3.Components.Log;

namespace ExampleApp
{
    class Program
    {
        static void Main(string[] args)
        {
            var logger = new ElasticSearchLogger();

            logger.Configure(ConfigParams.FromTuples(
                "connection.protocol", "http",
                "connection.host", "localhost",
                "connection.port", 9200
            ));

            logger.OpenAsync("123").Wait();

            var mycomponent = new MyComponentA(logger);
            for (var i = 0; i < 10; i++)
                mycomponent.myMethod();
        }
    }

    class MyComponentA
    {
        private ILogger _logger;
        private bool _console_log = true;

        public MyComponentA(ElasticSearchLogger logger)
        {
            this._logger = logger;

            this._logger.Info("123", "MyComponentA has been created.");
        }

        public void myMethod()
        {
            try
            {
                if (this._console_log)
                {
                    Console.WriteLine("Hola amigo");
                    Console.WriteLine("Bonjour mon ami");
                }
                this._logger.Info("123", "Greetings created.");
        }
            finally
            {
                this._logger.Info("123", "Finally reached.");
          }
        }
    }

}

import (
	"context"
	"fmt"

	cconf "github.com/pip-services3-gox/pip-services3-commons-gox/config"
	ellog "github.com/pip-services3-gox/pip-services3-elasticsearch-gox/log"
)

func main() {
	logger := ellog.NewElasticSearchLogger()

	logger.Configure(context.Background(), cconf.NewConfigParamsFromTuples(
		"connection.protocol", "http",
		"connection.host", "localhost",
		"connection.port", 9200,
	))

	err := logger.Open(context.Background(), "123")
	if err != nil {
		panic(err)
	}

	mycomponent := NewMyComponentA(logger)
	for i := 0; i < 10; i++ {
		mycomponent.MyMethod(context.Background())
	}

}

type MyComponentA struct {
	_logger      *ellog.ElasticSearchLogger
	_console_log bool
}

func NewMyComponentA(logger *ellog.ElasticSearchLogger) *MyComponentA {
	c := &MyComponentA{}
	c._logger = logger
	c._console_log = true

	c._logger.Info(context.Background(), "123", "MyComponentA has been created.")
	return c
}

func (c *MyComponentA) Configure(ctx context.Context, config *cconf.ConfigParams) {
	c._logger.Configure(ctx, config)
}

func (c *MyComponentA) IsOpen() bool {
	return c._logger.IsOpen()
}

func (c *MyComponentA) Open(ctx context.Context, correlationId string) (err error) {
	return c._logger.Open(ctx, correlationId)
}

func (c *MyComponentA) Close(ctx context.Context, correlationId string) (err error) {
	return c._logger.Close(ctx, correlationId)
}

func (c *MyComponentA) MyMethod(ctx context.Context) {
	if c._console_log {
		fmt.Println("Hola amigo")
		fmt.Println("Bonjour mon ami")
	}
	c._logger.Info(context.Background(), "123", "Greetings created.")

	defer c._logger.Info(ctx, "123", "Finally reached.")
}

import 'package:pip_services3_commons/pip_services3_commons.dart';
import 'package:pip_services3_components/pip_services3_components.dart';
import 'package:pip_services3_elasticsearch/pip_services3_elasticsearch.dart';

void main(List<String> argument) async {
  var logger = ElasticSearchLogger();

  logger.configure(ConfigParams.fromTuples([
    'connection.protocol',
    'http',
    'connection.host',
    'localhost',
    'connection.port',
    9200
  ]));

  await logger.open('123');

  var mycomponent = MyComponentA(logger);
  for (var i = 0; i < 10; i++) {
    mycomponent.myMethod();
  }
}

class MyComponentA {
  ILogger _logger;
  bool _console_log = true;

  MyComponentA(ElasticSearchLogger logger) : _logger = logger {
    _logger.info('123', 'MyComponentA has been created.');
    }
  }

  void myMethod() {
    try {
      if (_console_log) {
        print('Hola amigo');
        print('Bonjour mon ami');
      }
      _logger.info('123', 'Greetings created.');
    } finally {
      _logger.info('123', 'Finally reached.');
    }
  }
}

from pip_services3_elasticsearch.log import ElasticSearchLogger
from pip_services3_commons.run import IOpenable
from pip_services3_commons.config import IConfigurable, ConfigParams
from typing import Optional
import time

class MyComponentA(IConfigurable, IOpenable):

    _console_log = True
    
    def __init__(self):
        self._logger = ElasticSearchLogger()

        self._logger.info("123" , "MyComponentA has been created.")

    def configure(self, config: ConfigParams):
        self._logger.configure(config)
        
    def open(self, correlation_id: Optional[str]):
        self._logger.open(correlation_id)
       
    def close(self, correlation_id: Optional[str]):
        self._logger.close(correlation_id)
        
    def mymethod(self):

        try:
            if self._console_log:
                print("Hola amigo")
                print("Bonjour mon ami")
                self._logger.info("123" , "Greetings created.")
        finally:
                self._logger.info("123" , "Finally reached.")

Not available

Elasticsearch versions

At present, Pip.Services support versions 6 and 7 of Elaticsearch.

Wrapping up

In this tutorial we have seen how to use the ElasticSearchLogger component to send log messages to Elasticsearch. First, we created a custom component that included a method and a logger. Then, we created instances of ElasticSearchLogger and of the custom component, connected our logger to Elasticsearch and run the method in order to generate log messages. After successfully running the method, we verified that Elasticsearch had received the messages by using Elasticvue, an online application that provided us with a UI. We ended the tutorial with an improved version of the custom component.