Adding logging to a component

Key takeaways

Logging Logging is the capacity to create tagged messages from events in our code.
Logging levels Logging levels: nothing, fatal, error, warn, info, debug, and trace.
ConsoleLogger PIP.Services component for displaying logging messages on the console.
CachedLogger PIP.Services component that caches log messages in memory.
CompositeLogger PIP.Services component for aggregating logging messages.
DataDogLogger, ElasticSearchLogger, CloudWatchLogger PIP.Services logger implementations for Datadog, Elasticsearch, and Amazon CloudWatch components.

Introduction

In this tutorial, you will learn how to add logging capacity to a microservice. First, we will understand what logging consists of. Then, we will use the microservice we created in the “Creating a component” tutorial, replace the printed messages with logger messages and create an exception in our business process (my_task). After running the code, we will see the tagged messages from the logger.

Once we have seen how to create a logger that displays messages on our console, we will learn how to create a composite logger, which will add the capacity to aggregate the log messages from different sources and centralize their display on our console.

Finally, we will see how to add loggers for Datadog, Elasticsearch, and Amazon CloudWatch components.

What is logging?

Logging is the capacity to create tagged messages from events in our code. These messages can inform us about the running process.

There are different logging levels. PIP.Services defines them as:

Level name Level number Description
Nothing 0 Nothing to be logged.
Fatal 1 Logs only fatal errors that cause a microservice to fail.
Error 2 Logs all errors - fatal or recoverable.
Warn 3 Logs errors and warnings.
Info 4 Logs errors and important information messages.
Debug 5 Logs everything up to high-level debugging information.
Trace 6 Logs everything down to fine-granular debugging messages.

Once generated, log messages need to be stored or displayed. PIP.Services provides specific tools for this: CachedLogger and ConsoleLogger. The first class stores log messages in memory. The second class displays them on a console. The toolkit also provides us with the CompositeLogger, which allows for message aggregation and thus, creating a centralized logging point.

Additionally, PIP.Services provides implementations of loggers for CloudWatch, ElasticSearch, and DataDog.

Once generated, log messages need to be stored or displayed. PIP.Services provides specific tools for this: CachedLogger and ConsoleLogger. The first class stores log messages in memory. The second class displays them on a console. The toolkit also provides us with the CompositeLogger, which allows for message aggregation and thus, creating a centralized logging point.

Additionally, PIP.Services provides implementations of loggers for CloudWatch, ElasticSearch, and DataDog.

Once generated, log messages need to be stored or displayed. PIP.Services provides specific tools for this: CachedLogger and ConsoleLogger. The first class stores log messages in memory. The second class displays them on a console. The toolkit also provides us with the CompositeLogger, which allows for message aggregation and thus, creating a centralized logging point.

Additionally, PIP.Services provides implementations of loggers for CloudWatch, ElasticSearch, and DataDog.

Once generated, log messages need to be stored or displayed. PIP.Services provides specific tools for this: CachedLogger and ConsoleLogger. The first class stores log messages in memory. The second class displays them on a console. The toolkit also provides us with the CompositeLogger, which allows for message aggregation and thus, creating a centralized logging point.

Additionally, PIP.Services provides implementations of loggers for CloudWatch, ElasticSearch, and DataDog.

Once generated, log messages need to be stored or displayed. PIP.Services provides specific tools for this: CachedLogger and ConsoleLogger. The first class stores log messages in memory. The second class displays them on a console. The toolkit also provides us with the CompositeLogger, which allows for message aggregation and thus, creating a centralized logging point.

Additionally, PIP.Services provides implementations of loggers for CloudWatch, ElasticSearch, and DataDog.

Not available

Now, we will see how to create a console logger and a composite logger.

Adding a console logger to our component

In our example, we will add a logger that sends messages to our console. For this, we will use the ConsoleLogger class. After we created an instance of this class, we will set the logging level to five, which will allow us to log everything up to debug level.

var logger = new ConsoleLogger();
logger.setLevel(5);



import (
	clog "github.com/pip-services4/pip-services4-go/pip-services4-observability-go/log"
)

// Logger setting
logger := *clog.NewConsoleLogger()
logger.SetLevel(5)



# Logger setting
from pip_services4_observability.log import ConsoleLogger

logger = ConsoleLogger()
logger.set_level(5)
Not available

Then, we will replace our print messages with info-level log messages. For example, print(“MyComponentA has been created.") will be replaced with logger.info(None, “MyComponentA has been created.").

Finally, we will force an exception in the my_task method. As we had explained in the “Creating a component” tutorial, this method performs business-related tasks. Thus, we can simulate a problem within it by forcibly raising an exception. This method will look like this:

public myTask(ctx: Context): void {
    // create an artificial problem
    try{
        throw Error('Logging demo. Error created');
    }
    catch (ex) {
        logger.error(ctx, ex, "Error created.")  
    }
        
}



func MyTask() {
	// create an artificial problem
	err := errors.New("Logging demo. Error created")
	logger.Error(context.Background(), err, "Error created.")
}



def my_task(self, context):
    # create an artificial problem        
    try:
        raise Exception('Logging demo', 'Error created')
    except Exception as inst:
        logger.error(context, inst, "Error created.")
Not available

And, our final code will look like this:

a) Our components

import { ConsoleLogger } from 'pip-services4-observability-node';
import { 
    ConfigParams, ICleanable, IConfigurable, IOpenable, 
    IReferenceable, IReferences, IUnreferenceable, Context, Descriptor, IContext 
} from "pip-services4-components-node";

export class MyComponentB implements IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable {
    private _param1: string = 'ABC2';
    private _param2: number = 456;
    private _opened:boolean = false;

    private _status: string;

    private _logger = new ConsoleLogger();
     

    /**
     * Creates a new instance of the component.
     */
    public constructor(){
        this._status = 'Created';
        this._logger.setLevel(5);
        this._logger.info(null, "MyComponentB has been configured.");

    }
    
    public configure(config: ConfigParams): void {
        this._param1 = config.getAsStringWithDefault('param1', this._param1);
        this._param2 = config.getAsIntegerWithDefault('param2', this._param2);

        this._logger.info(null, "MyComponentB has been configured.")
    }

    public setReferences(refs: IReferences): void {
        // pass
    }

    public isOpen(): boolean {
        // pass
    }
    public open(ctx: Context): Promise<void> {
        // pass
    }
    public close(ctx: Context): Promise<void> {
        // pass
    }

    public myTask(ctx: Context): void {
        // create an artificial problem
        try {
            throw Error('Logging demo. Error created');
        }
        catch (ex) {
            this._logger.error(ctx, ex, "Error created.")
        }

    }

    /**
     * Unsets (clears) previously set references to dependent components.
     */
    public unsetReferences(): void {
        // pass
    }

    /**
     * Clears component state.
     * @param correlationId (optional) transaction id to trace execution through call chain.
     */
    public clear(ctx: Context): Promise<void> {
        // pass
        return Promise.resolve();;
    }
}

export class MyComponentA implements IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable {
    private _param1: string = 'ABC';
    private _param2: number = 123;
    private _opened: boolean = false;

    private _status: string;

    private _another_component: MyComponentB;

    public dummy_variable = "resource variable";

    private _logger = new ConsoleLogger();


    /**
     * Creates a new instance of the component.
     */
    public constructor() {
        this._status = 'Created';
        this._logger.setLevel(5);
        this._logger.info(null, "MyComponentA has been configured.");

    }

    public configure(config: ConfigParams): void {
        this._param1 = config.getAsStringWithDefault('param1', this._param1);
        this._param2 = config.getAsIntegerWithDefault('param2', this._param2);
        this._status = "Configured";

        this._logger.info(null, "MyComponentA has been configured.")
    }

    public setReferences(refs: IReferences): void {
        this._another_component = refs.getOneRequired(
            new Descriptor("myservice", "mycomponent-b", "*", "*", "1.0")
        )
        this._status = "Configured"
        this._logger.info(null, "MyComponentA's references have been defined.")
    }

    public isOpen(): boolean {
        return this._opened;
    }
    public open(ctx: Context): Promise<void> {
        this._opened = true;
        this._status = "Open";
        this._logger.info(ctx, "MyComponentA has been opened.")
        // artificial problem
        this.myTask(ctx);
        return Promise.resolve();
    }
    public close(ctx: Context): Promise<void> {
        this._opened = false;
        this._status = "Closed";
        this._logger.info(ctx, "MyComponentA has been closed.");
        return Promise.resolve();
    }

    public myTask(ctx: Context): void {
        // create an artificial problem
        try {
            throw Error('Logging demo. Error created');
        }
        catch (ex) {
            this._logger.error(ctx, ex, "Error created.")
        }

    }

    /**
     * Unsets (clears) previously set references to dependent components.
     */
    public unsetReferences(): void {
        this._another_component = null;
        this._status = "Un-referenced";
        this._logger.info(null, "References cleared");
    }

    /**
     * Clears component state.
     * @param correlationId (optional) transaction id to trace execution through call chain.
     */
    public clear(ctx: Context): Promise<void> {
        this.dummy_variable = null;
        this._status = null;
        this._logger.info(ctx, "Resources cleared");
        return Promise.resolve();
    }
}



import (
	"context"
	"errors"

	cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
	crefer "github.com/pip-services4/pip-services4-go/pip-services4-components-go/refer"
	crun "github.com/pip-services4/pip-services4-go/pip-services4-components-go/run"
	clog "github.com/pip-services4/pip-services4-go/pip-services4-observability-go/log"
)

type MyComponentB struct {
	crefer.IReferences
	crefer.IUnreferenceable
	cconfig.IConfigurable
	crun.IOpenable
	crun.ICleanable

	_status string
	_param1 string
	_param2 int
	_opened bool
	_logger clog.ILogger

	dummy_variable interface{}
}

// Creates a new instance of the component.
func NewMyComponentB() *MyComponentB {
	component := &MyComponentB{
		_status: "Created",
		_param1: "ABC2",
		_param2: 456,
		_opened: false,
		_logger: clog.NewConsoleLogger(),

		dummy_variable: "resource variable",
	}

	component._logger.SetLevel(5)
	component._logger.Info(context.Background(), "MyComponentB has been created.")

	return component
}

func (c *MyComponentB) Configure(ctx context.Context, config *cconfig.ConfigParams) {
	c._param1 = config.GetAsStringWithDefault("param1", c._param1)
	c._param2 = config.GetAsIntegerWithDefault("param2", c._param2)

	c._logger.Info(ctx, "MyComponentB has been configured.")
}

func (c *MyComponentB) SetReferences(ctx context.Context, references *crefer.References) {
	// pass
}

func (c *MyComponentB) isOpen() bool {
	// pass
	return true
}

func (c *MyComponentB) Open(ctx context.Context) {
	// pass
}

func (c *MyComponentB) Close(ctx context.Context) {
	// pass
}

func (c *MyComponentB) MyTask(ctx context.Context) {
	// pass
}

// Unsets (clears) previously set references to dependent components.
func (c *MyComponentB) UnsetReferences(ctx context.Context) {
	// pass
}

// Clears component state.
// - correlationId: (optional) transaction id to trace execution through call chain.
func (c *MyComponentB) Clear(ctx context.Context) {
	// pass
}

type MyComponentA struct {
	crefer.IReferences
	crefer.IUnreferenceable
	cconfig.IConfigurable
	crun.IOpenable
	crun.ICleanable

	_logger clog.ILogger

	_status string
	_param1 string
	_param2 int
	_opened bool

	dummy_variable interface{}

	_another_component interface{}
}

// Creates a new instance of the component.
func NewMyComponentA() *MyComponentA {
	component := &MyComponentA{
		_status: "Created",
		_param1: "ABC2",
		_param2: 456,
		_opened: false,
		_logger: clog.NewConsoleLogger(),

		dummy_variable: "dummy_variable setted",
	}

	component._logger.SetLevel(5)
	component._logger.Info(context.Background(), "MyComponentA has been created.")

	return component
}

func (c *MyComponentA) Configure(ctx context.Context, config *cconfig.ConfigParams) {
	c._param1 = config.GetAsStringWithDefault("param1", c._param1)
	c._param2 = config.GetAsIntegerWithDefault("param2", c._param2)
	c._status = "Configured"

	c._logger.Info(ctx, "MyComponentB has been configured.")
}

func (c *MyComponentA) SetReferences(ctx context.Context, references *crefer.References) {
	_another_component, err := references.GetOneRequired(crefer.NewDescriptor("myservice", "mycomponent-b", "*", "*", "1.0"))

	if err != nil {
		panic("Error: Another Component is not in refs")
	}

	c._another_component = _another_component.(MyComponentB)
	c._status = "Configured"

	c._logger.Info(ctx, "MyComponentA's references have been defined.")
}

func (c *MyComponentA) isOpen() bool {
	return c._opened
}

func (c *MyComponentA) Open(ctx context.Context, correlationId string) {
	c._opened = true
	c._status = "Open"
	c._logger.Info(ctx, "MyComponentA has been opened.")
	// artificial problem
	c.MyTask(ctx)
}

func (c *MyComponentA) Close(ctx context.Context) {
	c._opened = false
	c._status = "Closed"
	c._logger.Info(ctx, "MyComponentA has been closed.")
}

func (c *MyComponentA) MyTask(ctx context.Context) {
	// create an artificial problem
	c._logger.Error(ctx, errors.New("Logging demo. Error created"), "Error created.")
}

// Unsets (clears) previously set references to dependent components.
func (c *MyComponentA) UnsetReferences(ctx context.Context) {
	c._another_component = nil
	c._status = "Un-referenced"
	c._logger.Info(ctx, "References cleared")
}

// Clears component state.
// - correlationId: (optional) transaction id to trace execution through call chain.
func (c *MyComponentA) Clear(ctx context.Context) {
	c.dummy_variable = nil
	c._status = ""
	c._logger.Info(ctx, "Resources cleared")
}
Not available

b) Our factory

// Creating a factory

import { Factory } from 'pip-services4-components-node';

let MyFactory1 = new Factory();

MyFactory1.registerAsType(new Descriptor("myservice", "mycomponentA", "default", "*", "1.0"), MyComponentA)
MyFactory1.registerAsType(new Descriptor("myservice", "mycomponent-b", "default", "*", "1.0"), MyComponentB)



import (
      cbuild "github.com/pip-services4/pip-services4-go/pip-services4-components-go/build"
)

MyFactory1 := cbuild.NewFactory()

MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponentA", "default", "*", "1.0"), NewMyComponentA)
MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponent-b", "default", "*", "1.0"), NewMyComponentB)



# Creating a factory

from pip_services4_components.build import Factory 
MyFactory1 = Factory()

MyFactory1.register_as_type(Descriptor("myservice", "mycomponentA", "default", "*", "1.0"), MyComponentA)
MyFactory1.register_as_type(Descriptor("myservice", "mycomponent-b", "default", "*", "1.0"), MyComponentB)
Not available

c) Our service

// Creating a process container

class MyProcess extends ProcessContainer{
    public constructor() {
        super('myservice', 'My service running as a process');
        this._configPath = './config.yaml'
        this._factories.add(MyFactory1)
    }
}





import (
      cproc "github.com/pip-services4/pip-services4-go/pip-services4-container-go/container"
)

// Creating a process container
type MyProcess struct {
	*cproc.ProcessContainer
}

func NewMyProcess() *MyProcess {
	c := MyProcess{}
	c.ProcessContainer = cproc.NewProcessContainer("myservice", "My service running as a process")
	c.SetConfigPath("./config/config.yml")
	MyFactory1 := cbuild.NewFactory()

	MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponentA", "default", "*", "1.0"), NewMyComponentA)
	MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponent-b", "default", "*", "1.0"), NewMyComponentB)

	c.AddFactory(MyFactory1)
	return &c
}



# Creating a process container

from pip_services4_container.container import ProcessContainer

class MyProcess(ProcessContainer):
    def __init__(self):
        super(MyProcess, self).__init__('myservice', 'My service running as a process')
        self._config_path = 'C:\\Users\\Eugenio\\2024-PIP-Tutorials\\config2.yaml'
        self._factories.add(MyFactory1)
Not available

d) The dynamic configuration file for the components:

config.yaml

---
- descriptor: "myservice:mycomponentA:*:default:1.0"
  param1: XYZ
  param2: 987

- descriptor: myservice:mycomponent-b:*:*:1.0
  param1: XYZ
  param2: 987

e) Running our service

let MyProcess = require('./obj/example').MyProcess;

try {
    let proc = new MyProcess();
    proc.run(process.argv);
} catch (ex) {
    console.error(ex);
}




func main() {
	proc := NewMyProcess()
	proc.Run(context.Background(), os.Environ())
}



# -*- coding: utf-8 -*-

if __name__ == '__main__':
    runner = MyProcess()
    print("Run")
    try:
        runner.run()
    except Exception as ex:
        print(ex)
Not available

After running this code, we will get the following result:

Console logger messages

As we can see from the above figure, the program has logged all messages with level info and from our artificial error.

This concludes our first task.

Adding a composite logger to our component

Now, we will extend our logging capacity by adding a composite logger. This logger will allow us to aggregate all loggers from our component’s references into a centralized log. Our code will remain the same, except that now we need to create a composite logger for MyComponentA. For this, we will create an instance of this logger and set the logging level to 5.

Then, we will use the configure and set_references methods to let our composite logger know where to ask for log messages. Our factory and process container code sections will remain the same, but we will have to add a reference to our console logger in our configuration file. The syntax will be:

# Console logger
- descriptor: "pip-services:logger:console:default:1.0"
  level: {{LOG_LEVEL}}{{^LOG_LEVEL}}info{{/LOG_LEVEL}}

Finally, we will add a console logger to MyComponentB. After these changes, our component section will look like this:

import { ConsoleLogger, CompositeLogger } from 'pip-services4-observability-node';
import { 
    ConfigParams, ICleanable, IConfigurable, IOpenable, 
    IReferenceable, IReferences, IUnreferenceable, Context, Descriptor, IContext 
} from "pip-services4-components-node";

export class MyComponentB implements IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable {
    private _param1: string = 'ABC2';
    private _param2: number = 456;
    private _opened:boolean = false;

    private _status: string;

    private _logger = new ConsoleLogger();
     

    /**
     * Creates a new instance of the component.
     */
    public constructor(){
        this._status = 'Created';
        this._logger.setLevel(5);
        this._logger.info(null, "MyComponentB has been configured.");

    }
    
    public configure(config: ConfigParams): void {
        this._param1 = config.getAsStringWithDefault('param1', this._param1);
        this._param2 = config.getAsIntegerWithDefault('param2', this._param2);

        this._logger.info(null, "MyComponentB has been configured.")
    }

    public setReferences(refs: IReferences): void {
        // pass
    }

    public isOpen(): boolean {
        // pass
    }
    public open(correlationId: string): Promise<void> {
        // pass
    }
    public close(correlationId: string): Promise<void> {
        // pass
    }

    public myTask(ctx: Context): void {
        // create an artificial problem
        try {
            throw Error('Logging demo. Error created');
        }
        catch (ex) {
            this._logger.error(ctx, ex, "Error created.")
        }

    }

    /**
     * Unsets (clears) previously set references to dependent components.
     */
    public unsetReferences(): void {
        // pass
    }

    /**
     * Clears component state.
     * @param correlationId (optional) transaction id to trace execution through call chain.
     */
    public clear(correlationId: string): Promise<void> {
        // pass
    }
}

export class MyComponentA implements IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable {
    private _param1: string = 'ABC';
    private _param2: number = 123;
    private _opened: boolean = false;

    private _status: string;

    private _another_component: MyComponentB;

    public dummy_variable = "resource variable";

    private _logger = new CompositeLogger();


    /**
     * Creates a new instance of the component.
     */
    public constructor() {
        this._status = 'Created';
        this._logger.setLevel(5);
        this._logger.info(null, "MyComponentA has been configured.");

    }

    public configure(config: ConfigParams): void {
        this._param1 = config.getAsStringWithDefault('param1', this._param1);
        this._param2 = config.getAsIntegerWithDefault('param2', this._param2);
        this._status = "Configured";

        this._logger.info(null, "MyComponentA has been configured.")
    }

    public setReferences(refs: IReferences): void {
        this._another_component = refs.getOneRequired(
            new Descriptor("myservice", "mycomponent-b", "*", "*", "1.0")
        )
        this._status = "Configured"
        this._logger.setReferences(refs);

        this._logger.info(null, "MyComponentA's references have been defined.")
    }

    public isOpen(): boolean {
        return this._opened;
    }
    public open(ctx: Context): Promise<void> {
        this._opened = true;
        this._status = "Open";
        this._logger.info(ctx, "MyComponentA has been opened.")
        // artificial problem
        this.myTask(ctx);
    }
    public close(ctx: Context): Promise<void> {
        this._opened = false;
        this._status = "Closed";
        this._logger.info(ctx, "MyComponentA has been closed.");
    }

    public myTask(ctx: Context): void {
        // create an artificial problem
        try {
            throw Error('Logging demo. Error created');
        }
        catch (ex) {
            this._logger.error(ctx, ex, "Error created.")
        }

    }

    /**
     * Unsets (clears) previously set references to dependent components.
     */
    public unsetReferences(): void {
        this._another_component = null;
        this._status = "Un-referenced";
        this._logger.info(null, "References cleared");
    }

    /**
     * Clears component state.
     * @param correlationId (optional) transaction id to trace execution through call chain.
     */
    public clear(ctx: Context): Promise<void> {
        this.dummy_variable = null;
        this._status = null;
        this._logger.info(ctx, "Resources cleared");
    }
}




import (
	"context"
	"errors"
	"os"

	cbuild "github.com/pip-services4/pip-services4-go/pip-services4-components-go/build"
	cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
	crefer "github.com/pip-services4/pip-services4-go/pip-services4-components-go/refer"
	crun "github.com/pip-services4/pip-services4-go/pip-services4-components-go/run"
	cproc "github.com/pip-services4/pip-services4-go/pip-services4-container-go/container"
	clog "github.com/pip-services4/pip-services4-go/pip-services4-observability-go/log"
)

type MyComponentB struct {
	crefer.IReferences
	crefer.IUnreferenceable
	cconfig.IConfigurable
	crun.IOpenable
	crun.ICleanable

	_status string
	_param1 string
	_param2 int
	_opened bool
	_logger clog.ILogger

	dummy_variable interface{}
}

// Creates a new instance of the component.
func NewMyComponentB() *MyComponentB {
	component := &MyComponentB{
		_status: "Created",
		_param1: "ABC2",
		_param2: 456,
		_opened: false,
		_logger: clog.NewConsoleLogger(),

		dummy_variable: "resource variable",
	}

	component._logger.SetLevel(5)
	component._logger.Info(context.Background(), "MyComponentB has been created.")

	return component
}

func (c *MyComponentB) Configure(ctx context.Context, config *cconfig.ConfigParams) {
	c._param1 = config.GetAsStringWithDefault("param1", c._param1)
	c._param2 = config.GetAsIntegerWithDefault("param2", c._param2)

	c._logger.Info(ctx, "MyComponentB has been configured.")
}

func (c *MyComponentB) SetReferences(ctx context.Context, references *crefer.References) {
	// pass
}

func (c *MyComponentB) isOpen() bool {
	// pass
	return true
}

func (c *MyComponentB) Open(ctx context.Context) {
	// pass
}

func (c *MyComponentB) Close(ctx context.Context) {
	// pass
}

func (c *MyComponentB) MyTask(ctx context.Context) {
	// pass
}

// Unsets (clears) previously set references to dependent components.
func (c *MyComponentB) UnsetReferences(ctx context.Context) {
	// pass
}

// Clears component state.
// - correlationId: (optional) transaction id to trace execution through call chain.
func (c *MyComponentB) Clear(ctx context.Context) {
	// pass
}

type MyComponentA struct {
	crefer.IReferences
	crefer.IUnreferenceable
	cconfig.IConfigurable
	crun.IOpenable
	crun.ICleanable

	_logger clog.ILogger

	_status string
	_param1 string
	_param2 int
	_opened bool

	dummy_variable interface{}

	_another_component interface{}
}

// Creates a new instance of the component.
func NewMyComponentA() *MyComponentA {
	component := &MyComponentA{
		_status: "Created",
		_param1: "ABC2",
		_param2: 456,
		_opened: false,
		_logger: clog.NewConsoleLogger(),

		dummy_variable: "dummy_variable setted",
	}

	component._logger.SetLevel(5)
	component._logger.Info(context.Background(), "MyComponentA has been created.")

	return component
}

func (c *MyComponentA) Configure(ctx context.Context, config *cconfig.ConfigParams) {
	c._param1 = config.GetAsStringWithDefault("param1", c._param1)
	c._param2 = config.GetAsIntegerWithDefault("param2", c._param2)
	c._status = "Configured"

	c._logger.Info(ctx, "MyComponentB has been configured.")
}

func (c *MyComponentA) SetReferences(ctx context.Context, references *crefer.References) {
	_another_component, err := references.GetOneRequired(crefer.NewDescriptor("myservice", "mycomponent-b", "*", "*", "1.0"))

	if err != nil {
		panic("Error: Another Component is not in refs")
	}

	c._another_component = _another_component.(MyComponentB)
	c._status = "Configured"

	c._logger.Info(ctx, "MyComponentA's references have been defined.")
}

func (c *MyComponentA) isOpen() bool {
	return c._opened
}

func (c *MyComponentA) Open(ctx context.Context, correlationId string) {
	c._opened = true
	c._status = "Open"
	c._logger.Info(ctx, "MyComponentA has been opened.")
	// artificial problem
	c.MyTask(ctx)
}

func (c *MyComponentA) Close(ctx context.Context) {
	c._opened = false
	c._status = "Closed"
	c._logger.Info(ctx, "MyComponentA has been closed.")
}

func (c *MyComponentA) MyTask(ctx context.Context) {
	// create an artificial problem
	c._logger.Error(ctx, errors.New("Logging demo. Error created"), "Error created.")
}

// Unsets (clears) previously set references to dependent components.
func (c *MyComponentA) UnsetReferences(ctx context.Context) {
	c._another_component = nil
	c._status = "Un-referenced"
	c._logger.Info(ctx, "References cleared")
}

// Clears component state.
// - correlationId: (optional) transaction id to trace execution through call chain.
func (c *MyComponentA) Clear(ctx context.Context) {
	c.dummy_variable = nil
	c._status = ""
	c._logger.Info(ctx, "Resources cleared")
}

// Creating a process container
type MyProcess struct {
	*cproc.ProcessContainer
}

func NewMyProcess() *MyProcess {
	c := MyProcess{}
	c.ProcessContainer = cproc.NewProcessContainer("myservice", "My service running as a process")
	c.SetConfigPath("./config/config.yml")
	MyFactory1 := cbuild.NewFactory()

	MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponentA", "default", "*", "1.0"), NewMyComponentA)
	MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponent-b", "default", "*", "1.0"), NewMyComponentB)

	c.AddFactory(MyFactory1)
	return &c
}

func main() {
	proc := NewMyProcess()
	proc.Run(context.Background(), os.Environ())
}




from pip_services4_components.config import IConfigurable, ConfigParams
from pip_services4_components.refer import IReferenceable, IReferences, Descriptor, IUnreferenceable
from pip_services4_components.run import IOpenable, ICleanable
from pip_services4_observability.log import ConsoleLogger, LogLevel, CompositeLogger

class MyComponentB(IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable):
    _param1 = 'ABC2'
    _param2 = 456
    _opened = False

    __logger = ConsoleLogger()
    __logger.set_level(5)

    dummy_variable = "resource variable"
    
    def __init__(self):
        """
        Creates a new instance of the component.
        """
        self._status = "Created"
        self.__logger.info(None, "MyComponentB has been created.")
        
    def configure(self, config):
        self._param1 = config.get_as_string_with_default("param1", self._param1)
        self._param2 = config.get_as_integer_with_default("param2", self._param2)
        
    def set_references(self, references):
        pass
        
    def is_open(self):
        pass

    def open(self, context):
        pass

    def close(self, context):
        pass
        
    def my_task(self, context):
        pass

    def unset_references(self):
        """
        Unsets (clears) previously set references to dependent components.
        """
        pass
    
    def clear(self, context):
        """
        Clears component state.
        :param context: (optional) transaction id to trace execution through call chain.
        """
        pass



class MyComponentA(IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable):
    _param1 = 'ABC'
    _param2 = 123
    _another_component: MyComponentB
    _open = False
    _status = None
    __logger = CompositeLogger()
    __logger.set_level(5)

    def __init__(self):
        """
        Creates a new instance of the component.
        """
        self._status = "Created"
        
        self.__logger.info(None, "MyComponentA has been created.")

            
    def configure(self, config):
        self._param1 = config.get_as_string_with_default("param1", 'ABC')
        self._param2 = config.get_as_integer_with_default("param2", 123)
        self._status = "Configured"
        self.__logger.configure(config)
        

    def set_references(self, references):
        self._another_component = references.get_one_required(
            Descriptor("myservice", "mycomponent-b", "*", "*", "1.0")
        )
        self._status = "Configured"
        self.__logger.set_references(references)
        self.__logger.info(None,"MyComponentA's references have been defined.")
        
    def is_open(self):
        return self._open

    def open(self, context):
        self._open = True
        self._status = "Open"
        self.__logger.info(None,"MyComponentA has been opened.")
        # artificial problem
        self.my_task(context)

    def close(self, context):
        self._opened = False
        self._status = "Closed"
        self.__logger.info(context,"MyComponentA has been closed.")
        
    def my_task(self, context):
        # create an artificial problem        
        try:
            raise Exception('Logging demo', 'Error created')
        except Exception as inst:
            self.__logger.error(context, inst, "Error created.")  

    def unset_references(self):
        """
        Unsets (clears) previously set references to dependent components.
        """
        self._another_component = None
        self._status = "Un-referenced"
        self.__logger.info(None, "References cleared")
    
    def clear(self, context):
        """
        Clears component state.
        :param context: (optional) transaction id to trace execution through call chain.
        """
        self.dummy_variable = None
        self._status = None
        self.__logger.info(context, "Resources cleared")
Not available

Once we run our service with the re-defined components, we will get the following results:

Composite logger messages

As we can see, we have log messages received from both MyComponentA and MyComponentB.

Adding specific loggers

As we said earlier, PIP.Services has specific loggers for Datadog, Elasticsearch, and Amazon CloudWatch. The process to add any of them to a component is similar to what we saw in our console logger example: we need to declare an instance of the logger, configure it, set the message level, and add the messages we need. Here below are examples of how to define each of them.

a) Datadog

import { DataDogLogger } from 'pip-services4-datadog-node';

let logger = new DataDogLogger();

logger.configure(ConfigParams.fromTuples(
    "credential.access_key", "827349874395872349875493"
));

logger.setLevel(5);

await logger.open(ctx);

logger.info(ctx, "My message");




import (
	"context"

	cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
	logdatadog "github.com/pip-services4/pip-services4-go/pip-services4-datadog-go/log"
)

func main() {

	logger := logdatadog.NewDataDogLogger()

	logger.Configure(context.Background(), cconfig.NewConfigParamsFromTuples(
		"credential.access_key", "827349874395872349875493",
	))

	logger.SetLevel(5)
	_ = logger.Open(context.Background())
	logger.Info(context.Background(), "My message")

}




from pip_services4_components.context import IContext
from pip_services4_datadog.log import DataDogLogger
from pip_services4_components.config import ConfigParams

logger = DataDogLogger()

logger.configure(ConfigParams.from_tuples(
   "credential.access_key", "827349874395872349875493"
))

logger.set_level(5)

logger.open("123")

context = IContext

context={}

logger.info(context , "My message")
Not available

b) Elasticsearch

import { ElasticSearchLogger } from 'pip-services4-elasticsearch-node';

let logger = new ElasticSearchLogger();

logger.configure(ConfigParams.fromTuples(
    "connection.protocol", "http",
    "connection.host", "localhost",
    "connection.port", 9200
));

logger.setLevel(5);

await logger.open(ctx);

logger.info(ctx, "My message");



import (
	"context"

	cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
	logelastic "github.com/pip-services4/pip-services4-go/pip-services4-elasticsearch-go/log"
)

func main() {

	logger := logelastic.NewElasticSearchLogger()
	logger.Configure(context.Background(), cconfig.NewConfigParamsFromTuples(
		"connection.protocol", "http",
		"connection.host", "localhost",
		"connection.port", 9200,
	))

	logger.SetLevel(5)
	_ = logger.Open(context.Background())
	logger.Info(context.Background(), "My message")

}



from pip_services4_elasticsearch.log import ElasticSearchLogger
from pip_services4_components.config import ConfigParams

logger = ElasticSearchLogger()
logger.configure(ConfigParams.from_tuples(
    "connection.protocol", "http",
    "connection.host", "localhost",
    "connection.port", 9200
))

logger.set_level(5)

logger.open("123")

context = IContext

context={}
logger.info(context , "My message")
Not available

c) Amazon CloudWatch

import { CloudWatchLogger } from 'pip-services4-aws-node';

let logger = new CloudWatchLogger();

logger.configure(ConfigParams.fromTuples(
    "stream", "mystream",
    "group", "mygroup",
    "connection.region", "us-east-1",
    "connection.access_id", "XXXXXXXXXXX",
    "connection.access_key", "XXXXXXXXXXX"
));

logger.setLevel(5);

await logger.open(ctx);

logger.info(ctx, "My message");



import (
	"context"

	cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
	logaws "github.com/pip-services4/pip-services4-go/pip-services4-aws-go/log"
)

func main() {

	logger := logaws.NewCloudWatchLogger()
	logger.Configure(context.Background(), cconfig.NewConfigParamsFromTuples(
		"stream", "mystream",
		"group", "mygroup",
		"connection.region", "us-east-1",
		"connection.access_id", "XXXXXXXXXXX",
		"connection.access_key", "XXXXXXXXXXX",
	))

	logger.SetLevel(5)
	_ = logger.Open(context.Background())
	logger.Info(context.Background(), "My message")
}
Not available

Example

In this example, we will combine two features: displaying log information on our console and sending log information to ElasticSearch.

Pre-requisites

In order to be able to create a composite logger for both outputs, we need to import the following components:

import { CompositeLogger, ConsoleLogger, LogLevel } from 'pip-services4-observability-node';
import { ConfigParams, Context, Descriptor } from "pip-services4-components-node";
import { ElasticSearchLogger } from 'pip-services4-elasticsearch-node';



import (
	"errors"
	"fmt"

	cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
	cref "github.com/pip-services4/pip-services4-go/pip-services4-components-go/refer"
	clog "github.com/pip-services4/pip-services4-go/pip-services4-observability-go/log"
	elasticlog "github.com/pip-services4/pip-services4-go/pip-services4-elasticsearch-go/log"
)



from typing import Optional

from pip_services4_components.config import IConfigurable, ConfigParams
from pip_services4_components.refer import IReferenceable, IReferences, Descriptor, References
from pip_services4_components.run import IOpenable
from pip_services4_components.build import Factory
from pip_services4_observability.log import ConsoleLogger, CompositeLogger, LogLevel
from pip_services4_container import ProcessContainer
from pip_services4_elasticsearch.build import DefaultElasticSearchFactory
from pip_services4_elasticsearch.log import ElasticSearchLogger
Not available

Component creation

The next step is to create our two logging components, namely our console and ElasticSearch. For this, we instantiate the ConsoleLogger and ElasticSearchLogger classes. Our code is:


// ElasticSearchLogger sends a log to ElasticSearch
let elasticSearchLogger = new ElasticSearchLogger();
// Console logger writes messages to console
let consoleLogger = new ConsoleLogger();



// ElasticSearchLogger sends a log to ElasticSearch
elasticSearchLogger := elasticlog.NewElasticSearchLogger()
// Console logger writes messages to console
consoleLogger := clog.NewConsoleLogger()



# ElasticSearchLogger sends a log to ElasticSearch
elasticsearch_logger: ElasticSearchLogger = ElasticSearchLogger()
# Console logger writes messages to console
console_logger: ConsoleLogger = ConsoleLogger()
Not available

Once these instances have been created, we can configure them. To address this, we create a common configuration object that contains all the necessary parameters and their values:

let config = ConfigParams.fromTuples(
    // Common config
    "source", "my_component_log",
    "level", LogLevel.Debug,
    // Elastic config
    "index", "log",
    "daily", true,
    "date_format", "YYYYMMDD",
    "connection.host", "localhost",
    "connection.port", 9200,
    "connection.protocol", "http",
);



config := cconf.NewConfigParamsFromTuples(
	// Common config
	"source", "my_component_log",
	"level", clog.LevelDebug,

	// Elastic config
	"index", "log",
	"daily", true,
	"date_format", "yyyyMMdd",
	"connection.host", "localhost",
	"connection.port", 9200,
	"connection.protocol", "http",
)



config = ConfigParams.from_tuples(
    # Common config
    'source', 'my_component_log',
    'level', LogLevel.Debug,
    
    # Elastic config
    'index', 'log',
    'daily', True,
    "date_format", 'YYYYMMDD',
    'connection.host', 'localhost',
    'connection.port', 9200,
    "connection.protocol", "http",
)
Not available

and we assign these values to the respective objects

elasticSearchLogger.configure(config);
await elasticSearchLogger.open(ctx);



elasticSearchLogger.Configure(context.Background(), config)
_ = elasticSearchLogger.Open(context.Background(), "123")



elasticsearch_logger.configure(config)
elasticsearch_logger.open(context)
Not available

Next, we connect our ElasticSearch logger:

consoleLogger.configure(config);



consoleLogger.Configure(context.Background(), config)



console_logger.configure(config)
Not available
Composite logger

After defining our two loggers, we define a composite logger that manages both. For this, we first declare a reference object that points to both loggers:

let references = References.fromTuples(
    new Descriptor("my-component", "logger", "console", "default", "1.0"), elasticSearchLogger,
    new Descriptor("my-component", "logger", "elasticsearch", "default", "1.0"), consoleLogger
);



references := cref.NewReferencesFromTuples(context.Background(),
	cref.NewDescriptor("my-component", "logger", "console", "default", "1.0"), elasticSearchLogger,
	cref.NewDescriptor("my-component", "logger", "elasticsearch", "default", "1.0"), consoleLogger,
)




references = References.from_tuples(
    Descriptor('my-component', 'logger', 'console', 'default', '1.0'), elasticsearch_logger,
    Descriptor('my-component', 'logger', 'elasticsearch', 'default', '1.0'), console_logger
)
Not available

Then, we create an instance of the CompositeLogger class:


let logger = new CompositeLogger();



logger := clog.NewCompositeLogger()



logger = CompositeLogger()
Not available

And, we add our references to it:

logger.setReferences(references);



logger.SetReferences(context.Background(), references)



logger.set_references(references)
Not available

Adding log messages:

Now that our structure is complete, we can create different log messages, which will be sent to ElasticSearch and the console after executing our code:

logger.info(ctx, "Composite logger is configured and ready for using");
logger.warn(ctx, "Example warning");
logger.error(ctx, new Error("Example error"), "Error message");
logger.debug(ctx, "Debug info");
logger.fatal(ctx, new Error("Fatal error"), "Error that crash the process");



logger.Info(context.Background(), "Composite logger is configured and ready for using")
logger.Warn(context.Background(), "Example warning")
logger.Error(context.Background(), errors.New("Example error"), "Error message")
logger.Debug(context.Background(), "Debug info")
logger.Fatal(context.Background(), errors.New("Fatal error"), "Error that crash the process")



logger.info(context, "Composite logger is configured and ready for using")
logger.warn(context, "Example warning")
logger.error(context, Exception("Example error"), "Error message")
logger.debug(context, "Debug info")
logger.fatal(context, Exception("Fatal error"), "Error that crash the process")
Not available

After code execution, the following messages will appear on our console:

Console logger messages

And, we can see the ElasticSearch messages by using the URL:

http://localhost:9200/index_name/_search?pretty

where index_name is is “log” + “-” + current date (in date_format from config params).

For example, for January 25, 2022, our URL is

http://localhost:9200/log-20220125/_search?pretty

which will show:

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 5,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "log-20220127",
        "_type" : "_doc",
        "_id" : "10399e390171415ca122dbb307f67eea",
        "_score" : 1.0,
        "_source" : {
          "time" : "2022-01-26T23:12:01.954Z",
          "level" : "INFO",
          "source" : "my_component_log",
          "correlation_id" : "123",
          "error" : null,
          "message" : "Composite logger is configured and ready for using"
        }
      },
      {
        "_index" : "log-20220127",
        "_type" : "_doc",
        "_id" : "ed1d25a2051c444783f4539861dd44f1",
        "_score" : 1.0,
        "_source" : {
          "time" : "2022-01-26T23:12:01.956Z",
          "level" : "WARN",
          "source" : "my_component_log",
          "correlation_id" : "123",
          "error" : null,
          "message" : "Example warning"
        }
      },
      {
        "_index" : "log-20220127",
        "_type" : "_doc",
        "_id" : "0cc6fdd3d16447e6a9fa5f0ec111edc9",
        "_score" : 1.0,
        "_source" : {
          "time" : "2022-01-26T23:12:01.957Z",
          "level" : "ERROR",
          "source" : "my_component_log",
          "correlation_id" : "123",
          "error" : {
            "type" : "Error",
            "category" : "Unknown",
            "status" : 500,
            "code" : "UNKNOWN",
            "message" : "Example error",
            "stack_trace" : "Error: Example error\n    at C:\\Users\\user1\\OneDrive\\Desktop\\examples\\node\\obj\\elastic.js:35:29\n    at Generator.next (<anonymous>)\n    at fulfilled (C:\\Users\\user1\\OneDrive\\Desktop\\examples\\node\\obj\\elastic.js:5:58)"
          },
          "message" : "Error message"
        }
      },
      {
        "_index" : "log-20220127",
        "_type" : "_doc",
        "_id" : "039cf650b46b4ca49c513df617055dc3",
        "_score" : 1.0,
        "_source" : {
          "time" : "2022-01-26T23:12:01.958Z",
          "level" : "DEBUG",
          "source" : "my_component_log",
          "correlation_id" : "123",
          "error" : null,
          "message" : "Debug info"
        }
      },
      {
        "_index" : "log-20220127",
        "_type" : "_doc",
        "_id" : "7f8e1ef9a82643f382cffa52f9110ce3",
        "_score" : 1.0,
        "_source" : {
          "time" : "2022-01-26T23:12:01.959Z",
          "level" : "FATAL",
          "source" : "my_component_log",
          "correlation_id" : "123",
          "error" : {
            "type" : "Error",
            "category" : "Unknown",
            "status" : 500,
            "code" : "UNKNOWN",
            "message" : "Fatal error",
            "stack_trace" : "Error: Fatal error\n    at C:\\Users\\user1\\OneDrive\\Desktop\\examples\\node\\obj\\elastic.js:37:29\n    at Generator.next (<anonymous>)\n    at fulfilled (C:\\Users\\user1\\OneDrive\\Desktop\\examples\\node\\obj\\elastic.js:5:58)"
          },
          "message" : "Error that crash the process"
        }
      }
    ]
  }
}








{
  "took" : 5,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 5,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "log-20220125",
        "_type" : "_doc",
        "_id" : "8cd3ef625b804ffdb995827e59bac01f",
        "_score" : 1.0,
        "_source" : {
          "correlation_id" : "123",
          "error" : null,
          "level" : 4,
          "message" : "Composite logger is configured and ready for using",
          "source" : null,
          "time" : "2022-01-25T17:47:53.274931"
        }
      },
      {
        "_index" : "log-20220125",
        "_type" : "_doc",
        "_id" : "e636b3a8b76e44cabea31e9e92f0b451",
        "_score" : 1.0,
        "_source" : {
          "correlation_id" : "123",
          "error" : null,
          "level" : 3,
          "message" : "Example warning",
          "source" : null,
          "time" : "2022-01-25T17:47:53.274931"
        }
      },
      {
        "_index" : "log-20220125",
        "_type" : "_doc",
        "_id" : "144d963d95c74b4a84e1558dde3704f9",
        "_score" : 1.0,
        "_source" : {
          "correlation_id" : "123",
          "error" : {
            "category" : "Unknown",
            "cause" : null,
            "code" : "UNKNOWN",
            "correlation_id" : null,
            "details" : null,
            "message" : "Example error",
            "stack_trace" : null,
            "status" : 500,
            "type" : null
          },
          "level" : 2,
          "message" : "Error message",
          "source" : null,
          "time" : "2022-01-25T17:47:53.274931"
        }
      },
      {
        "_index" : "log-20220125",
        "_type" : "_doc",
        "_id" : "25226536425246fab9b78a3dca4919fd",
        "_score" : 1.0,
        "_source" : {
          "correlation_id" : "123",
          "error" : null,
          "level" : 5,
          "message" : "Debug info",
          "source" : null,
          "time" : "2022-01-25T17:47:53.275937"
        }
      },
      {
        "_index" : "log-20220125",
        "_type" : "_doc",
        "_id" : "d49a4fda786241af98638b0d82c93ec9",
        "_score" : 1.0,
        "_source" : {
          "correlation_id" : "123",
          "error" : {
            "category" : "Unknown",
            "cause" : null,
            "code" : "UNKNOWN",
            "correlation_id" : null,
            "details" : null,
            "message" : "Fatal error",
            "stack_trace" : null,
            "status" : 500,
            "type" : null
          },
          "level" : 1,
          "message" : "Error that crash the process",
          "source" : null,
          "time" : "2022-01-25T17:47:53.275937"
        }
      }
    ]
  }
}
Not available

Wrapping up

In this tutorial, we have learned what logging is, the different logging levels, and how to use the ConsoleLogger and CompositeLogger from PIP.Services to display log messages. The main advantage of the composite logger is its capacity to aggregate all logging messages, thus creating a centralized logging point.

We have also learned that PIP.Services provides several implementations of loggers, such as CloudWatchLogger, ElasticSearchLogger, and DataDogLogger.

Although the examples presented here are quite general, the concepts learned continue to apply to the development of more complex applications.