Adding logging to a component
Key takeaways
Logging | Logging is the capacity to create tagged messages from events in our code. |
Logging levels | Logging levels: nothing, fatal, error, warn, info, debug, and trace. |
ConsoleLogger | PIP.Services component for displaying logging messages on the console. |
CachedLogger | PIP.Services component that caches log messages in memory. |
CompositeLogger | PIP.Services component for aggregating logging messages. |
DataDogLogger, ElasticSearchLogger, CloudWatchLogger | PIP.Services logger implementations for Datadog, Elasticsearch, and Amazon CloudWatch components. |
Introduction
In this tutorial, you will learn how to add logging capacity to a microservice. First, we will understand what logging consists of. Then, we will use the microservice we created in the “Creating a component” tutorial, replace the printed messages with logger messages and create an exception in our business process (my_task). After running the code, we will see the tagged messages from the logger.
Once we have seen how to create a logger that displays messages on our console, we will learn how to create a composite logger, which will add the capacity to aggregate the log messages from different sources and centralize their display on our console.
Finally, we will see how to add loggers for Datadog, Elasticsearch, and Amazon CloudWatch components.
What is logging?
Logging is the capacity to create tagged messages from events in our code. These messages can inform us about the running process.
There are different logging levels. PIP.Services defines them as:
Level name | Level number | Description |
---|---|---|
Nothing | 0 | Nothing to be logged. |
Fatal | 1 | Logs only fatal errors that cause a microservice to fail. |
Error | 2 | Logs all errors - fatal or recoverable. |
Warn | 3 | Logs errors and warnings. |
Info | 4 | Logs errors and important information messages. |
Debug | 5 | Logs everything up to high-level debugging information. |
Trace | 6 | Logs everything down to fine-granular debugging messages. |
Once generated, log messages need to be stored or displayed. PIP.Services provides specific tools for this: CachedLogger and ConsoleLogger. The first class stores log messages in memory. The second class displays them on a console. The toolkit also provides us with the CompositeLogger, which allows for message aggregation and thus, creating a centralized logging point.
Additionally, PIP.Services provides implementations of loggers for CloudWatch, ElasticSearch, and DataDog.
Once generated, log messages need to be stored or displayed. PIP.Services provides specific tools for this: CachedLogger and ConsoleLogger. The first class stores log messages in memory. The second class displays them on a console. The toolkit also provides us with the CompositeLogger, which allows for message aggregation and thus, creating a centralized logging point.
Additionally, PIP.Services provides implementations of loggers for CloudWatch, ElasticSearch, and DataDog.
Once generated, log messages need to be stored or displayed. PIP.Services provides specific tools for this: CachedLogger and ConsoleLogger. The first class stores log messages in memory. The second class displays them on a console. The toolkit also provides us with the CompositeLogger, which allows for message aggregation and thus, creating a centralized logging point.
Additionally, PIP.Services provides implementations of loggers for CloudWatch, ElasticSearch, and DataDog.
Once generated, log messages need to be stored or displayed. PIP.Services provides specific tools for this: CachedLogger and ConsoleLogger. The first class stores log messages in memory. The second class displays them on a console. The toolkit also provides us with the CompositeLogger, which allows for message aggregation and thus, creating a centralized logging point.
Additionally, PIP.Services provides implementations of loggers for CloudWatch, ElasticSearch, and DataDog.
Once generated, log messages need to be stored or displayed. PIP.Services provides specific tools for this: CachedLogger and ConsoleLogger. The first class stores log messages in memory. The second class displays them on a console. The toolkit also provides us with the CompositeLogger, which allows for message aggregation and thus, creating a centralized logging point.
Additionally, PIP.Services provides implementations of loggers for CloudWatch, ElasticSearch, and DataDog.
Now, we will see how to create a console logger and a composite logger.
Adding a console logger to our component
In our example, we will add a logger that sends messages to our console. For this, we will use the ConsoleLogger class. After we created an instance of this class, we will set the logging level to five, which will allow us to log everything up to debug level.
var logger = new ConsoleLogger();
logger.setLevel(5);
import (
clog "github.com/pip-services4/pip-services4-go/pip-services4-observability-go/log"
)
// Logger setting
logger := *clog.NewConsoleLogger()
logger.SetLevel(5)
# Logger setting
from pip_services4_observability.log import ConsoleLogger
logger = ConsoleLogger()
logger.set_level(5)
Then, we will replace our print messages with info-level log messages. For example, print(“MyComponentA has been created.") will be replaced with logger.info(None, “MyComponentA has been created.").
Finally, we will force an exception in the my_task method. As we had explained in the “Creating a component” tutorial, this method performs business-related tasks. Thus, we can simulate a problem within it by forcibly raising an exception. This method will look like this:
public myTask(ctx: Context): void {
// create an artificial problem
try{
throw Error('Logging demo. Error created');
}
catch (ex) {
logger.error(ctx, ex, "Error created.")
}
}
func MyTask() {
// create an artificial problem
err := errors.New("Logging demo. Error created")
logger.Error(context.Background(), err, "Error created.")
}
def my_task(self, context):
# create an artificial problem
try:
raise Exception('Logging demo', 'Error created')
except Exception as inst:
logger.error(context, inst, "Error created.")
And, our final code will look like this:
a) Our components
import { ConsoleLogger } from 'pip-services4-observability-node';
import {
ConfigParams, ICleanable, IConfigurable, IOpenable,
IReferenceable, IReferences, IUnreferenceable, Context, Descriptor, IContext
} from "pip-services4-components-node";
export class MyComponentB implements IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable {
private _param1: string = 'ABC2';
private _param2: number = 456;
private _opened:boolean = false;
private _status: string;
private _logger = new ConsoleLogger();
/**
* Creates a new instance of the component.
*/
public constructor(){
this._status = 'Created';
this._logger.setLevel(5);
this._logger.info(null, "MyComponentB has been configured.");
}
public configure(config: ConfigParams): void {
this._param1 = config.getAsStringWithDefault('param1', this._param1);
this._param2 = config.getAsIntegerWithDefault('param2', this._param2);
this._logger.info(null, "MyComponentB has been configured.")
}
public setReferences(refs: IReferences): void {
// pass
}
public isOpen(): boolean {
// pass
}
public open(ctx: Context): Promise<void> {
// pass
}
public close(ctx: Context): Promise<void> {
// pass
}
public myTask(ctx: Context): void {
// create an artificial problem
try {
throw Error('Logging demo. Error created');
}
catch (ex) {
this._logger.error(ctx, ex, "Error created.")
}
}
/**
* Unsets (clears) previously set references to dependent components.
*/
public unsetReferences(): void {
// pass
}
/**
* Clears component state.
* @param correlationId (optional) transaction id to trace execution through call chain.
*/
public clear(ctx: Context): Promise<void> {
// pass
return Promise.resolve();;
}
}
export class MyComponentA implements IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable {
private _param1: string = 'ABC';
private _param2: number = 123;
private _opened: boolean = false;
private _status: string;
private _another_component: MyComponentB;
public dummy_variable = "resource variable";
private _logger = new ConsoleLogger();
/**
* Creates a new instance of the component.
*/
public constructor() {
this._status = 'Created';
this._logger.setLevel(5);
this._logger.info(null, "MyComponentA has been configured.");
}
public configure(config: ConfigParams): void {
this._param1 = config.getAsStringWithDefault('param1', this._param1);
this._param2 = config.getAsIntegerWithDefault('param2', this._param2);
this._status = "Configured";
this._logger.info(null, "MyComponentA has been configured.")
}
public setReferences(refs: IReferences): void {
this._another_component = refs.getOneRequired(
new Descriptor("myservice", "mycomponent-b", "*", "*", "1.0")
)
this._status = "Configured"
this._logger.info(null, "MyComponentA's references have been defined.")
}
public isOpen(): boolean {
return this._opened;
}
public open(ctx: Context): Promise<void> {
this._opened = true;
this._status = "Open";
this._logger.info(ctx, "MyComponentA has been opened.")
// artificial problem
this.myTask(ctx);
return Promise.resolve();
}
public close(ctx: Context): Promise<void> {
this._opened = false;
this._status = "Closed";
this._logger.info(ctx, "MyComponentA has been closed.");
return Promise.resolve();
}
public myTask(ctx: Context): void {
// create an artificial problem
try {
throw Error('Logging demo. Error created');
}
catch (ex) {
this._logger.error(ctx, ex, "Error created.")
}
}
/**
* Unsets (clears) previously set references to dependent components.
*/
public unsetReferences(): void {
this._another_component = null;
this._status = "Un-referenced";
this._logger.info(null, "References cleared");
}
/**
* Clears component state.
* @param correlationId (optional) transaction id to trace execution through call chain.
*/
public clear(ctx: Context): Promise<void> {
this.dummy_variable = null;
this._status = null;
this._logger.info(ctx, "Resources cleared");
return Promise.resolve();
}
}
import (
"context"
"errors"
cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
crefer "github.com/pip-services4/pip-services4-go/pip-services4-components-go/refer"
crun "github.com/pip-services4/pip-services4-go/pip-services4-components-go/run"
clog "github.com/pip-services4/pip-services4-go/pip-services4-observability-go/log"
)
type MyComponentB struct {
crefer.IReferences
crefer.IUnreferenceable
cconfig.IConfigurable
crun.IOpenable
crun.ICleanable
_status string
_param1 string
_param2 int
_opened bool
_logger clog.ILogger
dummy_variable interface{}
}
// Creates a new instance of the component.
func NewMyComponentB() *MyComponentB {
component := &MyComponentB{
_status: "Created",
_param1: "ABC2",
_param2: 456,
_opened: false,
_logger: clog.NewConsoleLogger(),
dummy_variable: "resource variable",
}
component._logger.SetLevel(5)
component._logger.Info(context.Background(), "MyComponentB has been created.")
return component
}
func (c *MyComponentB) Configure(ctx context.Context, config *cconfig.ConfigParams) {
c._param1 = config.GetAsStringWithDefault("param1", c._param1)
c._param2 = config.GetAsIntegerWithDefault("param2", c._param2)
c._logger.Info(ctx, "MyComponentB has been configured.")
}
func (c *MyComponentB) SetReferences(ctx context.Context, references *crefer.References) {
// pass
}
func (c *MyComponentB) isOpen() bool {
// pass
return true
}
func (c *MyComponentB) Open(ctx context.Context) {
// pass
}
func (c *MyComponentB) Close(ctx context.Context) {
// pass
}
func (c *MyComponentB) MyTask(ctx context.Context) {
// pass
}
// Unsets (clears) previously set references to dependent components.
func (c *MyComponentB) UnsetReferences(ctx context.Context) {
// pass
}
// Clears component state.
// - correlationId: (optional) transaction id to trace execution through call chain.
func (c *MyComponentB) Clear(ctx context.Context) {
// pass
}
type MyComponentA struct {
crefer.IReferences
crefer.IUnreferenceable
cconfig.IConfigurable
crun.IOpenable
crun.ICleanable
_logger clog.ILogger
_status string
_param1 string
_param2 int
_opened bool
dummy_variable interface{}
_another_component interface{}
}
// Creates a new instance of the component.
func NewMyComponentA() *MyComponentA {
component := &MyComponentA{
_status: "Created",
_param1: "ABC2",
_param2: 456,
_opened: false,
_logger: clog.NewConsoleLogger(),
dummy_variable: "dummy_variable setted",
}
component._logger.SetLevel(5)
component._logger.Info(context.Background(), "MyComponentA has been created.")
return component
}
func (c *MyComponentA) Configure(ctx context.Context, config *cconfig.ConfigParams) {
c._param1 = config.GetAsStringWithDefault("param1", c._param1)
c._param2 = config.GetAsIntegerWithDefault("param2", c._param2)
c._status = "Configured"
c._logger.Info(ctx, "MyComponentB has been configured.")
}
func (c *MyComponentA) SetReferences(ctx context.Context, references *crefer.References) {
_another_component, err := references.GetOneRequired(crefer.NewDescriptor("myservice", "mycomponent-b", "*", "*", "1.0"))
if err != nil {
panic("Error: Another Component is not in refs")
}
c._another_component = _another_component.(MyComponentB)
c._status = "Configured"
c._logger.Info(ctx, "MyComponentA's references have been defined.")
}
func (c *MyComponentA) isOpen() bool {
return c._opened
}
func (c *MyComponentA) Open(ctx context.Context, correlationId string) {
c._opened = true
c._status = "Open"
c._logger.Info(ctx, "MyComponentA has been opened.")
// artificial problem
c.MyTask(ctx)
}
func (c *MyComponentA) Close(ctx context.Context) {
c._opened = false
c._status = "Closed"
c._logger.Info(ctx, "MyComponentA has been closed.")
}
func (c *MyComponentA) MyTask(ctx context.Context) {
// create an artificial problem
c._logger.Error(ctx, errors.New("Logging demo. Error created"), "Error created.")
}
// Unsets (clears) previously set references to dependent components.
func (c *MyComponentA) UnsetReferences(ctx context.Context) {
c._another_component = nil
c._status = "Un-referenced"
c._logger.Info(ctx, "References cleared")
}
// Clears component state.
// - correlationId: (optional) transaction id to trace execution through call chain.
func (c *MyComponentA) Clear(ctx context.Context) {
c.dummy_variable = nil
c._status = ""
c._logger.Info(ctx, "Resources cleared")
}
b) Our factory
// Creating a factory
import { Factory } from 'pip-services4-components-node';
let MyFactory1 = new Factory();
MyFactory1.registerAsType(new Descriptor("myservice", "mycomponentA", "default", "*", "1.0"), MyComponentA)
MyFactory1.registerAsType(new Descriptor("myservice", "mycomponent-b", "default", "*", "1.0"), MyComponentB)
import (
cbuild "github.com/pip-services4/pip-services4-go/pip-services4-components-go/build"
)
MyFactory1 := cbuild.NewFactory()
MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponentA", "default", "*", "1.0"), NewMyComponentA)
MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponent-b", "default", "*", "1.0"), NewMyComponentB)
# Creating a factory
from pip_services4_components.build import Factory
MyFactory1 = Factory()
MyFactory1.register_as_type(Descriptor("myservice", "mycomponentA", "default", "*", "1.0"), MyComponentA)
MyFactory1.register_as_type(Descriptor("myservice", "mycomponent-b", "default", "*", "1.0"), MyComponentB)
c) Our service
// Creating a process container
class MyProcess extends ProcessContainer{
public constructor() {
super('myservice', 'My service running as a process');
this._configPath = './config.yaml'
this._factories.add(MyFactory1)
}
}
import (
cproc "github.com/pip-services4/pip-services4-go/pip-services4-container-go/container"
)
// Creating a process container
type MyProcess struct {
*cproc.ProcessContainer
}
func NewMyProcess() *MyProcess {
c := MyProcess{}
c.ProcessContainer = cproc.NewProcessContainer("myservice", "My service running as a process")
c.SetConfigPath("./config/config.yml")
MyFactory1 := cbuild.NewFactory()
MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponentA", "default", "*", "1.0"), NewMyComponentA)
MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponent-b", "default", "*", "1.0"), NewMyComponentB)
c.AddFactory(MyFactory1)
return &c
}
# Creating a process container
from pip_services4_container.container import ProcessContainer
class MyProcess(ProcessContainer):
def __init__(self):
super(MyProcess, self).__init__('myservice', 'My service running as a process')
self._config_path = 'C:\\Users\\Eugenio\\2024-PIP-Tutorials\\config2.yaml'
self._factories.add(MyFactory1)
d) The dynamic configuration file for the components:
config.yaml
---
- descriptor: "myservice:mycomponentA:*:default:1.0"
param1: XYZ
param2: 987
- descriptor: myservice:mycomponent-b:*:*:1.0
param1: XYZ
param2: 987
e) Running our service
let MyProcess = require('./obj/example').MyProcess;
try {
let proc = new MyProcess();
proc.run(process.argv);
} catch (ex) {
console.error(ex);
}
func main() {
proc := NewMyProcess()
proc.Run(context.Background(), os.Environ())
}
# -*- coding: utf-8 -*-
if __name__ == '__main__':
runner = MyProcess()
print("Run")
try:
runner.run()
except Exception as ex:
print(ex)
After running this code, we will get the following result:
![Console logger messages](figure1.png)
As we can see from the above figure, the program has logged all messages with level info and from our artificial error.
This concludes our first task.
Adding a composite logger to our component
Now, we will extend our logging capacity by adding a composite logger. This logger will allow us to aggregate all loggers from our component’s references into a centralized log. Our code will remain the same, except that now we need to create a composite logger for MyComponentA. For this, we will create an instance of this logger and set the logging level to 5.
Then, we will use the configure and set_references methods to let our composite logger know where to ask for log messages. Our factory and process container code sections will remain the same, but we will have to add a reference to our console logger in our configuration file. The syntax will be:
# Console logger
- descriptor: "pip-services:logger:console:default:1.0"
level: {{LOG_LEVEL}}{{^LOG_LEVEL}}info{{/LOG_LEVEL}}
Finally, we will add a console logger to MyComponentB. After these changes, our component section will look like this:
import { ConsoleLogger, CompositeLogger } from 'pip-services4-observability-node';
import {
ConfigParams, ICleanable, IConfigurable, IOpenable,
IReferenceable, IReferences, IUnreferenceable, Context, Descriptor, IContext
} from "pip-services4-components-node";
export class MyComponentB implements IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable {
private _param1: string = 'ABC2';
private _param2: number = 456;
private _opened:boolean = false;
private _status: string;
private _logger = new ConsoleLogger();
/**
* Creates a new instance of the component.
*/
public constructor(){
this._status = 'Created';
this._logger.setLevel(5);
this._logger.info(null, "MyComponentB has been configured.");
}
public configure(config: ConfigParams): void {
this._param1 = config.getAsStringWithDefault('param1', this._param1);
this._param2 = config.getAsIntegerWithDefault('param2', this._param2);
this._logger.info(null, "MyComponentB has been configured.")
}
public setReferences(refs: IReferences): void {
// pass
}
public isOpen(): boolean {
// pass
}
public open(correlationId: string): Promise<void> {
// pass
}
public close(correlationId: string): Promise<void> {
// pass
}
public myTask(ctx: Context): void {
// create an artificial problem
try {
throw Error('Logging demo. Error created');
}
catch (ex) {
this._logger.error(ctx, ex, "Error created.")
}
}
/**
* Unsets (clears) previously set references to dependent components.
*/
public unsetReferences(): void {
// pass
}
/**
* Clears component state.
* @param correlationId (optional) transaction id to trace execution through call chain.
*/
public clear(correlationId: string): Promise<void> {
// pass
}
}
export class MyComponentA implements IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable {
private _param1: string = 'ABC';
private _param2: number = 123;
private _opened: boolean = false;
private _status: string;
private _another_component: MyComponentB;
public dummy_variable = "resource variable";
private _logger = new CompositeLogger();
/**
* Creates a new instance of the component.
*/
public constructor() {
this._status = 'Created';
this._logger.setLevel(5);
this._logger.info(null, "MyComponentA has been configured.");
}
public configure(config: ConfigParams): void {
this._param1 = config.getAsStringWithDefault('param1', this._param1);
this._param2 = config.getAsIntegerWithDefault('param2', this._param2);
this._status = "Configured";
this._logger.info(null, "MyComponentA has been configured.")
}
public setReferences(refs: IReferences): void {
this._another_component = refs.getOneRequired(
new Descriptor("myservice", "mycomponent-b", "*", "*", "1.0")
)
this._status = "Configured"
this._logger.setReferences(refs);
this._logger.info(null, "MyComponentA's references have been defined.")
}
public isOpen(): boolean {
return this._opened;
}
public open(ctx: Context): Promise<void> {
this._opened = true;
this._status = "Open";
this._logger.info(ctx, "MyComponentA has been opened.")
// artificial problem
this.myTask(ctx);
}
public close(ctx: Context): Promise<void> {
this._opened = false;
this._status = "Closed";
this._logger.info(ctx, "MyComponentA has been closed.");
}
public myTask(ctx: Context): void {
// create an artificial problem
try {
throw Error('Logging demo. Error created');
}
catch (ex) {
this._logger.error(ctx, ex, "Error created.")
}
}
/**
* Unsets (clears) previously set references to dependent components.
*/
public unsetReferences(): void {
this._another_component = null;
this._status = "Un-referenced";
this._logger.info(null, "References cleared");
}
/**
* Clears component state.
* @param correlationId (optional) transaction id to trace execution through call chain.
*/
public clear(ctx: Context): Promise<void> {
this.dummy_variable = null;
this._status = null;
this._logger.info(ctx, "Resources cleared");
}
}
import (
"context"
"errors"
"os"
cbuild "github.com/pip-services4/pip-services4-go/pip-services4-components-go/build"
cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
crefer "github.com/pip-services4/pip-services4-go/pip-services4-components-go/refer"
crun "github.com/pip-services4/pip-services4-go/pip-services4-components-go/run"
cproc "github.com/pip-services4/pip-services4-go/pip-services4-container-go/container"
clog "github.com/pip-services4/pip-services4-go/pip-services4-observability-go/log"
)
type MyComponentB struct {
crefer.IReferences
crefer.IUnreferenceable
cconfig.IConfigurable
crun.IOpenable
crun.ICleanable
_status string
_param1 string
_param2 int
_opened bool
_logger clog.ILogger
dummy_variable interface{}
}
// Creates a new instance of the component.
func NewMyComponentB() *MyComponentB {
component := &MyComponentB{
_status: "Created",
_param1: "ABC2",
_param2: 456,
_opened: false,
_logger: clog.NewConsoleLogger(),
dummy_variable: "resource variable",
}
component._logger.SetLevel(5)
component._logger.Info(context.Background(), "MyComponentB has been created.")
return component
}
func (c *MyComponentB) Configure(ctx context.Context, config *cconfig.ConfigParams) {
c._param1 = config.GetAsStringWithDefault("param1", c._param1)
c._param2 = config.GetAsIntegerWithDefault("param2", c._param2)
c._logger.Info(ctx, "MyComponentB has been configured.")
}
func (c *MyComponentB) SetReferences(ctx context.Context, references *crefer.References) {
// pass
}
func (c *MyComponentB) isOpen() bool {
// pass
return true
}
func (c *MyComponentB) Open(ctx context.Context) {
// pass
}
func (c *MyComponentB) Close(ctx context.Context) {
// pass
}
func (c *MyComponentB) MyTask(ctx context.Context) {
// pass
}
// Unsets (clears) previously set references to dependent components.
func (c *MyComponentB) UnsetReferences(ctx context.Context) {
// pass
}
// Clears component state.
// - correlationId: (optional) transaction id to trace execution through call chain.
func (c *MyComponentB) Clear(ctx context.Context) {
// pass
}
type MyComponentA struct {
crefer.IReferences
crefer.IUnreferenceable
cconfig.IConfigurable
crun.IOpenable
crun.ICleanable
_logger clog.ILogger
_status string
_param1 string
_param2 int
_opened bool
dummy_variable interface{}
_another_component interface{}
}
// Creates a new instance of the component.
func NewMyComponentA() *MyComponentA {
component := &MyComponentA{
_status: "Created",
_param1: "ABC2",
_param2: 456,
_opened: false,
_logger: clog.NewConsoleLogger(),
dummy_variable: "dummy_variable setted",
}
component._logger.SetLevel(5)
component._logger.Info(context.Background(), "MyComponentA has been created.")
return component
}
func (c *MyComponentA) Configure(ctx context.Context, config *cconfig.ConfigParams) {
c._param1 = config.GetAsStringWithDefault("param1", c._param1)
c._param2 = config.GetAsIntegerWithDefault("param2", c._param2)
c._status = "Configured"
c._logger.Info(ctx, "MyComponentB has been configured.")
}
func (c *MyComponentA) SetReferences(ctx context.Context, references *crefer.References) {
_another_component, err := references.GetOneRequired(crefer.NewDescriptor("myservice", "mycomponent-b", "*", "*", "1.0"))
if err != nil {
panic("Error: Another Component is not in refs")
}
c._another_component = _another_component.(MyComponentB)
c._status = "Configured"
c._logger.Info(ctx, "MyComponentA's references have been defined.")
}
func (c *MyComponentA) isOpen() bool {
return c._opened
}
func (c *MyComponentA) Open(ctx context.Context, correlationId string) {
c._opened = true
c._status = "Open"
c._logger.Info(ctx, "MyComponentA has been opened.")
// artificial problem
c.MyTask(ctx)
}
func (c *MyComponentA) Close(ctx context.Context) {
c._opened = false
c._status = "Closed"
c._logger.Info(ctx, "MyComponentA has been closed.")
}
func (c *MyComponentA) MyTask(ctx context.Context) {
// create an artificial problem
c._logger.Error(ctx, errors.New("Logging demo. Error created"), "Error created.")
}
// Unsets (clears) previously set references to dependent components.
func (c *MyComponentA) UnsetReferences(ctx context.Context) {
c._another_component = nil
c._status = "Un-referenced"
c._logger.Info(ctx, "References cleared")
}
// Clears component state.
// - correlationId: (optional) transaction id to trace execution through call chain.
func (c *MyComponentA) Clear(ctx context.Context) {
c.dummy_variable = nil
c._status = ""
c._logger.Info(ctx, "Resources cleared")
}
// Creating a process container
type MyProcess struct {
*cproc.ProcessContainer
}
func NewMyProcess() *MyProcess {
c := MyProcess{}
c.ProcessContainer = cproc.NewProcessContainer("myservice", "My service running as a process")
c.SetConfigPath("./config/config.yml")
MyFactory1 := cbuild.NewFactory()
MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponentA", "default", "*", "1.0"), NewMyComponentA)
MyFactory1.RegisterType(crefer.NewDescriptor("myservice", "mycomponent-b", "default", "*", "1.0"), NewMyComponentB)
c.AddFactory(MyFactory1)
return &c
}
func main() {
proc := NewMyProcess()
proc.Run(context.Background(), os.Environ())
}
from pip_services4_components.config import IConfigurable, ConfigParams
from pip_services4_components.refer import IReferenceable, IReferences, Descriptor, IUnreferenceable
from pip_services4_components.run import IOpenable, ICleanable
from pip_services4_observability.log import ConsoleLogger, LogLevel, CompositeLogger
class MyComponentB(IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable):
_param1 = 'ABC2'
_param2 = 456
_opened = False
__logger = ConsoleLogger()
__logger.set_level(5)
dummy_variable = "resource variable"
def __init__(self):
"""
Creates a new instance of the component.
"""
self._status = "Created"
self.__logger.info(None, "MyComponentB has been created.")
def configure(self, config):
self._param1 = config.get_as_string_with_default("param1", self._param1)
self._param2 = config.get_as_integer_with_default("param2", self._param2)
def set_references(self, references):
pass
def is_open(self):
pass
def open(self, context):
pass
def close(self, context):
pass
def my_task(self, context):
pass
def unset_references(self):
"""
Unsets (clears) previously set references to dependent components.
"""
pass
def clear(self, context):
"""
Clears component state.
:param context: (optional) transaction id to trace execution through call chain.
"""
pass
class MyComponentA(IReferenceable, IUnreferenceable, IConfigurable, IOpenable, ICleanable):
_param1 = 'ABC'
_param2 = 123
_another_component: MyComponentB
_open = False
_status = None
__logger = CompositeLogger()
__logger.set_level(5)
def __init__(self):
"""
Creates a new instance of the component.
"""
self._status = "Created"
self.__logger.info(None, "MyComponentA has been created.")
def configure(self, config):
self._param1 = config.get_as_string_with_default("param1", 'ABC')
self._param2 = config.get_as_integer_with_default("param2", 123)
self._status = "Configured"
self.__logger.configure(config)
def set_references(self, references):
self._another_component = references.get_one_required(
Descriptor("myservice", "mycomponent-b", "*", "*", "1.0")
)
self._status = "Configured"
self.__logger.set_references(references)
self.__logger.info(None,"MyComponentA's references have been defined.")
def is_open(self):
return self._open
def open(self, context):
self._open = True
self._status = "Open"
self.__logger.info(None,"MyComponentA has been opened.")
# artificial problem
self.my_task(context)
def close(self, context):
self._opened = False
self._status = "Closed"
self.__logger.info(context,"MyComponentA has been closed.")
def my_task(self, context):
# create an artificial problem
try:
raise Exception('Logging demo', 'Error created')
except Exception as inst:
self.__logger.error(context, inst, "Error created.")
def unset_references(self):
"""
Unsets (clears) previously set references to dependent components.
"""
self._another_component = None
self._status = "Un-referenced"
self.__logger.info(None, "References cleared")
def clear(self, context):
"""
Clears component state.
:param context: (optional) transaction id to trace execution through call chain.
"""
self.dummy_variable = None
self._status = None
self.__logger.info(context, "Resources cleared")
Once we run our service with the re-defined components, we will get the following results:
![Composite logger messages](figure2.png)
As we can see, we have log messages received from both MyComponentA and MyComponentB.
Adding specific loggers
As we said earlier, PIP.Services has specific loggers for Datadog, Elasticsearch, and Amazon CloudWatch. The process to add any of them to a component is similar to what we saw in our console logger example: we need to declare an instance of the logger, configure it, set the message level, and add the messages we need. Here below are examples of how to define each of them.
a) Datadog
import { DataDogLogger } from 'pip-services4-datadog-node';
let logger = new DataDogLogger();
logger.configure(ConfigParams.fromTuples(
"credential.access_key", "827349874395872349875493"
));
logger.setLevel(5);
await logger.open(ctx);
logger.info(ctx, "My message");
import (
"context"
cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
logdatadog "github.com/pip-services4/pip-services4-go/pip-services4-datadog-go/log"
)
func main() {
logger := logdatadog.NewDataDogLogger()
logger.Configure(context.Background(), cconfig.NewConfigParamsFromTuples(
"credential.access_key", "827349874395872349875493",
))
logger.SetLevel(5)
_ = logger.Open(context.Background())
logger.Info(context.Background(), "My message")
}
from pip_services4_components.context import IContext
from pip_services4_datadog.log import DataDogLogger
from pip_services4_components.config import ConfigParams
logger = DataDogLogger()
logger.configure(ConfigParams.from_tuples(
"credential.access_key", "827349874395872349875493"
))
logger.set_level(5)
logger.open("123")
context = IContext
context={}
logger.info(context , "My message")
b) Elasticsearch
import { ElasticSearchLogger } from 'pip-services4-elasticsearch-node';
let logger = new ElasticSearchLogger();
logger.configure(ConfigParams.fromTuples(
"connection.protocol", "http",
"connection.host", "localhost",
"connection.port", 9200
));
logger.setLevel(5);
await logger.open(ctx);
logger.info(ctx, "My message");
import (
"context"
cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
logelastic "github.com/pip-services4/pip-services4-go/pip-services4-elasticsearch-go/log"
)
func main() {
logger := logelastic.NewElasticSearchLogger()
logger.Configure(context.Background(), cconfig.NewConfigParamsFromTuples(
"connection.protocol", "http",
"connection.host", "localhost",
"connection.port", 9200,
))
logger.SetLevel(5)
_ = logger.Open(context.Background())
logger.Info(context.Background(), "My message")
}
from pip_services4_elasticsearch.log import ElasticSearchLogger
from pip_services4_components.config import ConfigParams
logger = ElasticSearchLogger()
logger.configure(ConfigParams.from_tuples(
"connection.protocol", "http",
"connection.host", "localhost",
"connection.port", 9200
))
logger.set_level(5)
logger.open("123")
context = IContext
context={}
logger.info(context , "My message")
c) Amazon CloudWatch
import { CloudWatchLogger } from 'pip-services4-aws-node';
let logger = new CloudWatchLogger();
logger.configure(ConfigParams.fromTuples(
"stream", "mystream",
"group", "mygroup",
"connection.region", "us-east-1",
"connection.access_id", "XXXXXXXXXXX",
"connection.access_key", "XXXXXXXXXXX"
));
logger.setLevel(5);
await logger.open(ctx);
logger.info(ctx, "My message");
import (
"context"
cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
logaws "github.com/pip-services4/pip-services4-go/pip-services4-aws-go/log"
)
func main() {
logger := logaws.NewCloudWatchLogger()
logger.Configure(context.Background(), cconfig.NewConfigParamsFromTuples(
"stream", "mystream",
"group", "mygroup",
"connection.region", "us-east-1",
"connection.access_id", "XXXXXXXXXXX",
"connection.access_key", "XXXXXXXXXXX",
))
logger.SetLevel(5)
_ = logger.Open(context.Background())
logger.Info(context.Background(), "My message")
}
Example
In this example, we will combine two features: displaying log information on our console and sending log information to ElasticSearch.
Pre-requisites
In order to be able to create a composite logger for both outputs, we need to import the following components:
import { CompositeLogger, ConsoleLogger, LogLevel } from 'pip-services4-observability-node';
import { ConfigParams, Context, Descriptor } from "pip-services4-components-node";
import { ElasticSearchLogger } from 'pip-services4-elasticsearch-node';
import (
"errors"
"fmt"
cconfig "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
cref "github.com/pip-services4/pip-services4-go/pip-services4-components-go/refer"
clog "github.com/pip-services4/pip-services4-go/pip-services4-observability-go/log"
elasticlog "github.com/pip-services4/pip-services4-go/pip-services4-elasticsearch-go/log"
)
from typing import Optional
from pip_services4_components.config import IConfigurable, ConfigParams
from pip_services4_components.refer import IReferenceable, IReferences, Descriptor, References
from pip_services4_components.run import IOpenable
from pip_services4_components.build import Factory
from pip_services4_observability.log import ConsoleLogger, CompositeLogger, LogLevel
from pip_services4_container import ProcessContainer
from pip_services4_elasticsearch.build import DefaultElasticSearchFactory
from pip_services4_elasticsearch.log import ElasticSearchLogger
Component creation
The next step is to create our two logging components, namely our console and ElasticSearch. For this, we instantiate the ConsoleLogger and ElasticSearchLogger classes. Our code is:
// ElasticSearchLogger sends a log to ElasticSearch
let elasticSearchLogger = new ElasticSearchLogger();
// Console logger writes messages to console
let consoleLogger = new ConsoleLogger();
// ElasticSearchLogger sends a log to ElasticSearch
elasticSearchLogger := elasticlog.NewElasticSearchLogger()
// Console logger writes messages to console
consoleLogger := clog.NewConsoleLogger()
# ElasticSearchLogger sends a log to ElasticSearch
elasticsearch_logger: ElasticSearchLogger = ElasticSearchLogger()
# Console logger writes messages to console
console_logger: ConsoleLogger = ConsoleLogger()
Once these instances have been created, we can configure them. To address this, we create a common configuration object that contains all the necessary parameters and their values:
let config = ConfigParams.fromTuples(
// Common config
"source", "my_component_log",
"level", LogLevel.Debug,
// Elastic config
"index", "log",
"daily", true,
"date_format", "YYYYMMDD",
"connection.host", "localhost",
"connection.port", 9200,
"connection.protocol", "http",
);
config := cconf.NewConfigParamsFromTuples(
// Common config
"source", "my_component_log",
"level", clog.LevelDebug,
// Elastic config
"index", "log",
"daily", true,
"date_format", "yyyyMMdd",
"connection.host", "localhost",
"connection.port", 9200,
"connection.protocol", "http",
)
config = ConfigParams.from_tuples(
# Common config
'source', 'my_component_log',
'level', LogLevel.Debug,
# Elastic config
'index', 'log',
'daily', True,
"date_format", 'YYYYMMDD',
'connection.host', 'localhost',
'connection.port', 9200,
"connection.protocol", "http",
)
and we assign these values to the respective objects
elasticSearchLogger.configure(config);
await elasticSearchLogger.open(ctx);
elasticSearchLogger.Configure(context.Background(), config)
_ = elasticSearchLogger.Open(context.Background(), "123")
elasticsearch_logger.configure(config)
elasticsearch_logger.open(context)
Next, we connect our ElasticSearch logger:
consoleLogger.configure(config);
consoleLogger.Configure(context.Background(), config)
console_logger.configure(config)
Composite logger
After defining our two loggers, we define a composite logger that manages both. For this, we first declare a reference object that points to both loggers:
let references = References.fromTuples(
new Descriptor("my-component", "logger", "console", "default", "1.0"), elasticSearchLogger,
new Descriptor("my-component", "logger", "elasticsearch", "default", "1.0"), consoleLogger
);
references := cref.NewReferencesFromTuples(context.Background(),
cref.NewDescriptor("my-component", "logger", "console", "default", "1.0"), elasticSearchLogger,
cref.NewDescriptor("my-component", "logger", "elasticsearch", "default", "1.0"), consoleLogger,
)
references = References.from_tuples(
Descriptor('my-component', 'logger', 'console', 'default', '1.0'), elasticsearch_logger,
Descriptor('my-component', 'logger', 'elasticsearch', 'default', '1.0'), console_logger
)
Then, we create an instance of the CompositeLogger class:
let logger = new CompositeLogger();
logger := clog.NewCompositeLogger()
logger = CompositeLogger()
And, we add our references to it:
logger.setReferences(references);
logger.SetReferences(context.Background(), references)
logger.set_references(references)
Adding log messages:
Now that our structure is complete, we can create different log messages, which will be sent to ElasticSearch and the console after executing our code:
logger.info(ctx, "Composite logger is configured and ready for using");
logger.warn(ctx, "Example warning");
logger.error(ctx, new Error("Example error"), "Error message");
logger.debug(ctx, "Debug info");
logger.fatal(ctx, new Error("Fatal error"), "Error that crash the process");
logger.Info(context.Background(), "Composite logger is configured and ready for using")
logger.Warn(context.Background(), "Example warning")
logger.Error(context.Background(), errors.New("Example error"), "Error message")
logger.Debug(context.Background(), "Debug info")
logger.Fatal(context.Background(), errors.New("Fatal error"), "Error that crash the process")
logger.info(context, "Composite logger is configured and ready for using")
logger.warn(context, "Example warning")
logger.error(context, Exception("Example error"), "Error message")
logger.debug(context, "Debug info")
logger.fatal(context, Exception("Fatal error"), "Error that crash the process")
After code execution, the following messages will appear on our console:
![Console logger messages](figure3.png)
And, we can see the ElasticSearch messages by using the URL:
http://localhost:9200/index_name/_search?pretty
where index_name is is “log” + “-” + current date (in date_format from config params).
For example, for January 25, 2022, our URL is
http://localhost:9200/log-20220125/_search?pretty
which will show:
Code Example
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 5,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "log-20220127",
"_type" : "_doc",
"_id" : "10399e390171415ca122dbb307f67eea",
"_score" : 1.0,
"_source" : {
"time" : "2022-01-26T23:12:01.954Z",
"level" : "INFO",
"source" : "my_component_log",
"correlation_id" : "123",
"error" : null,
"message" : "Composite logger is configured and ready for using"
}
},
{
"_index" : "log-20220127",
"_type" : "_doc",
"_id" : "ed1d25a2051c444783f4539861dd44f1",
"_score" : 1.0,
"_source" : {
"time" : "2022-01-26T23:12:01.956Z",
"level" : "WARN",
"source" : "my_component_log",
"correlation_id" : "123",
"error" : null,
"message" : "Example warning"
}
},
{
"_index" : "log-20220127",
"_type" : "_doc",
"_id" : "0cc6fdd3d16447e6a9fa5f0ec111edc9",
"_score" : 1.0,
"_source" : {
"time" : "2022-01-26T23:12:01.957Z",
"level" : "ERROR",
"source" : "my_component_log",
"correlation_id" : "123",
"error" : {
"type" : "Error",
"category" : "Unknown",
"status" : 500,
"code" : "UNKNOWN",
"message" : "Example error",
"stack_trace" : "Error: Example error\n at C:\\Users\\user1\\OneDrive\\Desktop\\examples\\node\\obj\\elastic.js:35:29\n at Generator.next (<anonymous>)\n at fulfilled (C:\\Users\\user1\\OneDrive\\Desktop\\examples\\node\\obj\\elastic.js:5:58)"
},
"message" : "Error message"
}
},
{
"_index" : "log-20220127",
"_type" : "_doc",
"_id" : "039cf650b46b4ca49c513df617055dc3",
"_score" : 1.0,
"_source" : {
"time" : "2022-01-26T23:12:01.958Z",
"level" : "DEBUG",
"source" : "my_component_log",
"correlation_id" : "123",
"error" : null,
"message" : "Debug info"
}
},
{
"_index" : "log-20220127",
"_type" : "_doc",
"_id" : "7f8e1ef9a82643f382cffa52f9110ce3",
"_score" : 1.0,
"_source" : {
"time" : "2022-01-26T23:12:01.959Z",
"level" : "FATAL",
"source" : "my_component_log",
"correlation_id" : "123",
"error" : {
"type" : "Error",
"category" : "Unknown",
"status" : 500,
"code" : "UNKNOWN",
"message" : "Fatal error",
"stack_trace" : "Error: Fatal error\n at C:\\Users\\user1\\OneDrive\\Desktop\\examples\\node\\obj\\elastic.js:37:29\n at Generator.next (<anonymous>)\n at fulfilled (C:\\Users\\user1\\OneDrive\\Desktop\\examples\\node\\obj\\elastic.js:5:58)"
},
"message" : "Error that crash the process"
}
}
]
}
}
Code Example
{
"took" : 5,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 5,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "log-20220125",
"_type" : "_doc",
"_id" : "8cd3ef625b804ffdb995827e59bac01f",
"_score" : 1.0,
"_source" : {
"correlation_id" : "123",
"error" : null,
"level" : 4,
"message" : "Composite logger is configured and ready for using",
"source" : null,
"time" : "2022-01-25T17:47:53.274931"
}
},
{
"_index" : "log-20220125",
"_type" : "_doc",
"_id" : "e636b3a8b76e44cabea31e9e92f0b451",
"_score" : 1.0,
"_source" : {
"correlation_id" : "123",
"error" : null,
"level" : 3,
"message" : "Example warning",
"source" : null,
"time" : "2022-01-25T17:47:53.274931"
}
},
{
"_index" : "log-20220125",
"_type" : "_doc",
"_id" : "144d963d95c74b4a84e1558dde3704f9",
"_score" : 1.0,
"_source" : {
"correlation_id" : "123",
"error" : {
"category" : "Unknown",
"cause" : null,
"code" : "UNKNOWN",
"correlation_id" : null,
"details" : null,
"message" : "Example error",
"stack_trace" : null,
"status" : 500,
"type" : null
},
"level" : 2,
"message" : "Error message",
"source" : null,
"time" : "2022-01-25T17:47:53.274931"
}
},
{
"_index" : "log-20220125",
"_type" : "_doc",
"_id" : "25226536425246fab9b78a3dca4919fd",
"_score" : 1.0,
"_source" : {
"correlation_id" : "123",
"error" : null,
"level" : 5,
"message" : "Debug info",
"source" : null,
"time" : "2022-01-25T17:47:53.275937"
}
},
{
"_index" : "log-20220125",
"_type" : "_doc",
"_id" : "d49a4fda786241af98638b0d82c93ec9",
"_score" : 1.0,
"_source" : {
"correlation_id" : "123",
"error" : {
"category" : "Unknown",
"cause" : null,
"code" : "UNKNOWN",
"correlation_id" : null,
"details" : null,
"message" : "Fatal error",
"stack_trace" : null,
"status" : 500,
"type" : null
},
"level" : 1,
"message" : "Error that crash the process",
"source" : null,
"time" : "2022-01-25T17:47:53.275937"
}
}
]
}
}
Wrapping up
In this tutorial, we have learned what logging is, the different logging levels, and how to use the ConsoleLogger and CompositeLogger from PIP.Services to display log messages. The main advantage of the composite logger is its capacity to aggregate all logging messages, thus creating a centralized logging point.
We have also learned that PIP.Services provides several implementations of loggers, such as CloudWatchLogger, ElasticSearchLogger, and DataDogLogger.
Although the examples presented here are quite general, the concepts learned continue to apply to the development of more complex applications.