Microservice configuration

How microservices manage configurations.

Key takeaways

Environment variables	Variables that are a part of the running environment and whose value can affect the way processes function on a machine. It is considered a best practice to use environment variables for configuring applications.
Configuration file	File containing information about how to configure the container (i.e. which components to include).
Factory	Program that aids in the creation of components.
Descriptor	Component locator, consisting of the component’s group, type, kind, name, and version.
References	Special component that is used to store and locate components by their descriptors.

Introduction

This tutorial will explore the microservice configuration process. For this, we will first see an example that contains the main configuration aspects that most microservices have. Then, we will analyze how this process triggers and works. Finally, we will summarize what was learned.

Example

The code below will be used to analyze how configurations work. It has two parts. The first contains three components: ComponentA1, ComponentA2, and ComponentB. The first two classes are basically the same except for their names. Both have ComponentB as a dependency.

The second part contains the code used to package part 1 into a process container. This type of container acts as a system process and is based on the inversion of control pattern. Included in this part is a factory used by the container to create the components defined in part 1.

Each of these features will be explained in detail in the analysis section.

Part 1: Components

import { 
    ConfigParams, Descriptor, IConfigurable, IOpenable, 
    IReferenceable, IReferences, IUnreferenceable, Context 
} from "pip-services4-components-node"

export class ComponentB implements IReferenceable, IConfigurable, IOpenable, IUnreferenceable {
    private _param1: string = 'ABC2'
    private _param2: number = 456
    private _opened = false
    private _status: string

    /**
     * Creates a new instance of the component.
     */
    public constructor() {
        this._status = "Created";
        console.log("ComponentB has been created.");
    }

    public configure(config: ConfigParams): void {
        this._param1 = config.getAsStringWithDefault("param1", this._param1);
        this._param2 = config.getAsIntegerWithDefault("param2", this._param2);
        console.log("ComponentB has been configured.");
    }

    public setReferences(references: IReferences): void {
        throw new Error("Method not implemented.");
    }

    public isOpen(): boolean {
        throw new Error("Method not implemented.");
    }
  
    public open(ctx: Context): Promise<void> {
        throw new Error("Method not implemented.");
    }

    public close(ctx: Context): Promise<void> {
        throw new Error("Method not implemented.");
    }

    /**
     * Unsets (clears) previously set references to dependent components.
     */
    public unsetReferences(): void {
        throw new Error("Method not implemented.");
    }
}

export class ComponentA1 implements IReferenceable, IConfigurable, IOpenable, IUnreferenceable {
    private _param1: string = 'ABC';
    private _param2: number = 123;
    private _another_component: ComponentB;
    private _opened: boolean = false;
    private dummy_variable: string;

    /**
    * Creates a new instance of the component.
    */
    public constructor() {
        console.log("ComponentA1 has been created.");
    }

    public setReferences(references: IReferences): void {
        this._another_component = references.getOneRequired(
            new Descriptor("myservice", "component-b", "*", "*", "1.0")
        )
        console.log("ComponentA1's references have been defined.");
    }

    public configure(config: ConfigParams): void {
        this._param1 = config.getAsStringWithDefault("param1", 'ABC');
        this._param2 = config.getAsIntegerWithDefault("param2", 123);
        console.log("ComponentA1 has been configured.");
    }

    public isOpen(): boolean {
        return this._opened;
    }

    public async open(ctx: Context): Promise<void> {
        this._opened = true;
        console.log("ComponentA1 has been opened.");
    }

    public async close(ctx: Context): Promise<void> {
        this._opened = false;
        console.log("ComponentA1 has been closed.");
    }

    public async myTask(correlationId: string) {
        console.log("Doing my business task");
        this.dummy_variable = "dummy value";
    }

    /**
     * Unsets (clears) previously set references to dependent components.
     */
    public unsetReferences(): void {
        this._another_component = null;
        console.log("References cleared");
    }
}

export class ComponentA2 implements IReferenceable, IConfigurable, IOpenable, IUnreferenceable {
    private _param1 = 'ABC';
    private _param2 = 123;
    private _another_component: ComponentB;
    private _opened = false;
    private _status = "";
    private dummy_variable: string;

    /**
     * Creates a new instance of the component.
     */
    public constructor() {
        this._status = "Created";
        console.log("ComponentA2 has been created.");
    }

    public setReferences(references: IReferences): void {
        this._another_component = references.getOneRequired(
            new Descriptor("myservice", "component-b", "*", "*", "1.0")
        )
        this._status = "Configured";
        console.log("ComponentA2's references have been defined.");
    }

    public configure(config: ConfigParams): void {
        this._param1 = config.getAsStringWithDefault("param1", 'ABC');
        this._param2 = config.getAsIntegerWithDefault("param2", 123);
        this._status = "Configured";
        console.log("ComponentA2 has been configured.");
    }

    public isOpen(): boolean {
        return this._opened;
    }

    public async open(ctx: Context): Promise<void> {
        this._opened = true;
        this._status = "Open";
        console.log("ComponentA2 has been opened.");
    }

    public async close(ctx: Context): Promise<void> {
        this._opened = false
        this._status = "Closed"
        console.log("ComponentA2 has been closed.")
    }

    public async myTask(correlationId): Promise<void> {
        console.log("Doing my business task");
        this.dummy_variable = "dummy value";
    }
  
    /**
     * Unsets (clears) previously set references to dependent components.
     */
    public unsetReferences(): void {
        this._another_component = null;
        this._status = "Un-referenced";
        console.log("References cleared");
    }
}

Not available

import (
	"context"
	"fmt"

	cconf "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
	cref "github.com/pip-services4/pip-services4-go/pip-services4-components-go/refer"
)

type ComponentB struct {
	_param1 string
	_param2 int
	_opened bool
	_status string
}

func NewComponentB() *ComponentB {
	c := &ComponentB{}
	c._param1 = "ABC2"
	c._param2 = 456
	c._opened = false
	c._status = "Created"
	fmt.Println("ComponentB has been created.")
	return c
}

func (c *ComponentB) Configure(ctx context.Context, config *cconf.ConfigParams) {
	c._param1 = config.GetAsStringWithDefault("param1", c._param1)
	c._param2 = config.GetAsIntegerWithDefault("param2", c._param2)
	fmt.Println("ComponentB has been configured.")
}

func (c *ComponentB) SetReferences(ctx context.Context, references cref.IReferences) {

}

func (c *ComponentB) IsOpen() bool {
	return c._opened
}

func (c *ComponentB) Open(ctx context.Context, correlationId string) (err error) {
	return nil
}

func (c *ComponentB) Close(ctx context.Context, correlationId string) (err error) {
	return nil
}

// Unsets (clears) previously set references to dependent components.
func (c *ComponentB) UnsetReferences() {

}

type ComponentA1 struct {
	_param1            string
	_param2            int
	_another_component *ComponentB
	_opened            bool
	_status            string
	dummy_variable     string
}

// Creates a new instance of the component.
func NewComponentA1() *ComponentA1 {
	c := &ComponentA1{}
	c._param1 = "ABC2"
	c._param2 = 456
	c._opened = false
	c._status = "Created"
	fmt.Println("ComponentA1 has been created.")
	return c
}

func (c *ComponentA1) Configure(ctx context.Context, config *cconf.ConfigParams) {
	c._param1 = config.GetAsStringWithDefault("param1", "ABC")
	c._param2 = config.GetAsIntegerWithDefault("param2", 123)
	c._status = "Configured"
	fmt.Println("ComponentA1 has been configured.")
}

func (c *ComponentA1) SetReferences(ctx context.Context, references cref.IReferences) {
	res, descrErr := references.GetOneRequired(
		cref.NewDescriptor("myservice", "component-b", "*", "*", "1.0"),
	)
	if descrErr != nil {
		panic(descrErr)
	}
	c._another_component = res.(*ComponentB)
	c._status = "Configured"
	fmt.Println("ComponentA1's references have been defined.")
}

func (c *ComponentA1) IsOpen() bool {
	return c._opened
}

func (c *ComponentA1) Open(ctx context.Context, correlationId string) (err error) {
	c._opened = true
	c._status = "Open"
	fmt.Println("ComponentA1 has been opened.")
	return nil
}

func (c *ComponentA1) Close(ctx context.Context, correlationId string) (err error) {
	c._opened = false
	c._status = "Closed"
	fmt.Println("ComponentA1 has been closed.")
	return nil
}

func (c *ComponentA1) MyTask() {
	fmt.Println("Doing my business task")
	c.dummy_variable = "dummy value"
}

// Unsets (clears) previously set references to dependent components.
func (c *ComponentA1) UnsetReferences() {
	c._another_component = nil
	c._status = "Un-referenced"
	fmt.Println("References cleared")
}

type ComponentA2 struct {
	_param1            string
	_param2            int
	_another_component *ComponentB
	_opened            bool
	_status            string
	dummy_variable     string
}

// Creates a new instance of the component.
func NewComponentA2() *ComponentA2 {
	c := &ComponentA2{}
	c._param1 = "ABC"
	c._param2 = 123
	c._opened = false
	c._status = "Created"
	fmt.Println("ComponentA2 has been created.")
	return c
}

func (c *ComponentA2) Configure(ctx context.Context, config *cconf.ConfigParams) {
	c._param1 = config.GetAsStringWithDefault("param1", "ABC")
	c._param2 = config.GetAsIntegerWithDefault("param2", 123)
	c._status = "Configured"
	fmt.Println("ComponentA2 has been configured.")
}

func (c *ComponentA2) SetReferences(ctx context.Context, references cref.IReferences) {
	res, descrErr := references.GetOneRequired(
		cref.NewDescriptor("myservice", "component-b", "*", "*", "1.0"),
	)
	if descrErr != nil {
		panic(descrErr)
	}
	c._another_component = res.(*ComponentB)
	c._status = "Configured"
	fmt.Println("ComponentA2's references have been defined.")
}

func (c *ComponentA2) IsOpen() bool {
	return c._opened
}

func (c *ComponentA2) Open(ctx context.Context, correlationId string) (err error) {
	c._opened = true
	c._status = "Open"
	fmt.Println("ComponentA2 has been opened.")
	return nil
}

func (c *ComponentA2) Close(ctx context.Context, correlationId string) (err error) {
	c._opened = false
	c._status = "Closed"
	fmt.Println("ComponentA2 has been closed.")
	return nil
}

func (c *ComponentA2) MyTask() {
	fmt.Println("Doing my business task")
	c.dummy_variable = "dummy value"
}

// Unsets (clears) previously set references to dependent components.
func (c *ComponentA2) UnsetReferences() {
	c._another_component = nil
	c._status = "Un-referenced"
	fmt.Println("References cleared")
}

Not available

from pip_services4_components.config import IConfigurable, ConfigParams
from pip_services4_components.refer import IReferenceable, IReferences, Descriptor, IUnreferenceable
from pip_services4_components.run import IOpenable, ICleanable

class ComponentB(IReferenceable, IConfigurable, IOpenable):
    _param1 = 'ABC2'
    _param2 = 456
    _opened = False

    def __init__(self):
        """
        Creates a new instance of the component.
        """
        self._status = "Created"
        print("ComponentB has been created.")
        
    def configure(self, config):
        self._param1 = config.get_as_string_with_default("param1", self._param1)
        self._param2 = config.get_as_integer_with_default("param2", self._param2)
        print("ComponentB has been configured.")
        
    def set_references(self, references):
        pass
        
    def is_open(self):
        pass

    def open(self, correlation_id):
        pass

    def close(self, correlation_id):
        pass
        
    def my_task(self, correlation_id):
        pass

    def unset_references(self):
        """
        Unsets (clears) previously set references to dependent components.
        """
        pass


class ComponentA1(IReferenceable, IConfigurable, IOpenable):
    _param1 = 'ABC'
    _param2 = 123
    _another_component: ComponentB
    _open = False
    _status = None
    
    def __init__(self):
        """
        Creates a new instance of the component.
        """
        self._status = "Created"
        print("ComponentA1 has been created.")
            
    def configure(self, config):
        self._param1 = config.get_as_string_with_default("param1", 'ABC')
        self._param2 = config.get_as_integer_with_default("param2", 123)
        self._status = "Configured"
        print("ComponentA1 has been configured.")

    def set_references(self, references):
        self._another_component = references.get_one_required(
            Descriptor("myservice", "component-b", "*", "*", "1.0")
        )
        self._status = "Configured"
        print("ComponentA1's references have been defined.")
        
    def is_open(self):
        return self._open

    def open(self, correlation_id):
        self._open = True
        self._status = "Open"
        print("ComponentA1 has been opened.")

    def close(self, correlation_id):
        self._opened = False
        self._status = "Closed"
        print("ComponentA1 has been closed.")
        
    def my_task(self, correlation_id):
        print("Doing my business task")
        dummy_variable = "dummy value"

    def unset_references(self):
        """
        Unsets (clears) previously set references to dependent components.
        """
        self._another_component = None
        self._status = "Un-referenced"
        print("References cleared")
        
        
class ComponentA2(IReferenceable, IConfigurable, IOpenable):
    _param1 = 'ABC'
    _param2 = 123
    _another_component: ComponentB
    _open = False
    _status = None
    
    def __init__(self):
        """
        Creates a new instance of the component.
        """
        self._status = "Created"
        print("ComponentA2 has been created.")
            
    def configure(self, config):
        self._param1 = config.get_as_string_with_default("param1", 'ABC')
        self._param2 = config.get_as_integer_with_default("param2", 123)
        self._status = "Configured"
        print("ComponentA2 has been configured.")

    def set_references(self, references):
        self._another_component = references.get_one_required(
            Descriptor("myservice", "component-b", "*", "*", "1.0")
        )
        self._status = "Configured"
        print("ComponentA2's references have been defined.")
        
    def is_open(self):
        return self._open

    def open(self, correlation_id):
        self._open = True
        self._status = "Open"
        print("ComponentA2 has been opened.")

    def close(self, correlation_id):
        self._opened = False
        self._status = "Closed"
        print("ComponentA2 has been closed.")
        
    def my_task(self, correlation_id):
        print("Doing my business task")
        dummy_variable = "dummy value"

    def unset_references(self):
        """
        Unsets (clears) previously set references to dependent components.
        """
        self._another_component = None
        self._status = "Un-referenced"
        print("References cleared")

Not available

Part 2: Container

import { Descriptor, Factory } from "pip-services4-components-node";
import { ProcessContainer } from "pip-services4-container-node";


/**
 * Creating a process container
 */
export class MyProcess extends ProcessContainer {
    public constructor() {
        super('myservice', 'My service running as a process')
        this._configPath = './configV4.yaml'

        let MyFactory1 = new Factory();

        MyFactory1.registerAsType(new Descriptor("myservice", "component-a1", "default", "*", "1.0"), ComponentA1);
        MyFactory1.registerAsType(new Descriptor("myservice", "component-a2", "default", "*", "1.0"), ComponentA2);
        MyFactory1.registerAsType(new Descriptor("myservice", "component-b", "default", "*", "1.0"), ComponentB);

        this._factories.add(MyFactory1)
    }
}

/**
 * Running the container
 */
function main() {
    let runner = new MyProcess();
    console.log("run");
    try {
        runner.run(process.argv);
    }
    catch(ex){
        console.log(ex)
    }
}

Not available

import (
	"context"
	"os"

	cref "github.com/pip-services4/pip-services4-go/pip-services4-components-go/refer"

	cbuild "github.com/pip-services4/pip-services4-go/pip-services4-components-go/build"
	ccont "github.com/pip-services4/pip-services4-go/pip-services4-container-go/container"
)

// Running the container
func main() {
	proc := NewMyProcess()
	proc.Run(context.Background(), os.Args)
}

type MyProcess struct {
	*ccont.ProcessContainer
}

func NewMyProcess() *MyProcess {
	c := &MyProcess{}
	c.ProcessContainer = ccont.NewProcessContainer("myservice", "My service running as a process")
	c.SetConfigPath("./configV4.yaml")

	myFactory1 := cbuild.NewFactory()

	myFactory1.RegisterType(cref.NewDescriptor("myservice", "component-a1", "default", "*", "1.0"), NewComponentA1)
	myFactory1.RegisterType(cref.NewDescriptor("myservice", "component-a2", "default", "*", "1.0"), NewComponentA2)
	myFactory1.RegisterType(cref.NewDescriptor("myservice", "component-b", "default", "*", "1.0"), NewComponentB)

	c.AddFactory(myFactory1)

	return c
}

Not available

from pip_services4_components.build import Factory 
from pip_services4_container.container import ProcessContainer

# Creating a process container
class MyProcess(ProcessContainer):
    def __init__(self):
        super(MyProcess, self).__init__('myservice', 'My service running as a process')
        self._config_path = './configV4.yaml'

        # Creating a factory
        MyFactory1 = Factory()

        MyFactory1.register_as_type(Descriptor("myservice", "component-a1", "default", "*", "1.0"), ComponentA1)
        MyFactory1.register_as_type(Descriptor("myservice", "component-a2", "default", "*", "1.0"), ComponentA2)
        MyFactory1.register_as_type(Descriptor("myservice", "component-b", "default", "*", "1.0"), ComponentB)

        self._factories.add(MyFactory1)

# Running the container

import os

os.environ["COMPA2_ENABLED"] = "True"

if __name__ == '__main__':
    runner = MyProcess()
    print("run")
    try:
        runner.run()
    except Exception as ex:
        print(ex)

Not available

Analysis

Let’s now analyze the execution process happening in the above example. For this, we will follow a bottom-up approach and start from the code lines where the container execution is triggered. Then, we will continue up to the point where our component’s dependencies are created.

Environment variables

Pip.Services’ containerization approach allows us to perform component selections using the environment variables set in the execution environment. For example, if ComponentA1 is an in-memory persistence and ComponentA2 is a database persistence, we can select which to use in our container by setting the corresponding environment variable. Thus, our program starts by setting the environment variable COMPA1_ENABLED to true, which tells the container to include ComponentA1. Next, it triggers the execution of the container with the run() method.

Configuration file

Once the execution of the container is triggered, the program obtains its component configuration information from the configuration file, whose location is defined via the config_path variable. This will be a yaml file, containing information on the different components that the container must create.

The figure below shows the file for our example. It describes three components: a logger, ComponentA1, and ComponentA2. The logger is part of the set of components whose factories are called by the container by default. In this case, we select a console logger.

Then, we have the other two components, each inside a conditional statement. This allows us to choose the one we need using the environment variables. Since we’ve defined COMPA1_ENABLED as true, the container selects ComponentA1 and ignores ComponentA2.

Factory

With the information gathered from the environment variables and the configuration file, the container creates the required components via two kinds of factories: the default factories* that are part of the container and the custom factories we defined in our code.

(*for more information on default factories, see the default container factory page of Pip.Services Docs for your programming language of choice: Python, Node.js, .NET, Golang or Dart)

In our example, we create a factory for ComponentB, ComponentA1, and ComponentA2, and we register these components in it via descriptors. This step provides a link between what was defined in the configuration file (using the same descriptors, just in a colon-separated format) and our components. Note that, even though our config file does not contain descriptors for ComponentB, we still register it in the factory. This is because ComponentB is a dependency for ComponentA1 and ComponentA2 and will be created by the program at a later step, when we start to set references.

References

Finally, when creating either ComponentA1 or ComponentA2, the program detects that this class has implemented the IReferenceable interface. Then, from the setReferences() method, it obtains the necessary information to create all required dependencies, which would be ComponentB in our case. This information is obtained from an instance of the References class, which retrieves information from the factory’s registered components.

Additionally, by implementing the IConfigurable interface, we can set the values of the component’s parameters using the configure() method, which accepts a ConfigParams object as a parameter.

Results

The described process results in all required components being created and the following messages displayed on our screen:

Moreover, if we stop the process, we obtain the following messages on our screen:

Alternatively, if we choose to use ComponentA2 (e.g. by setting the COMPA2_ENABLED environment variable), we get the following results:

And, after stopping the process:

Wrapping up

In this tutorial, we have explored a basic microservice configuration process. We started with an example containing two code parts. The first presents three classes, one of them acting as a component dependency for the other two. The second part of the code creates a factory and a container, which is then run.

Then, in the following section, we analyzed each of the steps included in the configuration process. We saw how the container selects variables from the environment, obtains information about the components to be created from a configuration file, and creates them via the use of factories.

Finally, we learned that the program obtains information about dependent components from the setReferences() method and creates the components defined there.

The final result is a microservice, running inside a container, that uses environment variables to create certain components at runtime and references to create additional components as dependencies.