Component configuration

  • by Aleksey Dvoykin

Introduction

The Pip.Services Toolkit offers a simple but very flexible mechanism for component configuration. Configurations can be loaded from various sources - configuration files, command line parameters, environment variables, configuration services, etc. Once loaded, they are passed to the specific component, which configures itself accordingly. In this recipe, we’ll be taking a look at this mechanism’s capabilities and how it can be utilized.

Configuration

The configurable interface

A component can be made configurable by adding the IConfigurable interface and implementing its configure method. This method will be called by the container right after container creation, with the loaded configuration being passed as a parameter.

See: IConfigurable

interface IConfigurable {
	configure(config: ConfigParams): void;
}

See: IConfigurable

public interface IConfigurable
{
  	void Configure(ConfigParams config);
}

See: IConfigurable

type IConfigurable interface {

  Configure(ctx context.Context, config *ConfigParams)
}

See: IConfigurable

abstract class IConfigurable {

  void configure(ConfigParams config);
}

See: IConfigurable

class IConfigurable(ABC):

    def configure(self, config: ConfigParams):
        raise NotImplementedError('Method from interface definition')
Not available

The ConfigParams object

The only parameter that is passed to the configure method is ConfigParams object. Simply put - this is a map that allows us to get a configuration parameter value by its corresponding key. Although various programming languages have unique syntax for initializing maps and objects, ConfigParams support initialization that is independent of the language being used.

See: ConfigParams

let config = ConfigParams.fromTuples(
  	"param1", 123,
  	"param2", "2020-01-01T11:00:00.0Z"
);

See: ConfigParams

var config = ConfigParams.FromTuples(
  "param1", 123,
  "param2", "2020-01-01T11:00:00.0Z"
);

See: ConfigParams

import cconfig "github.com/pip-services3-gox/pip-services3-commons-gox/config"

config := cconfig.NewConfigParamsFromTuples(
	"param1", 123,
	"param2", "2020-01-01T11:00:00.0Z",
)

See: ConfigParams

var config = ConfigParams.fromTuples([
	'param1', 123, 
	'param2', '2020-01-01T11:00:00.0Z'
]);

See: ConfigParams

config = ConfigParams.From_tuples(
  "param1", 123,
  "param2", "2020-01-01T11:00:00.0Z"
)
Not available

ConfigParams also provide some additional functionality. All keys and values are stored as strings, but ConfigParams supports performing data type conversion when extracting values. Another option available is the opportunity to set default values.

let param1 = config.getAsInteger("param1");
let param2 = config.getAsDateTimeWithDefault("param2", new Date());


var param1 = config.GetAsInteger("param1");
var param2 = config.GetAsDateTimeWithDefault("param2", new DateTime());


param1 := config.GetAsInteger("param1")
param2 := config.GetAsDateTimeWithDefault("param2", time.Now())

var param1 = config.getAsInteger('param1');
var param2 = config.getAsDateTimeWithDefault('param2', DateTime.now());


param1 = config.get_as_integer("param1")
param2 = config.get_as_datetime_with_default("param2", datetime.datetime.now())
Not available

The parameter kets can have a complex structure, grouped by sections using dot notation. ConfigParams can be used to work with entire sections as well.

let configWithSections = ConfigParams.fromTuples(
  	"param1", 123
  	"options.param1", "ABC",
  	"options.param2", "XYZ"
);
let options = configWithSections.getSection("options");


var configWithSections = ConfigParams.FromTuples(
  "param1", 123,
  "options.param1", "ABC",
  "options.param2", "XYZ"
);
var options = configWithSections.GetSection("options");


configWithSections := cconfig.NewConfigParamsFromTuples(
	"param1", 123,
	"options.param1", "ABC",
	"options.param2", "XYZ",
)
options := configWithSections.GetSection("options")

var configWithSections = ConfigParams.fromTuples([
	'param1', 123, 
	'options.param1', 'ABC', 
	'options.param2', 'XYZ'
]);

var options = configWithSections.getSection('options');


config_with_sections = ConfigParams.from_tuples(
  "param1", 123
  "options.param1", "ABC",
  "options.param2", "XYZ"
)
options = config_with_sections.get_section("options")
Not available

Setting a default configuration

Another helpful option is the ability to set a default configuration.


let defaultConfig = ConfigParams.fromTuples(
  	"param1", 1,
  	"param2", "Default Value"
);
config = config.setDefaults(defaultConfig);



var defaultConfig = ConfigParams.FromTuples(
  "param1", 1,
  "param2", "Default Value"
);
config = config.SetDefaults(defaultConfig);


defaultConfig := cconfig.NewConfigParamsFromTuples(
		"param1", 1,
		"param2", "Default Value",
)
config = config.SetDefaults(defaultConfig)


var defaultConfig = ConfigParams.fromTuples([
		'param1', 1, 
	  	'param2', 'Default Value'
	]);

config = config.setDefaults(defaultConfig);



default_config = ConfigParams.from_tuples(
  "param1", 1,
  "param2", "Default Value"
)
config = config.set_defaults(default_config)
Not available

Serializing the ConfigParams object

Lastly, ConfigParams objects can be serialized/deserialized to/from JSON, YAML, or a plain string.

let anotherConfig = ConfigParams.fromString("param1=123;param2=ABC");


var anotherConfig = ConfigParams.FromString("param1=123;param2=ABC");


anotherConfig := cconfig.NewConfigParamsFromString("param1=123;param2=ABC")

var anotherConfig = ConfigParams.fromString('param1=123;param2=ABC');


another_config = ConfigParams.from_string("param1=123;param2=ABC")
Not available

To read more about what functionality is available through ConfigParams, be sure to check out the Commons documentation.

Example

Below is an example of a configurable component:

See: Commons module’s

export class DataController implements IConfigurable {
   	private _max_page_size: number = 5;
   	public constructor() { }

   	public configure(config: ConfigParams): void {
		this._max_page_size = config.getAsIntegerWithDefault('max_page_size', this._max_page_size);
   	}

   	public getData(correlationId: string, filter: FilterParams, paging: PagingParams): Promise<DataPage<MyData>> {
		paging.take = Math.min(paging.take, this._max_page_size);    
   	  	// Get data using max page size constraint.
   	}
}

See: Commons module’s

public class DataController: IConfigurable
{
	int _maxPageSize = 5;

   	public void Configure(ConfigParams config)
   	{
		this._maxPageSize = config.GetAsIntegerWithDefault("max_page_size", this._maxPageSize);
   	}
		
   	public DataPage get_data(string correlationId, FilterParams filter, PagingParams paging)
   	{
		paging.Take = Math.Min(paging.Take, this._maxPageSize);
		// Get data using max page size constraint.
   	}
	        
}

See: Commons module’s

import (
	"math"

	cconfig "github.com/pip-services3-gox/pip-services3-commons-gox/config"
	cdata "github.com/pip-services3-gox/pip-services3-commons-gox/data"
)

type DataController struct {
	_maxPageSize int
}

func NewDataController() *DataController {
	return &DataController{
		_maxPageSize: 5,
	}
}

func (c *DataController) Configure(ctx context.Context, config *cconfig.ConfigParams) {
	c._maxPageSize = config.GetAsIntegerWithDefault("max_page_size", c._maxPageSize)
}

func (c *DataController) GetData(ctx context.Context, correlationId string, filter *cdata.FilterParams, paging *cdata.PagingParams) (page *cdata.DataPage, err error) {
	*paging.Take = int64(math.Min(float64(*paging.Take), float64(c._maxPageSize)))
	// Get data using max page size constraint.
	return page, err
}

See: Commons module’s

class DataController implements IConfigurable {
  int _max_page_size = 5;
  DataController();

  @override
  void configure(ConfigParams config) {
    _max_page_size =
        config.getAsIntegerWithDefault('max_page_size', _max_page_size);
  }

  Future<DataPage<MyData>> getData(
      String? correlationId, FilterParams filter, PagingParams paging) async {
    paging.take = min(paging.take ?? 0, _max_page_size);
    // Get data using max page size constraint.
	
  }
}

See: Commons module’s

class DataController(IConfigurable):
   __max_page_size: int = 5


   def configure(self, config: ConfigParams):
		self.__max_page_size = config.get_as_integer_with_default('max_page_size', self.__max_page_size)
   
   def get_data(self, correlation_id: str, filter: FilterParams, paging: PagingParams) -> DataPage: 
	paging.take = min(paging.take, self.__max_page_size)   
        # Get data using max page size constraint.
Not available

Manual configuration

Manual configuration can be done in the following manner:

let component = new DataController();
let config = ConfigParams.fromTuples("max_page_size", 100);
component.configure(config);



var component = DataController();
var config = ConfigParams.FromTuples("max_page_size", 100);
component.Configure(config);


component := configurationexample.NewDataController()
config := cconfig.NewConfigParamsFromTuples("max_page_size", 100)
component.Configure(context.Background(), config)

var component = DataController();
var config = ConfigParams.fromTuples(['max_page_size', 100]);
component.configure(config);


component = DataController()
config = ConfigParams.from_tuples("max_page_size", 100)
component.configure(config)
Not available

However, a component’s configuration is usually stored in the microservice’s configuration file. The configure method will receive the parameters for the specific component (in the example below - everything between the component’s descriptor and the next descriptor or end of the file). To get more info on microservice configuration, read our Component Container recipe.

...
# Controller
- descriptor: "beacons:controller:default:default:1.0"
  max_page_size: 10
...

Resolvers

The NameResolver and OptionsResolver classes are helper classes that simplify the use of configurations.

NameResolver

NameResolver is a helper class that allows extraction of the component’s name from the configuration parameters. The name can be stored as a “id”/“name” parameter or inside a component’s descriptor.

Below is a simple example of how it can be used:

See: NameResolver, OptionsResolver

let config = ConfigParams.fromTuples(
	"descriptor", "myservice:connector:aws:connector1:1.0",
	"param1", "ABC",
	"param2", 123
);

let name = NameResolver.resolve(config); // Result: connector1

See: NameResolver, OptionsResolver

var config = ConfigParams.FromTuples(
	"descriptor", "myservice:connector:aws:connector1:1.0",
	"param1", "ABC",
	"param2", 123
);
string name = NameResolver.Resolve(config); // Result: connector1

See: NameResolver, OptionsResolver

config := cconfig.NewConfigParamsFromTuples(
	"descriptor", "myservice:connector:aws:connector1:1.0",
	"param1", "ABC",
	"param2", 123,
)
name := cconfig.NameResolver.Resolve(config) // Result: connector1

See: NameResolver, OptionsResolver

var config = ConfigParams.fromTuples([
  'descriptor',
  'myservice:connector:aws:connector1:1.0',
  'param1',
  'ABC',
  'param2',
  123
]);

var name = NameResolver.resolve(config); // Result: connector1

See: NameResolver, OptionsResolver

config = ConfigParams.from_tuples(
	"descriptor", "myservice:connector:aws:connector1:1.0",
	"param1", "ABC",
	"param2", 123
)
name = NameResolver.resolve(config) # Result: connector1
Not available

OptionsResolver

OptionsResolver is a helper class that extracts parameters from the “options” configuration section.

let config = ConfigParams.fromTuples(
	...
	"options.param1", "ABC",
	"options.param2", 123
);
let options = OptionsResolver.resolve(config); // Result: param1=ABC;param2=123


var config = ConfigParams.FromTuples(
    ...
	"options.param1", "ABC",
	"options.param2", 123
);
var options = OptionsResolver.Resolve(config)   // Result: param1=ABC;param2=123


config := cconfig.NewConfigParamsFromTuples(
	...
	"options.param1", "ABC",
	"options.param2", 123,
)
options := cconfig.OptionsResolver.Resolve(config) // Result: param1=ABC;param2=123

var config = ConfigParams.fromTuples([
  // ...
  'options.param1', 'ABC',
  'options.param2', 123
]);
var options = OptionsResolver.resolve(config); // Result: param1=ABC;param2=123


config = ConfigParams.from_tuples(
    ...
	"options.param1", "ABC",
	"options.param2", 123
)
options = OptionsResolver.resolve(config)   # Result: param1=ABC;param2=123
Not available

Configuration readers

Configuration parameters can be stored in microservice configurations, configuration files, or in configuration services. To help with configuration extraction, the Pip.Services Toolkit offers two special ConfigReader components. The interface for these components is defined in the Components module.

See: Components

interface IConfigReader {
	readConfig(correlationId: string, parameters: ConfigParams): Promise<ConfigParams>;
}

See: Components

public interface IConfigReader
{
    ConfigParams ReadConfig(string correlationId, ConfigParams parameters);
}

See: Components

type IConfigReader interface {
	
	ReadConfig(ctx context.Context, correlationId string, parameters *c.ConfigParams) (*c.ConfigParams, error)
}

See: Components

abstract class IConfigReader {
  Future<ConfigParams> readConfig(String? correlationId, ConfigParams parameters);
}

See: Components

class IConfigReader(ABC):

    def read_config_(self, correlation_id: Optional[str], parameters: ConfigParams) -> ConfigParams:
        raise NotImplementedError('Method from interface definition')
Not available

MemoryConfigReader

The MemoryConfigReader is a ConfigReader that stores configuration data in memory.

See: MemoryConfigReader, ConfigReader

let config = ConfigParams.fromTuples(
	"connection.host", "localhost",
	"connection.port", "8080"
);

let configReader = new MemoryConfigReader();
configReader.configure(config);

let parameters = ConfigParams.fromValue(process.env);
let result = await configReader.readConfig("123", parameters); 
// Result: connection.host=localhost;connection.port=8080

See: MemoryConfigReader, ConfigReader

var config = ConfigParams.FromTuples(
	"connection.host", "localhost",
	"connection.port", "8080"
);
var configReader = MemoryConfigReader();
configReader.configure(config);
var parameters = ConfigParams.FromValue(args);
configReader.ReadConfig("123", parameters); // Result: connection.host=localhost;connection.port=8080

See: MemoryConfigReader, ConfigReader

config := cconfig.NewConfigParamsFromTuples(
	"connection.host", "localhost",
	"connection.port", "8080",
)

configReader := creader.NewEmptyMemoryConfigReader()
configReader.Configure(ctx context.Context, config)
parameters := cconfig.NewConfigParamsFromValue(os.Args)
configReader.ReadConfig(ctx context.Context, "123", parameters) // Result: connection.host=localhost;connection.port=8080

See: MemoryConfigReader, ConfigReader

var config = ConfigParams.fromTuples(['connection.host', 'localhost', 'connection.port', '8080']);

var configReader = MemoryConfigReader();
configReader.configure(config);

var parameters = ConfigParams.fromValue(Platform.environment);
var result = await configReader.readConfig('123', parameters);

See: MemoryConfigReader, ConfigReader

config = ConfigParams.from_tuples(
	"connection.host", "localhost",
	"connection.port", "8080"
)
config_reader = MemoryConfigReader()
config_reader.configure(config)
parameters = ConfigParams.from_value(sys.argv)
config_reader.read_config("123", parameters) # Result: connection.host=localhost;connection.port=8080
Not available

JsonConfigReader

The JsonConfigReader is a ConfigReader that can read configurations from a JSON file.

{ "key1": "{{KEY1_VALUE}}", "key2": "{{KEY2_VALUE}}" }

See: JsonConfigReader

let configReader = new JsonConfigReader("config.json");
let parameters = ConfigParams.fromTuples("KEY1_VALUE", 123, "KEY2_VALUE", "ABC");
let result = await configReader.readConfig("correlationId", parameters); // Result: key1=123;key2=ABC

See: JsonConfigReader

var configReader = JsonConfigReader("config.json");
var parameters = ConfigParams.FromTuples("KEY1_VALUE", 123, "KEY2_VALUE", "ABC");
configReader.ReadConfig("123", parameters);    // Result: key1=123;key2=ABC

See: JsonConfigReader

configReader := creader.NewJsonConfigReader("config.json")
parameters := cconfig.NewConfigParamsFromTuples("KEY1_VALUE", 123, "KEY2_VALUE", "ABC")
configReader.ReadConfig(ctx context.Context, "123", parameters) // Result: key1=123;key2=ABC

See: JsonConfigReader

var configReader = JsonConfigReader('config.json');
var parameters = ConfigParams.fromTuples(['KEY1_VALUE', 123, 'KEY2_VALUE', 'ABC']);

var result = await configReader.readConfig('correlationId', parameters); // Result: key1=123;key2=ABC

See: JsonConfigReader

configReader = JsonConfigReader("config.json")
parameters = ConfigParams.from_tuples("KEY1_VALUE", 123, "KEY2_VALUE", "ABC")
configReader.read_config_("123", parameters)    # Result: key1=123;key2=ABC
Not available

YamlConfigReader

The YamlConfigReader is a ConfigReader that can read configurations from a YAML file.

key1: "1234"
key2: "ABCD"

See: YamlConfigReader

let configReader = new YamlConfigReader("config.yml");
let parameters = ConfigParams.fromTuples("KEY1_VALUE", 123, "KEY2_VALUE", "ABC");
let result = await configReader.readConfig("correlationId", parameters); // Result: key1=1234;key2=ABCD

See: YamlConfigReader

configReader = YamlConfigReader("config.yml");
parameters = ConfigParams.FromTuples("KEY1_VALUE", 123, "KEY2_VALUE", "ABC");;
configReader.ReadConfig("123", parameters);    // Result: key1=1234;key2=ABCD

See: YamlConfigReader

configReader := creader.NewYamlConfigReader("config.yml")
parameters := cconfig.NewConfigParamsFromTuples("KEY1_VALUE", 123, "KEY2_VALUE", "ABC")
configReader.ReadConfig(ctx context.Context, "123", parameters) // Result: key1=123;key2=ABC

See: YamlConfigReader

var configReader = YamlConfigReader('config.yml');
var parameters = ConfigParams.fromTuples(['KEY1_VALUE', 123, 'KEY2_VALUE', 'ABC']);
var result = await configReader.readConfig('correlationId', parameters); // Result: key1=1234;key2=ABCD

See: YamlConfigReader

configReader = YamlConfigReader("config.yml")
parameters = ConfigParams.from_tuples("KEY1_VALUE", 123, "KEY2_VALUE", "ABC")
configReader.read_config_("123", parameters)    # Result: key1=1234;key2=ABCD
Not available