Connectivity

Microservices systems are distributed by design, and connecting services to each other and the infrastructure is an important task. Pip.Services provides a few abstractions and patterns to make this easier.

Connection parameters

ConnectionParams is an object in the Pip.Services toolkit that holds connection information. It is based on ConfigParams and offers a few convenient methods to get and set the most common connection parameters like protocol, host or port number. It is defined in the connect package in the components module.

The same package contains an IDiscovery interface that shall be implemented by components that connect to discovery services. Through this interface, developers can retrieve a ConnectionParams object to connect to a particular service or a list of ConnectionParams to connect to cluster nodes. Also, this interface allows registering connection endpoints for starting services, so they can be stored in the discovery service and then retrieved by consumers.

MemoryDiscovery is the simplest implementation of a discovery component. It allows storing connection information in a single place in the config file. Typically, connection parameters are set in the connection (or connections for clusters) section in configuration parameters and passed to components via the configure method. Components can read that information by themselves.

# MongoDb persistence component
descriptor: "myservice:mypersistance:mongodb:default:1.0"
connection:
  host: mongo
  port: 27017

import { ConfigParams, IConfigurable } from "pip-services3-commons-nodex";
import { ConnectionParams } from "pip-services3-components-nodex";


class MyPersistence implements IConfigurable {
    private _host: string;
    private _port: number;

    public configure(config: ConfigParams) {
        let connection = ConnectionParams.fromConfig(config.getSection("connection"));
        this._host = connection.getHost();
        this._port = connection.getPortWithDefault(27017);
    }
}


using PipServices3.Commons.Config;
using PipServices3.Components.Connect;


class MyPersistence : IConfigurable
{
    private string _host;
    private int _port;

    public void Configure(ConfigParams config)
    {
        var connection = ConnectionParams.FromConfig(config.GetSection("connection"));
        _host = connection.Host;
        _port = connection.GetPortWithDefault(27017);
    }
}



import (
	"context"

	cconf "github.com/pip-services3-gox/pip-services3-commons-gox/config"
	cconn "github.com/pip-services3-gox/pip-services3-components-gox/connect"
)


type MyPersistence struct {
	host string
	port int
}

func (c *MyPersistence) Configure(ctx context.Context, config *cconf.ConfigParams) {
	connection := cconn.NewConnectionParamsFromConfig(config.GetSection("connection"))
	c.host = connection.Host()
	c.port = connection.Port()
}



import 'package:pip_services3_commons/pip_services3_commons.dart';
import 'package:pip_services3_components/pip_services3_components.dart';

class MyPersistence implements IConfigurable {
  String? _host;
  int? _port;

  void configure(ConfigParams config) {
    var connection =
        ConnectionParams.fromConfig(config.getSection('connection'));
    _host = connection!.getHost();
    _port = connection.getPortWithDefault(27017);
  }
}


from pip_services3_commons.config import ConfigParams, IConfigurable
from pip_services3_components.connect import ConnectionParams


class MyPersistence(IConfigurable):
    _host: str
    _port: int

    def configure(self, config: ConfigParams):
        connection = ConnectionParams.from_config(config.get_section("connection"))
        self._host = connection.get_host()
        self._port = connection.get_port_with_default(27017)
Not available

Hardcoding connection parameters in the component configurations works well for static connections. However, when the connection information is dynamic and has to be retrieved from a discovery service, a discovery_key parameter shall be used to specify the key that shall be employed to get the desired connection parameters. Pip.Services has the ConnectionResolver helper class to make this scenario easier.

The example below is similar to the previous one but shows how to get mongo connection parameters from the in-memory discovery service using the ConnectionResolver helper.

# In-memory discovery service
descriptor: "pip-services:discovery:memory:default:1.0"
mongo:
  host: mongo
  port: 27017

# MongoDb persistence component
descriptor: "myservice:mypersistance:mongodb:default:1.0"
connection:
  discovery_key: mongo

import { 
    ConfigParams, IConfigurable, 
    IReferenceable, IReferences 
} from "pip-services3-commons-nodex";

import { CredentialResolver } from "pip-services3-components-nodex";


class MyPersistence implements IConfigurable, IReferenceable {
    ...
    private _credentialResolver = new CredentialResolver();
    private _username: string;
    private _password: string;

    public configure(config: ConfigParams) {
        ...
        this._connectionResolver.configure(config);
    }

    public setReferences(refs: IReferences) {
        ...
        this._credentialResolver.setReferences(refs);

        let credentials = this._credentialResolver.lookup(null);
        this._username = credentials.getUsername();
        this._password = credentials.getPassword();
    }
}


using PipServices3.Commons.Config;
using PipServices3.Commons.Refer;
using PipServices3.Components.Auth;


class MyPersistence : IConfigurable, IReferenceable
{
    ...
    private CredentialResolver _credentialResolver = new CredentialResolver();
    private string _username;
    private string _password;

    public void Configure(ConfigParams config)
    {
        ...
        this._connectionResolver.Configure(config);
    }

    public async void SetReferences(IReferences references)
    {
        ...
        this._credentialResolver.SetReferences(references);

        var credentials = await _credentialResolver.LookupAsync(null);
        this._username = credentials.Username;
        this._password = credentials.Password;
    }
}



import (
	"context"

	cconf "github.com/pip-services3-gox/pip-services3-commons-gox/config"
	cref "github.com/pip-services3-gox/pip-services3-commons-gox/refer"
	cauth "github.com/pip-services3-gox/pip-services3-components-gox/auth"
)

func main() {

}

type MyPersistence struct {
	credentialResolver *cauth.CredentialResolver
	username           string
	password           string
}

func (c *MyPersistence) Configure(ctx context.Context, config *cconf.ConfigParams) {
	// ...
	c.credentialResolver.Configure(ctx, config)
}

func (c *MyPersistence) SetReferences(ctx context.Context, references cref.IReferences) {
	// ...
	c.credentialResolver.SetReferences(ctx, references)

	credentials, err := c.credentialResolver.Lookup(ctx, "")
	c.username = credentials.Username()
	c.password = credentials.Password()
}


import 'package:pip_services3_commons/pip_services3_commons.dart';
import 'package:pip_services3_components/pip_services3_components.dart';

class MyPersistence implements IConfigurable, IReferenceable {
    ...
    CredentialResolver _credentialResolver = new CredentialResolver();
    String? _username;
    String? _password;

    @override
    void configure(ConfigParams config) {
        ...
        this._connectionResolver.configure(config);
    }

    @override
    void setReferences(IReferences refs) async {
        ...
        _credentialResolver.setReferences(refs);

        var credentials = await _credentialResolver.lookup(null);
        _username = credentials!.getUsername();
        _password = credentials.getPassword();
    }
}



from pip_services3_commons.config import ConfigParams, IConfigurable
from pip_services3_commons.refer import IReferenceable, IReferences
from pip_services3_components.auth import CredentialResolver


class MyPersistence(IConfigurable, IReferenceable):
    ...
    _credential_resolver = CredentialResolver()
    _username: str
    _password: str

    def configure(self, config: ConfigParams):
        ...
        self._connection_resolver.configure(config)

    def set_references(self, refs: IReferences):
        ...
        self._credential_resolver.set_references(refs)

        credentials = self._credential_resolver.lookup(None)
        self._username = credentials.get_username()
        self._password = credentials.get_password()
Not available

Credential parameters

It is a good practice to store sensitive data, like usernames, passwords and API keys in a protected vault where they cannot be easily found by hackers. To support this scenario, the Pip.Services toolkit offers a set of abstractions that work similarly to connections. These are the CredentialParams class and the IcredentialStore interface.

CredentialParams is an object in the Pip.Services toolkit that holds the credentials required to connect to external services. Just like ConnectionParams, it is based on ConfigParams and offers a few convenient methods to get and set common credentials like username, password, access id, access key, and so on. It is defined in the auth package in the components module.

The IcredentialStore interface in the auth package is used to connect to vaults. Through this interface, developers can retrieve CredentialParams with the required credentials.

The MemoryCredentialStore class is the simplest implementation of a credential store component. It allows storing credential information in a single place in the config file and then sharing it across other components.

Similar to connection parameters, credentials are set in the credential section of the configuration parameters and passed to components via the configure method. Components can read that information by themselves.

# MongoDb persistence component
descriptor: "myservice:mypersistance:mongodb:default:1.0"
credential:
  username: admin
  password: pass123

import { ConfigParams, IConfigurable } from "pip-services3-commons-nodex";
import { CredentialParams } from "pip-services3-components-nodex";


class MyPersistence implements IConfigurable {
    ...
    private _username: string;
    private _password: string;

    public configure(config: ConfigParams) {
        ...
        let credentials = CredentialParams.fromConfig(config.getSection("credential"));
        this._username = credentials.getUsername();
        this._password = credentials.getPassword();
    }
}


using PipServices3.Commons.Config;
using PipServices3.Components.Auth;


class MyPersistence : IConfigurable, IReferenceable
{
    ...
    private string _username;
    private string _password;

    public void Configure(ConfigParams config)
    {
        ...
        var credentials = CredentialParams.FromConfig(config.GetSection("credential"));
        _username = credentials.Username;
        _password = credentials.Password;
    }
}


import (
	"context"

	cconf "github.com/pip-services3-gox/pip-services3-commons-gox/config"
	cref "github.com/pip-services3-gox/pip-services3-commons-gox/refer"
	cauth "github.com/pip-services3-gox/pip-services3-components-gox/auth"
)

type MyPersistence struct {
	// ...
	username string
	password string
}

func (c *MyPersistence) Configure(ctx context.Context, config *cconf.ConfigParams) {
	// ...
	credentials := cauth.NewCredentialParamsFromConfig(config)
	c.username = credentials.Username()
	c.password = credentials.Password()
}

func (c *MyPersistence) SetReferences(ctx context.Context, references cref.IReferences) {
	// ...
}


import 'package:pip_services3_commons/pip_services3_commons.dart';
import 'package:pip_services3_components/pip_services3_components.dart';

void main(List<String> arguments) async {}

class MyPersistence implements IConfigurable {
  ...
  String? _username;
  String? _password;

  @override
  void configure(ConfigParams config) {
    ...
    var credentials = CredentialParams.fromConfig(config.getSection('credential'));
    _username = credentials!.getUsername();
    _password = credentials.getPassword();
  }
}


from pip_services3_commons.config import ConfigParams, IConfigurable
from pip_services3_components.auth import CredentialParams


class MyPersistence(IConfigurable):
    ...
    _username: str
    _password: str

    def configure(self, config: ConfigParams):
        ...
        credentials = CredentialParams.from_config(config.get_section("credential"))
        self._username = credentials.get_username()
        self._password = credentials.get_password()
Not available

When credentials have to be retrieved from an external credential store, a store_key parameter shall be applied to specify the key that shall be used to get the desired credential parameters. Pip.Services has the CredentialResolver helper class to make this scenario easier.

The example below demonstrates how to get mongo credentials from an in-memory credential store using the CredentialResolver helper.

# In-memory credential store
descriptor: "pip-services:credential-store:memory:default:1.0"
mongo:
  username: admin
  password: pass123

# MongoDb persistence component
descriptor: "myservice:mypersistance:mongodb:default:1.0"
connection:
  ...
credential:
  discovery_key: mongo

import { 
  ConfigParams, IConfigurable, 
  IReferenceable, IReferences 
} from "pip-services3-commons-nodex";

import { ConnectionResolver } from "pip-services3-components-nodex";


class MyPersistence implements IConfigurable, IReferenceable {
    private _connectionResolver = new ConnectionResolver();
    private _host: string;
    private _port: number;

    public configure(config: ConfigParams) {
        this._connectionResolver.configure(config);
    }

    public setReferences(refs: IReferences) {
        this._connectionResolver.setReferences(refs);

        let connection = this._connectionResolver.resolve(null);
        this._host = connection.getHost();
        this._port = connection.getPortWithDefault(27017);
    }
}


using PipServices3.Commons.Config;
using PipServices3.Commons.Refer;
using PipServices3.Components.Connect;


class MyPersistence : IConfigurable, IReferenceable
{
    private ConnectionResolver _connectionResolver = new ConnectionResolver();
    private string _host;
    private int _port;

    public void Configure(ConfigParams config)
    {
        this._connectionResolver.Configure(config);
    }

    public async void SetReferences(IReferences references)
    {
        this._connectionResolver.SetReferences(references);

        var connection = await _connectionResolver.ResolveAsync(null);
        this._host = connection.Host;
        this._port = connection.GetPortWithDefault(27017);
    }
}



import (
	"context"

	cconf "github.com/pip-services3-gox/pip-services3-commons-gox/config"
	cref "github.com/pip-services3-gox/pip-services3-commons-gox/refer"
	cconn "github.com/pip-services3-gox/pip-services3-components-gox/connect"
)


type MyPersistence struct {
	// ...
	connectionResolver *cconn.ConnectionResolver
	host               string
	port               int
}

func (c *MyPersistence) Configure(ctx context.Context, config *cconf.ConfigParams) {
	// ...
	c.connectionResolver.Configure(ctx, config)
}

func (c *MyPersistence) SetReferences(ctx context.Context, references cref.IReferences) {
	c.connectionResolver.SetReferences(ctx, references)
	connection, err := c.connectionResolver.Resolve("")
	c.host = connection.Host()
	c.port = connection.Port()
}



import 'package:pip_services3_commons/pip_services3_commons.dart';
import 'package:pip_services3_components/pip_services3_components.dart';

void main(List<String> arguments) async {}

class MyPersistence implements IConfigurable, IReferenceable {
  var _connectionResolver = ConnectionResolver();
  String? _host;
  int? _port;

  @override
  void configure(ConfigParams config) {
    _connectionResolver.configure(config);
  }

  @override
  void setReferences(IReferences refs) async {
    _connectionResolver.setReferences(refs);

    var connection = await _connectionResolver.resolve(null);
    _host = connection!.getHost();
    _port = connection.getPortWithDefault(27017);
  }
}



from pip_services3_commons.config import IConfigurable, ConfigParams
from pip_services3_commons.refer import IReferenceable, IReferences
from pip_services3_components.connect import ConnectionResolver


class MyPersistence(IConfigurable, IReferenceable):
    _connectionResolver = ConnectionResolver()
    _host: str
    _port: int

    def configure(self, config: ConfigParams):
        self._connectionResolver.configure(config)

    def set_references(self, refs: IReferences):
        self._connectionResolver.set_references(refs)

        connection = self._connectionResolver.resolve(None)
        self._host = connection.get_host()
        self._port = connection.get_port_with_default(27017)
Not available

Specialized connection resolvers

Similar to standard ConnectionResolver and CredentialResolver, different modules contain specialized resolvers that can provide both connection and credential information, check for completeness and generate technology-specific connection URLs. Some examples are:

  • HttpConnectionResolver defined in RPC module
  • AwsConnectionResolver defined in AWS module
  • AzureFunctionConnectionResolver defined in the Azure module
  • MongoDbConnectionResolver defined in Mongo module
  • PostgresConnectionResolver defined in Postgres module
  • KafkaConnectionResolver defined in Kafka module
  • MqttConnectionResolver defined MQTT module

References

For more information about connectivity see: