MongoDB Persistence

How to persist data using a MongoDB database.

Key takeaways

MongoDbPersistence	Pip.Services component used to create MongoDB persistence objects that accept any type of data.
IdentifiableMongoDbPersistence	Pip.Services component used to create MongoDB persistence objects that accept identifiable data objects.

Introduction

This tutorial will help you understand how to create persistence components for MongoDB. In particular, you will learn how to use two components, namely MongoDbPersistence and IdentifiableMongoDbPersistence. The explanations will include practical examples.

Persisting data with MongoDB

The Pip.Services toolkit provides two different components for MongoDB persistence. They are the MongoDbPersistence and the IdentifiableMongoDbPersistence classes respectively. The first can be used to persist objects of any type. The second is aimed at data items with unique ids. Both classes are part of the MongoDB module, persistence

General pre-requisites

In order to use any of these two components, we need to install the MongoDB module. This can be done with the following command:

npm install pip-services4-mongodb-node --save

Not available

go get -u github.com/pip-services4/pip-services4-go/pip-services4-mongodb-go@latest

Not available

pip install pip-services4-mongodb

Not available

Data object

Throughout the examples, we will use the data structure that appears below. It contains an id field, which can be used to identify each document. The next two fields (key and content) are generic and represent any type of content that we want to persist.

import { IStringIdentifiable } from 'pip-services4-data-node';


export class MyData implements IStringIdentifiable {
    public id: string;
    public key: string;
    public content: string;
}

Not available

type MyData struct {
	Id      string `bson:"_id" json:"id"`
	Key     string `bson:"key" json:"key"`
	Content string `bson:"content" json:"content"`
}

type MyDataPage struct {
	Total *int64   `bson:"total" json:"total"`
	Data  []MyData `bson:"data" json:"data"`
}

Not available

from pip_services4_data.data import IStringIdentifiable

class MyData(IStringIdentifiable): 
    def __init__(self, id: str = None, key: str = None, content: str = None): 
        self.id = id 
        self.key = key 
        self.content = content

Not available

In addition, we create three instances of this class, which we will use in the examples for CRUD operations.

let data1: MyData = { id: "1", key: "key 1", content: "content 1" };
let data2: MyData = { id: "2", key: "key 2", content: "content 2" };
let data3: MyData = { id: "3", key: "key 3", content: "content 3" };

Not available

data1 := MyData{Id: "1", Key: "key 1", Content: "content 1"}
data2 := MyData{Id: "2", Key: "key 2", Content: "content 2"}
data3 := MyData{Id: "3", Key: "key 3", Content: "content 3"}

Not available

data1 = MyData('1', 'key 1', 'content 1') 
data2 = MyData('2', 'key 2', 'content 2') 
data3 = MyData('3', 'key 3', 'content 3')

Not available

MongoDbPersistence

This component can be used with any type of data object. However, all documents stored in MongoDB are identifiable, that is, they have a unique id value. This means that even if we don’t assign a unique identifier to our object, MongoDB will assign one automatically. That is the reason behind having an id field in our data structure.

Pre-requisites

To use the MongoDbPersistence component we need to insert it first. This can be done with the following command:

import { MongoDbPersistence } from 'pip-services4-mongodb-node';

Not available

import (
	mpersist "github.com/pip-services4/pip-services4-go/pip-services4-mongodb-go/persistence"
)

Not available

from pip_services4_mongodb.persistence import MongoDbPersistence

Not available

Component creation

To create our MongoDB persistence component, we create a class that extends the MongoDbPersistence class. We also define an instance of this class and configure it using the configure method available from its parent class. As this method requires an input of type ConfigParams, we import this component and define the host, port, and database. Finally, we open the persistence component. Our code will look something like this:

export class MyMongoDbPersistence extends MongoDbPersistence<MyData> {
    public constructor() {
        super('mydata');
    }
}
        
let persistence = new MyMongoDbPersistence();

let config = ConfigParams.fromTuples(
    "connection.host", "localhost",
    "connection.port", 27017,
    "connection.database", "pipdatabase"
);
persistence.configure(config);

await persistence.open(null);

Not available

type MyMongoDbPersistence struct {
	*mpersist.MongoDbPersistence[MyData]
}

func NewMyMongoDbPersistence() *MyMongoDbPersistence {
	c := &MyMongoDbPersistence{}
	c.MongoDbPersistence = mpersist.InheritMongoDbPersistence(c, "mydata")
	return c
}

persistence := NewMyMongoDbPersistence()
config := conf.NewConfigParamsFromTuples(
	"connection.host", "localhost",
	"connection.port", 27017,
	"connection.database", "pipdatabase",
)
persistence.Configure(context.Background(), config)

err := persistence.Open(context.Background())

Not available

class MyMongoDbPersistence(MongoDbPersistence):
    
    def __init__(self):
        super(MyMongoDbPersistence, self).__init__("mydata")
        
persistence = MyMongoDbPersistence()

from pip_services4_components.config import ConfigParams

config = ConfigParams.from_tuples(
    'connection.host', 'localhost', 
    'connection.port', 27017, 
    'connection.database', 'pipdatabase'
)

persistence.configure(config)

persistence.open("123")

Not available

Later on, once all operations have been completed, we can close our persistence component with the close() method.

await persistence.open(null);

Not available

err := persistence.Open(context.Background())

Not available

persistence.close(None)

Not available

CRUD operations

Our class inherits several methods from its parent class that can be used to perform CRUD operations. This section explores them.

Create

To store a document, we use the create method. This method asks for the correlationId and the data object. In the following example, we create a document based on the previously defined data1 object.

let result = await persistence.create(null, data1);

Not available

item, err := persistence.Create(context.Background(), data1)

Not available

result = persistence.create(None, data1)

Not available

Which returns:

result.id;      // Returns '1'
result.key;     // Returns 'key 1'
result.content; // Returns 'content 1'

Not available

result.Id      // Returns '1'
result.Key     // Returns 'key 1'
result.Content // Returns 'content 1'

Not available

result.id       # Returns '1'
result.key     # Returns 'key 1'
result.content # Returns 'content 1

Not available

Read

The MongoDbPersistence class offers several options to extract documents from a database.

getOneRandom()

As its name suggests, this method retrieves a random document based on a given filter. In the following example, we ask to retrieve a component with a key value of ‘key 3’.

import { FilterParams } from 'pip-services4-data-node';

let item = await persistence.getOneRandom(null, FilterParams.fromTuples("key", "key 3"));

Not available

import (
    cquery "github.com/pip-services4/pip-services4-go/pip-services4-data-go/query"
)

item, err = persistence.GetOneRandom(context.Background(), cquery.NewFilterParamsFromTuples("key", "key 3"))

Not available

from pip_services4_data.query import FilterParams

result = persistence.get_one_random(None, FilterParams.from_tuples('key', 'key 3'))

Not available

Which returns:

result.id;      // Returns '3'
result.key;     // Returns 'key 3'
result.content; // Returns 'content 3'

Not available


result.Id      // Returns '3'
result.Key     // Returns 'key 3'
result.Content // Returns 'content 3'

Not available

result.id      # Returns '3'
result.key     # Returns 'key 3'
result.content # Returns 'content 3'

Not available

getListByFilter()

This method gets a list of data items retrieved according to a given filter. In order to use it, we override this method. This action allows us to introduce any specific aspects that we may need. Our function will look something like this:

public getListByFilter(ctx: Context, filter: FilterParams, sort: SortParams): Promise<MyData[]> {
    return super.getListByFilter(ctx, this.composeFilter(filter), this.composeSort(sort), null);
}

Not available


func (c *MyMongoDbPersistence) GetListByFilter(ctx context.Context, filter *cquery.FilterParams, sort *cquery.SortParams) (items []MyData, err error) {
	return c.MongoDbPersistence.GetListByFilter(ctx, c.composeFilter(filter), c.composeSort(sort), nil)
}

Not available

from typing import Optional, List
from pip_services4_components.context import IContext
from pip_services4_data.query import SortParams

def get_list_by_filter(self, context: Optional[IContext], filter: FilterParams, sort: SortParams) -> List[MyData]:
         return super().get_list_by_filter(trace_id, self._compose_filter(filter), None, self._compose_sort(sort))

Not available

Once we have our class defined, we can call it to get our search results. For example, to get all the elements with a key value of ‘key 3’ we can write:

let list = await persistence.getListByFilter(ctx, FilterParams.fromTuples("key", "key 3"), null);

Not available

list, err := persistence.GetListByFilter(context.Background(), cdata.NewFilterParamsFromTuples("key", "key 3"), nil)

Not available

result = persistence.get_list_by_filter(None, FilterParams.from_tuples('key', 'key 3'), None)

Not available

Which returns:

result[0].id;      // Returns '3'
result[0].key;     // Returns 'key 3'
result[0].content; // Returns 'content 3'

Not available

result[0].Id      // Returns '3'
result[0].Key     // Returns 'key 3'
result[0].Content // Returns 'content 3'

Not available

result[0].id      # Returns '3'
result[0].key     # Returns 'key 3'
result[0].content # Returns 'content 3'

Not available

getPageByFilter()

This method gets a page of data items retrieved according to a given filter. It also allows adding a sorting parameter and a projection object. Similar to what we did in the previous example, we override this method in our persistence class. Besides, we add two methods, namely composeFilter and composeSort. These two methods are used to define aspects that are specific to the database we are using (In our case MongoDB). An example of both methods is:

    private composeFilter(filter: FilterParams): any {
        filter = filter || new FilterParams();
        let criteria = [];

        let key = filter.getAsNullableString('key');
        if (key != null)
            criteria.push({ key: key });

        return criteria.length > 0 ? { $and: criteria } : null;
    }

    private composeSort(sort: SortParams): any {
        sort = sort || new SortParams();
        let sortCondition = {};

        for (let field of sort) {
            sortCondition[field.name] = field.ascending ? 1 : 0;
        }

        return sortCondition;
    }

Not available

func (c *MyMongoDbPersistence) composeFilter(filter *cquery.FilterParams) bson.M {
	if &filter == nil || filter == nil {
		filter = cquery.NewEmptyFilterParams()
	}

	key, _ := filter.GetAsNullableString("key")
	var filterObj bson.M
	if key != "" {
		filterObj = bson.M{"key": key}
	} else {
		filterObj = bson.M{}
	}

	return filterObj
}

func (c *MyMongoDbPersistence) composeSort(sort *cquery.SortParams) bson.M {
	if &sort == nil || sort == nil {
		sort = cquery.NewEmptySortParams()
	}

	sortObj := bson.M{}

	for _, field := range *sort {
		if field.Ascending {
			sortObj[field.Name] = 1
		} else {
			sortObj[field.Name] = -1
		}

	}

	return sortObj
}

Not available

   def _compose_filter(self, filter: FilterParams):
        filter = filter or FilterParams()
        key = filter.get_as_nullable_string('key')

        filter_condition = {}

        if key is not None:
            filter_condition['key'] = key
    
        return filter_condition

    def _compose_sort(self, sort: SortParams):
        sort = sort or SortParams()
        compose_sort = ''

        for i, filed in enumerate(sort):
            compose_sort += filed.name + (' ASC' if filed.ascending else ' DESC')

        return compose_sort

Not available

And, an example of get_page_by_filter() is:

public getPageByFilter(ctx: Context, filter: FilterParams, paging: PagingParams, sort: SortParams): Promise<DataPage<MyData>> {
    return super.getPageByFilter(ctx, this.composeFilter(filter), paging, this.composeSort(sort), null);
}

Not available

func (c *MyMongoDbPersistence) GetPageByFilter(ctx context,Context, correlationId string, filter *cdata.FilterParams, paging *cdata.PagingParams, sort *cdata.SortParams) (page *DataPage[MyData], err error) {

	return c.MongoDbPersistence.GetPageByFilter(ctx, correlationId,
		c.composeFilter(filter), paging,
		c.composeSort(sort), nil)
}

Not available

from pip_services4_data.query import PagingParams, DataPage
def get_page_by_filter(self, context: Optional[IContext], filter: FilterParams, paging: PagingParams,
                           sort: SortParams) -> DataPage:
        return super().get_page_by_filter(context, self._compose_filter(filter), paging, self._compose_sort(sort), None)

Not available

Now, we can call this method from our persistence object. For example, to obtain all the records with a key value of ‘key 3’, we can write:

let page = await persistence.getPageByFilter(null, FilterParams.fromTuples("key", "key 3"), null, null);

Not available

page, err := persistence.GetPageByFilter(context.Background(), cquery.NewFilterParamsFromTuples("key", "key 3"), nil, nil)

Not available

result = persistence.get_page_by_filter(None, FilterParams.from_tuples("key", "key 3"), None, None)

Not available

which returns the searched values in a DataPage object:

result.data[0].id;       // Returns '3'
result.data[0].key;      // Returns 'key 3'
result.data[0].content;  // Returns 'content 3'

Not available

result.Data[0].Id       // Returns '3'
result.Data[0].Key      // Returns 'key 3'
result.Data[0].Content  // Returns 'content 3'

Not available

result.data[0].id       # Returns '3'
result.data[0].key      # Returns 'key 3'
result.data[0].content  # Returns 'content 3'

Not available

getCountByFilter()

This method gets the number of data items that will be retrieved based on a given filter. Because it is a private method in other languages – such as Node.js - we need to override it. Our added method will look similar to

public getCountByFilter(ctx: Context, filter: FilterParams): Promise<number> {
    return super.getCountByFilter(ctx, this.composeFilter(filter));
}

Not available

func (c *MyMongoDbPersistence) GetCountByFilter(ctx context.Context, filter *cquery.FilterParams) (count int64, err error) {
	return c.MongoDbPersistence.GetCountByFilter(ctx, c.composeFilter(filter))
}

Not available

    def get_count_by_filter(self, context: Optional[IContext], filter: FilterParams) -> int:
        return super().get_count_by_filter(context, self._compose_filter(filter))

Not available

Now, we can call it from our code and get the returned amount of records that comply with a given condition, such as key equal to ‘key 3’.

let count = await persistence.getCountByFilter(null, FilterParams.fromTuples("key", "key 3"));

Not available

count, err := persistence.GetCountByFilter(context.Background(), cquery.NewFilterParamsFromTuples("key", "key 3")) // Returns 1

Not available

result = persistence.get_count_by_filter(None, FilterParams.from_tuples('key', 'key 3')) # Returns 1

Not available

Update

As MongoDbPersistence doesn’t have an update method, we need to define it in our class. We will see how to do this in the Example section.

Delete

The MongoDbPersistence class provides the deleteByFilter() method, which deletes all those documents that comply with a given condition. The following example shows how to delete all the elements with a key value equal to ‘key 3’:

await persistence.deleteByFilter(null, FilterParams.fromTuples("key", "key 3"));

Not available

err = persistence.DeleteByFilter(context.Background(), cquery.NewFilterParamsFromTuples("key", "key 3"))

Not available

persistence.delete_by_filter(None, FilterParams.from_tuples('key', 'key 3'))

Not available

Component’s final version

After overriding and adding the methods specified in the previous examples, our MongoDb persistence component looks like this:

export class MyMongoDbPersistence extends MongoDbPersistence<MyData> {
    public constructor() {
        super('mydata');
    }

    private composeFilter(filter: FilterParams): any {
        filter = filter || new FilterParams();
        let criteria = [];

        let key = filter.getAsNullableString('key');
        if (key != null)
            criteria.push({ key: key });

        return criteria.length > 0 ? { $and: criteria } : null;
    }

    private composeSort(sort: SortParams): any {
        sort = sort || new SortParams();
        let sortCondition = {};

        for (let field of sort) {
            sortCondition[field.name] = field.ascending ? 1 : 0;
        }

        return sortCondition;
    }

    public getOneRandom(ctx: Context, filter: FilterParams): Promise<MyData> {
        return super.getOneRandom(ctx, this.composeFilter(filter));
    }

    public getListByFilter(ctx: Context, filter: FilterParams, sort: SortParams): Promise<MyData[]> {
        return super.getListByFilter(ctx, this.composeFilter(filter), this.composeSort(sort), null);
    }

    public getPageByFilter(ctx: Context, filter: FilterParams, paging: PagingParams, sort: SortParams): Promise<DataPage<MyData>> {
        return super.getPageByFilter(ctx, this.composeFilter(filter), paging, this.composeSort(sort), null);
    }

    public getCountByFilter(ctx: Context, filter: FilterParams): Promise<number> {
        return super.getCountByFilter(ctx, this.composeFilter(filter));
    }

    public deleteByFilter(ctx: Context, filter: FilterParams): Promise<void> {
        return super.deleteByFilter(ctx, this.composeFilter(filter));
    }
}

Not available

import (
	"context"

	cquery "github.com/pip-services4/pip-services4-go/pip-services4-data-go/query"
	"go.mongodb.org/mongo-driver/bson"

	mpersist "github.com/pip-services4/pip-services4-go/pip-services4-mongodb-go/persistence"
)

type MyMongoDbPersistence struct {
	*mpersist.MongoDbPersistence[MyData]
}

func NewMyMongoDbPersistence() *MyMongoDbPersistence {
	c := &MyMongoDbPersistence{}
	c.MongoDbPersistence = mpersist.InheritMongoDbPersistence(c, "mydata")
	return c
}

func (c *MyMongoDbPersistence) composeFilter(filter *cquery.FilterParams) bson.M {
	if &filter == nil || filter == nil {
		filter = cquery.NewEmptyFilterParams()
	}

	key, _ := filter.GetAsNullableString("key")
	var filterObj bson.M
	if key != "" {
		filterObj = bson.M{"key": key}
	} else {
		filterObj = bson.M{}
	}

	return filterObj
}

func (c *MyMongoDbPersistence) composeSort(sort *cquery.SortParams) bson.M {
	if &sort == nil || sort == nil {
		sort = cquery.NewEmptySortParams()
	}

	sortObj := bson.M{}

	for _, field := range *sort {
		if field.Ascending {
			sortObj[field.Name] = 1
		} else {
			sortObj[field.Name] = -1
		}

	}

	return sortObj
}

func (c *MyMongoDbPersistence) GetListByFilter(ctx context.Context, filter *cquery.FilterParams, sort *cquery.SortParams) (items []MyData, err error) {
	return c.MongoDbPersistence.GetListByFilter(ctx, c.composeFilter(filter), c.composeSort(sort), nil)
}

func (c *MyMongoDbPersistence) GetPageByFilter(ctx context.Context, filter *cquery.FilterParams, paging *cquery.PagingParams, sort *cquery.SortParams) (page cquery.DataPage[MyData], err error) {

	return c.MongoDbPersistence.GetPageByFilter(ctx,
		c.composeFilter(filter), *paging,
		c.composeSort(sort), nil)
}

func (c *MyMongoDbPersistence) GetCountByFilter(ctx context.Context, filter *cquery.FilterParams) (count int64, err error) {
	return c.MongoDbPersistence.GetCountByFilter(ctx, c.composeFilter(filter))
}

func (c *MyMongoDbPersistence) DeleteByFilter(ctx context.Context, filter *cquery.FilterParams) error {
	return c.MongoDbPersistence.DeleteByFilter(ctx, c.composeFilter(filter))
}

Not available

class MyMongoDbPersistence(MongoDbPersistence):
    
    def __init__(self):
        super(MyMongoDbPersistence, self).__init__("mydata2")
   
    def _compose_filter(self, filter: FilterParams):
        filter = filter or FilterParams()
        key = filter.get_as_nullable_string('key')

        filter_condition = {}

        if key is not None:
            filter_condition['key'] = key
    
        return filter_condition

    def _compose_sort(self, sort: SortParams):
        sort = sort or SortParams()
        compose_sort = ''

        for i, filed in enumerate(sort):
            compose_sort += filed.name + (' ASC' if filed.ascending else ' DESC')

        return compose_sort


    def get_one_random(self, context: Optional[str], filter: FilterParams) -> MyData:
        return super().get_one_random(context, self._compose_filter(filter))

    def get_list_by_filter(self, context: Optional[IContext], filter: FilterParams, sort: SortParams) -> List[MyData]:
         return super().get_list_by_filter(context, self._compose_filter(filter), None, self._compose_sort(sort))
         
    def get_page_by_filter(self, context: Optional[IContext], filter: FilterParams, paging: PagingParams,
                           sort: SortParams) -> DataPage:
        return super().get_page_by_filter(context, self._compose_filter(filter), paging, self._compose_sort(sort), None)

    def get_count_by_filter(self, context: Optional[IContext], filter: FilterParams) -> int:
        return super().get_count_by_filter(context, self._compose_filter(filter))

    def delete_by_filter(self, context: Optional[IContext], filter: FilterParams):
        super().delete_by_filter(context, self._compose_filter(filter))

Not available

Example

Now, we will see a simple example that puts most of the learned concepts together. It starts by importing the necessary libraries and creating a MongoDB persistence class that includes an update method. Then, it performs CRUD operations and prints the results. The code is:

import { FilterParams, IStringIdentifiable, SortParams } from "pip-services4-data-node";
import { ConfigParams, Context } from "pip-services4-components-node";
import { MongoDbPersistence } from 'pip-services4-mongodb-node';

export class MyData implements IStringIdentifiable {
    public id: string;
    public key: string;
    public content: string;
}

export class MyMongoDbPersistence extends MongoDbPersistence<MyData> {
    public constructor() {
        super('mydata');
    }

    private composeFilter(filter: FilterParams): any {
        filter = filter || new FilterParams();
        let criteria = [];

        let key = filter.getAsNullableString('key');
        if (key != null)
            criteria.push({ key: key });

        return criteria.length > 0 ? { $and: criteria } : null;
    }

    public getListByFilter(ctx: Context, filter: FilterParams, sort: SortParams): Promise<MyData[]> {
        return super.getListByFilter(ctx, this.composeFilter(filter), null, null);
    }

    public async update(ctx: Context, item: MyData): Promise<MyData> {
        if (item == null || item.id == null) {
            return null;
        }

        let newItem = Object.assign({}, item);
        delete newItem.id;
        newItem = this.convertFromPublic(newItem);

        let filter = { _id: item.id };
        let update = { $set: newItem };
        let options = {
            returnDocument: "after"
        };

        let result = await new Promise<any>((resolve, reject) => {
            this._collection.findOneAndUpdate(filter, update, options, (err, result) => {
                if (err == null) resolve(result);
                else reject(err);
            });
        });

        newItem = result ? this.convertToPublic(result.value) : null;
        return newItem;
    }

    public deleteByFilter(ctx: Context, filter: FilterParams): Promise<void> {
        return super.deleteByFilter(ctx, this.composeFilter(filter));
    }
}

export function printResult(operationName: string, res: MyData) {
    console.log(`==================== ${operationName} ====================`);
    console.log(`MyData with id: ${res.id}`);
    console.log(`MyData key: ${res.key}`);
    console.log(`MyData content: ${res.content}`);
}

let data1: MyData = { id: "1", key: "key 1", content: "content 1" };

let persistence = new MyMongoDbPersistence();

let config = ConfigParams.fromTuples(
    "connection.host", "localhost",
    "connection.port", 27017,
    "connection.database", "pipdatabase"
);
persistence.configure(config);

await persistence.open(null);
await persistence.clear(null);

// 1 - Create
let result = await persistence.create(null, data1);
printResult("Create", result);

// 2 - Retrieve
let items = await persistence.getListByFilter(null, FilterParams.fromTuples("key", "key 1"), null);
printResult("Get by id", items[0]);

// 3 - Update
items[0].content = "new content 2";
items[0].key = "key 2";

let update = await persistence.update(null, items[0]);
printResult("Update", update);

// 4 - Delete
await persistence.deleteByFilter(null, FilterParams.fromTuples("key", "key 1"));
await persistence.close(null);

Not available

import (
	"context"
	"fmt"

	conf "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
	cquery "github.com/pip-services4/pip-services4-go/pip-services4-data-go/query"
	"go.mongodb.org/mongo-driver/bson"
	"go.mongodb.org/mongo-driver/mongo"
	mngoptions "go.mongodb.org/mongo-driver/mongo/options"

	mpersist "github.com/pip-services4/pip-services4-go/pip-services4-mongodb-go/persistence"
)

type MyMongoDbPersistence struct {
	*mpersist.MongoDbPersistence[MyData]
}

func NewMyMongoDbPersistence() *MyMongoDbPersistence {
	c := &MyMongoDbPersistence{}
	c.MongoDbPersistence = mpersist.InheritMongoDbPersistence(c, "mydata")
	return c
}

func (c *MyMongoDbPersistence) composeFilter(filter *cquery.FilterParams) bson.M {
	if &filter == nil || filter == nil {
		filter = cquery.NewEmptyFilterParams()
	}

	key, _ := filter.GetAsNullableString("key")
	var filterObj bson.M
	if key != "" {
		filterObj = bson.M{"key": key}
	} else {
		filterObj = bson.M{}
	}

	return filterObj
}

func (c *MyMongoDbPersistence) composeSort(sort *cquery.SortParams) bson.M {
	if &sort == nil || sort == nil {
		sort = cquery.NewEmptySortParams()
	}

	sortObj := bson.M{}

	for _, field := range *sort {
		if field.Ascending {
			sortObj[field.Name] = 1
		} else {
			sortObj[field.Name] = -1
		}

	}

	return sortObj
}

func (c *MyMongoDbPersistence) Update(ctx context.Context, correlationId string, item MyData) (result MyData, err error) {
	newItem, err := c.ConvertFromPublic(item)
	id := newItem["_id"]
	filter := bson.M{"_id": id}
	update := bson.D{{"$set", newItem}}
	var options mngoptions.FindOneAndUpdateOptions
	retDoc := mngoptions.After
	options.ReturnDocument = &retDoc
	fuRes := c.Collection.FindOneAndUpdate(ctx, filter, update, &options)
	if fuRes.Err() != nil {
		return result, fuRes.Err()
	}
	c.Logger.Trace(context.Background(), "Updated in %s with id = %s", c.CollectionName, id)
	var docPointer *MyData
	err = fuRes.Decode(&docPointer)
	if err != nil {
		if err == mongo.ErrNoDocuments {
			return result, nil
		}
		return result, err
	}

	result, err = c.Overrides.ConvertToPublic(docPointer)
	if err != nil {
		return MyData{}, nil
	}
	return result, nil
}

func (c *MyMongoDbPersistence) GetListByFilter(ctx context.Context, filter *cquery.FilterParams, sort *cquery.SortParams) (items []MyData, err error) {
	return c.MongoDbPersistence.GetListByFilter(ctx, c.composeFilter(filter), c.composeSort(sort), nil)
}

func (c *MyMongoDbPersistence) DeleteByFilter(ctx context.Context, filter *cquery.FilterParams) error {
	return c.MongoDbPersistence.DeleteByFilter(ctx, c.composeFilter(filter))
}

type MyData struct {
	Id      string `bson:"_id" json:"id"`
	Key     string `bson:"key" json:"key"`
	Content string `bson:"content" json:"content"`
}

type MyDataPage struct {
	Total *int64   `bson:"total" json:"total"`
	Data  []MyData `bson:"data" json:"data"`
}

func PrintResult(operationName string, res MyData) {
	fmt.Println("==================== " + operationName + " ====================")
	fmt.Println("MyData with Id: " + res.Id)
	fmt.Println("MyData with Key: " + res.Key)
	fmt.Println("MyData with Content: " + res.Content)
}

func main() {
	data1 := MyData{Id: "1", Key: "key 1", Content: "content 1"}

	persistence := NewMyMongoDbPersistence()
	config := conf.NewConfigParamsFromTuples(
		"connection.host", "localhost",
		"connection.port", 27017,
		"connection.database", "pipdatabase",
	)
	persistence.Configure(context.Background(), config)

	_ = persistence.Open(context.Background())
	_ = persistence.Clear(context.Background())

	// 1 - Create
	result, _ := persistence.Create(context.Background(), data1)
	PrintResult("Create", result)

	// 2 - Retrieve
	items, _ := persistence.GetListByFilter(context.Background(), cquery.NewFilterParamsFromTuples("key", "key 1"), nil)
	PrintResult("Get by id", items[0])

	// 3 - Update
	items[0].Content = "new content 2"
	items[0].Key = "key 2"

	update, _ := persistence.Update(context.Background(), "123", items[0])
	PrintResult("Update", update)

	// 4 - Delete
	_ = persistence.DeleteByFilter(context.Background(), cquery.NewFilterParamsFromTuples("key", "key 1"))

	_ = persistence.Close(context.Background())
}

Not available

from copy import deepcopy
from typing import Optional, Any

import pymongo
from pip_services4_components.config import ConfigParams
from pip_services4_data.data import IStringIdentifiable 
from pip_services4_data.query import FilterParams
from pip_services4_mongodb.persistence import MongoDbPersistence


class MyData(IStringIdentifiable):
    def __init__(self, id: str = None, key: str = None, content: str = None):
        self.id = id
        self.key = key
        self.content = content

class MyMongoDbPersistence(MongoDbPersistence):

    def __init__(self):
        super(MyMongoDbPersistence, self).__init__("mydata")

    def _compose_filter(self, filter: FilterParams):
        filter = filter or FilterParams()
        key = filter.get_as_nullable_string('key')

        filter_condition = {}

        if key is not None:
            filter_condition['key'] = key
    
        return filter_condition
    
    def get_list_by_filter(self, context: Optional[IContext], filter: FilterParams, sort: SortParams) -> List[MyData]:

         return super().get_list_by_filter(context, self._compose_filter(filter), None, None)
        
    def update(self, context: Optional[IContext], item: Any) -> Any:
        if item is None or item.id is None:
            return
        new_item = deepcopy(item)
        new_item = self._convert_from_public(new_item)
        _id = item.id

        result = self._collection.find_one_and_update(
            {'_id': _id}, {'$set': new_item},
            return_document=pymongo.ReturnDocument.AFTER
        )

        new_item = self._convert_to_public(result)

        return new_item

    def delete_by_filter(self, context: Optional[IContext], filter: FilterParams):
        super().delete_by_filter(context, self._compose_filter(filter))


data1 = MyData(None, 'key 1', 'content 1')

persistence = MyMongoDbPersistence()

config = ConfigParams.from_tuples(
    'connection.host', 'localhost', 
    'connection.port', 27017, 
    'connection.database', 'mydb'
)

persistence.configure(config)

persistence.open("123")
persistence.clear("123")


def print_result(operation_name: str, res: MyData):
    print(f"==================== {operation_name} ====================")
    print(f'MyData with Id: {res.id}')
    print(f'MyData Key: {res.key}')
    print(f'MyData Content: {res.content}')


# CRUD
# 1 - Create
result = persistence.create(None, data1)
print_result('Create', result)

# 2 - Retrieve
items = persistence.get_list_by_filter('123', FilterParams.from_tuples('key', 'key 1'), None)  
print_result('Get by id', items[0])

# 3 - Update
items[0].content = 'new content 2'
items[0].key = 'key 2'

update = persistence.update(None, items[0])  
print_result('Update', update)

# 4 - Delete
persistence.delete_by_filter(None, FilterParams.from_tuples('key', 'key 1')) 

persistence.close("123")

Not available

And the output is:

IdentifiableMongoDbPersistence

This component is used to perform CRUD operations with identifiable data objects, that is, objects that can be identified via a unique id.

Pre-requisites

To use the IdentifiableMongoDbPersistence component we need to import it first. This can be done with the following command:

import { IdentifiableMongoDbPersistence } from 'pip-services4-mongodb-node';

Not available

import (
	mpersist "github.com/pip-services4/pip-services4-go/pip-services4-mongodb-go/persistence"
)

Not available

from pip_services4_mongodb.persistence import IdentifiableMongoDbPersistence

Not available

Component creation

To create an identifiable MongoDB persistence component, we create a subclass of the IdentifiableMongoDbPersistence class where we specify the name of the table we will be using (In our example: mydata). We also define an instance of it and, via the configure() method, we add the connection parameters. In our example, we use a local database and we connect to it through the default port 27017. We also define a database named “pipdatabase”.

import { ConfigParams } from 'pip-services4-components-node';

export class MyIdentifiableMongoDbPersistence extends IdentifiableMongoDbPersistence<MyData, string> {
    public constructor() {
        super("mydata");
    }
}
let persistence = new MyIdentifiableMongoDbPersistence();

let config = ConfigParams.fromTuples(
    "connection.host", "localhost",
    "connection.port", 27017,
    "connection.database", "pipdatabase"
);

persistence.configure(config);

Not available

import (
	"context"

	conf "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"

	mpersist "github.com/pip-services4/pip-services4-go/pip-services4-mongodb-go/persistence"
)

type MyIdentifiableMongoDbPersistence struct {
	mpersist.IdentifiableMongoDbPersistence[MyData, string]
}

func NewMyIdentifiableMongoDbPersistencee() *MyIdentifiableMongoDbPersistence {
	c := &MyIdentifiableMongoDbPersistence{}
	c.IdentifiableMongoDbPersistence = *mpersist.InheritIdentifiableMongoDbPersistence[MyData, string](c, "mydata")
	return c
}


persistence := NewMyIdentifiableMongoDbPersistencee()
config := conf.NewConfigParamsFromTuples(
	"connection.host", "localhost",
	"connection.port", 27017,
	"connection.database", "pipdatabase",
)
persistence.Configure(context.Background(), config)

Not available

class MyIdentifiableMongoDbPersistence(IdentifiableMongoDbPersistence):
    
    def __init__(self):
        super(MyIdentifiableMongoDbPersistence, self).__init__("mydata")
        
persistence = MyIdentifiableMongoDbPersistence()

from pip_services4_components.config import ConfigParams

config = ConfigParams.from_tuples(
    'connection.host', 'localhost', 
    'connection.port', 27017, 
    'connection.database', 'pipdatabase'
)

persistence.configure(config)

Not available

And, after creating it, we open the connection.

await persistence.open(null);

Not available

_ = persistence.Open(context.Background())

Not available

persistence.open("123")

Not available

Later on, once we have finished using this persistence component, we can close it with the close() method.

await persistence.clear(null);

Not available

_ = persistence.Clear(context.Background())

Not available

persistence.close(None)

Not available

CRUD operations

This class presents a set of methods for CRUD operations. This section explains their usage and provides examples for each of them.

Create

This component presents two methods that allow us to create a document in MongoDB. They are:

Read

create()

To add a new document to our collection, we can use the create() method, which accepts the correlationId and the data item as inputs. The example below shows how to use it.

let result = await persistence.create(null, data1);

Not available

result, _ := persistence.Create(context.Background(), data1)

Not available

result = persistence.create(None, data1)

Not available

Which returns:

result.id;      // Returns '1'
result.key;     // Returns 'key 1'
result.content; // Returns 'content 1'

Not available

aresult.Id      // Returns '1'
result.Key     // Returns 'key 1'
result.Content // Returns 'content 1'

Not available

result.id      # Returns '1'
result.key     # Returns 'key 1'
result.content # Returns 'content 1'

Not available

set()

This method updates an existing data item. If the item doesn’t exist, it creates it. The example below shows how to use it.

data1 = { id: "1", key: "key 1", content: "new content 1" };
result = await persistence.set(null, data1);

Not available

data1 = MyData{Id: "1", Key: "key 1", Content: "new content 1"}
result, _ = persistence.Set(context.Background(), data1)

Not available

data1 = MyData('1', 'key 1', 'new content 1') 
result = persistence.set(None, data1)

Not available

Which returns:

result.id;      // Returns '1'
result.key;     // Returns 'key 1'
result.content; // Returns 'new content 1'

Not available


result.Id      // Returns '1'
result.Key     // Returns 'key 1'
result.Content // Returns 'new content 1'

Not available

result.id      # Returns '1'
result.key     # Returns 'key 1'
result.content # Returns 'new content 1'

Not available

get_one_by_id()

To retrieve a data object, we can use the get_one_by_id method, which allows for the selection of a data object based on its id. In the following example, we obtain the item with key = ‘1’.

result = await persistence.getOneById(null, "1");

Not available

result, _ = persistence.GetOneById(context.Background(), "1")

Not available

result = persistence.get_one_by_id(None, '1')

Not available

Which returns:

result.id;      // Returns '1'
result.key;     // Returns 'key 1'
result.content; // Returns 'content 1'

Not available


result.Id      // Returns '1'
result.Key     // Returns 'key 1'
result.Content // Returns 'content 1'

Not available

result.id      # Returns '1'
result.key     # Returns 'key 1'
result.content # Returns 'content 1'

Not available

getListByIds()

We can also use the getListByIds() method, which is similar to the previous one, but accepts a list containing ids and retrieves the documents related to those ids. In the following example, we search for those items with id equal to ‘1’ and ‘2’.

let resList = await persistence.getListByIds(null, ["1", "2"]);

Not available

list, _ := persistence.GetListByIds(context.Background(), []string{"1", "2"})

Not available

result = persistence.get_list_by_ids(None, ['1','2'])

Not available

Which returns:

result[0].id;      // Returns '1'
result[0].key;     // Returns 'key 1'
result[0].content; // Returns 'content 1'
result[1].id;      // Returns '2'
result[1].key;     // Returns 'key 2'
result[1].content; // Returns 'content 2'

Not available

list[0].Id      // Returns '1'
list[0].Key     // Returns 'key 1'
list[0].Content // Returns 'content 1'
list[1].Id      // Returns '2'
list[1].Key     // Returns 'key 2'
list[1].Content // Returns 'content 2'

Not available

result[0].id      # Returns '1'
result[0].key     # Returns 'key 1'
result[0].content # Returns 'content 1'
result[1].id      # Returns '2'
result[1].key     # Returns 'key 2'
result[1].content # Returns 'content 2'

Not available

Update

update()

This method updates the data stored in a record. It accepts the correlationId and the id of the record to be updated as input parameters. In the example below, we change the value of content to ‘new content 2’ for a record with id equal to ‘2’.

let updated = await persistence.update(null, { id: "2", key: "key 2", content: "new content 2" });

Not available

updated, _ := persistence.Update(context.Background(), MyData{Id: "2", Key: "key 2", Content: "new content 2"})

Not available

result = persistence.update(None, MyData('2', 'key 2', 'new content 2') )

Not available

Which returns:

result.id;      // Returns '2'
result.key;     // Returns 'key 2'
result.content; // Returns 'new content 2'

Not available


result.Id      // Returns '2'
result.Key     // Returns 'key 2'
result.Content // Returns 'new content 2'

Not available

result.id      # Returns '2'
result.key     # Returns 'key 2'
result.content # Returns 'new content 2'

Not available

updatePartially()

This method also updates an item, but only the specified fields. It takes the id of the item to be updated and an AnyValueMap object containing the fields to be modified and their updated values as input parameters. The following example shows how to update the content field for a record with id equal to ‘3’.

import { AnyValueMap } from 'pip-services4-commons-node';


updated = await persistence.updatePartially(null, "2", AnyValueMap.fromTuples("content", "new content 2 - partially updated"));

Not available

import cdata "github.com/pip-services4/pip-services4-go/pip-services4-commons-go/data"

updated, _ = persistence.UpdatePartially(context.Background(), "2", *cdata.NewAnyValueMapFromTuples(
	"content", "new content 2 - partially updated",
))

Not available

from pip_services4_commons.data import AnyValueMap

result = persistence.update_partially(None, '2', AnyValueMap({'content': 'new content 2 - partially updated'}))

Not available

Which returns:

result.id;      // Returns '2'
result.key;     // Returns 'key 2'
result.content; // Returns 'new content 2 - partially updated'

Not available

result.Id      // Returns '2'
result.Key     // Returns 'key 2'
result.Content // Returns 'new content 2 - partially updated'

Not available

result.id      # Returns '2'
result.key     # Returns 'key 2'
result.content # Returns 'new content 2 - partially updated'

Not available

Delete

deleteById()

We can delete a stored data object by using the delete() method. Here, we need to indicate the correlationId and the id of the object to be deleted. The following example deletes a record with an id equal to ‘1’.

let deleted = await persistence.deleteById(null, "1");

Not available

_ = persistence.DeleteByIds(context.Background(), []string{"1", "2"})

Not available

result = persistence.delete_by_id(None, '1')

Not available

Which returns:

result.id;       // Returns '1'
result.key;      // Returns 'key 1'
result.content;  // Returns 'content 1'

Not available

result.Id       // Returns '1'
result.Key      // Returns 'key 1'
result.Content  // Returns 'content 1'

Not available

result.id       # Returns '1'
result.key      # Returns 'key 1'
result.content  # Returns 'content 1'

Not available

deleteByIds

This method accepts a list containing the ids of the documents to be deleted. The following example shows how to delete the records with ids equal to ‘1’ and ‘2’.

await persistence.deleteByIds(null, ["1", "2"]);

Not available

_ = persistence.DeleteByIds(context.Background(), []string{"1", "2"})

Not available

persistence.delete_by_ids(None, ['1', '2'])

Not available

Example

To summarize, we put everything together in one comprehensive example. In it, we first create a data class with a field named id. Then, we create our persistence object, configure it and open the connection. Once we are connected to the database “mydb”, we perform the four CRUD operations and print the results. The code is:

import { IStringIdentifiable } from "pip-services4-data-node";
import { ConfigParams, Context } from "pip-services4-components-node";
import { IdentifiableMongoDbPersistence } from 'pip-services4-mongodb-node';

export class MyIdentifiableMongoDbPersistence extends IdentifiableMongoDbPersistence<MyData, string> {
    public constructor() {
        super("mydata");
    }
}

export class MyData implements IStringIdentifiable {
    public id: string;
    public key: string;
    public content: string;
}

export function printResult(operationName: string, res: MyData) {
    console.log(`==================== ${operationName} ====================`);
    console.log(`MyData with id: ${res.id}`);
    console.log(`MyData key: ${res.key}`);
    console.log(`MyData content: ${res.content}`);
}


let data1: MyData = { id: "1", key: "key 1", content: "content 1" };

let persistence = new MyIdentifiableMongoDbPersistence();
let config = ConfigParams.fromTuples(
    "connection.host", "localhost",
    "connection.port", 27017,
    "connection.database", "pipdatabase"
);
persistence.configure(config);

await persistence.open(null);
await persistence.clear(null);

// CRUD
// 1 - Create
let result = await persistence.create(null, data1);
printResult("Create", result);

// 2 - Retrieve
let item = await persistence.getOneById(ctx, "1");
printResult("Get by id", item);

// 3 - Update
let update = await persistence.update(null, { id: "2", key: "key 2", content: "new content 2" });
printResult("Update", update);

// 4 - Delete
let deleted = await persistence.deleteById(null, "1");
printResult("Delete by id", deleted);

await persistence.close(null);

Not available

import (
	"context"
	"fmt"

	conf "github.com/pip-services4/pip-services4-go/pip-services4-components-go/config"
	mpersist "github.com/pip-services4/pip-services4-go/pip-services4-mongodb-go/persistence"
)

type MyData struct {
	Id      string `bson:"_id" json:"id"`
	Key     string `bson:"key" json:"key"`
	Content string `bson:"content" json:"content"`
}

type MyIdentifiableMongoDbPersistence struct {
	mpersist.IdentifiableMongoDbPersistence[MyData, string]
}

func NewMyIdentifiableMongoDbPersistencee() *MyIdentifiableMongoDbPersistence {
	c := &MyIdentifiableMongoDbPersistence{}
	c.IdentifiableMongoDbPersistence = *mpersist.InheritIdentifiableMongoDbPersistence[MyData, string](c, "mydata")
	return c
}

func PrintResult(operationName string, res MyData) {
	fmt.Println("==================== " + operationName + " ====================")
	fmt.Println("MyData with Id: " + res.Id)
	fmt.Println("MyData with Key: " + res.Key)
	fmt.Println("MyData with Content: " + res.Content)
}

func main() {
	data1 := MyData{Id: "1", Key: "key 1", Content: "content 1"}

	persistence := NewMyIdentifiableMongoDbPersistencee()
	config := conf.NewConfigParamsFromTuples(
		"connection.host", "localhost",
		"connection.port", 27017,
		"connection.database", "pipdatabase",
	)
	persistence.Configure(context.Background(), config)

	_ = persistence.Open(context.Background())
	_ = persistence.Clear(context.Background())

	// CRUD
	// 1 - Create
	result, _ := persistence.Create(context.Background(), data1)
	PrintResult("Create", result)

	// 2 - Retrieve
	item, _ := persistence.GetOneById(context.Background(), "1")
	PrintResult("Get by id", item)

	// 3 - Update
	update, _ := persistence.Update(context.Background(), MyData{Id: "1", Key: "key 2", Content: "new content 2"})
	PrintResult("Update", update)

	// 4 - Delete
	delete, _ := persistence.DeleteById(context.Background(), "1")
	PrintResult("Delete by id", delete)

	_ = persistence.Close(context.Background())
}

Not available

from pip_services4_mongodb.persistence import IdentifiableMongoDbPersistence
from pip_services4_components.config import ConfigParams
import pymongo

class MyIdentifiableMongoDbPersistence(IdentifiableMongoDbPersistence):
    
    def __init__(self):
        super(MyIdentifiableMongoDbPersistence, self).__init__("mydata")
		
persistence = MyIdentifiableMongoDbPersistence()

from pip_services3_commons.data import IStringIdentifiable

class MyData(IStringIdentifiable): 
    def __init__(self, id: str = None, key: str = None, content: str = None): 
        self.id = id 
        self.key = key 
        self.content = content 

data1 = MyData('1', 'key 1', 'content 1') 

config = ConfigParams.from_tuples(
    'connection.host', 'localhost', 
    'connection.port', 27017, 
    'connection.database', 'mydb'
)
persistence.configure(config)

persistence.open("123")
persistence.clear("123")

def print_result(operation_name: str, res: MyData):
    print(f"==================== {operation_name} ====================")
    print(f'MyData with ID: {res.id}')
    print(f'MyData Key: {res.key}')
    print(f'MyData Content: {res.content}')

# CRUD
# 1 - Create
result = persistence.create(None, data1)
print_result('Create', result)

# 2 - Retrieve
item = persistence.get_one_by_id('123','1')
print_result('Get by id', item)

# 3 - Update
update = persistence.update(None, MyData('1', 'key 2', 'new content 2') )
print_result('Update', update)

# 4 - Delete
delete = persistence.delete_by_id(None, "1") 
print_result('Delete by id', delete)

persistence.close(None)

Not available

And the output is:

Wrapping up

In this tutorial, we have explored how to create MongoDB persistence components. We saw two different components, namely the MongoDbPersistence and the IdentifiableMongoDbPersistence classes, and how to perform CRUD operations with them. Finally, we saw a comprehensive example for each component.