Creating a memory persistence component
Key takeaways
| Memory persistence | Create a memory persistence component and perform CRUD operations. |
Introduction
In this tutorial, you will learn how to create a persistence component, which will store some objects in memory. Then, we will see how to perform CRUD operations, such as adding data, reading it, updating stored values and deleting them. We will use a dummy object, which has the characteristic of being identifiable via an id parameter. All concepts learned here can be expanded to other more complex objects.
Create a memory persistence component
In order to create our memory persistence component, we will follow these two steps.
Step 1 - Creating a dummy class
We will create a dummy class, which represents an object that is identifiable via an id. PIP.Services provides us with the IStringdentifiable interface that can be used to create data objects with this characteristic. We will also define a content parameter for the class, which can include any text.
Once we created our class, we will create three instances of it, each with a different id. For one of the objects, we will use None to let the program define its id. The code will look something like this:
// get all item
page, _ = persistence.GetPageByFilter(context.Background(), cdata.NewFilterParamsFromTuples("id", "1"), cdata.NewPagingParams(0, nil, true))
fmt.Printf("Has %v items \n", page.Total)
from pip_services4_data.data import IStringIdentifiable
class Dummy(IStringIdentifiable):
def __init__(self, id: str = None, key: str = None, content: str = None):
self.id = id
self.key = key
self.content = content
dummy1 = Dummy('1', 'key 1', 'content 1')
dummy2 = Dummy('id 1', 'key 2', 'content 2')
dummy3 = Dummy(None, 'key 3', 'content 3')
Step 2 – Create a memory persistence object
The next step is to create a memory persistence object. Here, we need to use the IdentifiableMemoryPersistence class, which is an abstract persistence component that stores data in memory and implements CRUD operations over data items with unique ids. We will also define two methods namely, getPageByFilter and getOneByKey, which will be used to read the persisted values.
import (
"context"
"fmt"
"strings"
cquery "github.com/pip-services4/pip-services4-go/pip-services4-data-go/query"
cpersist "github.com/pip-services4/pip-services4-go/pip-services4-persistence-go/persistence"
)
type Dummy struct {
Id string `json:"id"`
Key string `json:"key"`
Content string `json:"content"`
}
type MyMemoryPersistence struct {
*cpersist.IdentifiableMemoryPersistence[Dummy, string]
}
func NewMyMemoryPersistence() *MyMemoryPersistence {
return &MyMemoryPersistence{IdentifiableMemoryPersistence: cpersist.NewIdentifiableMemoryPersistence[Dummy, string]()}
}
func composeFilter(filter *cquery.FilterParams) func(item Dummy) bool {
if filter == nil {
filter = cquery.NewFilterParams(make(map[string]string))
}
id, _ := filter.GetAsNullableString("id")
temp_ids, idsOk := filter.GetAsNullableString("ids")
var ids []string
if idsOk {
ids = strings.Split(temp_ids, ",")
}
key, _ := filter.GetAsNullableString("key")
return func(dummy Dummy) bool {
if id != "" && dummy.Id != id {
return false
}
if key != "" && dummy.Key != key {
return false
}
if len(ids) > 0 {
for _, v := range ids {
if dummy.Id == v {
return true
}
}
return false
}
return true
}
}
func (c *MyMemoryPersistence) GetOneByKey(ctx context.Context, key string) (item Dummy, err error) {
for _, val := range c.Items {
if val.Key == key {
item = val
break
}
}
return item, err
}
func (c *MyMemoryPersistence) GetPageByFilter(ctx context.Context, filter *cquery.FilterParams, paging *cquery.PagingParams) (page cquery.DataPage[Dummy], err error) {
if &filter == nil {
filter = cquery.NewEmptyFilterParams()
}
tempPage, err := c.IdentifiableMemoryPersistence.GetPageByFilter(ctx, composeFilter(filter), *paging, nil, nil)
return tempPage, err
}
func NewDummyPage(len *int64, data []Dummy) cquery.DataPage[Dummy] {
return cquery.DataPage[Dummy]{Total: int(*len), Data: data}
}
// ...
persistence := NewMyMemoryPersistence()
from typing import Callable, Optional, Any
from pip_services4_persistence.persistence import IdentifiableMemoryPersistence
from pip_services4_data.query import FilterParams, PagingParams, DataPage
class MyMemoryPersistence(IdentifiableMemoryPersistence):
def __init__(self):
super(MyMemoryPersistence, self).__init__()
def __compose_filter(self, filter_params: FilterParams) -> Callable[[Dummy], bool]:
filter_params = filter_params or FilterParams()
id = filter_params.get_as_nullable_string("id")
temp_ids = filter_params.get_as_nullable_string("ids")
ids = temp_ids.split(",") if temp_ids is not None else None
key = filter_params.get_as_nullable_string("key")
def inner(item: Dummy) -> bool:
if id is not None and item.id != id:
return False
if ids is not None and item.id in ids:
return False
if key is not None and item.key != key:
return False
return True
return inner
def get_page_by_filter(self, correlation_id: Optional[str], filter: Any, paging: PagingParams, sort: Any = None,
select: Any = None) -> DataPage:
return super().get_page_by_filter(correlation_id, self.__compose_filter(filter), paging, sort, select)
def get_one_by_key(self, correlation_id, key):
for item in self._items:
if item.key == key:
self._logger.trace(correlation_id, "Found object by key={}", key)
return item
self._logger.trace(correlation_id, "Cannot find by key={}", key)
persistence = MyMemoryPersistence()
CRUD operations
Now that we have a persistence object, we will perform CRUD operations.
Create the persisted objects
To add values to the persistence object, we will use the create method. This method asks for two parameters: context and the object to persist. For the context we will use None as in our example we are not interested in following a sequence of operations.
item, _ := persistence.Create(context.Background(), dummy1)
fmt.Println("Created Dummy with ID: " + item.Id)
item, _ = persistence.Create(context.Background(), dummy2)
fmt.Println("Created Dummy with ID: " + item.Id)
item, _ = persistence.Create(context.Background(), dummy3)
fmt.Println("Created Dummy with ID: " + item.Id)
result = persistence.create(None, dummy1)
print(f'Created Dummy with ID: {result.id}')
result = persistence.create(None, dummy2)
print(f'Created Dummy with ID: {result.id}')
result = persistence.create(None, dummy3)
print(f'Created Dummy with ID: {result.id}')
After creating the persisted objects, we will obtain the following output:
As we can see, the memory persistence object allocated a value to the id of dummy3, which we had declared as None.
Read the values from the persistence object
To read the persisted values, we can use the getPageByFilter method that we defined when we created the memory persistence object. Here, we will use a filter to indicate that we are only looking for the dummy2 object.
page, _ := persistence.GetPageByFilter(context.Background(), cquery.NewFilterParamsFromTuples("id", "id 1", "key", "key 2"), cquery.NewPagingParams(0, 1, true))
fmt.Printf("Has %v items \n", page.Total)
for _, v := range page.Data {
fmt.Printf("%v, %v, %v \n", v.Id, v.Key, v.Content)
}
# get one item
result = persistence.get_page_by_filter(None,
FilterParams.from_tuples('key', 'key 2'),
PagingParams(0, None,True))
print(f'Has {result.total} items')
for item in result.data:
print(f'{item.id}, {item.key}, {item.content}')
The result object is of type DataPage, which has two fields: data and total. The first is a list containing the items on the retrieved page, and the second is the total number of items in our request. After running this code, we will see the following output with the values of the obtained object.
Similarly, we can obtain all the persisted objects by using None as our filter.
// all items
items := persistence.Items
for _, v := range items {
fmt.Printf("%v, %v, %v \n", v.Id, v.Key, v.Content)
}
# get all items
result = persistence.get_page_by_filter(None,
None,
PagingParams(0, None, True))
print(f'Has {result.total} items')
for item in result.data:
print(f'{item.id}, {item.key}, {item.content}')
After running the above code, we will obtain the following result:
Update a value in the persistence object
To update a value in the persistence object, we need to use the update method. For example, we can change the content of the dummy2 persisted object to “new content 2”.
result, _ := persistence.Update(context.Background(), Dummy{"id 1", "key 2", "new content 2"})
result = persistence.update(None, Dummy('id 1', 'key 2', 'new content 2') )
To verify the change, we can extract the dummy2 object by applying a filter:
// get all items
result, _ = persistence.GetPageByFilter(context.Background(), cdata.NewFilterParamsFromTuples("id", "id 1"), cdata.NewPagingParams(0, 3, false))
fmt.Printf("Has %v items \n", page.Total)
for _, v := range page.Data {
fmt.Printf("%v , %v, %v \n", v.Id, v.Key, v.Content)
}
# get all items
result = persistence.get_page_by_filter(None,
FilterParams.from_tuples('id', 'id 1'),
PagingParams(0, 3))
for item in result.data:
print(f'{item.id}, {item.key}, {item.content}')
And get the updated object:
We can also use the updatePartially function. In this case, we need to specify the id of the object to be updated and a dictionary (map) containing the field to be updated and its new value.
// update patially
updateMap := cdata.NewAnyValueMap(map[string]interface{}{"content": "new new content 2"})
item, _ := persistence.UpdatePartially(context.Background(), "id 1", updateMap)
fmt.Printf("%v , %v, %v \n", item.Id, item.Key, item.Content)
result = persistence.update_partially(None, 'id 1', {'content' : 'new new content 2'})
To verify the change, we can use the filter defined earlier.
// get all items
page, _ = persistence.GetPageByFilter(context.Background(), cdata.NewFilterParamsFromTuples("id", "id 1"), cdata.NewPagingParams(0, nil, true))
fmt.Printf("get all item\n")
fmt.Printf("Has %v items \n", page.Total)
for _, v := range page.Data {
fmt.Printf("%v , %v, %v \n", v.Id, v.Key, v.Content)
}
# get all items
result = persistence.get_page_by_filter(None,
FilterParams.from_tuples('id', 'id 1'),
PagingParams(0, None, True))
for item in result.data:
print(f'{item.id}, {item.key}, {item.content}')
And, we will obtain the updated persisted object.
Delete a value from the persistence object
Similarly, we can delete an object stored in the persistence object by using the deleteById function. In our example, we ask to delete dummy1 by indicating its id.
result, _ = persistence.DeleteById(context.Background(), "1")
result = persistence.delete_by_id(None, "1")
To verify that the object has been deleted, we can apply a filter and search for it.
# get all items
result = persistence.get_page_by_filter(None,
FilterParams.from_tuples('id', '1'),
PagingParams(0, None, True))
print(f'Has {result.total} items')
As expected, the answer will be:
Wrapping up
In this tutorial, we have seen how to create a memory persistence component and apply CRUD operations to it. Although we used a simple dummy object to create an example, the principles explained continue to apply to more complex objects.