Slawomir Stec

Storing JSON as a map attribute in the DynamoDB

PynamoDB defines a MapAttribute class to store raw key-value pairs. It is possible to use it in two ways. The first option is to define each map attribute as an instance of the Attribute class. For example, I've used this approach for the appointment item definition like below:

The second option is to define a Map and fill it with raw data, it can be for example a serialized object in the JSON format passed as python dict. This post contains the code I've written for it.

What I wanted to achieve is a quite common scenario for CRUD operations on simple entities. The javascript client-side defines a model object that should be stored on the backend. Let's assume it is implemented in TypeScript and all field types are defined. Such an object is serialized to the JSON format and send to the backend. The backend has the same definition of the model implemented in python. The field types are well defined, same as on the client-side, the model class is annotated with @dataclass_json and @dataclass annotations. We get working IDE autocomplete and a type check support on both, client and backend side. However, there is another mapping required, from the backend model to a DynamoDB entity model. If I would use the model with fields described with the Attribute class, like in AppointmentItem, I have to maintain a mapping for it. So the backend model is like following:

The MapAttribute raw values can be mapped automatically when being passed a dict instance. Empty values are not allowed and cause an error on entity save action. I've defined a testcase for CRUD operations, a strategy to implement storing raw MapAttributes and a DAO layer to use it. Sample code below.

I've decided to use a trick for converting nested objects to python dict. Firstly, I convert it to JSON, then to python dict. Empty values and None values are deleted from the model. Please note, this is testing code only, however, allowed me to save automatically my client objects without maintaining any mappings.

Additionally, I've defined a strategy for storing entities with MapAttribute column, each row in DynamoDb consist of three columns, PK (prefix + unique client id), SK (prefix + entity id), client_model_json (MapAttribute). This is for modeling a 1-to-many relationship for entities created by the user. This is another common scenario that is nice to have automatically handled on the backend side.