Handling arbitrary set of document types in a Symfony application

Sometimes you as a backend developer need to store attributes of different documents in database— personal identity, vehicle identity, house/vehicle/boat ownership, etc. The problem is that the relational model is not fully suitable for that purpose. If you decide to store documents in a single table and started from say Passport in the nearest future you ought to add new columns for the other types like Driving License and get a table full of null values. Ugh! If you decide to employ separate table for each document type sooner or later you’ll face with a lot of JOINs, UNIONs and IFs to get the full collection of documents for a given person.

There is another issue. Document attributes can vary from time to time and from region to region. To be honest it’s a rare issue but living in a big country you definitely come across it at the worst possible time.

First time I faced with similar problem few years ago building SuiteCRM-based project in a bank. That time I created a custom solution using “brand new” JSON Data Type in MySQL 5.7.16. Nowadays it’s extremely simple to create a similar solution using the well-known Composer packages.

First of all let’s define some purposes of developing the solution. We need to store any document type in a single data table. We also need to freely and easy add (or remove) any new document type without affecting the application and/or database. We’d like to avoid a lot of frontend work for every document type registered in our application.

Let’s start from a simple API-like Symfony project:

$ symfony new documents-in-symfony
$ cd documents-in-symfony
$ symfony composer require orm api

I advise you at least for experimenting purposes to add a database to the project with Docker Compose recipe given in “Symfony 5: The Fast Track” book. Don’t forget to set up DATABASE_URL environment variable in your .env file. Launch database service to check it’s working OK:

$ docker compose up -d

Add dunglas/doctrine-json-odm package which will do a lot of work for us:

$ symfony composer require dunglas/doctrine-json-odm

Add MakerBundle to ease entity and migration generation:

$ symfony composer require --dev symfony/maker-bundle

Now we are ready to create an entity which holds a document payload. Pay attention to using json_document type for Document::$payload property. While employing PostgreSQL you may use JSONB data type by adding options={“jsonb”: true} to column definition.

Create a migration for the entity and apply it to the database:

$ symfony console make:migration
$ symfony console doctrine:migrations:migrate

To expose Document entity to the World add class-level annotation @ApiResource , launch Symfony CLI web-server and point your browser to http://localhost:8000/api. So far so good!

Now I suggest you to create a couple of document types to play a little with them and check how they are represented in the table after POST’ing JSON string through exposed entry point. Have a look at this commit to make it a little bit clearer. There you can find a couple of HTTP requests which can be launched with PhpStorm’ embedded HTTP client. While experimenting sending different requests don’t forget to change type hint or @var annotation for Document::$payload property. Can you see the difference of the raw data stored in document.payload column? Try to find a substring like “#type”: “App\\DocumentType\\DrivingLicense” which is the key for further deserialization the whole string to a fully-fledged object.

Seems like we have come up a little closer to our goal. But it’s still too early to celebrate. You might guess we can’t manually change type hinting for payload property all the time. Here’s where Symfony Serializer ability to work with abstract classes and interfaces comes into play. The documentation page says we should create an interface and add the class-level annotation @DiscriminatorMap to it. Add the IDocument interface and the annotation similar to the documentation:

Now you need to implement IDocument interface in both Passport and DrivingLicense classes. Make IDocument::getType() return a string passport for Passport class and driving_license for DrivingLicense . Use IDocument::getName() implementations to return a human-readable document title to display in UI. Modify type hint (or @var annotation) for Document::$payload property to IDocument. Slightly modify HTTP requests by adding “type” key with corresponding value under “payload” (Line 6 in the snippet below):

Run HTTP requests again one after another and look at the data table.Wow! Feel the pleasure of full dynamicity we can employ while storing different documents in the single database column! Seems like we are almost done!

But wait! How our frontend colleague will know which document types it could send, which set of fields should be used and which data type the data in them should be. Here’s the most tricky (but nevertheless easy!) part of the whole scenario.

In our ideal world I think we all would like to simply add or remove document type class (similar to Passport or DrivingLicense) to the source tree and let the magic do the rest. So let’s build the magic! We should solve two problems — provide our frontend colleague with a list of document types available in the system and the field definitions for every document type. We’ll start with field definition. For that purpose we’ll use the doctrine/annotations package which is already installed since doctrine/orm package depends on it. I suggest to create a field-level annotation DocumentField which will store the meta-information about the HTML form field to be rendered in UI:

Now annotate all the properties of Passport and DrivingLicense classes with that annotation. As you can see in a moment annotated properties will be collected into document form field definitions. In the end your Passport class should look something like this:

Here’s the final step remained: we still need two classes — a service that builds a form definition and a simple controller that can expose it to the frontend. We will employ Symfony Service Container autoconfiguration feature to collect all the document types existing in the system. I believe you remember that all our document types implement IDocument interface. So add the following snippets to services.yaml:

Now create DocumentSchemaService

and DocumentSchemaController

Try to navigate to http://localhost:8000/document-schemas and you’ll see JSON with a) human-readable document title to display in UI, b) a discriminator to be sent along with form field values and c) form field definitions to generate HTML form on client-side with either vanilla JS or Angular Reactive Forms.

There is a lot to be improved in the given solution and adapted to your exact needs. But now you have a simple idea how to add any type and number of documents to your project: create a class representing the document type and implementing IDocument interface, annotate its fields with @DocumentField annotation and register it in @DiscriminatorMap on IDocument interface.

You can clone repository with working project described above here. Happy documenting!