Does each microservice really need its own database?

The short answer is yes. However, before you start hyperventilating about the cost of all those extra Oracle licenses, lets first explore why it is essential to do this and then discuss what is meant by the term ‘database’.

The main benefit of the microservice architecture is that it dramatically improves agility and velocity. That’s because when you  correctly decompose a system into microservices, you can develop and deploy each microservice independently and in parallel with the other services. In order to be able to independently develop microservices , they must be loosely coupled. Each microservice’s persistent data must be private to that service and only accessible via it’s API . If two or more microservices were to share persistent data then you need to carefully coordinate changes to the data’s schema, which would slow down development.

There are a few different ways to keep a service’s persistent data private. You do not need to provision a database server for each service. For example,  if you are using a relational database then the options are:

  • Private-tables-per-service – each service owns a set of tables that must only be accessed by that service
  • Schema-per-service – each service has a database schema that’s private to that service
  • Database-server-per-service – each service has it’s own database server.

Private-tables-per-service and schema-per-service have the lowest overhead.  Using a schema per service is appealing since it makes ownership clearer. For some applications, it might make sense for database intensive services to have their own database server.

It is a good idea to create barriers that enforce this modularity. You could, for example, assign a different database user id to each service and use a database access control mechanism such as grants. Without some kind of barrier to enforce encapsulation, developers will always be tempted to bypass a service’s API and access it’s data directly.

It might also make sense to have a polyglot persistence architecture. For each service you choose the type of database that is best suited to that service’s requirements. For example, a service that does text searches could use ElasticSearch. A service that manipulates a social graph could use Neo4j. It might not make sense to use a relational database for every service.

There are some downsides to keeping a service’s persistent data private. Most notably, it can be challenging to implement business transactions that update data owned by multiple services. Rather than using distributed transaction, you typically must use an eventually consistent, event-driven approach to maintain database consistency.

Another problem, is that it is difficult to implement some queries because you can’t do database joins across the data owned by multiple services. Sometimes, you can join the data within a service. In other situations, you will need to use Command Query Responsibility Segregation (CQRS) and maintain denormalizes views.

Another challenge is that  services sometimes need to share data. For example, let’s imagine that several services need access to user profile data. One option is to encapsulate the user profile data with a service, that’s then called by other services. Another option is to use an event-driven mechanism to replicate data to each service that needs it.

In summary,  it is important that each service’s persistent data is private. There are, however,  a few different ways to accomplish this such as a schema-per-service. Some applications benefit from a polyglot persistence architecture that uses a mixture of database types.  A downside of not sharing databases is that maintaining data consistency and implementing queries is more challenging.

This entry was posted in architecture, microservices and tagged , , . Bookmark the permalink.

16 Responses to Does each microservice really need its own database?

  1. seogjoonlee says:

    This topic reminds me what we discussed 2 weeks ago. Thank you for the insightful answer. 🙂

  2. nikolazivkov says:

    Great article. I was wondering if a more precise definition would be to say that each Bounded Context (as per DDD) requires its own database? A Bounded Context on the other hand can have one more microservices, all of them sharing the same domain model i.e. the database schema.

  3. I’m in agreement in terms of more traditional backing stores, however, i’ve been wondering about this in the context of microservices combined with event sourcing/cqrs. Would one have an event store per micro service? It seems like that might become problematic from the perspective of replays and what have you. If service A builds views/projections/etc based upon it’s own events as well as those service B and C, it seems like aggregating them (or failing to) would cause issues with having the proper state represented. While on the other hand a ‘global’ event store would clearly violate some of the principles set forth here.

    • ceracm says:

      I think it is fine to have a global event store. What is important, however, is that each aggregate that is stored in the event store is considered to be private to the service. Other services can subscribe to the events but only the owner can update and load an aggregate’s events.

      • but I think this will offend one basic principle of a microservice having its own database / persistence in case mircoservice- and domain model specific events like “OrderCreatedEvent” are stored in one global event store database (and will lead to a monolithic database schema which actually should be avoided)

  4. ceracm says:

    Not really. The essence of the Database per service pattern is that each service’s data is private to that service. That can be achieved with one database.

  5. Serban Alexandru says:

    Great article.
    I was wondering if I have to choose the Private-tables-per-service method because I have a monolith on a single server that I am trying to break intro microservices , is there a way to reuse the database connection among microservices and still keep them independently of one another and be able at some point if I want for some of them to change the database .
    I ask this because I think is an overhead to make a new database connection for each microservice if they all use the same database.

  6. Anand c says:

    Excellent article

  7. Shefali Agrawal says:

    Great Article

    I am a beginner on implementing micro-service and this article has helped me to move forward. Appreciated

    Question : We are converting our monolithic database and services in to micro-services on premises (NOT CLOUD). I am bit confused on how to achieve scalability if I opt for Private-tables-per-service OR Schema-per-service

  8. What is the best way to implement the “service per datastore” architecture and ensuring data consistency and data queries? Hearing a lot about Saga, event sourcing etc but no concrete information on how to achieve a production grade solution. Any help will be appreciated.

  9. rikkiprince says:

    I feel like there’s a little bit of ambiguity in your conclusion. There’s a risk that it could be implied that schema-per-service is a way of “sharing a database”, but I feel like the intention of your post is to say that the services should never share data through the database, even if the schemas are technically on the same server (for administrative and financial convenience).

    Is that right? The services should still communicate all data through APIs or message queues, even though the schemas might reside on the same physical database server?

    • ceracm says:

      The key idea is that each service’s tables are private to that service and inaccessible to other services. All communication is via the service API (REST, messaging, etc).

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s