This is going to be the first in a series of blog post, detailing my experience and thought around the development of a new open source microservice architecture. It is the result of several years of work across a number of public and private sector organisations in many different industries. In that time I have developed some strong opinions around microservice architectures, and I hope to capture some of that both in this series of blog post, and as a new open source protect; The Radical Microservice Architecture (RADMA)
Starting at the begining
My first exposure to microservices was in the spring of 2015. I was working at a well known global financial institution as a team lead and solution architect for their global innovation team. It was our job to create prototypes for new product ideas and get them to market quickly for market testing. Very much inspired by the lean startup philosophy. Create a hypothesis and test the market to see if your hypothesis is correct. If it is you build on it with your next hypothesis, if not you pivot in a new direction based on a different hypothesis. The desired result of which is that you end up creating the product that your customers actually want.
It was during this time that another architect introduced me to microservices and proposed an initial design for me to build on. I of course began by going directly to the source and reading Sam Newman’s book Building Microservices to gain a good grasp of what it was all about and why. As a result of that I came back with a design for a loosely coupled microservice architecture that I have been refining ever since, and that I would now like to share with the world. In order to do that, I am going to blog about it here whilst building a brand new version of it from scratch called the “Radical Microservice Architecture” or RADMA.
What does a microservice architecture look like?
The first thing I did was try to identify from Sam Newman’s book exactly what characterises a microservice system/architecture. This became the basis for the requirements of my design.
- A Microservice architecture should be technology agnostic
- Microservices should be platform agnostic
- Communication between services should be loosely coupled
- Prefer choreography over orchestration.
- Microservices should be Independently scaleable.
- A Microservice should own it’s own database
- A Microservice architecture requires mature CI/CD, testing etc.
- Metrics and logging should be considered from the beginning.
In addition to these requirement, the book it’s self explicitly gives us a number of principles for a microservice architecture.
- Model around business concepts
- Adopt a culture of automation
- Hide internal implementation details
- Decentralise all the things
- Independently deployable
- Isolate failure
- Highly observable
Over the course of the coming posts I will discuss all of these concept and principles in great detail and show how RADMA has been designed to satisfy all of these requirements, but lets explore requirments a little more first.
A Microservice architecture should be technology agnostic
When designing a Microservice system we should design it in such a way that it is technology agnostic. What this means is that is should not really matter whether you pick MySQL, MSSQL or PostgreSQL, what you a using is an RDMS and if you want to swap one for another later, you should be able to do so. Likewise with all the other parts of your system. ActiveMQ, RabbitMQ, StormMQ….no, an AMPQ compliant messaging system. The point of this is to try to avoid vendor lock in. There are of course times where you need to make choices, and these choices will most certainly bind you to one vendor or another to a greater or lesser degree. But try to keep this in mind throughout the design of any microservice architecture.
Microservices should be platform agnostic.
This is similar but subtly different from the previous requirement, and exists to support it. In order for our system to be technology agnostic it must be platform agnostic. If at some time in the future you choose to change platform, you do not want to have to re-write all of your services. Back in 2015 what I really meant by this was that it should not matter whether I want to run my service on Windows or Linux. Back then for someone with a .NET development background was bold move and meant that if I wanted to do this I had to embrace Mono to run .NET code on Linux or choose another language with better cross platform support. In 2021 however this takes on a slightly new angle. Netcore has been with us for a few years now and running .NET code on Linux is now common place. We should still ensure that Microservice can run on multiple operating systems, but now “platform” also has wider connotations. The rise of the ARM processor in smart phones and last years announcement from Apple that they will be moving the Macintosh over to ARM means that when we are being “platform agnostic” we should also consider it we could switch processor architecture.
Communication between services should be loosely coupled
A microservice on it’s own doesn’t do very much (or shouldn’t). Instead it participates with other services to provide a wider more useful set of features. In order to do that services need to communicate with each other, and send messages to each other that all the parties understand. When one of the services involved in this communication needs to change it may require a change to the contract between services. Your services need to be resilient to these changes. If deploying a new version of service A requires you to also make changes to Service B for the system to continue to operate, then your services are tightly couple and you have not built microservices, you have build a very complicated and distributed monolith. If, however, you can deploy your new service and the system carries on as normal, allowing you to update other services to take the new change at their own pace, then your services are loosely coupled. I believe that this, along with the next requirement are the most difficult to satisfy.
Prefer choreography over orchestration.
I feel this is the one that most microservice architectures either miss or ignore. Orchestration is the idea that a component has knowledge of the existence of other components and coordinates the calls to these components in order to achieve some objective. Choreography on the other hand is when a component does not care about the behaviours of other components, and focus only on it’s own job, and with many such components participating to complete a wider task.
In practice this means we need a system where services have no knowledge of each other’s existence and that the contract with which they communicate may change at any time.
Microservices should be Independently scaleable.
Each service should be independently deployable, and thus scaleable. If one part of the system is experiencing high load, we should be able to scale only that part of the system to deal with it.
A Microservice should own it’s own database
Another common mistake is for multiple services to access the same data. This is another form of coupling because if one service requires an update to the schema of the data, then any other also accessing that data will also have to be updated. There should be no shared databases in a microservice system. Anything that want to query the data has to go through the service that owns that data. It is suprising how many times now I have gone into an organisation who are “already doing microservices” only to discover that all the services exist in one large solution and are accessing each others data directly.
A Microservice architecture requires mature CI/CD, testing etc.
The nature of a microservice system is that you have many more components, all changing at their own pace. Trying to handle that manually is impossible. It’s fine when you have two services, but when you grow to 10? 20? 50? 200? Embrace that culture of automation and know what your CI/CD pipeline looks like before you start work on your first service.
Metrics and logging should be considered from the beginning.
In fact all cross cutting concerns should be. Data access, queue communications etc etc. But Metrics and logs are very important to understand what is happening on a distributed system. So make sure your services have good logging and metrics right from the get go, but further more ensure you have the tooling in place to aggregate this data and view it in a coherent way.
Introducing RADMA a way to solve them all.
The Radical Microservice Architecture (RADMA) addresses all of these challenges, and aims to provide a low friction way for developers and organisations to get up and running with microservices quickly. There are 4 parts to the Architecture that we will explore in more detail in further articles.
RADMA.Infrastructure
RADMA.Infrastructure gives you everything you need to set up a RADMA environment for development testing or production. Regardless of whether you want to run your environment on AWS, Azure or a cluster of Raspberry Pis. RADMA.Infrastructure exists as a set of git repositories with all the scripts you need to get started.
RADMA.Framework
RADMA.Framework is a collection of nuget packages that allow you to quickly create RADMA microservices that target RADMA environments created using RADMA.Infratructure. This includes the service chassis that encapsulates all of the cross cutting concerns of a RADMA microservice operating in a RADMA environment.
RADMA.Presentation
RADMA.Platform
RADMA.Platform is a collection of docker images that can be used to deploy a series of pre-built services you can build on to quickly get a new system up and running. This includes both popular open source tooling along side RADMA services, all pre configured to get you going as quickly as possible.
In closing
I am aware there is not much substance here and this is rather high level. Over the course of the coming weeks I will blog further on this project as I build out more of the RADMA system. In the future I am sure this article will become the basis for some documentation on the project.
