A few months into my job at Shoprocket, I was getting frustrated. We had a PHP application being worked on by a small team of sharp developers spread across the globe, and to me it seemed bug fixes spawned more bugs, integrating new features wasn't as simple as it should be, and deployments were infrequent and stressful. The reason: highly coupled code.
We had a monolithic application, almost all of it in one GitHub repository. Updates or fixes to one feature often required changes in several places. For testing, we had a peer review process where at least two other developers had to sign off on a branch before it merged into master and deployed. All of this meant that when a bug slipped in, it was often hard to locate the breaking change. Once we did, the fix went through the same process, and the whole application had to be redeployed.
Being an eager young developer always looking for the new thing, when I came across microservices I thought this was going to solve all of our problems.
What I thought microservices were
There isn't a formal definition of microservices, but from my research I took away that microservice architecture is a way of designing software applications as distributed systems of independent services, each highly specialised and with its own lifecycle. For a detailed introduction, Martin Fowler's article is excellent.
Two aspects had me hooked. First, the idea of splitting the application into services with strict boundaries. This lowers the chance of logic specific to one business context leaking between components, because the interfaces between boundaries have to be carefully planned out. It's not as simple as making local function calls. Second, each component can have its own independent lifecycle. Teams can own their code and use a stack best suited to the service, and changes can be deployed one service at a time and rolled back without affecting the rest.
I was sold. Shoprocket was going to break up the monolith. This is part 1 of a short series following the journey. My hope is that readers planning or going through a similar transition learn from our successes and failures.
The first challenge was human, not technical
I had to sell the idea to the rest of the team. They were curious but didn't share the enthusiasm, and they were reluctant to give up dev hours at this early stage of the product's life. Luckily, the CTO wanted me to look into our session management, so I seized the moment: I convinced him I could split off a session service.
In retrospect, this was a poor choice of first service. Our product includes a JavaScript widget that enables ecommerce on any website. We treat sessions as a resource, persisting session information to the database for analysing cart abandonment and other KPIs. But this resource isn't as tangible as something like orders or line items, and we used sessions on almost every change in application state. That meant the service would be inevitably chatty with the rest of the application, which I didn't foresee but which made my task harder.
What I learned actually building it
I ramped up my research. Alongside Fowler, Sam Newman's Building Microservices turned out to be an indispensable resource.
Newman's advice, in hindsight, was the advice I should have followed: first restructure the existing application into modules that reflect the domain's bounded contexts. Once the developers are happy with the boundaries, then split modules off into their own services. The reason is that it's far easier to break down and rewrite modules with bad boundaries than it is to break down and rewrite entire services with all their extra plumbing code and persistent resources.
I didn't follow that advice. Eager to build a component of my own, I went ahead with the session service.
I chose a stack only marginally different to the monolith's and started building. The service itself came together quickly, taking a few weeks to produce a standalone application with a REST API. That wasn't where the hard part was. The hard part was getting the service to talk with the monolith.
Session management turned out to be tightly coupled to the rest of the codebase. Replacing function calls with API calls was slow work. Because the session service and the monolith had separate database schemas, I had to rewrite any logic in the monolith that used session resources from the old database, restructuring data at the application boundary to keep clients like our JavaScript widget compatible.
Integrating took longer than building. And that still didn't include most of the overhead microservices normally bring: service discovery, logging, monitoring, distributed tracing. Worse, tight coupling meant changes spread across many areas of the code. As the team had been working in parallel during that time, merging and getting the session service into production would have required a lot more coordination than I'd estimated.
The honest assessment
The service might soon go into production. But I now feel I should have followed Newman's advice and put my efforts into modularising the existing application first. The benefit of having one service living on its own is marginal. The real benefit comes from splitting up the whole thing, at least into coarse-grained services that chat as little as possible with each other. That requires well-thought-out component boundaries, and we weren't there yet.
I'm still sold on microservices as an idea. I see my attempt as a useful experiment. I learned a great deal about application architecture, domain modelling, and REST principles, and those lessons had knock-on effects across other parts of our code.
For anyone thinking of a similar transition, my honest recommendations:
- Read up as much as possible on the subject before committing
- Put off the work until you're genuinely confident with your existing codebase and your domain model
- Remember there are other ways to bring stability and resilience to a monolith. Automated unit and functional tests help enormously. Test-driven development reduces regression. Horizontal scaling makes deployments less painful
- Start by modularising your monolith before extracting services
Shoprocket will continue on its journey. Part 2 is where it gets more interesting.
Originally published as the first in a short series documenting Shoprocket's early migration away from a PHP monolith. The full journey, including the mistakes corrected in later iterations, has continued across every major rewrite of the platform since.
Developer on the Shoprocket team, writing about the engineering side of building an ecommerce platform.
