LogoTRUONG PHAM
Home
Projects
Blogs
YouTube
Contact

Newsletter

Stay updated with technical artifacts and engineering insights.

LogoTRUONG PHAM

Building scalable software and sharing insights on technology & life.

Sitemap

  • Home
  • Projects
  • Blogs
  • YouTube
  • Contact

Connect

  • GitHub
  • LinkedIn
  • Email
  • YouTube

© 2024 TRUONG PHAM. © All rights reserved.

Privacy PolicyTerms of Service
Back
Docker Compose is Not Enough - The Local Development Story
Microservice Journey: Lessons & Trade-offs

Docker Compose is Not Enough - The Local Development Story

When the number of services reaches 10, a 16GB RAM machine starts screaming. How do you develop locally without 'frying' your laptop?

TP
Truong PhamSoftware Engineer
PublishedMarch 24, 2024
Stack
microservice ·devops ·productivity

There's a secret pain in microservices: To fix a single line of code in Service A, you sometimes have to start a whole... universe of related services around it.


The Harsh Reality of "Docker Compose All"

In the beginning, we only had 2-3 services. docker-compose up was enough, and everything was smooth. But as the system grew to 7, 8, then 10 services, my 16GB Macbook Pro started becoming a portable heater. Docker hogged all the RAM, the CPU was dancing constantly, and every time I saved a file, hot-reload took dozens of seconds.

Developer creativity was suddenly stifled by... the sound of the cooling fan.

1. Context-based Startup Strategy

We realized it wasn't necessary to run everything at once. If I'm working on the Payment Service, I only need:

  1. Itself (running directly via npm run dev or go run for maximum speed).
  2. Its Database (running in Docker).
  3. Auth Service (needed to validate tokens).
  4. Other services? Mock them completely or point to the Staging environment on the Cloud.

Useful Tools: We wrote separate docker-compose.db.yml files for databases and docker-compose.infra.yml for Redis/RabbitMQ. When working on a specific area, we only turn on those parts.

2. Developing Directly on Staging with Telepresence?

I tried using Telepresence—a tool that allows connecting a local machine to a Kubernetes cluster on the Cloud. It makes the service on your machine "think" it's inside the cluster, able to call other services by their internal names.

Trade-off: Very cool but extremely complex configuration for a small team. Unstable network would cause the IDE to hang constantly. Ultimately, we decided to go back to a more traditional and simpler method.

3. Mock, Mock, and Mock

This is a survival skill. Instead of trying to run the real Service B, write a simple mock server that returns the sample data Service A needs.

Pros: Extremely fast response time, no RAM overhead, and you can easily test error cases (like Service B returning a 500) without actually making Service B fail.


Hard-earned Lessons

  1. Don't overuse Docker for code you are currently editing: Run the code you are writing directly on the OS (Native). It compiles faster and uses resources much more efficiently than running through Docker's virtualization layer on Mac/Windows.
  2. Invest in a Makefile: We have commands like make dev-auth, make dev-order. It helps new members skip having to remember dozens of complex Docker commands.
  3. 32GB RAM is not a luxury: If you're doing Microservices, consider it an essential investment in your work tools.

The ultimate goal of a Local environment is to give you the fastest Feedback loop. Anything that slows down saving code and seeing results is the enemy.


Conclusion

The local development environment is often overlooked when designing architecture, but it's the thing that most directly affects the team's morale and productivity. Don't let microservice architecture turn your computer into a "brick" every Monday morning.

Series • Part 7 of 20

Microservice Journey: Lessons & Trade-offs

NextDatabase Separation - A Painful but Necessary 'Divorce'
Authentication & Authorization - When API Gateway Carries the Team
02Microservices is Not Just About Splitting Services03Monorepo or Microservice? Lessons from a 2-Person Team04REST, gRPC, or NATS? Don't Let Choice Bury Progress05Distributed Transactions - Why 'Immediate Consistency' is a Luxury06Authentication & Authorization - When API Gateway Carries the Team07Docker Compose is Not Enough - The Local Development StoryReading08Database Separation - A Painful but Necessary 'Divorce'09BFF (Backend for Frontend) - The Savior of UX10Idempotency - Why Every API Should Be 'Stubborn'?11Infra as Code (IaC) - Automate or Die in a Mountain of Config12The Pain Named Cloud Bill - Cost Optimization for 'Broke Teams'
TP

Written by Truong Pham

Software Engineer passionate about building high-performance systems and meaningful experiences.

Read more articles