LogoTRUONG PHAM
Home
Projects
Blogs
YouTube
Contact

Newsletter

Stay updated with technical artifacts and engineering insights.

LogoTRUONG PHAM

Building scalable software and sharing insights on technology & life.

Sitemap

  • Home
  • Projects
  • Blogs
  • YouTube
  • Contact

Connect

  • GitHub
  • LinkedIn
  • Email
  • YouTube

© 2024 TRUONG PHAM. © All rights reserved.

Privacy PolicyTerms of Service
Back
The Maintenance Nightmare - When Microservice Systems Age
Microservice Journey: Lessons & Trade-offs

The Maintenance Nightmare - When Microservice Systems Age

Microservices increase maintenance costs exponentially. How to manage fragmentation and 'knowledge islands' in a small team.

TP
Truong PhamSoftware Engineer
PublishedApril 5, 2024
Stack
microservice ·maintenance ·legacy-code ·team-management

Anyone can build a new system. But only those with true grit can maintain that system reliably after 2 years, when the once-new code has now become 'legacy'.


1. The Pain Named "En Masse"

In a Monolith world, if you want to upgrade the Node.js version from 18 to 20, you fix exactly one place. In Microservices, you have to open 20 Repos (or 20 directories), modify 20 Dockerfiles, update 20 CI/CD files, and wait for 20 pipelines to run.

The price of fragmentation:

  • Security Patching: When a critical vulnerability appears in a popular library, "scanning" and patching in 20 services is an ordeal.
  • Drift: Service A uses this library version, while Service B uses another. Gradually, your system becomes a museum of software versions.

2. "Knowledge Islands" - When Trust is Misplaced

With a team of 2-3 people, each person usually "owns" a cluster of services.

  • Person A handles 5 Finance services.
  • Person B handles 5 Warehouse services.

What happens when Person A quits? Person B is suddenly faced with 5 extremely complex "black boxes." Because microservices separate so strictly, they sometimes also separate the understanding of the team members.

3. Survival Strategy for Small Teams

To avoid collapsing under the mountain of maintenance debt, we apply:

  1. Standardization: Use one Scaffold/Template for every service. Every service must have the same folder structure, the same logging, and the same deployment method.
  2. Shared CI/CD Templates: Don't rewrite pipelines for each service. Use templates so that when you need to fix something, you only fix it in one place for all.
  3. Cross Culture: Regularly perform cross-code reviews and rotate tasks among members. Ensure at least 2 people always understand any given service.
  4. Accept Controlled Technical Debt: Don't try to upgrade everything to the latest version immediately. Prioritize things related to security and performance.

Lessons Learned

  1. Easy to Build, Hard to Maintain: Calculate maintenance costs before deciding to split a new service. Adding 1 service adds a lifelong maintenance burden.
  2. Automation is No Longer an Option: If you don't have scripts for bulk updates, you'll soon give up.
  3. Documentation is the Lifeline: As the system ages, your memory will betray you. Those hastily written README lines from the past are your "lifebuoys" later.

Conclusion

Microservice maintenance is a marathon, not a sprint. Don't let the excitement of the early building days make you forget that: Code is a company asset, but it is also a debt for the one who wrote it.

Series • Part 19 of 20

Microservice Journey: Lessons & Trade-offs

NextMicroservice Journey Summary - Do You Really Need It?
Macroservice - When We Decide to 'Merge' Services Back
14Contract Testing - When Integration Testing Becomes a Burden15AI-Service Integration - Bringing LLM/RAG into Microservice Architecture16Feature Flags - Deploy and Release are No Longer One17Secret Management - Don't Let API Keys Wander Around18Macroservice - When We Decide to 'Merge' Services Back19The Maintenance Nightmare - When Microservice Systems AgeReading20Microservice Journey Summary - Do You Really Need It?
TP

Written by Truong Pham

Software Engineer passionate about building high-performance systems and meaningful experiences.

Read more articles