LogoTRUONG PHAM
Home
Projects
Blogs
YouTube
Contact

Newsletter

Stay updated with technical artifacts and engineering insights.

LogoTRUONG PHAM

Building scalable software and sharing insights on technology & life.

Sitemap

  • Home
  • Projects
  • Blogs
  • YouTube
  • Contact

Connect

  • GitHub
  • LinkedIn
  • Email
  • YouTube

© 2024 TRUONG PHAM. © All rights reserved.

Privacy PolicyTerms of Service
Back
The Pain Named Cloud Bill - Cost Optimization for 'Broke Teams'
Microservice Journey: Lessons & Trade-offs

The Pain Named Cloud Bill - Cost Optimization for 'Broke Teams'

Receiving the AWS bill at the end of the month and being shocked by the huge amount for tiny microservices. How we saved 50% on Cloud costs.

TP
Truong PhamSoftware Engineer
PublishedMarch 29, 2024
Stack
microservice ·cloud ·cost-optimization ·aws

Microservices have a 'side effect' rarely mentioned in architectural textbooks: It can burn your money many times faster than a Monolith if not tightly managed.


The First Bill Shock

The first month of running a full Microservice architecture on AWS, we received a bill 3 times higher than expected. Why?

  • 10 services running on 10 separate RDS instances, even though the data for each was only a few hundred MB.
  • Each service needed its own Load Balancer.
  • Data transfer between Availability Zones (Inter-AZ transfer) was being charged.

We realized: If we didn't optimize, this architecture would swallow all profits before the product even had a chance to succeed.

1. Sharing Resources While Maintaining Boundaries

Instead of a costly RDS instance for every service, we grouped them:

  • Run one slightly larger RDS instance.
  • Create multiple Logical Databases (or Schemas) within it for each service.
  • Trade-off: If this instance goes down, all services lose their DB. But for a small team in the early stages, this is an acceptable risk to save hundreds of dollars a month.

2. Serverless and Container Auto-scaling

Instead of renting EC2 servers that run 24/7 (even when no one is using them), we switched to AWS Fargate or Google Cloud Run. Pros: You only pay when code is processing a request. No traffic at night? Costs are almost zero.

Lesson: Don't run "always-on" instances for services with inconsistent traffic.

3. Turn Off the Lights When Leaving the Room (Dev/Staging environment)

Our Development and Staging environments accounted for 40% of costs but were only used 8 hours a day, Monday to Friday. Solution: Write a simple script (or use IaC) to automatically "turn off" the entire Staging cluster at 7 PM and "turn on" at 8 AM the next morning. Turn them off completely on weekends. Result: Immediate 60% savings on non-production environments.


Other Small "Tips"

  1. Spot Instances: Use the Cloud Provider's "excess" servers at 70-90% lower prices. Suitable for background workers (NATS consumers) where a few seconds of interruption isn't critical.
  2. Smart Free Tier Usage: Take advantage of every startup support program.
  3. Resource Limits: Always set cpu_limit and memory_limit for containers. Don't let a memory-leaking service eat up all the resources of the entire server cluster.

Advice for "Broke Teams"

Doing Microservices doesn't mean you have to spend like a tycoon. Start with maximum frugality. When traffic grows and brings in revenue, that's the time to put more money into buying stability and complete infrastructure isolation.

Remember: Good code is code that runs efficiently, but good infrastructure is infrastructure that brings the highest value at the lowest cost.


Conclusion

Managing Cloud costs is an inseparable part of a system architect's skill set. A technically beautiful system that bankrupts the company with monthly bills is still a failure.

Series • Part 12 of 20

Microservice Journey: Lessons & Trade-offs

NextWhen the System Goes Silent - Lessons on Post-mortem and Blame-free Culture
Infra as Code (IaC) - Automate or Die in a Mountain of Config
07Docker Compose is Not Enough - The Local Development Story08Database Separation - A Painful but Necessary 'Divorce'09BFF (Backend for Frontend) - The Savior of UX10Idempotency - Why Every API Should Be 'Stubborn'?11Infra as Code (IaC) - Automate or Die in a Mountain of Config12The Pain Named Cloud Bill - Cost Optimization for 'Broke Teams'Reading13When the System Goes Silent - Lessons on Post-mortem and Blame-free Culture14Contract Testing - When Integration Testing Becomes a Burden15AI-Service Integration - Bringing LLM/RAG into Microservice Architecture16Feature Flags - Deploy and Release are No Longer One17Secret Management - Don't Let API Keys Wander Around
TP

Written by Truong Pham

Software Engineer passionate about building high-performance systems and meaningful experiences.

Read more articles