Legacy Modernization

Microservices first breaks prod on legacy apps you must run

Most teams make microservices the default answer to monolith pain, and that default is wrong because production complexity usually becomes the constraint before code structure does. If you are the DevOps engineer carrying the pager, a dozen services with vague ownership is not modernization; it is a larger blast radius with better branding.

Microservices are the wrong default when operations maturity is the real bottleneck

The post Monolith Scaling Pains Modernize to Microservices names real scaling pain, but treating those symptoms as an extraction order is a mistake because operational complexity grows before service ownership improves. A monolith that has slow releases, noisy deployments, and database contention will not automatically become safer after it is split, because the same teams will now debug HTTP timeouts, schema drift, and retry storms across the network.

The popular default says: carve out services, containerize them, put them on Kubernetes, add a service mesh, and let teams move independently. That works for teams that already have production ownership, reliable release automation, and clear domain boundaries. It fails for many teams because they replace a known failure mode, a large deployable artifact, with several less visible failure modes: partial deploys, incompatible contracts, duplicated authorization rules, and observability gaps.

I would not start by decomposing a monolith into many Kubernetes services, because each new service adds deployment, monitoring, security, capacity, and incident-response work before it removes a bottleneck. Kubernetes v1.29, Helm v3, Argo CD v2.10, Istio 1.22, Envoy, Prometheus v2.51, Grafana 10, OpenTelemetry Collector v0.99, and Jaeger v1.56 are solid tools, but they do not reduce cognitive load when nobody can say which team owns a failed trace at 02:00.

The production math is harsher than the architecture diagram. If a monolith has one release pipeline and one database, the pager path is short even when the codebase is ugly. If the same system becomes 18 services, a request can cross NGINX, an Istio sidecar, three Spring Boot 3.2 applications, Kafka 3.7, PostgreSQL 16, and Redis 7 before returning a 500, so the DevOps burden grows because every hop has its own saturation, configuration, and telemetry surface.

A number from a real incident review is more useful than an architecture preference: a measured p95 checkout latency of 480 ms during peak traffic points to a bottleneck, but it does not prove a service boundary is the fix because slow SQL, lock contention, garbage collection, or thread-pool starvation can produce the same symptom inside one deployable. Start with evidence, because distributed systems make wrong guesses more expensive.

The first outage will come from the new boundaries, not from the old code

Microservices convert in-process calls into network calls, and that conversion creates failure modes because TCP, DNS, TLS, retries, and load balancing now sit on the critical path. A method call in a Java monolith fails loudly and locally; a gRPC call over HTTP/2 can fail as DEADLINE_EXCEEDED, UNAVAILABLE, RESOURCE_EXHAUSTED, or a client-side timeout, and each status means different remediation during an incident.

Retries are a common example of a popular default that hurts production. A team adds retries in Envoy or Resilience4j because transient failures exist, but the same retries can multiply load on the slow dependency because every caller sends more work at the exact moment the dependency is least able to serve it. Use retry budgets and deadlines, because “try again” without a limit is an outage amplifier.

Kubernetes also gets credited with solving scaling pain it cannot solve alone. The vendor-published Kubernetes HPA controller sync period is 15 seconds through –horizontal-pod-autoscaler-sync-period, which means it reacts after metrics arrive rather than before the database pool is exhausted. HPA v2 can scale pods on CPU or custom metrics, but it cannot remove a PostgreSQL row lock, shrink a Hibernate transaction, or fix a hot Kafka partition.

Here is a small production gate I would run before approving any extraction candidate. It is intentionally boring, because a boring gate catches regressions earlier than a complicated platform promise:

#!/usr/bin/env bash
set -euo pipefail
URL="${1:-http://localhost:8080/health}"
LIMIT_MS="${LIMIT_MS:-300}"
for i in $(seq 1 20); do
  ms=$(curl -o /dev/null -s -w "%{time_total}" "$URL" | awk '{printf "%d", $1*1000}')
  [ "$ms" -le "$LIMIT_MS" ] || { echo "slow ${ms}ms on try $i"; exit 1; }
done
echo "20 checks under ${LIMIT_MS}ms"

The 300 ms limit above is a chosen guardrail to tune, not a universal truth, and it is useful because it forces a candidate service to meet an explicit latency expectation before it receives its own deployment lifecycle. The loop count of 20 is deliberately small for CI feedback, because a pre-merge signal should be cheap enough that developers do not bypass it.

Observability must exist before decomposition, because otherwise the first extracted service will remove local stack traces and replace them with guesses. Use OpenTelemetry semantic conventions, W3C Trace Context, RED metrics, USE metrics, and Prometheus histograms before splitting, because latency, error rate, saturation, and utilization tell you where the boundary should be. Use exemplars in Grafana and traces in Jaeger, because a p99 graph without a trace sample rarely tells the on-call engineer which dependency caused the tail.

SLO arithmetic should also govern the decision. A 99.9% monthly availability target permits roughly 43.2 minutes of unavailability in a 30-day month, so adding five independently deployable services can burn the same error budget faster because each service has its own rollout, dependency chain, and failure profile. Microservices are defensible when those teams can own the budget; they are reckless when the same two DevOps engineers own every alarm.

A modular monolith beats a service zoo until ownership is enforceable

The unpopular position is simple: most teams should make the monolith stricter before they make it smaller. A modular monolith is not a retreat, because enforcing package boundaries, API contracts, and database migration discipline inside one artifact reduces risk while preserving a short rollback path.

Use ArchUnit 1.2 for Java package rules, Maven Enforcer Plugin 3.4 or Gradle version catalogs for dependency control, OpenAPI 3.1 for external contracts, JSON Schema 2020-12 for event payloads, Flyway 10 or Liquibase 4 for migrations, and Testcontainers 1.19 for realistic integration tests. These tools help because they make illegal coupling visible before runtime, which is cheaper than discovering it through distributed production failures.

The comparison is not “old versus modern”; it is two operating models with different costs:

  • Option A: Modular monolith with enforced internal boundaries. This wins when one team or a small set of teams owns most changes, because one deployable keeps rollback, capacity planning, and incident triage simple. It costs discipline: developers must accept package rules, database migration reviews, feature flags, and slower extraction of independently scaling components.
  • Option B: Kubernetes-hosted microservices with independent pipelines. This wins when services map to stable teams with their own on-call rotations, because deploy autonomy only pays off when ownership follows the service. It costs platform work: container hardening, network policy, mTLS, service discovery, tracing, secrets rotation, and per-service SLOs.

That comparison matters because the microservices default often hides staffing costs inside platform tooling. Terraform 1.8 can provision the cluster, External Secrets Operator can sync credentials, and cert-manager 1.14 can rotate certificates, but someone still has to decide whether an Istio DestinationRule, an application timeout, or a PostgreSQL max_connections limit caused the incident.

PostgreSQL’s default max_connections is commonly 100 in many packaged deployments, and that concrete ceiling matters because 12 services with HikariCP pools of 10 connections each can exhaust the database before a single user-facing feature improves. PgBouncer 1.22 can help, but pooling is mitigation rather than proof that the architecture is healthy, because it manages connection churn without fixing chatty service boundaries.

Release safety also gets harder after decomposition. A monolith can use one blue-green deployment, one canary, or one rollback. A service graph needs version compatibility, consumer-driven contracts with Pact 4, backward-compatible database migrations, and often dual writes. Those are solvable problems, but they are not free because every compatibility window becomes another thing the on-call engineer must understand under pressure.

Extract only the path that earns its own production budget

I disagree with the sequence implied by the article Legacy Application Modernization: Monolith to Microservices whenever migration starts from the desired architecture rather than the production constraint, because extraction should be a budget decision as much as a design decision. A candidate service should earn independence through measurable isolation value: separate scaling pressure, separate release cadence, separate data ownership, or separate failure containment.

Start with the path that hurts production most, not the component that looks cleanest on a diagram. If invoice PDF generation consumes 70% of CPU during month-end, that reported share is a stronger extraction argument than a tidy domain noun, because background workloads can often move behind a queue without changing synchronous user flows. If a checkout transaction waits on one hot table, extracting “checkout” may make things worse because the database lock remains and the network path grows.

Kafka 3.7, Debezium 2.6, and the outbox pattern are useful when the extracted capability can tolerate asynchronous consistency, because they reduce direct database coupling while giving operations a replayable event log. They are a bad default for synchronous invariants, because eventual consistency turns simple support questions into timeline reconstruction across topics, offsets, and consumer groups.

Data ownership is the hardest line to fake. A service that has its own REST endpoint but still shares 40 tables with the monolith is a distributed monolith, because schema changes require lockstep coordination while latency and failure modes are already distributed. Prefer a narrow extraction with one owned schema, one migration stream, and one rollback plan, because production support depends on knowing what can change independently.

Set entry criteria before extraction. A service should have a named owner, a runbook, alerts with symptoms rather than guesses, dashboards for RED and saturation metrics, OpenTelemetry traces across the old and new boundary, and a deployment pipeline with automatic rollback or manual rollback documented. Argo CD sync waves can order deployment, but they do not replace compatibility design because Kubernetes can start pods faster than downstream clients can adapt.

Network security should be designed for supportability, not ceremony. Use Kubernetes NetworkPolicy, mTLS through Istio or Linkerd 2.15, and short-lived credentials from Vault 1.15 or cloud workload identity, because service boundaries without identity create lateral movement risk. Do not enable a mesh only to get a diagram, because sidecars consume CPU and memory while adding configuration states that can fail independently.

Capacity tests should precede the cutover. A tuned target might say the extracted service must handle 250 requests per second with p95 under 200 ms and error rate below 0.1% for 30 minutes in a staging environment that uses production-like data volume. Those numbers are examples to calibrate, not universal thresholds, and they matter because vague “it scales better” claims do not help during a deployment freeze.

Also define the kill switch. Use LaunchDarkly, Unleash, or a simple database-backed feature flag if that is what the team can operate, because a reversible route from NGINX or an API gateway back to the monolith is often the difference between a short incident and a customer-visible outage. AWS documents a 900-second maximum Lambda timeout, which is a reminder that managed platforms have hard ceilings; production designs need explicit escape hatches rather than hope.

Start by proving the monolith is the problem, not the scapegoat

Your first concrete move should be to instrument one painful request path end to end, write down its p95, p99, error rate, database waits, queue time, and rollback path, then ask whether a service boundary removes the bottleneck. If it does not, tighten the monolith first, because the pager will punish architecture theater long before users reward architectural purity.

Eugene started his career at Chudovo in 2015. The expert is responsible for advisory services that include but are not limited to close communication with the clients, analyzing their needs, designing tailored software strategies, etc. Throughout the practice at Chudovo, an IT consultant took part in over 17 projects, covering a wide range of client services and solving customer pains from the business perspective. With Eugene’s involvement, the customers were able to build an efficient strategy for business growth as well as to set up a long-term technology roadmap. Eugene takes an active part in software development projects in the role of IT consultant and composes proposals for the customers.