# Production-Grade B2B SaaS Microservices Architecture ## Principal Cloud Architect Design ## Stack --------------------------------------------------------------- API Gateway : Kong Enterprise Ingress : Kong Ingress Controller Protocol : gRPC (HTTP/2) + Apache Kafka Rate Limiting : Redis Sliding Window Counter Service Mesh : Istio Security : mTLS + HashiCorp Vault Container Platform : Kubernetes Container Runtime : containerd Orchestration : Kubernetes Service Discovery : Istio + Kubernetes DNS Secrets : HashiCorp Vault Observability : Prometheus + Grafana + Loki + Tempo/Jaeger + OpenTelemetry Distributed Cache : Redis Cluster Database : PostgreSQL Cluster Search : Elasticsearch/OpenSearch Message Broker : Apache Kafka Object Storage : S3 Compatible CI/CD : GitHub Actions / GitLab CI GitOps : ArgoCD --------------------------------------------------------------- ============================================================== 1. API Gateway & Routing Topology ============================================================== Internet │ Cloud Load Balancer │ Kong Enterprise │ +-------------+--------------+ | | | Authentication Routing Rate Limiter | | | +-------------+--------------+ │ Istio Ingress │ Kubernetes Cluster -------------------------------------------------------------- Gateway Responsibilities -------------------------------------------------------------- ✓ SSL Termination ✓ JWT Validation ✓ OAuth2 ✓ OpenID Connect ✓ API Key Authentication ✓ Request Validation ✓ Request Transformation ✓ Response Transformation ✓ Rate Limiting ✓ Logging ✓ Metrics ✓ Canary Routing ✓ Blue Green Routing ✓ Dynamic Upstream Selection ✓ Header Injection ✓ Correlation ID Generation ✓ Tenant Identification ✓ Request Size Validation ✓ IP Allow/Deny List ✓ Bot Detection ✓ WAF Integration ✓ gRPC Proxy ✓ REST Proxy -------------------------------------------------------------- Routing Rules -------------------------------------------------------------- /api/v1/auth/* → Authentication Service /api/v1/users/* → User Service /api/v1/orders/* → Order Service /api/v1/catalog/* → Catalog Service /api/v1/payments/* → Payment Service /api/v1/notifications/* → Notification Service /api/v1/admin/* → Admin Service -------------------------------------------------------------- Authentication Middleware -------------------------------------------------------------- Incoming Request ↓ TLS Validation ↓ JWT Validation ↓ Tenant Validation ↓ Subscription Validation ↓ Redis Cache Lookup ↓ Permission Validation ↓ Route Authorization ↓ Forward to Service -------------------------------------------------------------- Dynamic Routing -------------------------------------------------------------- Enterprise Customer → Premium Cluster Standard Customer → Shared Cluster Canary Header Present → Canary Deployment Blue-Green Header → Blue Environment Geo Routing → Nearest Region ============================================================== 2. Microservices Decomposition & Protocol Design ============================================================== Each service owns its own database. Database per Service Pattern No Shared Database Communication Rules External Clients ↓ REST Internal Services ↓ gRPC Event Driven ↓ Kafka -------------------------------------------------------------- Core Services -------------------------------------------------------------- Identity Service Tenant Service Organization Service User Service Billing Service Subscription Service Catalog Service Inventory Service Pricing Service Order Service Payment Service Invoice Service Notification Service Audit Service Reporting Service Analytics Service Search Service Gateway Service Configuration Service Feature Flag Service Logging Service Monitoring Service File Service Email Service SMS Service Webhook Service Scheduler Service -------------------------------------------------------------- REST Usage -------------------------------------------------------------- Public APIs Mobile Apps Frontend Third Party Integrations -------------------------------------------------------------- gRPC Usage -------------------------------------------------------------- Low Latency Calls Internal Services Streaming Binary Payload Contract First Design -------------------------------------------------------------- Kafka Event Topics -------------------------------------------------------------- UserCreated UserUpdated TenantCreated TenantDeleted OrderCreated OrderPaid OrderCancelled PaymentSuccess PaymentFailed InvoiceGenerated InventoryUpdated NotificationSent EmailSent AuditLogged -------------------------------------------------------------- Kafka Patterns -------------------------------------------------------------- Event Sourcing CQRS Dead Letter Queue Retry Queue Outbox Pattern Idempotent Consumer Exactly Once Processing -------------------------------------------------------------- Payload Validation -------------------------------------------------------------- JSON Schema Protocol Buffers OpenAPI Validation Business Rule Validation Data Sanitization Schema Registry ============================================================== 3. Traffic Control, Rate Limiting & Throttling ============================================================== Redis Sliding Window Counter ↓ Tenant Key ↓ API Key ↓ User ID ↓ Redis Counter ↓ Decision Engine ↓ Allow / Reject -------------------------------------------------------------- Rate Limit Levels -------------------------------------------------------------- Anonymous 10 Requests / Minute Free 100 Requests / Minute Professional 1000 Requests / Minute Enterprise 10000 Requests / Minute Unlimited Enterprise Custom Policy -------------------------------------------------------------- Tenant Scoped Rate Limiting -------------------------------------------------------------- Tenant ID ↓ Application ID ↓ User ID ↓ API Endpoint ↓ Redis Key -------------------------------------------------------------- Algorithms -------------------------------------------------------------- Sliding Window Counter Token Bucket Leaky Bucket Adaptive Rate Limiting Burst Protection Global Rate Limit Regional Rate Limit Per Endpoint Rate Limit Per Tenant Rate Limit Per User Rate Limit -------------------------------------------------------------- DDoS Protection -------------------------------------------------------------- Cloudflare AWS Shield IP Reputation Geo Blocking Challenge Response Bot Detection Connection Limits Slowloris Protection Request Size Limit Header Validation TLS Enforcement ============================================================== 4. Service Mesh, mTLS & Zero Trust Security ============================================================== Istio Control Plane │ +-------------+-------------+ | | | Service A Service B Service C | | | Envoy Envoy Envoy -------------------------------------------------------------- Service Discovery -------------------------------------------------------------- Kubernetes DNS Istio Service Registry Envoy Sidecars Automatic Discovery -------------------------------------------------------------- mTLS -------------------------------------------------------------- STRICT Mode Certificate Rotation Automatic Certificate Renewal SPIFFE Identity Workload Identity -------------------------------------------------------------- Authorization -------------------------------------------------------------- RBAC Role ↓ Permission ↓ Endpoint ↓ Action -------------------------------------------------------------- ABAC -------------------------------------------------------------- User Attributes Tenant Attributes Resource Attributes Environment Attributes Time Based Policies -------------------------------------------------------------- Zero Trust Model -------------------------------------------------------------- Never Trust Always Verify Authenticate Every Request Authorize Every Request Encrypt Every Connection -------------------------------------------------------------- HashiCorp Vault -------------------------------------------------------------- Dynamic Database Credentials Dynamic Secrets PKI Certificates API Keys JWT Signing Keys Encryption Keys Secret Rotation Secret Leasing Audit Logs ============================================================== 5. Observability, Circuit Breakers & Failover ============================================================== OpenTelemetry ↓ Jaeger / Tempo ↓ Prometheus ↓ Grafana ↓ AlertManager -------------------------------------------------------------- Metrics -------------------------------------------------------------- CPU Memory Latency Error Rate P99 P95 Kafka Lag Redis Latency Database Connections gRPC Errors HTTP Errors -------------------------------------------------------------- Logging -------------------------------------------------------------- Structured JSON Logs Correlation ID Request ID Trace ID Tenant ID User ID Loki ELK OpenSearch -------------------------------------------------------------- Distributed Tracing -------------------------------------------------------------- Client ↓ Gateway ↓ Service A ↓ Service B ↓ Kafka ↓ Database ↓ Response -------------------------------------------------------------- Circuit Breaker -------------------------------------------------------------- Istio Destination Rules Failure Threshold ↓ Circuit Open ↓ Retry ↓ Half Open ↓ Closed -------------------------------------------------------------- Retry Strategy -------------------------------------------------------------- Exponential Backoff Jitter Maximum Attempts Timeout Fallback Response -------------------------------------------------------------- Health Checks -------------------------------------------------------------- Readiness Probe Liveness Probe Startup Probe Synthetic Monitoring Heartbeat Events -------------------------------------------------------------- Multi Region Failover -------------------------------------------------------------- Region A ↓ Global Load Balancer ↓ Region B ↓ Region C Automatic Failover Database Replication Kafka MirrorMaker 2 Redis Replication S3 Cross Region Replication DNS Failover ============================================================== Recommended Kubernetes Namespace Layout ============================================================== kong istio-system monitoring logging security vault redis kafka postgres identity tenant user order payment catalog notification analytics audit ============================================================== High-Level End-to-End Request Flow ============================================================== Client ↓ Cloud Load Balancer ↓ Kong Enterprise ↓ Authentication Plugin ↓ Rate Limiter ↓ Istio Ingress Gateway ↓ Envoy Sidecar ↓ Target Microservice ↓ gRPC Internal Call ↓ Kafka Event ↓ Consumer Service ↓ Database ↓ Response ============================================================== Security Checklist ============================================================== ✔ TLS 1.3 ✔ mTLS STRICT ✔ JWT Authentication ✔ OAuth2 ✔ OIDC ✔ RBAC ✔ ABAC ✔ Vault Secrets ✔ Certificate Rotation ✔ Redis Rate Limiting ✔ WAF ✔ DDoS Protection ✔ CSP ✔ HSTS ✔ Secure Headers ✔ Audit Logs ✔ Immutable Infrastructure ✔ GitOps ✔ Signed Container Images ✔ SBOM ✔ Runtime Security ============================================================== Architecture Characteristics ============================================================== Scalable Highly Available Cloud Native Zero Trust Event Driven Low Latency Multi Region Fault Tolerant Secure by Default Observability First Production Ready Enterprise Grade Kubernetes Native Horizontal Scaling Resilient Future Proof