Enterprise integration is the backbone of modern IT landscapes. After 13 years of experience integrating SAP, Oracle, Salesforce, and custom systems at international corporations, I'm sharing proven patterns and anti-patterns.
Why Integration Is So Complex
The challenge: heterogeneous systems need to communicate — different technologies, data formats, protocols, and availability windows. All while meeting high demands for availability, performance, and data consistency.
Typical integration scenarios:
- SAP ERP ↔ E-Commerce Platform
- Legacy System ↔ Cloud Services
- Warehouse Management ↔ Transport Management
- CRM ↔ ERP ↔ Marketing Automation
The Evolution of Integration
1. Point-to-Point: The Beginning
Description: Direct connections between systems.
When it works:
- Few systems (2-3)
- Clear, stable interfaces
- Low complexity
The problem: With n systems, you potentially have n×(n-1)/2 connections. With 10 systems, that's already 45 connections — no longer maintainable!
My take: Only for very simple scenarios or as a transitional solution.
2. Hub-and-Spoke (ESB): The Classic
Description: A central hub (Enterprise Service Bus) mediates between all systems.
Advantages:
- Central management
- Reusable adapters
- Transformation & orchestration in one place
Disadvantages:
- Single point of failure
- Can become a monolith itself
- Performance bottleneck under high throughput
Practical tip:
For a retail client, we deployed Oracle SOA Suite as an ESB. Works well for complex BPMN workflows, but is maintenance-intensive. Today, I'd go with event-driven for new projects.
3. Event-Driven Architecture: The Modern Solution
Description: Systems communicate via asynchronous events through a message broker (Kafka, RabbitMQ).
Advantages:
- Decoupling: producer doesn't know consumer
- Scalability: parallel processing possible
- Resilience: retry mechanisms, dead letter queues
- Audit trail: all events are logged
Challenges:
- Eventual consistency (not always immediately consistent)
- Debugging complexity
- Event schema evolution
My preferred stack:
- Kafka for high-throughput, event sourcing
- RabbitMQ for classic message queues with routing
- CloudEvents Standard for event schemas
Concrete Integration Patterns
Pattern 1: Request-Reply (Synchronous)
Use case: Availability checks, price queries, customer lookups
Client → REST API → Service → Response
Best practices:
- Configure timeouts (e.g., 3 seconds)
- Implement circuit breakers
- Ensure idempotency
Technology: REST, gRPC, GraphQL
Pattern 2: Fire-and-Forget (Asynchronous)
Use case: Order confirmations, warehouse bookings, logging
Producer → Message Queue → Consumer (later)
Best practices:
- Guarantee at-least-once delivery
- Consumers must be idempotent
- Dead letter queue for errors
Technology: RabbitMQ, AWS SQS, Azure Service Bus
Pattern 3: Publish-Subscribe
Use case: Order events to multiple systems (warehouse, shipping, billing)
Publisher → Topic/Exchange → [Subscriber1, Subscriber2, ...]
Best practices:
- Version event schemas (Avro, JSON Schema)
- Consumer groups for scaling
- Replay capability for new consumers
Technology: Kafka, AWS SNS, Redis Pub/Sub
Pattern 4: Saga Pattern
Use case: Distributed transactions (Order → Payment → Shipping → Invoicing)
Step1 → Success → Step2 → Success → Step3
↓ Fail ↓ Fail
Compensate Compensate
Best practices:
- Choreography (events) vs. orchestration (coordinator)
- Compensation logic for each step
- Monitoring & alerting for saga state
Common mistake:
Implementing sagas without compensation logic. When step 3 fails, steps 1 and 2 need to be rolled back — this must be explicitly implemented!
SAP Integration: The Elephant in the Room
SAP is the core system in many enterprise landscapes. Integration is often complex, but very doable with the right approach.
SAP Integration Options
- RFC/BAPI: Synchronous function calls
- ✅ Direct access to business logic
- ❌ Tight coupling, performance limits
- IDoc: Asynchronous documents
- ✅ SAP-native, robust
- ❌ Complex configuration, proprietary format
- OData Services: RESTful API
- ✅ Modern, JSON-based
- ❌ Not all SAP modules support OData
- SAP PI/PO: SAP's own ESB
- ✅ SAP-optimized
- ❌ Expensive, complex, legacy
My recommendation:
For new integrations: OData when available, otherwise RFC/BAPI with a wrapper service as an anti-corruption layer. IDocs only when unavoidable.
API Gateway Pattern
Essential for microservices architectures:
Client → API Gateway → [Service1, Service2, Service3, ...]
Functions:
- Routing: Requests to the right service
- Authentication/Authorization: Centrally managed
- Rate Limiting: Protection against overload
- Transformation: Request/response mapping
- Aggregation: Combine multiple service calls
Tools:
- Kong: Open source, Lua-based
- NGINX: Proven, performant
- AWS API Gateway: Managed service
- Spring Cloud Gateway: Java stack
Integration Testing
The biggest challenge: how do you test a distributed system?
My approach:
- Contract Tests: Pact, Spring Cloud Contract
→ Ensure producer/consumer compatibility - Integration Tests: Testcontainers
→ Real dependencies (DB, Kafka) in Docker - End-to-End Tests: Minimal!
→ Only critical happy paths, too slow for CI/CD - Chaos Engineering: Production!
→ Test resilience with Chaos Monkey
Monitoring & Observability
In distributed systems, monitoring is mandatory:
The 3-Pillar Model:
- Metrics: Prometheus + Grafana
→ Latency, throughput, error rate - Logging: ELK Stack or Loki
→ Structured logs with correlation IDs - Tracing: Jaeger or Zipkin
→ Request flow across all services
Key insight:
Correlation IDs are the key! Every request gets a unique ID that propagates through all services. This makes debugging 100x easier.
Common Anti-Patterns to Avoid
- ❌ Distributed Monolith: Microservices with synchronous calls everywhere
- ❌ Chatty Services: 100 API calls for one business operation
- ❌ Shared Database: Multiple services on one DB — tight coupling!
- ❌ Missing Timeouts: One hanging service blocks everything
- ❌ No Idempotency: Retries lead to duplicate orders
Conclusion
Enterprise integration is complex, but manageable with the right patterns. My key takeaways:
- Async first: Event-driven where possible, synchronous only when necessary
- Anti-Corruption Layer: Protects against legacy complexity
- Observability: Build in from the start, not as an afterthought
- Start simple: Point-to-point is fine for 2-3 systems
- Idempotency: Makes retries safe and life easier
The choice of the right pattern depends on your context: is there already an ESB? Cloud or on-premise? Batch or real-time? Get expert advice — it saves time and money.