Real-Time and Streaming Analytics in SAP: Patterns, Trade-offs, and Practical Implementation

The Real-Time Analytics Spectrum: Defining Your Latency Tier

"Real-time" is a spectrum, not an absolute. The latency requirements of an operational dashboard for manufacturing quality differ radically from those of a sales analytics dashboard. Understanding where your use case sits on the spectrum is the first step to choosing the right architecture and tools.

Three Latency Tiers

Near-Real-Time (NRT) — Minutes is the most common and practical tier. Data flows from source systems (S/4HANA, BW/4HANA, non-SAP databases) to analytics every few minutes to an hour through delta replication. Datasphere replication flows fit here: after the initial load, a flow with load type Initial and Delta stays active and copies changes at a delta load frequency you set (default 60 minutes; anything from immediate to 24 hours), so analytics stays current without streaming infrastructure. Most enterprise analytics sits at this tier—daily, hourly, or sub-hourly refresh is sufficient. A sales dashboard refreshing every 15 minutes covers most business questions without requiring streaming infrastructure.

Operational Real-Time (ORT) — Seconds serves use cases requiring sub-minute updates. Manufacturing KPIs (cycle time, defect rate), warehouse operations (inventory movements, order fulfillment), or quality control dashboards need data that is seconds to a minute old. This tier demands continuous replication or events: change data capture from SAP systems (replication flows with an immediate delta load frequency, SLT, or real-time remote tables), event streams via SAP Event Mesh, or live queries against the source (federation). ORT is where most "real-time" enterprise analytics lives. The difference between a 30-minute and a near-instant refresh can determine whether a supervisor spots a line stoppage before thousands of units are scrapped, or after.

Streaming (Sub-Second) — Milliseconds is the specialized tier for latency-sensitive operations: algorithmic trading, real-time anomaly detection, IoT-driven quality monitoring. Streaming requires event-driven pipelines, message brokers (Apache Kafka, advanced event mesh in SAP Integration Suite), and sub-second data ingestion. While powerful, streaming adds significant operational complexity. Most enterprises should avoid this tier unless specific, measurable business impact justifies the complexity.

For operational analytics, operational real-time (seconds) is the practical sweet spot. It eliminates most decision delays, leverages SAP's CDC and federation capabilities, and avoids the operational burden of sub-second streaming infrastructure.

Change Data Capture (CDC): The Foundation of Real-Time Replication

Change Data Capture is the technical foundation enabling real-time analytics. Rather than extracting entire tables on a schedule, CDC captures only the rows that changed since the last extraction, dramatically reducing network and computational load.

SAP Datasphere Replication Flows: CDC in Action

Datasphere Replication Flows support multiple CDC patterns tailored to SAP's ecosystem:

Delta Extraction from S/4HANA and BW/4HANA uses SAP's own delta mechanisms: change data capture on extraction-enabled CDS views in S/4HANA, ODP providers (for example, SAP BW/4HANA DataStore objects or ECC extractors), and SLT for tables. With load type Initial and Delta, the first run extracts the full dataset; the flow then stays active and transfers only changed rows at the delta load frequency (default 60 minutes; from immediate to 24 hours). Flows on ODP sources run deltas at most hourly. Moving only changes instead of full extracts cuts data volume sharply for large tables, and because billing follows the replication job's duration, a longer interval also costs less.

SLT-Based Replication uses SAP Landscape Transformation Replication Server (SLT). Its change data capture is trigger-based: database triggers on the source tables record every insert, update, and delete in logging tables, which SLT reads and transfers. SLT can feed Datasphere replication flows for ABAP tables in SAP ECC or S/4HANA (a standalone SLT server is the recommended setup), giving continuous table-level replication where no suitable CDS view exists. End-to-end latency depends on SLT's processing, the delta load frequency, and the target, so measure it in your landscape rather than assume seconds. SLT is more invasive: it needs the DMIS add-on, an SLT configuration, RFC connections, and database triggers plus logging tables on the source.

CDS View Extraction with Delta applies to SAP's Core Data Services (CDS) views. Replication flows can only extract CDS views that carry the data-extraction annotation; for delta, the view also needs change data capture (or a generic timestamp delta), and views with parameters or table functions aren't supported. Released, extraction-enabled CDS views (filterable in the SAP Business Accelerator Hub) are the upgrade-stable choice. Datasphere extracts the entire view on the first run, then retrieves only the changed rows. This pattern combines simplicity (CDS views are predefined, governance-aligned) with efficiency (delta reduces payload).

Non-SAP Source CDC depends on the connection type, so check SAP's connection-type matrix before you design. Microsoft SQL Server, Azure SQL Database, and MySQL can be replication-flow sources, and Oracle and Snowflake were added in 2026 as sources for initial loads. Remote tables on SQL Server, Azure SQL Database, Oracle, and SAP HANA connections support real-time replication, so changes keep flowing without a schedule. For Salesforce and other SaaS platforms, Datasphere has no direct connection: changes arrive through SAP Integration Suite (timestamp-based queries or the platform's own change events) and are written into Datasphere. Hybrid landscapes can apply near-real-time patterns widely, but not with one uniform mechanism.

SAP HANA Cloud: In-Memory Federation for Real-Time Analytics

Not all real-time analytics requires copying data. SAP HANA Cloud enables federation—querying source systems directly without replication—via calculation views and smart data access.

When to Federate vs. Replicate

Federate (query live) when: data volumes are small to medium (millions, not billions of rows), freshness is critical (data must be current at query time), or regulatory constraints prohibit copying (data residency, privacy). Example: A supply chain dashboard querying live inventory levels from three regional ERP systems. Federated views from HANA Cloud join real-time data across systems without replication.

Replicate (copy to analytics) when: query complexity is high (heavy aggregations, multi-table joins), data volume is large (requires indexing and statistics), or the source system's performance cannot tolerate frequent analytical queries. Example: A revenue attribution dashboard analyzing 24 months of transactions, requiring dimensional joins and complex calculations. Replicated data in HANA allows pre-indexed, optimized aggregations.

Modern implementations often blend both: a hybrid model where hot, tactical data (current orders, recent transactions) feeds operational dashboards via federation, while historical data (past quarters' actuals) comes from replicated, indexed tables. HANA Cloud's calculation views make this seamless—a single SAC dashboard can consume both.

Real-Time Calculation Views and Transactional Analytics

HANA Cloud calculation views enable operational analytics by exposing S/4HANA's transactional data in analytics-friendly dimensions and measures. A calculation view on VBAK (sales order header) and VBAP (items) joins dimensions (customer, product, region) and aggregates measures (quantity, revenue, margin) in HANA's in-memory engine, delivering fast query response even across millions of orders.

When combined with federation, calculation views query source systems live. When combined with replication, views query replicated snapshots. The advantage: a single business metric (total orders, average order value) is defined once in a calculation view, consumed consistently across all downstream analytics, and updated according to the refresh strategy (federated = always current, replicated = as-of-last-refresh).

Event-Driven Architecture: SAP Event Mesh and Business Events

CDC picks up changes after they are written to the database; event-driven architecture publishes them as business events when they occur, so subscribers can react immediately.

SAP Event Mesh: Publish/Subscribe for Analytics

SAP Event Mesh — a multi-protocol broker (AMQP, MQTT, REST) on SAP BTP, with advanced event mesh in SAP Integration Suite (based on Solace) as the high-volume, enterprise-grade option — lets S/4HANA and other SAP systems publish business events. When a sales order is created or changed, S/4HANA publishes the standard events sap.s4.beh.salesorder.v1.SalesOrder.Created.v1 and SalesOrder.Changed.v1. Subscribers (an integration flow, a CAP service, or an SAP Build Process Automation trigger) receive them and raise alerts, update records, or start downstream processing. Neither Datasphere nor SAC subscribes to topics: to use events in analytics, a subscriber writes them into Datasphere.

Event-driven integration replaces polling for the systems that subscribe. Instead of asking "What changed?" every 10 minutes, the subscriber is notified: "An order was created." This push model is what makes second-level reactions possible: a quality alert can go out within seconds of the triggering event, rather than waiting for the next batch refresh.

Business Events from S/4HANA

S/4HANA ships standard business events for many objects (sales orders, business partners, and more; the SAP Business Accelerator Hub lists what your release publishes), and since release 2022 the ABAP RESTful Application Programming Model (RAP) can raise custom business events as well. Events use the CloudEvents format: metadata (type, source, ID, time) plus a data payload. Standard S/4HANA events are mostly notifications that carry the object key and little else, so the subscriber usually reads the details (customer, items, amounts, status) through an API before writing them anywhere.

To get events into Datasphere, use one of two landing paths. A subscriber writes them into Datasphere tables (for example, an integration flow writing through the JDBC adapter into an Open SQL schema). Or, where events already run through Kafka, Datasphere replication flows read Confluent topics (Confluent connections work as replication source and target; generic Apache Kafka connections only as a target). A sales dashboard on a live connection then shows the latest order counts at its next refresh, and a supply chain view reflects new inbound deliveries soon after their events land. That is event-driven analytics, not batch-mode reporting—with the caveat that the dashboard still refreshes on its own interval.

Event-Driven Patterns in Practice

Reactive Alerts: When an event signals a stock-relevant change (for example, a goods receipt for a critical material), the subscriber writes it into Datasphere, where a view compares stock against the threshold, and SAC shows the exception on a live dashboard. For time-critical cases, the same event can trigger an SAP Build Process Automation process or another subscriber that notifies the buyer directly: "Lead time for Part XYZ exceeded; expedite PO." Decision-makers act on deviations in minutes, not days.

Real-Time Aggregations: Instead of recalculating a sales dashboard hourly, landed events can feed running totals. Each created or changed order lands as a row, and a Datasphere view aggregates the current day; the dashboard shows the running total when opened or at its auto-refresh interval. True per-event counters (increment on create, decrement on cancel) belong in the subscriber or a stream processor, not in the warehouse.

Cross-System Correlation: Events from several systems can be correlated once they share a key and a timestamp. A manufacturing execution system (MES) reports production completion; S/4HANA records the delivery and goods issue. Correlating the two—in a stream processor for second-level detection, or in Datasphere once both have landed (as events or replicated deltas)—measures order-to-shipment time and exposes bottlenecks as they form. SAP Event Mesh transports the events; it doesn't correlate them.

Streaming Integration Patterns: Kafka, IoT, and Beyond

For enterprises where streaming is essential—high-frequency IoT data, Kafka-based data lakes, real-time threat detection—SAP provides multiple paths to integrate streaming data into analytics.

Apache Kafka Integration

Confluent connection in Datasphere: Organizations using Confluent can load Kafka topics into Datasphere with replication flows, which accept a Confluent connection as source or target. IoT sensors publishing temperature, pressure, and vibration data to a Kafka topic can land in Datasphere tables for analytics. Delivery guarantees, deduplication, and schema changes still need design work: test how the flow handles your message format before promising exactly-once results.

S/4HANA data to Kafka: For the inverse direction—exporting S/4HANA data to Kafka—Datasphere replication flows can write S/4HANA CDS view and table changes to Apache Kafka or Confluent targets. Loads into non-SAP targets need premium outbound integration blocks, sized by replicated data volume. Material master changes, purchase order updates, and inventory movements flow to Kafka topics, feeding data lakes and analytics platforms. This decouples S/4HANA from downstream consumers: Kafka becomes the event backbone, while S/4HANA stays the system of record.

SAP Integration Suite Kafka Adapter

Cloud Integration in SAP Integration Suite has a Kafka adapter (sender and receiver), so integration flows can consume Kafka topics and publish to them. That makes Integration Suite the bridge in hybrid landscapes where a Kafka cluster coexists with SAP Event Mesh: an integration flow can map S/4HANA events into the format Kafka consumers expect, or route Kafka records into SAP systems. For broker-to-broker links at volume, advanced event mesh can bridge to Kafka directly. Change capture itself happens at the source (CDS view change data capture, SLT, database logs) or in Datasphere replication flows; the integration layer moves and maps the changes.

ABAP RAP Events in Real-Time Scenarios

Modern ABAP development (SAP S/4HANA 2022 and later, SAP S/4HANA Cloud, and the SAP BTP ABAP environment) can raise RAP business events for custom business objects. When a RAP-based sales order application processes a state change (draft → submitted → approved), it raises a typed business event that is published through SAP Event Mesh. Subscribers—not Datasphere itself—consume these structured, typed payloads and write what analytics needs into Datasphere.

IoT Data from SAP Digital Manufacturing

For manufacturing organizations, SAP Plant Connectivity (PCo) collects machine data from the shop floor (for example, from OPC UA sources) and sends tag-based notifications to SAP Digital Manufacturing, which runs production execution. For real-time production analytics, keep second-level monitoring close to the line (in SAP Digital Manufacturing or a dedicated streaming stack) and land aggregated KPIs (OEE, cycle time, defect rate) and the events you need in Datasphere through an integration flow or a subscriber. SAC doesn't subscribe to event streams; a live SAC dashboard shows what has landed at its auto-refresh interval. Combined with quality data from SAP Quality Management (QM) in S/4HANA, you get near-real-time manufacturing visibility—valuable in high-velocity, capital-intensive operations.

Operational Dashboards: Live Data Connections and Refresh Strategies

Real-time analytics only matters if dashboards reflect real-time data. SAP Analytics Cloud (SAC) supports multiple connection patterns optimized for different latency requirements.

Live Data Connections vs. Imported Data

Live Connections query source systems on demand (federation pattern). When a user opens a dashboard, SAC queries HANA Cloud calculation views, Datasphere models, or S/4HANA live. This keeps data current at query time but adds query latency: every widget's query runs against the source, so complex pages with many widgets load more slowly. Suitable for executive dashboards with complex filters, where accuracy outweighs response time, or for low-traffic use cases where query load is minimal.

Imported Data copies data into an SAC model and refreshes it with scheduled import jobs—hourly at the most frequent (every 1, 2, 3, 4, 6, 8, or 12 hours), or daily, weekly, or monthly. Queries run against SAC's own copy, which suits high-concurrency, interactive dashboards. Suitable for dashboards with many concurrent users where an hour or more of staleness is acceptable; anything fresher needs a live connection.

Hybrid dashboards blend both: core KPIs (from replicated data, refreshed frequently) are live-responsive, while detailed drill-downs (from live connections) fetch fresher but slower data on-demand. This balances performance with freshness.

Automatic Refresh and Exception-Based Alerting

SAC stories can auto-refresh at an interval the designer sets (for example, every minute or every 30 minutes). Auto refresh runs only while the story is open in a browser tab, pauses when the tab is in the background, stops after five consecutive failed refreshes, and isn't supported in the SAC mobile app. SAC doesn't subscribe to Datasphere or Event Mesh events, so refresh stays interval-based: for critical KPIs (stock levels, quality metrics, order fulfillment), pair a live connection with a short auto-refresh interval on an always-open screen.

Exception-based alerting goes further, but know SAC's limits. Thresholds highlight problems visually, and data change insights notify subscribed users about significant changes (including values crossing thresholds) through SAC notifications, email, or the mobile app—at most daily, since a day is the smallest unit. For second-level alerts (stock below minimum, a defect-rate spike, an order slipping), raise the alert from the event itself: a subscriber or an SAP Build Process Automation process triggered by the S/4HANA event notifies the right person immediately.

Real-Time Analytics Architecture Patterns

Three real-time analytics architecture patterns in SAP: near-real-time hybrid (delta replication, typically 5-30 min), event-driven operational (events land in seconds; dashboards follow their refresh interval), and full streaming for IoT and manufacturing (alerts in seconds in the streaming layer).

Organizations don't choose one pattern—they implement all three, tailored to business criticality and latency requirements.

Pattern A: Batch + Near-Real-Time Hybrid

The most pragmatic pattern for most enterprises. Datasphere replication flows extract delta changes from S/4HANA and other sources at the delta load frequency you set (for example, every 15 minutes; ODP sources at most hourly). SAC dashboards on a live connection to Datasphere show the replicated data, and Datasphere can persist heavy views for fast queries. Latency: the delta interval plus processing, typically 5-30 minutes from event to visibility. Suitable for sales, revenue, inventory, and most financial analytics where intra-day decision-making is sufficient.

Pattern B: Event-Driven Operational Analytics

For use cases requiring sub-minute reaction time. S/4HANA publishes business events (for example, sales order created or changed) to SAP Event Mesh. A subscriber (an integration flow or CAP service) reads the details through the S/4HANA API and writes them into Datasphere, where views maintain running daily totals and exception lists. SAC dashboards on a live connection show the result at their auto-refresh interval, and time-critical alerts go out from the subscriber directly. Latency: seconds for events to land, plus the dashboard's refresh interval. Suitable for quality monitoring, exception management, and supply chain visibility where immediate reaction prevents losses.

Pattern C: Full Streaming Pipeline

For manufacturing and IoT-heavy scenarios. Sensor data streams to Apache Kafka (for example, on Confluent), where stream processing computes OEE, detects anomalies, and raises alerts within seconds. Replication flows load the topics or the computed results into Datasphere through a Confluent connection for history and cross-plant analytics, and SAC shows them on live dashboards. Latency: seconds for alerts in the streaming layer; dashboards follow their refresh interval. Operationally complex (Kafka clusters, streaming pipelines, schema registry) but necessary for predictive maintenance, yield optimization, and real-time quality control in capital-intensive operations.

Practical Guidance: Latency Requirements Analysis and Cost

Real-time analytics adds complexity and cost. Before committing to real-time, perform a latency requirements analysis:

The Latency Requirements Framework

For each analytics use case, answer these questions:

What is the decision window? How long can decision-makers wait for data before acting? A sales forecast can tolerate 24-hour latency. A warehouse optimization dashboard cannot tolerate more than 5 minutes. A machine anomaly detector cannot tolerate more than 30 seconds. This establishes your ceiling.

What is the cost of delay? Delayed visibility to a production line stoppage costs thousands per minute in unplanned downtime. Delayed visibility to inventory imbalance costs hundreds per day in expedited shipping. Delayed visibility to monthly revenue costs nothing—it's historical reporting. Quantify the cost of delay; high-cost delays justify real-time investment.

Is the latency requirement truly driven by business need, or is it habit? Many organizations assume daily reporting is insufficient without testing whether 4-hour or hourly refresh would change decisions. Pilot operational dashboards at different refresh frequencies and measure decision impact. Often, a 15-minute refresh removes most of the pain without streaming infrastructure.

What is your tolerance for complexity? Streaming pipelines, Kafka clusters, event-driven architectures, and CDC configurations require dedicated operational expertise. If your team lacks real-time infrastructure experience, the operational burden may exceed the business benefit. Near-real-time (5-30 min) with CDC is simpler and often sufficient.

Cost Implications

Near-Real-Time (5-30 min refresh): Low additional cost. Replication flows with delta extraction move only changed rows. They are billed by the replication job's duration rather than data volume, so longer delta intervals cost less, and loads into non-SAP targets also need premium outbound integration blocks. Suitable for cost-conscious organizations.

Operational Real-Time (seconds): Moderate cost increase. Event Mesh adds SAP BTP messaging cost, subscribers need a runtime (Cloud Integration or CAP), replication flows with immediate delta loads run longer jobs, Datasphere keeps more hot data in memory for real-time aggregations, and frequent dashboard refreshes add query load on the live source. Size it per use case rather than with a rule of thumb.

Streaming (sub-second): Significant cost. Kafka cluster management (cloud-hosted or on-premises), stream-processing infrastructure, high-frequency ingestion, and the people to run them all add operational and licensing costs. Justified only when business impact exceeds infrastructure investment.

When Batch is Actually Better

Monthly closing analytics, historical trend analysis, and annual reporting don't benefit from real-time. A nightly batch refresh (after close of business) is more suitable, cost-effective, and aligns with decision cadence. Similarly, regulatory reporting with legal signatures and approvals cannot be real-time; scheduled batch processes with audit trails are required.

Common Anti-Patterns to Avoid

Over-Engineering for Sub-Second Latency When Minutes Suffice: Building streaming pipelines for a sales dashboard that refreshes hourly adds cost and complexity without business value. Start with the minimum latency required to make better decisions, then optimize from there.

Streaming Everything: Not all data is high-value when fresh. Master data (customer, product hierarchies) can be replicated daily. Transactional data (orders, invoices) may require hourly refresh. Operational data (stock, in-transit shipments) requires real-time. Design each data domain's refresh strategy independently.

Ignoring Data Quality in Real-Time: Speed doesn't justify inaccuracy. A real-time dashboard showing incorrect inventory is worse than a delayed dashboard showing correct inventory. Ensure data validation, transformation logic, and quality checks are in place before operationalizing real-time feeds.

Duplicating Data Across Patterns: Organizations often maintain parallel batch and real-time pipelines, duplicating logic and creating consistency headaches. Standardize on one source of truth (Datasphere), then branch to different consumption patterns (imported vs. live, batch vs. event-driven) downstream.

Implementation Roadmap for Real-Time Analytics

Phase 1: Near-Real-Time Foundation (Months 1-6)

• Assess latency requirements for top 10 analytics use cases. Document decision windows and cost of delay.
• Implement Datasphere Replication Flows with delta extraction for critical source systems (S/4HANA transactional data, BW/4HANA InfoProviders).
• Build SAC dashboards on a live connection to the replicated Datasphere data, with a 15-minute delta load frequency on the critical flows.
• Measure impact: Did 15-minute latency enable decisions that daily refresh did not?

Phase 2: Event-Driven Ops (Months 7-12)

• Map S/4HANA business events for high-impact use cases (quality failures, order exceptions, supply chain events).
• Connect S/4HANA to SAP Event Mesh and build subscribers (integration flows or CAP services) that write events into Datasphere for real-time aggregations.
• Build operational dashboards with auto-refresh and exception alerts for quality and supply chain KPIs.
• Retire batch quality reports; shift to event-driven exception management.

Phase 3: Streaming (Selective, Month 13+)

• Identify manufacturing or IoT use cases where sub-5-second latency drives measurable business value.
• Implement Kafka integration via Confluent Cloud or on-premises Kafka cluster.
• Build streaming ingestion pipelines to Datasphere; create real-time OEE, anomaly, and predictive maintenance dashboards.
• Continuously monitor and optimize; streaming complexity is high, so ensure benefits justify operational burden.

Real-Time Analytics as Competitive Advantage

Real-time analytics shifts organizations from reactive (analyzing what happened) to proactive (detecting what is happening and acting immediately). A supervisor who learns of a line stoppage via a real-time alert can halt production and investigate in minutes; a supervisor who learns via tomorrow's report has already lost thousands in scrap. A supply chain planner with real-time shipment visibility can reroute inventory to avoid shortages; a planner with weekly reporting is always one crisis behind.

SAP's real-time capabilities—CDC with Datasphere, HANA Cloud federation, event-driven architecture, and streaming integration—provide a complete toolkit. The challenge is matching complexity to need. Start with near-real-time (delta extraction, 5-30 minute refresh), prove business value, and expand to event-driven and streaming only where that value justifies additional operational complexity.

Organizations that implement real-time analytics thoughtfully—focused on high-impact decisions, architected for cost efficiency, and operationally sustainable—unlock competitive advantage. Those that over-engineer for speed they don't need, or under-invest in data quality, create complexity without benefit. The latency requirements framework helps you strike the balance.