High Speed low latency real-time data processing and analytics for IOT data for a Large US Retailer

Problem Statement

  • Real-time streaming data from IOT sensors monitoring Cooking equipment like fryers and ovens, Hot and Cold Storage containers, Transport vehicles that are continuously sending data on Power, Temperature, HVAC, etc
  • High Speed low latency real-time data processing and analytics for IOT data
  • Order of 100K messages per second from IOT Devices for monitoring food storage and food processing in hot and cold environments, 24 x 365
  • Need for real-time parsing, enrichment, predictive monitoring and deliver data in near-real time to Advanced analytics and Data Science users.

Solution Overview

  • Real-time Data processing layer using Kafka, Spark, Cassandra, Redis
  • Data enrichment to add master data attributes, tagging for asset and process, Business rules-based Data Quality checks.
  • Real time Data Aggregation to build calculated aggregate metrics, Time window functions, moving averages and checks with Thresholds and allow Search and real time dashboard functionality
  • Analytics Layer for near-real time data analysis, AI/ML modeling
  • Persistent Data Storage on Cassandra for historical Data Analysis.

Outcomes

  • Seamless ingestion, processing of real time data with high volume/second
  • Technology agnostic robust architecture for high performance, scalability
  • Near real-time Data enrichment, in-process analytics, monitoring dashboards and enabling AI/ML based advanced analytics.
High Speed low latency real-time data processing
case studies

See More Case Studies

Contact us

Partner with Us for Comprehensive IT Resources

We’re happy to answer any questions you may have and help you determine which of our services best fit your needs.

Your benefits:
What happens next?
1

We Schedule a call at your convenience 

2

We do a discovery and consulting meting 

3

We prepare a proposal 

Schedule a Free Consultation