We determined user activity based on accelerometer activity to accurately associate them with individual visits to locations and time spent per location. This led us to create a solution for iOS and android which used less than 1% of the battery per day. Once the tracking SDK was deployed in enterprise apps, the volume of incoming data required scaling to many machines and a distributed database to handle it in real-time. We used innovative processing approaches to minimize AWS costs, including moving some processing to client-side code and clever data schemas in an elasticsearch database for queries.
Data was ingested via an AWS Firehose stream and processed from S3 in chunks and aggregated into an elasticsearch database. By using a custom, open street maps geocoding implementation, geo-clustering, and machine learning, data was able to generate consumer profiles and be accurately mapped to real-world locations. We built a custom analytics dashboard powered by this data in elasticsearch that allowed real-time aggregations, filtering, and data exploration.
Upon deployment, we were able to deliver a data-processing and machine-learning engine for Cintric. It was designed to process trillions of data points in real-time, in order to deliver consumer insights for retail brands. Users gained access to a powerful dashboard that can measure real-world impact of marketing campaigns, prevent store cannibalization, compare performance to competitors, and discover optimal locations to open and close.