Full Session Description

Walk into a store, grab the items you want, and walk out without having to interact with a cashier or even use a self-checkout system. That’s the no-hassle shopping experience showcasing the AI-powered checkout pioneered by Standard Cognition. The company makes use of Computer Vision to remove the need for checkout lines of any sort in physical retail locations. Streaming Vision-Data is converted into a stream of metadata and the need of the hour is to be able to do continuous analytics on this metadata. The variety and velocity of this metadata stream is very high and requires special purpose analytics tools that are geared for real-time analytics. Application developers need to be able to prototype rapidly on this metadata so that they can try out different analytical models quickly. This talk describes how Standard Cognition uses Rockset for rapid prototyping of application models on vision data. In specific, we first discuss multiple challenges associated with analysis of vision data, why a traditional database was insufficient for our needs and why we chose a realtime database to address the following challenges:* Describe the three flavors of velocity of vision data and the policies of keeping high frequency (~ 500 Hz) data on the store premises, immediately processing low-frequency (~ 5Hz) data in the cloud and streaming medium frequency (~ 50 Hz) data for realtime analytics * Describe why and how the schema of the generated metadata changes from day to day, which means that analytical tools we use need to be able to handle very frequent schema changes. These changes are typically the addition of new columns, columns with mixed types, complex objects inside a column, etc* Describe how we created an application-developer platform REST-api by encapsulating complex analytical SQL queries within Query Lambdas. This allows our application developer to rapidly iterate and build data powered applications on production data sets. We share with you the workflow we have created for analytical processing of vision data, the salient features of that workflow, and its uniqueness compared to traditional data processing systems.

Dhruba Borthakur

CTO @ Rockset

Tushar Dadlani

Computer Vision Engineering Manager @ Rockset

About the authors

Dhruba Borthakur is CTO and co-founder of Rockset, responsible for the company’s technical direction. He was an engineer on the database team at Facebook, where he was the founding engineer of the RocksDB data store. Earlier at Yahoo, he was one of the founding engineers of the Hadoop Distributed File System. He was also a contributor to the open source Apache HBase project. Dhruba previously held various roles at Veritas Software, founded an e-commerce startup, Oreceipt.com, and contributed to Andrew File System (AFS) at IBM-Transarc Labs.

Tushar Dadlani is a computer vision engineering manager, building the next-generation of perception algorithms at Standard Cognition. Tushar was previously the founding CTO of Explorer.ai where he built maps to enable the perception of self-driving cars.