Anveshak: A Platform for Distributed Video Analytics
The push for smarter and safer cities has led to the proliferation of video cameras in public spaces. Regions like London, New York, Singapore and China, have deployed camera-networks with 1000’s of feeds to help with urban safety, e.g., to detect abandoned objects, track miscreants, and for behavioral analysis. They are also used for citizen services, e.g., to identify open parking spots or count the traffic flow. Such many-camera networks, when coupled with sophisticated Computer Vision (CV) algorithms and Deep Learning (DL) models, can also serve as meta-sensors to replace other physical sensors for IoT applications and to complement on-board cameras for self-driving cars.
One canonical application domain to consume such ubiquitous video feeds is called tracking, where suspicious activities in public spaces need to be detected and followed by law enforcement to ensure safety. Here, the goal is to identify a target (e.g., a vehicle or a person of interest), based on a given sample image or feature vector, in video streams arriving from cameras distributed across the city, and to track that target’s movements across the many-camera network. Contemporary many-camera surveillance platforms are monolithic, proprietary and bespoke. They also lack tunable adaptivity and scaling.
Anveshak is a framework designed to address the shortcomings of existing surveillance platforms. We propose a novel domain-specific dataflow model with functional operators to plug-in different analytics, for current and emerging tracking applications. We design domain-sensitive heuristics for frame drops and batching, which enable users to tune accuracy, latency and scalability under dynamism. We implement the dataflow model and heuristics in our Anveshak platform to execute across distributed resources. We scale Anveshak to track users across a 1000 camera network using 10; 8 core 32 GB Virtual Machines, which is a 25x improvement over the baseline approach
Khochare A., and Simmhan S. “A scalable and composable analytics platform for distributed wide-area tracking.” PhD Forum, 20th International Conference on Distributed Computing and Networking. ACM, 2019.
Khochare A., and Simmhan Y. “A Scalable Framework for Distributed Object Tracking across a Many-camera Network.” arXiv preprint arXiv:1902.05577 (2019).
- Khochare A., Sheshadri K., Shriram R., and Simmhan Y. “Dynamic Scaling of Video Analytics for Wide-area Tracking in Urban Spaces.” To appear in 12th International SCALE Challenge 2019.
Resilient Storage on Distributed Edge Resources
Sheshadri K R and Sumit Monga
Edge and fog computing have grown popular as IoT deployments become wide-spread. While application composition and scheduling on such resources are being explored, there exists a gap in a distributed data storage service on the edge and fog layer, instead depending solely on the cloud for data persistence. Such a service should reliably store and manage data on fog and edge devices, even in the presence of failures, and offer transparent discovery and access to data for use by
edge computing applications.
Elf-store is an edge located federated data storage service, it provides reliable storage for streams of micro-batches of data. Edges host the actual data blocks, we present a novel technique to select edges on which to replicate blocks to guarantee a minimum reliability. In addition to this, the system supports recovery. The Edge resources are monitored by reliable Fog devices, which are like super-peers and form an overlay network. The fog layer provides metadata indexing using bloom-filters, locates data within 2 hops and maintains approximate global statistics about reliability and storage capacity of the Edge resources. Our experiments on two IoT virtual deployments with 20 and 272 devices show that ElfStore has low overheads, is bound only by the network bandwidth, has scalable performance, and offers tunable resilience.
- ElfStore: A Resilient Data Storage Service for Federated Edge and Fog Resources, Monga, S. K., Sheshadri K, R and Simmhan, Y., IEEE International Conference on Web Services (ICWS), 2019, pp. 1-9 (To Appear)
CoFEE: Distributed Orchestration over Edge, Fog and Cloud Resources
Prateeksha Varshney, Shayal Chhabra and Shriram R
The Internet of Things (IoT) has manifested in large scale deployment of sensors, actuators and computing devices at the edge of the network. These are complemented by Fog resources that offer federated infrastructure management and accelerated computing power. Analytics over the streams of data from the Edge devices lead to efficient transportation and utility management, healthcare and safety. These applications tend to be declarative and latency-sensitive.
There is a potential opportunity to leverage the availability of Edge and Fog computing resources that can supplement Cloud resources to perform cheaper and low latency analytics. The key challenge is that the Edge resources are prone to failures due to network, mobility or limited power capacity. Also, each resource have different computational constraints and associated costs. This motivates the need for reliable and efficient strategies for computation on Edge and Fog resources.
CoFEE stands for “A Scheduler for Cloud, Fog and Edge Execution”. In this work, we have proposed a novel declarative trigger-based dataflow execution model over data streams from wide-area sensor sources. A scalable runtime engine for resilient distributed execution of these dataflows on unreliable Edge, reliable Fog and Cloud resources has been developed which guarantees runtime strategies that could meet stringent deadlines for the dataflows. The engine also attempts to minimize the monetary cost for resources and can tolerate Edge device failures.
CoFEE uses a hierarchical model of Edge, Fog and Cloud resources for easier manageability. It performs just in time task placements on the resources chosen by our novel scheduling algorithm. We have also proposed several indexing strategies to efficiently locate the available data on the network. We have performed detailed experiments to demonstrate the scalability, resilience and lower costs of CoFEE relative to other baseline scheduling strategies.
- Under review