NNStreamer: Efficient and Agile Development of On-Device AI Systems
We propose NNStreamer, a software system that handles neural networks as filters of stream pipelines, applying the stream processing paradigm to deep neural network applications. A new trend with the wide-spread of deep neural network applications is on-device AI. It is to process neural networks on mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy issues, data transmission costs, and operational costs signify the need for on-device AI, especially if we deploy a massive number of devices. NNStreamer efficiently handles neural networks with complex data stream pipelines on devices, significantly improving the overall performance with minimal effort. Besides, NNStreamer simplifies implementations and allows reusing off-the-shelf media filters directly, which reduces developmental costs significantly. We are already deploying NNStreamer for a wide range of products and platforms, including the Galaxy series and various consumer electronic devices. The experimental results suggest a reduction in developmental costs and enhanced performance of pipeline architectures and NNStreamer. It is an open-source project incubated by Linux Foundation AI, available to the public and applicable to various hardware and software platforms.
Sangjung Woo (Samsung Electronics), Parichay Kapoor (Samsung Electronics), Gichan Jang (Samsung Electronics), Yongjoo Ahn (Samsung Electronics), Jihoon Lee (Samsung Electronics), Dongju Chae (Samsung Electronics), Wook Song (Samsung Electronics), Hyoungjoo Ahn (Samsung Electronics), MyungJoo Ham (Samsung Electronics), Geunsik Lim (Samsung Electronics), Jijoong Moon (Samsung Electronics), Jaeyun Jung (Samsung Electronics),
* IEEE Digital Library: https://www.computer.org/csdl/proceedings-article/icse-seip/2021/386900a198/1sET9mkqoBW
Created with Clowdr: https://clowdr.org/
by ICSE2021 Conference
linux foundation
