DSS online #4 : End-to-End Deep Learning Deployment with ONNX
End-to-End Deep Learning Deployment with ONNX
A deep learning model is often viewed as fully self-contained, freeing practitioners from the burden of data processing and feature engineering. However, in most real-world applications of AI, these models have similarly complex requirements for data pre-processing, feature extraction and transformation as more traditional ML models.
Any non-trivial use case requires care to ensure no model skew exists between the training-time data pipeline and the inference-time data pipeline. This is not simply theoretical – small differences or errors can be difficult to detect but can have dramatic impact on the performance and efficacy of the deployed solution. Despite this, there are currently few widely accepted, standard solutions for enabling simple deployment of end-to-end deep learning pipelines to production.
Recently, the Open Neural Network Exchange (ONNX) standard has emerged for representing deep learning models in a standardized format. While this is useful for representing the core model inference phase, we need to go further to encompass deployment of the end-to-end pipeline. In this talk Nick introduced ONNX for exporting deep learning computation graphs, as well as the ONNX-ML component of the specification, for exporting both “traditional” ML models as well as common feature extraction, data transformation and post-processing steps. He covered how to use ONNX and the growing ecosystem of exporter libraries for common frameworks (including TensorFlow, PyTorch, Keras, scikit-learn and Apache SparkML) to deploy complete deep learning pipelines. Finally, I will explore best practices for working with and combining these disparate exporter toolkits, as well as highlight the gaps, issues and missing pieces to be taken into account and still to be addressed.
Nick Pentreath (Open Source Developer, Developer Advocate) – Principal Engineer, IBM CODAIT – Nick is a Principal Engineer at IBM. He is an Apache Spark committer and PMC member and author of Machine Learning with Spark. Previously, he co-founded Graphflow, a startup focused on recommendations and customer intelligence. He has worked at Goldman Sachs, Cognitive Match, and led the Data Science team at Mxit, Africa’s largest social network. He is passionate about combining commercial focus with machine learning and cutting-edge technology to build intelligent systems that learn from data to add business value.
Full material : http://www.datascienceseed.com/2020/10/04/dss-online-4-tech-ethics-for-the-open-source-ai-the-linux-foundation-ai/
linux foundation
