Journey To The Center Of AI [CL80]
Journey To The Center Of AI
Episode: 80
Published: 10/31/2023
How to AI
AI has been in the headlines almost constantly for the last 18 months. But despite all that fanfare, ChatGPT���ing, and Faces being hugged; I have to admit that I still don���t know how it all works. Not that anyone really does, but beyond the prompt text I submit to a Chats GPT or Dalls ‘E’, I don���t know what is happening in the background. What is the software stack powering GPT 4? What type of operating system, orchestrator, and applications allow for the massive training models behind OpenAI? And what about the hardware? I know there���s a lot of GPUs, but there���s got to be more to it than that. I decided to dig into some of these layers to try and trace from the prompt down to the physical servers, and you get to come with me. So get in loser, we���re going to learn how to AI.
• I bet OpenAI has published something about their stack… sorta (https://openai.com/research/gpt-4)
• According to a post from 2016, OpenAI was using Python, TensorFlow, Numpy, Keras, and Anaconda (https://openai.com/research/infrastructure-for-deep-learning)
• TensorFlow is a machine learning library with APIs available for Python and C++ (https://github.com/tensorflow/tensorflow)
• Numpy is a Python package developed to support scientific computing (https://numpy.org/)
• Flyte, a graduated project from the Linux Foundation that leverages Kubernetes (https://github.com/flyteorg/flyte)
• Nvidia H100 Spec Sheet (https://resources.nvidia.com/en-us-tensor-core/gtc22-whitepaper-hopper)
• DGX SuperPod (https://docs.nvidia.com/https:/docs.nvidia.com/dgx-superpod-reference-architecture-dgx-h100.pdf)
• Tensor Cores and CUDA cores, what is the difference? (https://www.makeuseof.com/nvidia-cuda-cores-vs-tensor-cores-whats-the-difference/)
• Nvidia helpfully posted some example code using the cuDNN library (https://developer.nvidia.com/blog/programming-tensor-cores-cuda-9/)
• The Infiniband standard is a different protocol than Ethernet (https://community.fs.com/article/infiniband-vs-ethernet-which-is-right-for-your-data-center-network.html)
• Ultra Ethernet Consortium is focused on developing an open standard that meets or exceeds what Infiniband does today (https://ultraethernet.org/)
• Remember when Crypto was using as much power as the country of Argentina (https://www.pbs.org/wnet/peril-and-promise/2022/08/cryptocurrency-massive-energy-consumption/)
• AI is already on pace to match that by 2027 (https://www.nytimes.com/2023/10/10/climate/ai-could-soon-need-as-much-electricity-as-an-entire-country.html)
• Microsoft���s AI for Beginners site to get down some of the terminology and lab time (https://microsoft.github.io/AI-For-Beginners/)
• You may also want to take a beginner course on Python (https://www.learnpython.org/)
Intro and outro music by James Bellavance copyright 2022
by Ned in the Cloud
linux foundation
