Cosmic Conversations: Through AI with Ilya Sutskever and Jensen Huang
Exploring the AI Universe with NVIDIA's GTC Spring 2023 Fireside Chat
Today, we embark on a cosmic voyage through the AI universe, guided by two brilliant minds: Ilya Sutskever, Co-Founder and Chief Scientist at OpenAI, and Jensen Huang, Founder and CEO of NVIDIA. Video portal here.
Our mission: Delve deep into their recent fireside chat at NVIDIA's GTC Spring 2023 conference and extract valuable insights into the current state and future vision of AI.
Compression: The Key to AI's Black Hole
The chat began with the revelation that compression is the key to AI. Compression allows AI models to digest the vast universe of data, making sense of its complexities and drawing meaningful insights. In my mind, this is not very different from what humans do at a fundamental level (but the details are different). Humans just adjust our ideas to the feeling we get from a thought or action.
Large Data and Reinforcement Learning: Fueling AI's Exploration
Large-scale data and reinforcement learning are the rocket fuel for AI's journey. With GPT models, pretraining creates a “world” model from statistical learning. Then, reinforcement learning with human feedback (RLHF) is used to guide the model to the results the creator desires, much like parents guiding their children.
ChatGPT vs. GPT-4: AI Detectives in the Tech Galaxy
Ilya shed light on the difference between ChatGPT and GPT-4.
The foundation of GPT-4 predicts the next word better. He illustrates this with a thought experiment.
Imagine AI as a detective, tasked with predicting the culprit in a murder mystery. This is what AI is doing, predicting the next word based on its statiscal understanding of all the sequences of words (or really mathematical vectors) it has been trained on. Ilya hinted that this is potentially possible — if not already. This powerful capability can only be harnessed with the combination of fine-tuning and RLHF. This propelled me into the parallel universe of Minority Report. What do you think?
Reasoning, Images, and AI: A Cosmic Trifecta
The chat ventured into the realm of reasoning and whether neural nets can truly reason. Although the concept of what “reason” is remains fuzzy, GPT models excel at image processing and using its language model to “understand” the scene and task. This is no small task! Ilya confirmed that GPT — with vision — understands complex aspects of the world that was not been possible with only text, such as physics, by learning from videos.
Multi-Modality: The AI Game Changer
They both highlighted the importance of multi-modality — communication practices in terms of the textual, aural, linguistic, spatial, and visual resources — in AI. The ability to process and understand multiple data types, such as text, images, and videos, is a game changer for AI's potential impact.
AI Training Itself: The Final Frontier?
Can AI train itself?! While Ilya believes it's possible, he is more focused on consuming more human-made data. The universe of human data is vast, and AI has yet to reach its outer limits. OpenAI plans to feed the machine an “all it can eat buffet” and try to use bumper rails to guide the monster to benevolence.
GPT-4 Surprises: A Reliable AI Companion
What surprised Ilya about GPT-4?… Its astounding reliability. GPT-4 follows instructions extremely well, making it a valuable companion for humans. But, the concept of steerablility is one of the most complex problems for even human-to-human interactions. How do you teach the concept of love and human morals (and specifically who has the right morals to teach it)?
As we conclude our cosmic voyage with Ilya Sutskever and Jensen Huang, it's clear that the AI universe is vast, mysterious, and full of potential.
I will do a deep dive, in a detailed post, on OpenAI, GPT-4, and the significant impact of multi-modality AI models are and will continue to have on technology, work, creativity, and, in general, our lives. I will also provide a guide on how to use these technologies to rocket you into the future.
Thanks for this great informative read! I look forward to following your deep dive post on OpenAI!