Is scaling all you need for AI Large Language Models? Scaling laws and the Inverse Scaling Challenge. Featuring Ian McKenzie, FAR AI Research Scientist
- S4E1
- 47:37
- January 27th 2023
The last couple of years have been an AI model arms race.
The assumption is that the larger the model the better it will perform. But that may not always be the case.
FAR AI Research Scientist Ian McKenzie is a key member of the team organizing the Inverse Scaling Challenge, an initiative set up to investigate scaling laws. We discuss:
- Large Language Models and how they are trained
- The scaling laws and how they are being revised as research and development progresses
- The Inverse Scaling Challenge and its findings
Orchestrate all the Things
Connecting the dots with George Anadiotis: Analyst, Consultant, Engineer, Founder, Host, Researcher, and Writer.
Stories about Tech, Data, AI and Media, and how they flow into each other shaping our lives.
I’ve engaged from the likes of Gary Marcus and Andrew Ng to emerging thinkers and innovators across multiple domains.
My stories have been featured on ZDNet and VentureBeat, and are syndicated across DZone, Hackernoon, Medium and Substack.
Some might call this futurism; let’s just say it’s connecting the dots
Many conversations have a technical focus. Most also examine business perspectives and use cases, while others are socio-technical.
Some are analyses on emerging themes – picking them up early, featuring expert comment, or offering alternative takes.
Others cover breaking news, typically also featuring the people behind them plus some analysis. There are some book reviews as well.
I focus on the connection between data, analytics, data science, graphs, machine learning and AI and their impact on society and business.
I have been covering topics related to:
- AI and Machine Learning
- Data, Analytics and Data Science
- Knowledge Graphs, Graph Databases, Graph AI & Data Science
- Innovation, and a wide array of technologies such as Blockchain, Cloud, Observability, IoT, Open Data and Open Source, Social Media and Software Engineering.
For inquiries, please use https://linkeddataorchestration.com/contact/
Meet the Host
I've got Tech, Data, AI and Media, and i'm not afraid to use them.
My name is George Anadiotis, and i am a writer, a planner and a doer. I am an Onalytica Top 100 Influencer in Big Data and Cloud, a Knowledge Graph expert, and a VentureBeat and ZDNet contributor, among other things.
Linked Data Orchestration is my brand. This podcast is where i share my work, as well as conversations with people who bring interesting news and views to the table.