Foundations
Start your journey into local AI inference
3 modules8 lessons24 questions
01 What is Local Inference?
AI On Your Device
3 qImagine having a super-smart robot brain living right inside your phone! 🤖 It doesn't need to text anyone or use the internet to know the answers to your questions.
Cloud vs Local
3 qCloud AI is like asking a genius across the world for help via mail. 🌍 Local AI is like having that genius sit next to you in the room. 🧑🏫
Why Local AI Matters
3 qYour secrets stay with you! 🤫 You don't have to pay a toll every time you ask a question, and it works even when you're on an airplane. ✈️
02 The Three Axes
Capacity — What Fits
3 qYou can't fit an elephant into a Mini Cooper! 🐘🚗 Models take up space, and your computer needs enough RAM (memory) to hold the whole model before it can run.
Bandwidth — How Fast
3 qImagine drinking a milkshake through a tiny straw. 🥤 It doesn't matter how fast you can swallow; the straw slows you down. That's memory bandwidth!
The Software Stack
3 qYou wouldn't use a screwdriver to hammer a nail! 🛠️ Different AI tasks need different software tools to run smoothly on your computer.
03 Your First Mental Model
Tokens — The AI's Alphabet
3 qAI doesn't read words like we do. It chops words into Lego blocks called 'tokens'. 🧱 'Apple' might be one token, but 'Supercalifragilistic' might be five!
Parameters & Weights
3 qParameters are the AI's brain cells! 🧠 When an AI learns, it adjusts billions of tiny knobs (weights) until it gets the right answers.