Chapter 2: How AI Works

Chapter 2: How AI Works

Demystifying the Black Box

“Understanding AI isn’t about mathematics—it’s about mental models. Just as you don’t need to understand color wavelengths to use color theory, you don’t need differential calculus to design with AI.”

The Magic Is Just Math (But Don’t Panic)

Here’s a secret that might anger some engineers: AI is just pattern matching with extra steps. That’s it. No consciousness, no understanding, no genuine intelligence—just very sophisticated pattern recognition. And as a designer, you’re already an expert at patterns.

It’s All About Patterns

Think about how you learned design. You looked at hundreds of websites, apps, and interfaces. Your brain started noticing patterns:

You internalized these patterns so deeply that you can now spot “bad design” instantly, even if you can’t immediately articulate why. That intuition? That’s your biological neural network at work. AI systems do the same thing, just with silicon instead of neurons.

The Core Concept: Every AI system, no matter how complex, follows this basic process:

  1. Observe examples (training data)
  2. Find patterns (model creation)
  3. Apply patterns to new situations (inference)
  4. Get feedback and improve (iteration)

Sound familiar? It’s exactly how you learned design.

Data: The Foundation of Everything

If AI is like cooking, data is your ingredients. Use fresh, quality ingredients, and you might create something amazing. Use spoiled, biased, or insufficient ingredients, and no amount of chef skill will save the dish.

The Metaphor: Imagine teaching someone about “good design” by only showing them websites from 1995. They’d learn that good design means animated GIFs, visitor counters, and “Under Construction” signs. This is exactly what happens when AI trains on biased or limited data.

Quality Over Quantity: Everyone talks about “big data,” but for designers, the real issue is “right data.” Instagram’s Explore algorithm doesn’t need every photo ever taken—it needs photos that represent what users actually engage with.

The Bias Problem: Here’s where your design ethics matter. If you train a hiring AI only on current employees (mostly male engineers), it learns to prefer male candidates. If you train a facial recognition system mostly on white faces, it fails for people of color. As designers, we must ask: “Whose data is missing? Whose experience is excluded?”

Real-world impact: Amazon scrapped an AI recruiting tool because it downgraded resumes containing the word “women’s” (as in “women’s chess club”). The system learned from ten years of resumes—mostly from men—and concluded that male candidates were preferable.

Your Role: You’re the one who designs how data gets collected. Every form field, every permission request, every interaction is a data collection opportunity. You decide:

Where to dive deeper:

Models: The Recipe Book

A model is just a recipe that turns input into output. Feed it an image, get a classification. Feed it text, get a summary. Feed it user behavior, get a prediction.

The Metaphor: Think of models like Instagram filters. Each filter is a set of rules (increase contrast, shift colors, add vignette) that transforms any photo in a consistent way. AI models are similar—they’re transformation rules learned from examples instead of programmed manually.

Pre-trained vs. Custom: This is like choosing between using Bootstrap or creating a custom design system. Pre-trained models (GPT, BERT, ResNet) are like Bootstrap—powerful, general-purpose, ready to use. Custom models are like bespoke design systems—tailored to your specific needs but requiring more resources.

Model Size Matters: Bigger isn’t always better. It’s like image resolution—a 100MB header image might be incredibly detailed, but it’ll kill your load time. Similarly, huge AI models might be more accurate, but they’re slower and more expensive. Your job is finding the right balance for your use case.

Version Control: Models evolve like your designs do. Version 1 might be basic but functional. Version 2 adds features. Version 3 fixes bugs. As a designer, you need to plan for these updates:

Where to dive deeper:

The Training Process: Teaching Your AI

Training an AI is like training a new designer. You don’t just dump all design knowledge on them at once. You start with basics, provide examples, give feedback, and gradually increase complexity.

The Three-Phase Journey:

Training Phase: This is like design school. The AI sees thousands of examples with correct answers. “This is a button users clicked. This is one they ignored.” It adjusts its internal rules to get better at predicting the right answer.

Validation Phase: This is like design critiques. The AI is tested on examples it hasn’t seen before. If it performs well, great! If not, back to training. This prevents “overfitting”—the AI equivalent of designing only for yourself instead of your users.

Testing Phase: This is like user testing. The AI faces real-world scenarios. This reveals the true performance and edge cases you didn’t anticipate.

Overfitting: The Memorization Problem

The Metaphor: Imagine a designer who’s only ever worked on one client’s brand. They become amazing at that specific style but can’t adapt to new challenges. That’s overfitting—learning the training data too specifically instead of learning general principles.

Example: An AI trained to identify cats might memorize specific cats from training images instead of learning “cat-ness.” Show it a new cat, and it fails.

Your Design Response: When you notice AI being suspiciously perfect in demos but failing in real use, suspect overfitting. Design feedback mechanisms to catch this:

Where to dive deeper:

The AI Development Lifecycle: Your New Design Process

The AI development lifecycle parallels the design process you already know:

1. Problem Definition (Discovery Phase)

2. Data Collection (Research Phase)

3. Model Development (Design Phase)

4. Evaluation (Testing Phase)

5. Deployment (Launch Phase)

6. Monitoring (Post-Launch)

Understanding Limitations: The Edges of Possibility

Every design medium has constraints. Screens have edges. Attention has limits. AI has boundaries too.

The Fundamental Limitations:

AI Can’t Understand Meaning: It recognizes patterns, not significance. GPT can write a beautiful poem about loss without understanding sadness. It’s like a color-blind painter who’s learned which tubes of paint people call “red” and “green” but has never seen the colors.

AI Can’t Handle True Novelty: It can recombine learned patterns in new ways but can’t genuinely innovate. It’s like a DJ who can create amazing remixes but can’t compose original music.

AI Lacks Common Sense: It doesn’t understand cause and effect the way humans do. It might suggest ice cream for breakfast because the data shows people are happy when eating ice cream, missing the social context entirely.

AI Has No Ethics: It optimizes for whatever metric you give it, without moral consideration. Tell it to maximize engagement, and it might promote outrageous content. It’s like a navigation app that always chooses the fastest route, even if it goes through a kindergarten playground.

Your Design Challenge: Create interfaces that:

The Designer’s Mental Model

Forget neural networks and backpropagation. Here’s how to think about AI as a designer:

AI is a Prediction Machine: Every AI application is essentially predicting something:

Confidence Isn’t Certainty: When an AI is “90% confident,” it doesn’t mean it’s 90% right. It means that in its training, when it felt this confident, it was right 90% of the time. It’s like a weather forecast—70% chance of rain doesn’t mean 70% of the sky will rain.

Context Is Everything: AI doesn’t truly understand context—it pattern-matches to similar contexts it’s seen. This is why chatbots sometimes give tone-deaf responses and why you need to design context-setting interactions.

Conclusion: From Mystery to Mastery

AI isn’t magic—it’s patterns all the way down. Like typography, color theory, or interaction design, it’s a system with rules, capabilities, and limitations. The mystery isn’t in how it works but in how to use it well.

You don’t need to understand the math any more than you need to understand the physics of light to use color effectively. What you need is a mental model—a designer’s understanding of this new material.

Here’s what you’ve learned:

But more importantly, you’ve learned that AI development mirrors the design process you already know. The vocabulary is different, but the journey is familiar: understand the problem, gather insights, create solutions, test with users, iterate based on feedback.

The black box isn’t so black anymore. It’s translucent—you can see the shapes inside even if the details remain fuzzy. And that’s enough. Because your job isn’t to build the box; it’s to design how humans interact with it.

You’re not becoming an AI engineer. You’re becoming something more valuable: a designer who can bridge the gap between human needs and AI capabilities. You’re learning to design not just for AI, but with AI as a design material.

The next chapter is where it gets exciting. Now that you understand what AI is and how it works, let’s explore how to design experiences that make AI truly useful for humans.