Matrices.
Four measurements in. One score out. One operation in between.
Predict first
Patient three is the heaviest of the four, yet has the lowest risk. Could ‘multiply each measurement by a weight, and add’ produce that?
The recipe
Four measurements in. Find the weights.
Each risk score is every measurement times its own weight, added up. Turn the four dials until the scores match.
Turn the four dials until every score matches its risk.
Total miss 1.590
Turning dials until the predictions match: that is what a model does when it learns. Lecture 4 finds the dials without guessing.
Columns
A x is a journey. One column at a time.
Walk x₁ steps along the first column, then x₂ along the second. Then let A move every point at once.
Drag the tips of the columns
128 recipes
One multiplication. 128 recipes at once.
The first layer of the real network: 128 columns, each a 28 × 28 picture. Draw a digit and watch every one of them score it.
x1 × 784×W784 × 128=s1 × 128
Ink: positive weights. Blue: negative. Tap any recipe.
Add up all 784 entries of the last picture, and you have it.
100,352 multiplications in one product.The rest of the network turns the 128 scores into a point in the learned space. It says: –
Digits from MNIST, by Yann LeCun, Corinna Cortes and Christopher J. C. Burges (CC BY-SA 3.0). A real layer also adds a bias and bends each score; shown here is the product alone.
Column pickers
A single 1 picks a column. Build the rest.
Each column of B holds the weights for A’s columns. Build B to meet each goal.
The shape gate
Three views
By columns, by rows, by entries. One answer.
Build the same product three ways, and watch every entry come out the same.
The row lens on real data: a row of weights on the left blends patients
The race
Billions a second. Why AI runs on graphics cards.
Multiply two big matrices on this device twice: one entry after another on its processor, every entry at once on its graphics card.
multiply-adds
Your processor
One entry after another
Your graphics card
Every entry at once
Real timings on this device. The processor runs plain JavaScript on one core; the graphics card runs one small program per entry.
Prove it
Four questions. The reasons matter more than the score.
Question 1 of 4
A is 3 × 4. For A x to exist, what shape must x be?
Question 2 of 4
Which matrix P swaps the two columns of a 2 × 2 matrix A, in A P?
Question 3 of 4
What do you get if you transpose a matrix twice?
Question 4 of 4
The patient table has one patient per row. What does the row vector (½, ½, 0) times the table give?