Vectors and matrices

Welcome to multivariable calculus! There are a lot of exciting and fundamentally visual topics ahead. In order to best understand these ideas, there are two main prerequisites we recommend before starting:

• Single-variable calculus (such as AP Calculus AB/BC)
• Vectors and matrices

If you've already seen these, but you still want to quickly brush up on the essentials before beginning multivariable calculus, then this article is here to highlight a few of the more crucial skills that you should have going into the course.

Calculus

The biggest prerequisite for multivariable calculus is good old single-variable calculus. (Now that we're in multivariable land, we need this new adjective "single-variable" to keep track of which version we're talking about.) Specifically, be sure that you are comfortable with the following broad concepts:

• Limits
• Differentiation (including the power rule, product rule, quotient rule, and chain rule)
• What the derivative represents
• Critical points and optimization
• Integration and the Fundamental Theorem of Calculus
• Parametric equations and polar coordinates

Feel free to spend a little time reviewing any of these topics. The preparation will pay off as we get into multivariable calculus.

Vectors and matrices

The second big prerequisite for multivariable calculus is vectors and matrices. Both of these topics are super useful, because they let us talk about multi-dimensional coordinates and sometimes entire transformations with just one object, which we can then manipulate.

Here are the six concepts that we'll need:

• Vectors
• Dot products
• Cross products
• Matrices, intro
• Visualizing matrices
• Determinants

These concepts aren't always taught prior to taking single-variable calculus, so it's completely fine if some of them feel new. Whether you need to learn it for the first time or just want to brush up, each concept links to a review article that introduces the idea and gives practice using it. If you still feel uncomfortable after reading a review article, it will have links to other content that will help you learn.

Vectors are the building blocks of everything multivariable. We use them when we want to represent a coordinate in higher-dimensional space or, more generally, to write a list of anything. In this article, we'll cover what vectors are, different ways to write them, and the three basic vector operations.

What is a vector?

Most generally, a vector is a list of things. In multivariable calculus, "thing" typically ends up meaning "number," but not always. For example, we'll see a vector made up of derivative operators when we talk about multivariable derivatives. This generality is super useful down the line.

Vectors and points in space

When a vector is just a list of numbers, we can visualize it as an arrow in space. For example, we visualize the vector $(4,2)$ as an arrow whose tail is at the origin and whose tip is at the point $(4, 2)$. For this reason, we don't usually distinguish between points and vectors in multivariable calculus.

However, sometimes we draw a vector with the tail away from the origin. This changes nothing about the vector, only where we draw it. For example, we can also draw the vector $(4,2)$ starting at $(0, 2)$. Both arrows correspond to the vector $(4,2)$, even though one of them doesn't end at the point $(4,2)$.

This is why it can sometimes be confusing to write vectors exactly like points in space. For this reason, people have come up with other notations for vectors.

Notation

There are lots of ways to write vectors. Here are the three we'll use most in this course. The little arrow on top of $\vec{v}$ is a convention that indicates that $\vec{v}$ refers to a vector.

$\begin{align}
\vec{v} &= (1, 2, 3) = \left[ \begin{array}{c} 1 \ 2 \ 3 \end{array} \right] = 1 \blueD{\hat{\imath}} + 2 \maroonD{\hat{\jmath}} + 3 \greenD{\hat{k}}
\end{align}$

The first notation is what we discussed earlier. Technically it refers to a point, but we use it interchangeably to refer to a vector. This notation extends to any number of dimensions.

The second notation is matrix notation, which we can also extend to as many dimensions as we want. Matrix notation is particularly useful when we think about vectors interacting with matrices. We'll discuss matrices and how to visualize them in coming articles.

The third notation, unlike the previous ones, only works in 2D and 3D. The symbol $\blueD{\hat{\imath}}$ (pronounced "i hat") is the unit $x$ vector, so $\blueD{\hat{\imath}} = (1, 0, 0)$. Similarly, $\maroonD{\hat{\jmath}} = (0, 1, 0)$ and $\greenD{\hat{k}} = (0, 0, 1)$. This notation might make more sense once we cover vector addition.

In this course, we almost always use the $(1, 2, 3)$ notation in exercises, because it saves space when we need to define multiple variables. The videos use a mix of matrix notation and the $1 \blueD{\hat{\imath}} + 2 \maroonD{\hat{\jmath}} + 3 \greenD{\hat{k}} $ notation.

Addition

One of the basic vector operations is addition. In general, whenever we add two vectors, we add their corresponding components:

$(a, b, c) + (A, B, C) = (a + A, b + B, c + C)$

This works in any number of dimensions, not just three. We can visualize the sum $\blueD{\vec{a}} + \maroonD{\vec{b}}$ as sliding the tail of $\maroonD{\vec{b}}$ to the tip of $\blueD{\vec{a}}$. Here's an example of that in 2D.

Let's try a practice question.

[[☃ graded-group 1]]

If you want to go deeper, learn about how to understand vector addition visually with this video, and get practice with this exercise.

Scalar multiplication

The second basic vector operation is scalar multiplication, which is when we stretch or shrink a vector. Scalar is just a fancy word for number (same root as the word scaling). Here's an example of how it works:

$\begin{align}
\vec{b} &= (1, 2, 3) \ \
2\vec{b} &= (2, 4, 6) \ \
0.5\vec{b} &= (0.5, 1, 1.5) \ \
-\vec{b} &= (-1, -2, -3)
\end{align}$

In general, scaling a vector by a number means multiplying each of the vector's components by that number. That means

$\begin{align}
x \vec{a} = x (a, b, c) = (xa, xb, xc)
\end{align}$

Let's try an example.

[[☃ graded-group 1]]

The intuitive meaning of scaling a vector by a factor of $2$ is that we are making the vector twice as long. Here's what that looks like:

Scaling by a factor of $0.5$ makes a vector half as long. This would look like the above vector $(2, 4)$ becoming $(1, 2)$, instead of the other way around.

Scaling by a factor of $-1$ means flipping the direction of a vector, because each of its components becomes the opposite of what it used to be. Here's an example of what that looks like:

If you want to go deeper, check out this video, and get practice with this exercise.

Magnitude

When we visualize vectors as arrows, a natural question to ask might be, "How long is it?" The magnitude of a vector answers this question. We write the magnitude of a vector with double bars on both sides, or sometimes with just single bars: $| \vec{a} |$ or $| \vec{a} |$.

We calculate the magnitude with the Pythagorean theorem, because we can think of a vector as the hypotenuse of a triangle. This is equivalent to using the distance formula, so the magnitude of the vector $(a, b)$ is $\sqrt{a^2 + b^2}$.

Let's try an example.

[[☃ graded-group 1]]

Magnitude works the same in 3D and in higher dimensions.

[[☃ graded-group 2]]

If you want to go deeper, check out this video, and get practice with this exercise

What's next

Beyond addition, scalar multiplication, and magnitude, there are two more important operations between vectors. These are the dot product and the cross product, and we will cover them in the next two articles.

The dot product is a fundamental way we can combine two vectors. Intuitively, it tells us something about how much two vectors point in the same direction.

Definition and intuition

We write the dot product with a little dot $\cdot$ between the two vectors (pronounced "a dot b"):

$\vec{a} \cdot \vec{b} = | \vec{a} | | \vec{b} | \cos(\theta)$

If we break this down factor by factor, the first two are $| \vec{a} |$ and $| \vec{b} |$. These are the magnitudes of $\vec{a}$ and $\vec{b}$, so the dot product takes into account how long vectors are. The final factor is $\cos(\theta)$, where $\theta$ is the angle between $\vec{a}$ and $\vec{b}$. This tells us the dot product has to do with direction.

Specifically, when $\theta = 0$, the two vectors point in exactly the same direction. Not accounting for vector magnitudes, this is when the dot product is at its largest, because $\cos(0) = 1$. In general, the more two vectors point in the same direction, the bigger the dot product between them will be.

When $\theta = \dfrac{\pi}{2}$, the two vectors are precisely perpendicular to each other. This corresponds to the dot product between them being $0$, because $\cos\left( \dfrac{\pi}{2} \right) = 0$.

It's also possible for a dot product to be negative if the two vectors are pointing in opposite directions, which is when $\dfrac{\pi}{2} < \theta < \dfrac{3\pi}{2}$.

Another way to think about $\theta$ is to imagine one vector dropping a shadow on to the other. When the angle is small, the shadow lands far from the origin and the dot product is large.

When $\theta$ is close to $\dfrac{\pi}{2}$, the shadow lands close to the origin and the dot product is small.

Keep in mind that the dot product of two vectors is a number, not a vector. That means, for example, that it doesn't make sense to ask what $\vec{a} \cdot \vec{b} \cdot \vec{c}$ equals. Once we evaluated $\vec{a} \cdot \vec{b}$ to be some number, we would end up trying to take the dot product between a number and a vector, which isn't how the dot product works.

A better way to compute the dot product

When we need to find a dot product in multivariable calculus, we typically have only the coordinates of $\vec{a}$ and $\vec{b}$. Calculating $| \vec{a} | | \vec{b} | \cos(\theta)$ would force us to find two square roots and a cosine, which is a lot of work! Luckily, there is an easier way. Just multiply corresponding components and then add:

$\begin{align}
\vec{a} &= (a_1, a_2, a_3) \ \
\vec{b} &= (b_1, b_2, b_3) \ \
\vec{a} \cdot \vec{b} &= a_1b_1 + a_2b_2 + a_3b_3
\end{align}$

Although the example above features 3D vectors, this formula extends for vectors of any length.

[[☃ explanation 1]]

This makes calculating dot products straightforward if you know the components of each vector.

$\begin{align}
&\vec{v} = 4 \hat{\imath} - 1 \hat{\jmath} + 2 \hat{k} \ \
&\vec{w} = 1 \hat{\imath} + 2 \hat{\jmath} + 4 \hat{k} \ \
&\vec{v} \cdot \vec{w} = 4 \cdot 1 + (-1) \cdot 2 + 2 \cdot 4 = 10
\end{align}$

Let's try an example.

[[☃ graded-group 1]]

Even though we now have a nice formula in terms of coordinates, the intuition behind the dot product remains how it measures relative direction. Try to predict the sign of the dot product based on just a picture.

[[☃ graded-group-set 1]]

That's all we'll need to know about dot products for now. If you want to learn more, check out this video.

What's next

Now that we've covered the dot product, there's just one more vector operation to learn about: the cross product. As we'll see, the cross product nicely complements the dot product but is slightly more limited.

Like the dot product, the cross product is an operation between two vectors. Before getting to a formula for the cross product, let's talk about some of its properties.

Properties of the cross product

We write the cross product between two vectors as $\vec{a} \times \vec{b}$ (pronounced "a cross b"). Unlike the dot product, which returns a number, the result of a cross product is another vector. Let's say that $\vec{a} \times \vec{b} = \vec{c}$. This new vector $\vec{c}$ has two special properties.

First, it is perpendicular to both $\vec{a}$ and $\vec{b}$. Phrasing this in terms of the dot product, we could say that $\vec{c} \cdot \vec{a} = \vec{c} \cdot \vec{b} = 0$. This property alone makes the cross product quite useful. This is also why the cross product only works in three dimensions. In 2D, there isn't always a vector perpendicular to any pair of other vectors. In four and more dimensions, there are infinitely many vectors perpendicular to a given pair of other vectors.

Second, the length of $\vec{c}$ is a measure of how far apart $\vec{a}$ and $\vec{b}$ are pointing, augmented by their magnitudes.

$| \vec{c} | = | \vec{a} | | \vec{b} | \sin(\theta)$

It's similar to the dot product, but instead of $\cos(\theta)$ the cross product uses $\sin(\theta)$, where $\theta$ is the angle between $\vec{a}$ and $\vec{b}$. That way, when the angle is $90$ degrees, the cross product is at its largest. In this sense, the dot product and the cross product complement each other.

There's one interpretation of the length of $\vec{c}$ that is particularly useful. Think of the parallelogram formed by $\vec{a}$ and $\vec{b}$. The base of this parallelogram has length $| \vec{a} |$, and the height has length $| \vec{b} | \sin(\theta)$. That means the area of the parallelogram in total is precisely the magnitude of the cross product.

[[☃ image 1]]

The right-hand rule

Notice that in the image above the cross product is perpendicular to $\vec{a}$ and $\vec{b}$, as expected. But there are actually two vectors that could be perpendicular to $\vec{a}$ and $\vec{b}$. If $\vec{c} = \vec{a} \times \vec{b}$, then these two choices are $\vec{c}$ and $-\vec{c}$. How do we decide which of the two perfectly valid choices is the cross product?

We have a convention called the right-hand rule to resolve this ambiguity. If you hold up your right hand, point your index finger in the direction of $\vec{a}$, and point your middle finger in the direction of $\vec{b}$, then your thumb will point in the direction of $\vec{a} \times \vec{b}$.

[[☃ image 3]]

It's arbitrary that we define the cross product with the right-hand rule instead of a left-hand rule, but by using this convention the cross product no longer has any ambiguity.

The not so pretty formula

The most important takeaway from the cross product should be its properties, not its formula. But sometimes we do need to compute a cross product. Unfortunately, the formula for the cross product is not as nice as it was for the dot product. When we get to the article on determinants, we'll see a nicer way to remember the formula for the cross product. For now:

$\begin{align}
\vec{a} &= (a_1, a_2, a_3) \ \
\vec{b} &= (b_1, b_2, b_3) \ \
\vec{a} \times \vec{b} &= \begin{bmatrix}
a_2b_3 - a_3b_2 \ \
a_3b_1 - a_1b_3 \ \
a_1b_2 - a_2b_1
\end{bmatrix}
\end{align}$

Let's try an example using the formula.

[[☃ graded-group 1]]

Now let's see one of those properties we discussed in action.

[[☃ graded-group 2]]

Cross product vs. dot product

When we compare the dot product and the cross product, there are three main differences.

1. The dot product returns a number, but the cross product returns a vector.

2. The dot product works in any number of dimensions, but the cross product only works in 3D.

3. The dot product measures how much two vectors point in the same direction, but the cross product measures how much two vectors point in different directions.

This is all we need to know about cross products for now. If you want to learn more, check out this video.

What's next

Now that we have a solid foundation in vectors and the ways we can combine them, the last topic we'll cover is matrices. The next three articles will describe what matrices are, how to visualize them, and a useful property they have called the determinant.

In addition to vectors, matrices are a fundamental way we can talk about higher-dimensional space. They will show up all over multivariable calculus, so let's get into it.

What is a matrix?

A matrix is an array of numbers that we surround with square brackets. The dimension of a matrix is how many rows and columns it has, which we write as $\text{rows} \times \text{columns}$. For example, here is a $2 \times 3$ matrix (pronounced "two by three"). The convention is to use uppercase letters for a variable that is a matrix.

$A = \left[ \begin{array}{ccc} \blueD{3} & \maroonD{-2} & \greenD{5} \ \blueD{4} & \maroonD{2} & \greenD{1} \end{array} \right]$

When a matrix has the same number of rows as columns, we call it a square matrix. For example, the matrix $A$ is non-square. We call the numbers in a matrix its elements, or its entries. To refer to an entry in a matrix, we specify its row then its column in a subscript. The convention is to use lowercase letters for matrix elements.

$a_{2, 1} = \blueD{4}, ; ; ; a_{1,3} = \greenD{5}$

The transpose of a matrix is that matrix with its rows and columns switched. That is, the entry $a_{i,j}$ becomes $a_{j,i}$. We write $A^T$ to refer to the transpose of $A$. The transpose also reverses the dimensions of a matrix, so for example $A^T$ is $3 \times 2$.

$A^T = \left[ \begin{array}{cc} \blueD{3} & \blueD{4} \ \maroonD{-2} & \maroonD{2} \ \greenD{5} & \greenD{1} \end{array} \right]$

In general, the entries in a matrix don't have to be numbers. For example, some of the matrices we'll encounter in multivariable calculus have functions or even derivative operators as entries.

To learn more about what a matrix is, check out this video. For more on the transpose, check out this video. To get practice with matrix dimensions, try this exercise. To get practice with matrix elements, try this exercise.

Vectors as matrices

Because a vector is just a list of numbers, we can represent it as a matrix. This is why we can write vectors with matrix notation. We can always write a given vector as a row vector ($1 \times n$) or as a column vector ($n \times 1$). The only difference between row and column vectors is how they work in matrix multiplication.

$\vec{v} = \left[ \begin{array}{ccc} \blueD{1} & \blueD{2} & \blueD{3} \end{array} \right]^T = \left[ \begin{array}{c} \blueD{1} \ \blueD{2} \ \blueD{3} \end{array} \right]$

We use the transpose here because otherwise the row vector and column vector are not technically equal, because they have different dimensions.

To learn more about the transpose of vectors, check out this video.

Addition

We can add two matrices if they have equal dimensions. The way we add them is to add corresponding components. In symbols, this says that $A + B = C$ means $a_{i,j} + b_{i,j} = c_{i,j}$.

$\begin{align}
\left[ \begin{array}{cc} \blueD{-1} & \maroonD{3} \ \blueD{2} & \maroonD{5} \end{array} \right] + \left[ \begin{array}{cc} \blueD{3} & \maroonD{0} \ \blueD{-4} & \maroonD{-2} \end{array} \right] &= \left[ \begin{array}{cc} \blueD{-1 + 3} & \maroonD{3 + 0} \ \blueD{2 - 4} & \maroonD{5 - 2} \end{array} \right] \\
&= \left[ \begin{array}{cc} \blueD{2} & \maroonD{3} \ \blueD{-2} & \maroonD{3} \end{array} \right]
\end{align}$

To get practice with matrix addition, check out this exercise.

Scalar multiplication

We can multiply any matrix by a scalar, which is a fancy word for a number. When we do scalar multiplication, we multiply the scalar into every entry of the matrix. In symbols, this says that $xA = B$ means $xa_{i,j} = b_{i,j}$.

$\begin{align}
2 \left[ \begin{array}{cc} \blueD{2} & \maroonD{1} \ \blueD{4} & \maroonD{3} \end{array} \right] &= \left[ \begin{array}{cc} 2 \cdot \blueD{2} & 2 \cdot \maroonD{1} \ 2 \cdot \blueD{4} & 2 \cdot \maroonD{3} \end{array} \right] \ \
&= \left[ \begin{array}{cc} \blueD{4} & \maroonD{2} \ \blueD{8} & \maroonD{6} \end{array} \right]
\end{align}$

To get practice with scalar multiplication, check out this exercise.

Matrix multiplication (optional)

This section is optional because matrix multiplication is most useful when we need to talk about $n$-dimensional space, while in this course we focus on 2D and 3D. Nonetheless, learning this now provides a great foundation for extending into higher dimensions later.

Unlike with addition and scalar multiplication, we do not just multiply the corresponding entries of two matrices $A$ and $B$ to find their product. Instead, we calculate the dot product of a row vector of $A$ with a column vector of $B$ to find each entry. For example, to find $c_{1,2}$ of $C = AB$, we take the dot product of row $1$ of $A$ with column $2$ of $B$:

$\left[ \begin{array}{cc} \bold{\goldD{3}} & \bold{\goldD{1}} \ -1 & 2 \end{array} \right] \left[ \begin{array}{cc} 4 & \bold{\tealD{4}} \ 2 & \bold{\tealD{0}} \end{array} \right] = \left[ \begin{array}{cc} ? & \goldD{3} \cdot \tealD{4} + \goldD{1} \cdot \tealD{0} \ ? & ? \end{array} \right]$

In general, to find entry $c_{i,j}$ of $C = AB$, we take the dot product of the $i$th row of $A$ with the $j$th column of $B$. Here's the rest of the matrix multiplication above:

$\begin{align}
&\phantom{=}\left[ \begin{array}{cc} \blueD{3} & \maroonD{1} \ \blueD{-1} & \maroonD{2} \end{array} \right] \left[ \begin{array}{cc} \blueD{4} & \maroonD{4} \ \blueD{2} & \maroonD{0} \end{array} \right]
\\
&= \left[ \begin{array}{cc} \blueD{3} \cdot \blueD{4} + \maroonD{1} \cdot \blueD{2} & \blueD{3} \cdot \maroonD{4} + \maroonD{1} \cdot \maroonD{0} \ \blueD{(-1)} \cdot \blueD{4} + \maroonD{2} \cdot \blueD{2} & \blueD{-1} \cdot \maroonD{4} + \maroonD{2} \cdot \maroonD{0} \end{array} \right] \ \
&= \left[ \begin{array}{cc} 14 & 12 \ 0 & -4 \end{array} \right]
\end{align}$

As a consequence of this definition, we can only multiply two matrices $A$ and $B$ if $A$ has the same number of columns as $B$ has rows. Otherwise we would have to take the dot product of two vectors with different lengths.

We can always multiply two square matrices of the same dimension, such as a $3 \times 3$ with a $3 \times 3$. Other valid combinations include, for example, a $2 \times 5$ with a $5 \times 2$, or a $3 \times 1$ with a $1 \times 3$.

The dimension of the product matrix will be the rows of $A$ and the columns of $B$. Therefore, the product of a $1 \times 4$ matrix with a $4 \times 1$ matrix will be a $1 \times 1$ matrix, which is interchangeable with a number. On a sidenote, this leads to another way we can write the dot product:

$\begin{align}
\left[ \begin{array}{ccc} 1 & 2 & 3 \end{array} \right] \left[ \begin{array}{c} 4 \ 5 \ 6 \end{array} \right] &= (1, 2, 3) \cdot (4, 5, 6) \ \
&= 1 \cdot 4 + 2 \cdot 5 + 3 \cdot 6 \ \
&= 32
\end{align}$

To learn more about matrix multiplication, check out this video. For practice, try this exercise.

What's next

Now that we know matrix fundamentals, it's a good time to step back and think about what matrices mean. Otherwise the formulas and concepts surrounding them can feel unmotivated. The next article presents a way to visualize matrices that will give intuition into how they work. Specifically, it sheds light on how to think about the formula for matrix multiplication, and it prepares us for the final article about determinants.

In the last article we covered matrix fundamentals, but matrices are much more than tables of numbers. That's why in this article we'll discuss a way to think about matrices visually. This perspective makes a lot of what initially seems difficult about matrices become intuitive to understand. We'll only need this perspective for square matrices in multivariable calculus, so we will limit ourselves to those here.

Matrices as movement

What is the action of a matrix? What does a matrix look like? These questions may seem nonsensical, but we will answer them both by visualizing how $2\times 2$ matrices move the 2D plane.

Here is a drawing of the plane, along with the unit vectors $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$, which stand for $\blueD{(1, 0)}$ and $\maroonD{(0, 1)}$.

Let's consider matrix $A$:

$A = \left[ \begin{array}{cc} \blueD{1} & \maroonD{0} \ \blueD{1} & \maroonD{1} \end{array} \right]$

This is how the matrix acts upon the grid:

• The columns of the matrix tell us where it moves the unit vectors $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$, which again stand for $\blueD{(1, 0)}$ and $\maroonD{(0, 1)}$.

• The rest of the grid follows accordingly, always keeping grid lines parallel and evenly spaced. The origin stays frozen in place.

That means $A$ moves $\blueD{\hat{\imath}} \to \blueD{(1, 1)}$ and $\maroonD{\hat{\jmath}} \to \maroonD{(0, 1)}$. Here's what that looks like:

The unit vector $\maroonD{\hat{\jmath}}$ didn't move because it started at $\maroonD{(0, 1)}$. The unit vector $\blueD{\hat{\imath}}$ moved upward one unit, and this dragged the grid with it. Notice that there's a faint copy of the original lines in the background to help us stay oriented.

Let's see the same process for another matrix.

$B = \left[ \begin{array}{cc} \blueD{0} & \maroonD{-1} \ \blueD{-2} & \maroonD{1} \end{array} \right]$

We know $B$ moves $\blueD{\hat{\imath}} \to \blueD{(0, -2)}$ and $\maroonD{\hat{\jmath}} \to \maroonD{(-1, 1)}$. That looks like this:

Here's a question to practice.

[[☃ graded-group 2]]

To sum up, the action of a matrix is to move the entire grid. We can understand it by thinking about how it moves the unit vectors. We can visualize what this looks like by drawing a modified 2D grid.

These ideas extend into three dimensions as well. The third row of the matrix contains $z$-coordinates for all the unit vectors, and the third column of the matrix tells us where $\greenD{\hat{k}}$ lands.

If you'd like, play around with matrices as movement with this interactive demonstration. Drag the vectors to make the grid move, and see the matrix that corresponds to the movement in the top left corner.

[[☃ image 10]]

To learn more about matrices as transformations, check out this article. For more practice, go here.

How matrices move vectors

We already know how a given matrix moves the unit vectors $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$ (just look at the columns), but how can we find where a matrix moves any arbitrary vector? Let's consider a specific example using the first matrix from the previous section.

$A = \left[ \begin{array}{cc} \blueD{1} & \maroonD{0} \ \blueD{1} & \maroonD{1} \end{array} \right]$

How does $A$ move the non-unit vector $\goldD{(1, 2)}$? Before anything, let's get a feel for this visually. First, the vector with no matrix movement:

Now, the vector after the matrix moves the grid:

The vector just comes along for the ride as the matrix moves the grid, ultimately landing on $\goldD{(1, 3)}$. This is the essence of how matrices move vectors, which is formally called matrix-vector multiplication.

Now let's go over how we could calculate this. We represent $\goldD{(1, 2)}$ as a combination of the unit vectors by saying $\goldD{(1, 2)} = \goldD{1}\blueD{\hat{\imath}} + \goldD{2}\maroonD{\hat{\jmath}}$.

This combination remains the same after we apply $A$, but instead of using $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$, we use the result of applying $A$ to $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$:

Here's what the whole process looks like in symbols.

$\begin{align}
A \left[ \begin{array}{c} \goldD{1} \ \goldD{2} \end{array} \right] &= A\left( \goldD{1} \left[ \begin{array}{c} \blueD{1} \ \blueD{0} \end{array} \right] + \goldD{2} \left[ \begin{array}{c} \maroonD{0} \ \maroonD{1} \end{array} \right] \right) \ \
&= A\left( \goldD{1} \blueD{\hat{\imath}} + \goldD{2} \maroonD{\hat{\jmath}} \right) \ \
&= \goldD{1} A \blueD{\hat{\imath}} + \goldD{2} A \maroonD{\hat{\jmath}} \ \
&= \goldD{1} \left[ \begin{array}{c} \blueD{1} \ \blueD{1} \end{array} \right] + \goldD{2} \left[ \begin{array}{c} \maroonD{0} \ \maroonD{1} \end{array} \right] \ \
&= \left[ \begin{array}{c} 1 \ 3 \end{array} \right]
\end{align}$

The critical step is when we break $A\left( \goldD{1} \blueD{\hat{\imath}} + \goldD{2} \maroonD{\hat{\jmath}} \right)$ into $\goldD{1} A \blueD{\hat{\imath}} + \goldD{2} A \maroonD{\hat{\jmath}}$. That is when we are able to represent where $\goldD{(1, 2)}$ lands in terms of where $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$ land.

Let's try a couple practice questions.

[[☃ graded-group 1]]

[[☃ graded-group 3]]

Like in the section above, the idea behind matrices moving vectors extends into three dimensions. We just decompose our vector into a sum of $\blueD{\hat{\imath}}$, $\maroonD{\hat{\jmath}}$, and $\greenD{\hat{k}}$, then we use where the matrix takes all these unit vectors to find where it takes our vector.

Here's another interactive demonstration to play around with matrices moving vectors.

[[☃ image 1]]

To learn more about matrix-vector multiplication, check out this video. To go deeper, try these videos from linear algebra.

Matrix multiplication intuition (optional)

With the perspective of matrices as movement, we have the tools to understand what it means to multiply two matrices. The core idea is composition.

$\begin{align}
A &= \left[ \begin{array}{cc} \blueD{1} & \maroonD{1} \ \blueD{0} & \maroonD{1} \end{array} \right] \ \
B &= \left[ \begin{array}{cc} \blueD{0} & \maroonD{-1} \ \blueD{1} & \maroonD{0} \end{array} \right]
\end{align}$

The product $AB$ just means apply $B$, then apply $A$. When we apply $A$ second, we treat the transformed $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$ as regular vectors getting moved by $A$ the way we learned in the previous section.

[[☃ image 1]]

To calculate the end result, we follow $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$ along the two movements. First, $B$ takes $\blueD{\hat{\imath}} \to \blueD{(0, 1})$ and $\maroonD{\hat{\jmath}} \to \maroonD{(-1, 0)}$. Second, we find where $A$ takes these vectors:

$\begin{align}
\left[ \begin{array}{cc} \blueD{1} & \maroonD{1} \ \blueD{0} & \maroonD{1} \end{array} \right] \left[ \begin{array}{c} \blueD{0} \ \blueD{1} \end{array} \right] &= \left[ \begin{array}{c} \blueD{1} \ \blueD{1} \end{array} \right] \ \
\left[ \begin{array}{cc} \blueD{1} & \maroonD{1} \ \blueD{0} & \maroonD{1} \end{array} \right] \left[ \begin{array}{c} \maroonD{-1} \ \maroonD{0} \end{array} \right] &= \left[ \begin{array}{c} \maroonD{-1} \ \maroonD{0} \end{array} \right]
\end{align}$

Putting these in a matrix, we have the product. Notice that our calculations are reflected in the visual above.

$\left[ \begin{array}{cc} \blueD{1} & \maroonD{1} \ \blueD{0} & \maroonD{1} \end{array} \right] \left[ \begin{array}{cc} \blueD{0} & \maroonD{-1} \ \blueD{1} & \maroonD{0} \end{array} \right] = \left[ \begin{array}{cc} \blueD{1} & \maroonD{-1} \ \blueD{1} & \maroonD{0} \end{array} \right]$

To conclude, we can think of matrix multiplication as composing the movements each matrix represents. When we follow unit vectors along these movements, we can calculate the product.

As a challenge problem, try to derive the general formula for $2 \times 2$ matrix multiplication. Hint: follow $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$ as $B$ moves them, then follow where $A$ moves the already transformed vectors.

[[☃ explanation 1]]

As a bonus challenge problem, try to find the formula for multiplying $3 \times 3$ matrices. Hint: follow $\blueD{\hat{\imath}}$, $\maroonD{\hat{\jmath}}$, and $\greenD{\hat{k}}$.

[[☃ explanation 2]]

To learn more about matrix multiplication, check out this video. For practice, try this exercise.

What's next

Now that we have a solid grasp of how matrices move space, we are in a prime position to understand the final concept that we'll cover in this review series: the determinant.

When we interpret matrices as movement, there is a sense in which some matrices stretch space out and others squeeze it in. This scaling factor has a name: the determinant.

Determinant as a scaling factor

Let's go through a few examples to get a feel for how the determinant works. Here's a reminder of what the grid looks like before applying any matrices. The area of the little box starts as $1$.

If a matrix stretches things out, then its determinant is greater than $1$.

If a matrix doesn't stretch things out or squeeze them in, then its determinant is exactly $1$. An example of this is a rotation.

If a matrix squeezes things in, then its determinant is less than $1$.

Some matrices shrink space so much they actually flatten the entire grid on to a single line. This happens whenever a matrix maps the unit vectors $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$ to be multiples of each other, lying on the same line. These matrices have a determinant of $0$.

Even though determinants represent scaling factors, they are not always positive numbers. The sign of the determinant has to do with the orientation of $\blueD{\hat{\imath}}$ and $\maroonD{\hat{\jmath}}$. If a matrix flips the orientation, then its determinant is negative. Notice how $\blueD{\hat{\imath}}$ is on the left of $\maroonD{\hat{\jmath}}$ in the image below, when normally it is on the right of $\maroonD{\hat{\jmath}}$.

The same idea of scaling area extends to 3D matrices as well. The only difference is that in 3D we say the matrix scales volume rather than area. The unit square also becomes the unit cube, whose sides are the unit vectors $\blueD{\hat{\imath}}$, $\maroonD{\hat{\jmath}}$, and $\greenD{\hat{k}}$.

If you'd like, play around with determinants as scaling factors with this interactive demonstration. Notice how, whenever we flip the orientation of the unit vectors, we are forced to pass through a single moment in which the determinant is zero.

[[☃ image 2]]

One last important note is that the determinant only makes sense for square matrices. That's because square matrices move vectors from $n$-dimensional space to $n$-dimensional space, so we can talk about volume changing. For nonsquare matrices, linear algebra has the concepts of null space and range, but they are not multivariable calculus topics. All the formulas in the next section require a matrix with the same number of rows as columns.

How to calculate determinants

Now that we have a strong sense of what determinants represent, let's go over how we can find the determinant of a given matrix. We'll cover how to do this for $2 \times 2$ and $3 \times 3$ matrices.

Calculating 2D determinants

There are two ways to write the determinant.

$\det\left( \left[ \begin{array}{cc} \blueD{a} & \maroonD{b} \ \blueD{c} & \maroonD{d} \end{array} \right] \right) = \bigg| \begin{array}{cc} \blueD{a} & \maroonD{b} \ \blueD{c} & \maroonD{d} \end{array} \bigg|$

The formula for the 2D determinant is $ad - bc$. For example:

$\det\left( \left[ \begin{array}{cc} \blueD{1} & \maroonD{3} \ \blueD{5} & \maroonD{4} \end{array} \right] \right) = 1 \cdot 4 - 3 \cdot 5 = -11$

[[☃ explanation 7]]

Let's try a practice question.

[[☃ graded-group 3]]

For more practice calculating 2D determinants, check out this exercise.

Calculating 3D determinants

The general formula for the determinant of a $3 \times 3$ matrix is a mouthful, so let's start by walking through a specific example. The top row is bolded because we'll go along it one entry at a time to find the determinant.

$\det\left( \left[ \begin{array}{ccc} \bold{\blueD{2}} & \bold{\maroonD{1}} & \bold\greenD{{2}} \ \blueD{3} & \maroonD{3} & \greenD{1} \ \blueD{1} & \maroonD{4} & \greenD{2} \end{array} \right] \right) = ; ???$

First, consider the $\blueD{2}$ in the top left of the matrix. Let's call this our "anchor number." Imagine we ignore all the other entries that are in the same row or column as our anchor number. The matrix would look like this:

$\left[ \begin{array}{ccc} \blueD{2} & & \ & \maroonD{3} & \greenD{1} \ & \maroonD{4} & \greenD{2} \end{array} \right]$

Now we take the 2D determinant of the submatrix we found.

$\det\left( \left[ \begin{array}{cc} \maroonD{3} & \greenD{1} \ \maroonD{4} & \greenD{2} \end{array} \right] \right) = 3 \cdot 2 - 1 \cdot 4 = \goldD{2}$

Finally, we multiply the smaller determinant with the anchor number $\blueD{2}$ to get $\blueD{2} \cdot \goldD{2} = 4$. This $4$ is the first of three terms that we'll add to find the full 3D determinant.

$\det\left( \left[ \begin{array}{ccc} \blueD{2} & \maroonD{1} & \greenD{2} \ \blueD{3} & \maroonD{3} & \greenD{1} \ \blueD{1} & \maroonD{4} & \greenD{2} \end{array} \right] \right) = 4 ; + ; ??$

Let's do the next step. This time, our anchor number is $\maroonD{1}$.

$\left[ \begin{array}{ccc} & \maroonD{1} & \ \blueD{3} & & \greenD{1} \ \blueD{1} & & \greenD{2} \end{array} \right]$

We take the 2D determinant of our new submatrix to get $3 \cdot 2 - 1 \cdot 1 = \goldD{5}$. Now this is a bit odd, but we multiply the result by the negative of the anchor number to make our second term $-\maroonD{1} \cdot \goldD{5} = -5$. In general, we alternate multiplying the small determinant by the anchor number and by the negative of the anchor number, like a checkerboard pattern:

$\left[ \begin{array}{ccc} \greenD{+} & \redD{-} & \greenD{+} \ \redD{-} & \greenD{+} & \redD{-} \ \greenD{+} & \redD{-} & \greenD{+} \end{array} \right]$

Now we have two out of three terms.

$\det\left( \left[ \begin{array}{ccc} \blueD{2} & \maroonD{1} & \greenD{2} \ \blueD{3} & \maroonD{3} & \greenD{1} \ \blueD{1} & \maroonD{4} & \greenD{2} \end{array} \right] \right) = 4 - 5 ; + ; ?$

For the final step, the anchor number is $\greenD{2}$. According to the checkerboard pattern, we do not need to multiply by negative one at the end. Take a moment to try to imagine the submatrix we get this time. Its determinant is $3 \cdot 4 - 3 \cdot 1 = \goldD{9}$. We multiply this by the anchor number to get $\greenD{2} \cdot \goldD{9} = 18$.

At last, we can add together all the terms we've found to see that the determinant is $4 - 5 + 18 = 17$. Finding the determinant of a $3 \times 3$ matrix is a lot of work! Good job following along.

Here's another example, done all at once. Try to imagine crossing out the entries in the row and column of each anchor number to see where its submatrix comes from.

$\begin{align}
\det\left( \left[ \begin{array}{ccc} \blueD{4} & \maroonD{1} & \greenD{3} \ \blueD{0} & \maroonD{2} & \greenD{4} \ \blueD{3} & \maroonD{2} & \greenD{1} \end{array} \right] \right) &= \blueD{4} \det \left( \left[ \begin{array}{cc} \maroonD{2} & \greenD{4} \ \maroonD{2} & \greenD{1} \end{array} \right] \right) \ \
&- \maroonD{1} \det \left( \left[ \begin{array}{cc} \blueD{0} & \greenD{4} \ \blueD{3} & \greenD{1} \end{array} \right] \right) \ \
&+ \greenD{3} \det \left( \left[ \begin{array}{cc} \blueD{0} & \maroonD{2} \ \blueD{3} & \maroonD{2} \end{array} \right] \right) \ \
&= \blueD{4}(\goldD{-6}) - \maroonD{1}(\goldD{-12}) + \greenD{3}(\goldD{-6}) \ \
&= -30
\end{align}$

If we apply the same procedure to a general $3 \times 3$ matrix, we get a very long formula. What's most important to take away is the strategy we use to calculate the determinant, not the formula itself.

$\begin{align}
\det\left( \left[ \begin{array}{ccc} \blueD{a} & \maroonD{b} & \greenD{c} \ \blueD{d} & \maroonD{e} & \greenD{f} \ \blueD{g} & \maroonD{h} & \greenD{i} \end{array} \right] \right) &= \blueD{a} \det \left( \left[ \begin{array}{cc} \maroonD{e} & \greenD{f} \ \maroonD{h} & \greenD{i} \end{array} \right] \right) \ \
&- \maroonD{b} \det \left( \left[ \begin{array}{cc} \blueD{d} & \greenD{f} \ \blueD{g} & \greenD{i} \end{array} \right] \right) \ \
&+ \greenD{c} \det \left( \left[ \begin{array}{cc} \blueD{d} & \maroonD{e} \ \blueD{g} & \maroonD{h} \end{array} \right] \right) \ \
&= aei - afh - bdi + bfg + cdh - ceg
\end{align}$

[[☃ explanation 1]]

Let's do a practice problem.

[[☃ graded-group 1]]

To learn more about calculating 3D determinants, check out this video.

Connection to cross products

The formula for the cross product is not pretty, but there's a nice trick for deriving it on the fly. To find the cross product of $\vec{a} = (\blueD{a_1}, \maroonD{a_2}, \greenD{a_3})$ and $\vec{b} = (\blueD{b_1}, \maroonD{b_2}, \greenD{b_3})$, just evaluate the following $3 \times 3$ determinant, where the top row is the unit vectors $\blueD{\hat{\imath}}$, $\maroonD{\hat{\jmath}}$, and $\greenD{\hat{k}}$.

$\begin{align}
\det\left( \left[ \begin{array}{ccc} \blueD{\hat{\imath}} & \maroonD{\hat{\jmath}} & \greenD{\hat{k}} \ \blueD{a_1} & \maroonD{a_2} & \greenD{a_3} \ \blueD{b_1} & \maroonD{b_2} & \greenD{b_3} \end{array} \right] \right) &= \blueD{\hat{\imath}} \det \left( \left[ \begin{array}{cc} \maroonD{a_2} & \greenD{a_3} \ \maroonD{b_2} & \greenD{b_3} \end{array} \right] \right) \ \
&- \maroonD{\hat{\jmath}} \det \left( \left[ \begin{array}{cc} \blueD{a_1} & \greenD{a_3} \ \blueD{b_1} & \greenD{b_3} \end{array} \right] \right) \ \
&+ \greenD{\hat{k}} \det \left( \left[ \begin{array}{cc} \blueD{a_1} & \maroonD{a_2} \ \blueD{b_1} & \maroonD{b_2} \end{array} \right] \right) \ \
&= \vec{a} \times \vec{b}
\end{align}$

Technically, the $3 \times 3$ determinant above is not defined because it has vectors in the top row instead of numbers. But if we carry on evaluating it anyway, we arrive at the cross product of $\vec{a}$ and $\vec{b}$. Many students find it easier to remember the formula for the cross product in terms of the determinant.

What's next

The wide world of multivariable calculus is next! Congratulations on finishing up this series on vectors and matrices. Now we have all the concepts we need, and hopefully we've built an intuitive, visual understanding of each of them.