Cross products in the light of linear transformations | Essence of linear algebra chapter 11


Hey folks! Where we left off, I was talking about how to compute a three-dimensional
cross product between two vectors, v x w. It’s this funny thing where you write a matrix,
whose second column has the coordinates of v, whose third column has the coordinates of
w, but the entries of that first column, weirdly,
are the symbols i-hat, j-hat and k-hat where you just pretend like those guys are
numbers for the sake of computations. Then with that funky matrix in hand, you compute its determinant. If you just chug along with those computations,
ignoring the weirdness, you get some constant times i-hat + some constant
times j-hat + some constant times k-hat. How specifically you think about computing
that determinant is kind of beside the point. All that really matters here is that you’ll
end up with three different numbers that are interpreted as the coordinates of
some resulting vector. From here, students are typically told to
just believe that the resulting vector has the following geometric
properties. Its length equals the area of the parallelogram
defined by v and w. It points in a direction perpendicular to
both of v and w. And this direction obeys the right hand rule in the sense that if you point your forefinger
along v and your middle finger along w then when you stick up your thumb it’ll point in the direction of the new vector. There are some brute force computations that you could do to confirm these facts. But I want to share with you a really elegant
line of reasoning. It leverages a bit of background, though. So for this video I’m assuming that everybody
has watched chapter 5 on the determinant and chapter 7 where I introduce the idea of
duality. As a quick reminder, the idea of duality is
that anytime you have a linear transformation from
some space to the number line, it’s associated with a unique vector in
that space in the sense that performing the linear transformation is the same as taking a dot product with that
vector. Numerically, this is because one of those
transformations is described by a matrix with just one row where each column tells you the number that
each basis vector lands on. And multiplying this matrix by some vector
v is computationally identical to taking the dot product between v and the vector
you get by turning that matrix on its side. The takeaway is that whenever you’re out in
the mathematical wild and you find a linear transformation to the
number line you will be able to match it to some vector which is called the “dual vector” of that
transformation so that performing the linear transformation is the same as taking a dot product with that
vector. The cross product gives us a really slick
example of this process in action. It takes some effort, but it’s definitely
worth it. What I’m going to do is to define a certain
linear transformation from three dimensions to the number line. And it will be defined in terms of the two
vectors v and w. Then, when we associate that transformation
with its “dual vector” in 3D space that “dual vector” is going to be the
cross product of v and w. The reason for doing this will be that understanding
that transformation is going to make clear the connection between
the computation and the geometry of the cross product. So to back up a bit, remember in two dimensions what it meant to
compute the 2D version of the cross product? When you have two vectors v and w, you put the coordinates of v as the first
column of the matrix and the coordinates of w is the second column
of matrix then you just compute the determinant. There’s no nonsense with basis vectors stuck
in a matrix or anything like that. Just an ordinary determinant returning a number. Geometrically, this gives us the area of a
parallelogram spanned out by those two vectors with the possibility of being negative, depending
on the orientation of the vectors. Now, if you didn’t already know the 3D cross
product and you’re trying to extrapolate you might imagine that it involves taking
three separate 3D vectors u, v and w. And making their coordinates the columns of
a 3×3 matrix then computing the determinant of that matrix. And, as you know from chapter 5 geometrically, this would give you the volume
of a parallelepiped spanned out by those three vectors with the plus or minus sign depending on the right-hand rule orientation
of those three vectors. Of course, you all know that this is not the
3D cross product. The actual 3D cross product takes in two vectors
and spits out a vector. It doesn’t take in three vectors and spit
out a number. But this idea actually gets us really close
to what the real cross product is. Consider that first vector u to be a variable say, with variable entries x, y and z while v and w remain fixed. What we have then is a function from three
dimensions to the number line. You input some vector x, y, z and you get
out a number by taking the determinant of a matrix whose
first column is x, y, z and whose other two columns are the coordinates
of the constant vectors v and w. Geometrically, the meaning of this function
is that for any input vector x, y, z, you consider
the parallelepiped defined by this vector v and w then you return its volume with the plus or
minus sign depending on orientations. Now, this might feel like kind of a random
thing to do. I mean, where does this function come from? Why are we defining it this way? And I’ll admit at this stage of my kind of
feel like it’s coming out of the blue. But if you’re willing to go along with it and play around with the properties that this
guy has it’s the key to understanding the cross product. One really important fact about this function
is that it’s linear. I’ll actually leave it to you to work through
the details of why this is true based on properties of the determinant. But once you know that it’s linear we can start bringing in the idea of “duality”. Once you know that it’s linear you know that there’s some way to describe
this function as matrix multiplication. Specifically, since it’s a function that goes
from three dimensions to one dimension there will be a 1×3 matrix that encodes this
transformation. And the whole idea of duality is that the special thing about transformations
from several dimensions to one dimension is that you can turn that matrix on its side and, instead, interpret the entire transformation
as the dot product with a certain vector. What we’re looking for is the special 3D vector
that I’ll call p such that taking the dot product between p
and any other vector [x, y, z] gives the same result as plugging in [x, y,
z] as the first column of a 3×3 matrix whose other two columns have the coordinates
of v and w then computing the determinant. I’ll get to the geometry of this in just a
moment. But right now, let’s dig in and think about
what this means computationally. Taking the dot product between p and [x, y,
z] will give us something times x + something
times y + something times z where those somethings are the coordinates
of p. But on the right side here, when you compute
the determinant you can organize it to look like some constant
times x + some constant times y + some constant times z where those constants involve certain combinations
of the components of v and w. So, those constants, those particular combinations
of the coordinates of v and w are going to be the coordinates of the vector
p that we’re looking for. But what’s going on the right here should feel very familiar to anyone who’s actually worked through a cross-product
computation. Collecting the constant terms that are multiplied
by x, y and z like this is no different from plugging in the symbols
i-hat, j-hat and k-hat to that first column and seeing which coefficients aggregate on
each one of those terms. It’s just that plugging in i-hat, j-hat and
k-hat is a way of signaling that we should interpret
those coefficients as the coordinates of a vector. So, what all of this is saying is that this funky computation can be thought
of as a way to answer the following question: What vector p has the special property that when you take a dot product between p
and some vector [x, y, z] it gives the same result as plugging in [x,
y, z] to the first column of the matrix whose other two columns have the coordinates
of v and w then computing the determinant? That’s a bit of a mouthful. But it’s an important question to digest for
this video. Now for the cool part which ties all this
together with the geometric understanding of the cross
product that I introduced last video. I’m going to ask the same question again. But this time, we’re going to try to answer
it geometrically instead of computationally. What 3D vector p has the special property that when you take a dot product between p
and some other vector [x, y, z] it gives the same result as if you took the
signed volume of a parallelepiped defined by this vector [x, y, z] along with
v and w? Remember, the geometric interpretation of
a dot product between a vector p and some other vector is to project that other vector onto p then to multiply the length of that projection
by the length of p. With that in mind, let me show a certain way
to think about the volume of the parallelepiped that we care
about. Start by taking the area of the parallelogram
defined by v and w then multiply it, not by the length of [x,
y, z] but by the component of [x, y, z] that’s perpendicular
to that parallelogram. In other words, the way our linear function
works on a given vector is to project that vector onto a line that’s
perpendicular to both v and w then, to multiply the length of that projection
by the area of the parallelogram spanned by v and w. But this is the same thing as taking a dot
product between [x, y, z] and a vector that’s perpendicular
to v and w with a length equal to the area of that parallelogram. What’s more, if you choose the appropriate
direction for that vector the cases where the dot product is negative will line up with the cases where the right
hand rule for the orientation of [x, y, z], v and w is negative. This means that we just found a vector p so that taking a dot product between p and
some vector [x, y, z] is the same thing as computing that determinant
of a 3×3 matrix whose columns are [x, y, z], the coordinates
of v and w. So, the answer that we found earlier, computationally using that special notational trick must correspond geometrically to this vector. This is the fundamental reason why the computation and the geometric interpretation
of the cross product are related. Just to sum up what happened here I started by defining a linear transformation
from 3D space to the number line and it was defined in terms of the vectors
v and w then I went through two separate ways to think about the “dual vector” of this
transformation the vector such that applying the transformation is the same thing as taking a dot product
with that vector. On the one hand, a computational approach will lead you to the trick of plugging in
the symbols i-hat, j-hat and k-hat to the first column of the matrix and computing
the determinant. But, thinking geometrically we can deduce that this duel vector must be
perpendicular to v and w with a length equal to the area of the parallelogram
spanned out by those two vectors. Since both of these approaches give us a dual
vector to the same transformation they must be the same vector. So that wraps up dot products and cross products. And the next video will be a really important
concept for linear algebra “change of basis”




Comments
  1. At 1:05 is it an error while computing the det?
    Cause according to what I learnt it should be
    I(v2w3-v3w2) -j(v3w1-v1w3)+k(v1w2-v2w1)

  2. This light up understanding of how the computation of cross product works. But, what I still don't understand is that… why would we need to specially define a cross product? I meant, is it really comes up very often in real world application so that it deserve its own notation? Since the definition of cross product seem utterly specific to have any general use :/

  3. This essentially seems like how "extrusion" works in 3D softwares..
    Volume is basically n times the area
    N being how many times is the area being stacked over and over
    Makes absolute sense!!

  4. I love your video series ! They truly are phenomenal at helping me get an intuitive understanding.

    A helpful thought – I believe there is a typo at 1:43. For the third row of the matrix I believe you used the determinant corresponding to i_hat again instead of the determinant corresponding to k_hat. @3Blue1Brown

  5. But why do we need to take a perpendicular vector for dot product?and why the result of cross product is perpendicular ? can anyone explain?

  6. Had to watch it twice to get it, cause I'm just a lowly eng student, but holy hell is this proof brilliant. One question though, is it possible to extend this concept into Rn, and take the "cross product" of n-1 vectors? Because I was taught that the cross product is only applicable in R3, but it seems like it could work in R4 if you had 3 vectors to work off of. Or is that too useless of an application for anyone to bother with?

  7. The video is paused at 11:02 as I type this. At that exact second something in my brain clicked so hard I had to pause to take a breath. I literally went "oh, my god" out loud. I can't wait to watch it again.

  8. Isn’t the determinant of i j and k hat mean to be i(v2w3-v3w2) – j(v3w1-v1w3) + k(v1w2-v2w1)? In the video it is i(…) + j(…) + k(…), I’m confused as I learned 3×3 determinants go in the pattern of + – + – + etc.

  9. did you learn and thought all by yourself, because this level of understanding is not in school education . please tell me.

  10. i don't know why people find hard to understand math if they can't visualize it
    , i mean, math is all about abstraction, these topics are easy because we're working with 3d dimension but in practice is all about data and multiple dimension

  11. Hi, dont you have a policy of correcting the video if there is a mistake..like the one at 1:49 mins. For new viewers it would be more authentic.

  12. I feel like this sort of visualization is critical for those of us not gifted with the preternatural spatio-visual capabilities of the people who figured this stuff out

  13. You way of jumping back and forward really makes the topic hard to understand. the stuff is absolutely amazing. i mean all necessary info is there. but there is some sense of unstructuredness. Imho, the line of reasoning should be rather streamlined.

  14. Writing it down, really helped me to understand it. It might help you too.
    First let’s define all necessary vectors / matrices in correspondence with the video.
    I encourage you to draw them.

    h = [x, y, z]^T:
    The free-to-choose vector, named h for convenience.

    v = [v1, v2, v3]^T:
    The vector representing one base of the parallelepiped.

    w = [w1, w2, w3]^T:
    The vector representing the other base of the parallelepiped.

    p = [p1, p2, p3]^T:
    An unknown vector that is perpendicular to the base of the parallelepiped formed by v and w.

    p’ = [a1, a2, a3]^T:
    A vector from the components of h that is perpendicular to the base of the parallelepiped formed by v and w.
    Its length equals (h dot p) / |p|, because the dot product adds an unknown scaling factor for the vector we are projecting onto.
    Introduced at 10:27

    M = [h, v, w]:
    A 3 by 3 Matrix, representing the parallelepiped, named M for convenience.

    The first thing to understand is, that we can get Det(M), the signed volume of the parallelepiped, with the formula below.
    Because the volume of a parallelepiped is just width * depth * height and p’ is the component of h perpendicular to v and w.

    Det(M) = |p’| * |v| * |w|

    But if the length of p would be equal to |v| * |w|, the area of the parallelogram, then we could also just write it as below
    because the dot product is the length of the projection of h onto p, times |p| = |v| * |w|.

    Det(M) = (p dot h)

    Now, how can we find such a vector h and p such that this holds?
    We calculate the determinant of the matrix, leading us to:

    p1 = v2 * w3 – v3 * w2
    p2 = v3 * w1 – v1 * w3
    p3 = v1 * w2 – v2 * w1

    Which we can just calculate.

    Hope this helps someone, it was worth figuring out.

  15. I have a written linear algebra exam tomorrow and this series is an awesome supplement to books, exercises, and previous exam sets. Love it

  16. I really love the way you describe.But as a non native english speaker,sometimes I really challange with some sentences you form.For example there is a 17 seconds of only one sentence between 8.58 and 9.15 which is totally mathematical,and i'm lost…Maybe you think about this makes a little bit harder to comprehend.Anyway,I appreciate your studies deep in my heart…

  17. this explains the concept of why a determinant works to calculate area as well. determinants are dot products between a vector and the perpendicular of the other vector

  18. Changing the perspective from computational to geometrical really made the whole thing intuitive! Wish I had those lecture in uni

  19. Could you please explain to me why we use j i and k in the determinant? In the geometric observation part, you never mentioned i j and k.

  20. three years after you posted it, some idiot in a far away land watch. after watching it, he watched it in again, and the 10:23 minute mark, his brain snapped, all the pieces lined up. one cannot describe the joy the idiot felt as he witnessed the beauty of truth at that manifested at that moment. truly I say to you, thank you. you are an artist

  21. it burns as much as "mitochondrion is the power house of the cells" that "the cross product of two vectors is a vector that is perpendicular to both". But apart from that, I know nothing about this magical being…

  22. @58 seconds, should j hat not be + j(v1w3 – v3w1)? Also @8.04 seconds, should y hat not be + (v1w3 – v3w1)?

  23. i pretty much understood everything, except why do we multiply v2*w3-v3*w2 for the first point of the P vector and same goes for the second and third coordinate. Is it similar to the geometric explanation you gave in the determinant episode? that geometric photo of a (parallelogram?) and how it explains the (a*d-b*c)?

    if anyone could reply, it would mean a lot.

  24. Hi Grant, thank-you so much for sharing your insight – I cannot tell you how much I appreciate it!

    Can ask whether the natural extension to higher dimensions (e.g. in 4D, a ternary 'product' that returns a single vector) is useful? In particular, I've been told in the past that the cross product has a meaning in 7 dimensions as well as in 3. Is this the correct way to generalise from 3D to 7D and if so, why does it work only in 3D and 7D?

    Thank-you for reading!

  25. My god this was exhausting to follow..

    But there's still something I don't get:
    First he used a variable vector [x,y,z] as the first column of the matrix, I could (barely) follow all that stuff, but then at the end he replaced it with i, j, and k.

    Aren't those vectors themselves?
    I don't get what this implies.

    ————————————-
    How do you get from

    p→ • [x,y,z]
    = det( [ x,y,z / v1,v2,v3 / w1,w2,w3 ] )

    to

    p→
    = det( [ i,j,k / v1,v2,v3 / w1,w2,w3 ] )

    (where p→ = v→ x w→)

  26. This is really elegant.
    It uses a geometrical property of the cross product and writes it in a linear transformation , then by using a notation trick on the left hand side of plugging i-hat,j-hat and k-hat to let the dot product remain a vector,and thus we get a way of computing the cross product on the other side.

  27. Good video. I swore something was off until i rewatched it 4 times.
    Some things were just explained in an order that felt vocally fluent but not logically simplistic. The sequence of words chosen confused me at some points (geometry section, maybe I have an odd thinking order). To anyone confused, really pay attention to every word.
    Tomasz Kudłacz comment was somewhat more intuitive to me.

  28. Dot product is defined for vectors of higher dimensioms (like 4 dimensional or 5 dimensional vectors). Is cross product defined for vectors of higher dimensions?If yes, how is it defined and what‘s its geometrical meaning?

  29. My Engineering teacher regurgitates the computation trick from the textbook and gave the geometric proof of vectors in 3D space in a drawing. Later on through 3D engineering design software, I could see that the computation trick works. I suspected then that he had no insight and was jumping from a math formula to a concrete drawing. I have to watch the video many more times to let it sink until I can articulate it with math notations that it becomes second nature. Right now my intuition from what lies deeper like the duality, etc are just bubbling in my idiot box forming visual images. From an engineering perspective, a well developed intuitive visualization means you can now assess and tweak the engineering design tools. Amazing and You have enhanced the skills of yet another unexceptional. BTW – Anyone has been taught better in engineering colleges?

  30. FFFFFFFF it was very hard and painfully but at the end I have it. It takes me more an infinite number of time and i had to watch the other videos of the series many time and taking write note. So don't worry if you don't grasp it fast.

  31. Ow, it's because the area of the parallelogram is equal to the length of the p vector so that both multiplied by the projected vector gives a number line. Oh, that's elegant af.

  32. How many linear transformations to the number line are there? Duality shouldn't be limited to these two particular functions right? There have to be a ton of dual functions out there.

  33. I'd like to praise the video for its quality but also point out a thing I stumbled upon while thinking, hoping that others with the same problem will be able to solve it. The thing is: I was not convinced that the geometric interpretation of the transformation leads to a direct conclusion that p has to be both perpendicular to the base of the v,w parallelogram with the length equal to its base. The reason is: such a vector is a candidate for p, but is it the only possible option? Are there others vectors that could, coincidentally have their dot product with [x y z] be equal to the volume of that parallelepiped. The answer is: no, there are no other possibilities (keep in mind that p has to be able to fulfill the criteria for any given [x y z]), and the reason is: focus on arbitrary [x y z] and vary it's orientation in space while preserving its length. The dot product depends on the angle between [x y z] and p, but at the same time, the volume of the parallelepiped depends on the angle between [x y z] and the normal to the base parallelogram. For the equality to hold for any [x y z], it must be that p is in the same direction as the normal to the base parallelogram so the both dot product and the determinant depend on the same parameter, as they are always equal. Once this is realized, it is easy to reason why the length of p has to be equal to the area of the base parallelogram.

  34. I really want to thank you, it is amazing how much effort you put into your videos. This work really helps students like me to understand the real idea behind various mystical mathematical operations like Determinand, Crossproducts and so on. Furthermore, I think it is great that you do not go for stump combining two numbers, what I mean with that is, as an engineer most people can handle numbers, but only a few people know about the ideas behind them.

  35. I quickly understood the mathematical demonstration but I find the geometric demonstration not really convincing because I cannot "see" that the dot product result (a "length") is the same as the DET result (which is a volume).

  36. This geometric interpretation of the dot product is brilliant, and when applied to define the computation for the cross product is very insightful. This is literally what I was missing in my HS precal class: reasons for why these methods for computing this stuff works.

    On an unrelated note, I would appreciate it if you made an "Essence of Classical Mechanics" series!

  37. Aaaa… sorry? Sir? Spotted a mistake… at 6:03 you say ”it seems like coming out of the blue”, but actually… ahem… aaarr.. we see that it was coming out of the brown…

  38. Not gonna lie, this was pretty damn opaque. But fascinating nonetheless. This might be the first time I will have to re-watch the video several times. There were some "Aha!" moments scattered here and there, but I wouldn't able to explain it to someone else if I had to. So time to watch it again.

  39. I love this video, thanks Grant, but I'm having a problem.
    This question assumes we understand why cofactor expansion works to calculate the 3D determinant in the first place, and then rephrases it as a dot product.
    But, geometrically, why does cofactor expansion work to calculate the 3D determinant?!?!
    Grant doesn't have a video on it 🙁
    Thanks!

  40. Yesterday, after watching the previous videos on this topic, I got inspired to find an easier method to calculate de determinant of a 2×2 matrix. Given that it was the area of a parallelogram, I started with that. I took the vectors i(a,b) and j(c,d), between them the angle A, and the vector going from the tip of i to the tip of j (forming a triangle). I ended up with det= || i || * || j || *sin(A). Now, what is A ?

    I got from Al Kashi's theorem cos(A)= (|| u ||² – ( || i ||² + || j ||²)) / -2 || i || * || j ||

    Therefore, det= sin(arccos((|| u ||² – ( || i ||² + || j ||²)) / -2 || i || * || j ||
    )) * || i || * || j ||
    = sin(arccos( i . j / || i || * || j ||
    )) * || i || * || j ||

    (in my notebook it was a lot longer since I wrote everything in terms of vectors coordinates, not vector themselves, idk why, it just seemed easier to go through this thing that way

    Then I thought… wow that's really helpful, a lot easier than (ad-bc)

    Strangely, in this video I learn that the cross product's magnitude is exactly that, I felt relieved that I didn't screw up and a bit sad because I didn't invent a complicated formula that I'm not gonna use 🙁

  41. After watching this, I will immediately understand what a paper indicates when they use some weird vector dot product just by duality. This is the whole context behind the linear algebra calculation and can easily help relate what the author wants to achieve along their lines. This is amazing. I truly thank you.

  42. Interesting. I could visualize now that the gradient vector is a dual of Jacobian Matrix for scalar functions. Nice!

  43. You just covered my 6 months course by only 15 videos. HUGE RESPECT! We need more series like these. Thank you sir!

  44. Sir I am forever grateful. I reeeally hope that your channel is still around when my future kids are ready for it.

  45. I wish I had a Math teacher like you in my school. Nevertheless, it is never too late. Its an eye-opening of how math is so beautiful and I can't stop smiling at the concepts explained by you. Its total bliss. Thank You

  46. at 10.32 I have a question: how do you find the component of vector[xyz] which is perpendicular to the other two (which create the parallelogram) if those vectors do not create a flat plane?

  47. Why is the determinant geometrically equivalent to a transformation onto the number line? Or is it not equivalent, but simply a convenient computational trick

  48. Why don't you are changing signs accordingly to changing signs patterns (from cofactor martix if I'm not mistaken)?

  49. There is a vector which we may apply the dot product operation between it and x which gives the volume with x, v, and w. This is the dual, and it is unique (I need to check). By a basic geometric argument, a vector exists identical in the dot product function it induces. But then these are the same, or a linear transform has a non-unique dual. Whoa.

  50. 6:25 If [x,y,z] happens to be on the same plane as v and w the volume of the parallelepiped = 0. Does it mean this function isn't injective? And shoudn't linear function be injective?

Leave a Reply

Your email address will not be published. Required fields are marked *