David's Blog

The ramblings of a frustated Python Data Scientist ...

Linear Algebra, Vectors and spaces, Vectors


Introduction

This is the first blog post in my series on learning Linear Algebra. You can find the full set of post on this page:

Videos covered by this page

The following vidoes are included in the material on this page.

Vectors and spaces

  1. Vector intro for linear algebra
  2. Real coordinate spaces
  3. Adding vectors algebraically & graphically
  4. Multiplying vector by scalar
  5. Vector examples
  6. Introduction to unit vector notation
  7. Linear algebra parametric representations of lines

Let's get started

The obvious object to use to represent a vector is a List. However, knowing what's to come I'll actually implement my vectors as Numpy arrary.

In [1]:
# Define a vector as a list and output the results
list_vector = [1, 4]
list_vector
Out[1]:
[1, 4]
In [2]:
# import Numpy
import numpy as np

# Define a vertor as a Numpy array and output the results
simple_vector = np.array([5, 2])
simple_vector
Out[2]:
array([5, 2])

Sample vector in a markdown cell

As mentioned before I also want to be able to see the maths equations for the concepts I'm learning. Here are some sample vectors

$$\vec{vector} \in \Re^2 = \begin{bmatrix}5 \\ 2 \end{bmatrix}$$$$\vec{vector} \in \Re^3 = \begin{bmatrix}5 \\ 2 \\ 4 \end{bmatrix}$$

Vector Addition and Subtraction

To add or subtract two vectors, add or subtract the corresponding components. The sum of two or more vectors is called the resultant. The resultant of two vectors can be found using either the parallelogram method or the triangle method. For additional reference see Adding and Subtracting Vectors

Addition

$$ \vec{a} = \begin{bmatrix} 6 \\ -2 \end{bmatrix} \vec{b} = \begin{bmatrix} -4 \\ 4 \end{bmatrix} $$$$ \vec{a} + \vec{b} = \begin{bmatrix} 6 & + & -4 \\ -2 & + & 4 \end{bmatrix} $$$$ \vec{a} + \vec{b} = \begin{bmatrix} 2 \\ 2 \end{bmatrix} $$

Subtraction

$$ \vec{a} = \begin{bmatrix} 6 \\ -2 \end{bmatrix} \vec{b} = \begin{bmatrix} -4 \\ 4 \end{bmatrix} $$$$ \vec{a} - \vec{b} = \begin{bmatrix} 6 & - & -4 \\ -2 & - & 4 \end{bmatrix} $$$$ \vec{a} + \vec{b} = \begin{bmatrix} 10 \\ -6 \end{bmatrix} $$

Now let's implement in Python.

In [3]:
# Import Numpy - for refence, only need to import once per script / notebook
import numpy as np

# Create vectors a and b
a = np.array([6, -2])
b = np.array([-4, 4])

# Display vectors a and b
print(f'Vector a: {a}')
print(f'Vector b: {b}')
Vector a: [ 6 -2]
Vector b: [-4  4]
In [4]:
# Add vector b to vector a and print result
c = a + b
print(f'Vector c: {c}')
Vector c: [2 2]
In [5]:
# Add vector a to vector b and print result
d = b + a
print(f'Vector d: {d}')
Vector d: [2 2]
In [6]:
# Subtract vector b from vector a and print result
e = a - b
print(f'Vector e: {e}')
Vector e: [10 -6]
In [7]:
# Subtract vector a from vector b and print result
f = b - a
print(f'Vector f: {f}')
Vector f: [-10   6]

Whilst vector e and f appear different, if you plot them both by anchoring them at the origin you'll see the magnitude of each of the vetors is the same but they point in differenet directions.

From the Parallelogram Method and the also from the commutative property of real numbers under addition we can Show that addition of vectors is commutative, i.e. A+B = B+A. The same applied to subtraction.

Multipling a Vector by a Scalar

When you multiple a vector by a scalar you simply multiple each of the vector components by the scalar. For additional reference see How do I multiply a vector by a scalar?

$$ \vec{a} = \begin{bmatrix} 6 \\ -2 \end{bmatrix} $$$$ 2 . \vec{a} = \begin{bmatrix} 6 & . & 2 \\ -2 & . & 2 \end{bmatrix} = \begin{bmatrix} 12 \\ -4 \end{bmatrix} $$

Now let's implement in Python.

In [8]:
# Import Numpy - for refence, only need to import once per script / notebook
import numpy as np

# Create vector a
a = np.array([6, -2])

# Display vectors a and b
print(f'Vector a: {a}')
Vector a: [ 6 -2]
In [9]:
# Multiple vector by the scalar value 2
c = a * 2
print(f'Vector c: {c}')
Vector c: [12 -4]
In [10]:
# Multiple vector by the negative scalar value -2
d = a * -2
print(f'Vector d: {d}')
Vector d: [-12   4]

A more complicated $\Re^4$ example

$$ \vec{a} = \begin{bmatrix} 0 \\ -1 \\ 2 \\ 3 \end{bmatrix} \vec{b} = \begin{bmatrix} 4 \\ -2 \\ 0 \\ 5 \end{bmatrix} $$$$ 4\vec{a} - 2\vec{b} = 4\begin{bmatrix} 0 \\ -1 \\ 2 \\ 3 \end{bmatrix} - 2\begin{bmatrix} 4 \\ -2 \\ 0 \\ 5 \end{bmatrix} = \begin{bmatrix} 0 \\ -4 \\ 8 \\ 12 \end{bmatrix} - \begin{bmatrix} 8 \\ -4 \\ 0 \\ 10 \end{bmatrix} = \begin{bmatrix} -8 \\ 0 \\ 8 \\ 2 \end{bmatrix} $$

Unit Vector Notation

Unit vectors may be used to represent the axes of a Cartesian coordinate system. For instance, the unit vectors in the direction of the x, and y axes of a two dimensional Cartesian $\Re^2$ coordinate system are as follows:

$$ \vec{i} = \begin{bmatrix} 1 \\ 0 \\ \end{bmatrix} , \vec{j} = \begin{bmatrix} 0 \\ 1 \\ \end{bmatrix} $$

And the unit vectors in the direction of the x, y, and z axes of a three dimensional Cartesian $\Re^3$ coordinate system are as follows:

$$ \vec{i} = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} , \vec{j} = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} , \vec{k} = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} $$

Which all means that an $\Re^2$ vector can be written in the form of the unit vectors meaning:

$$ \vec{v} = \begin{bmatrix} 2 \\ 3 \end{bmatrix} = 2\vec{i} + 3\vec{j} = 2\begin{bmatrix} 1 \\ 0 \end{bmatrix} + 3 \begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 2 \\ 3 \end{bmatrix} $$

Or an $\Re^3$ vector can be written in the form of the unit vectors like this:

$$ \vec{v} = \begin{bmatrix} 2 \\ 3 \\ 4 \end{bmatrix} = 2\vec{i} + 3\vec{j} + 4\vec{k} = 2\begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} + 3 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} + 4 \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 2 \\ 3 \\ 4 \end{bmatrix} $$

Now let's implement in Python.

In [11]:
# Import Numpy - for refence, only need to import once per script / notebook
import numpy as np

# Create vectors i & j
i = np.array([1, 0])
j = np.array([0, 1])

# Display vectors i and j
print(f'Vector i: {i}')
print(f'Vector j: {j}')
Vector i: [1 0]
Vector j: [0 1]
In [12]:
# Now create vector v
v = (2 * i) + (3 * j)

print(f'Vector v: {v}')
Vector v: [2 3]
In [13]:
# Create R3 unit vectors i, j & k
i = np.array([1, 0, 0])
j = np.array([0, 1, 0])
k = np.array([0, 0, 1])

# Display vectors i and j
print(f'Vector i: {i}')
print(f'Vector j: {j}')
print(f'Vector k: {k}')
Vector i: [1 0 0]
Vector j: [0 1 0]
Vector k: [0 0 1]
In [14]:
# Now create vector v
v = (2 * i) + (3 * j) + (4 * k)

print(f'Vector v: {v}')
Vector v: [2 3 4]

Parametric representation of lines

Okay this needed a bit of thought an a lot of additional reference study. These are only very rough notes.

Standard position: The origin (0, 0)
Set of Co-linear Position Vectors in the standard position $S = \{c\vec{V} | c \in \Re \}$

Set of Co-linear Position Vectors $L = \{ \vec{X} + t\vec{V} | t \in \Re \}$ where:

  • $\vec{V}$ is a vector in the stand position
  • $\vec{X}$ is a position vector offset

So given two positional vectors $\vec{a}$ and $\vec{b}$ placed at the origin:

$$ \vec{a} = \begin{bmatrix} 2 \\ 1 \end{bmatrix} , \vec{b} \begin{bmatrix} 0 \\ 3 \end{bmatrix} $$

Give me the parametric representation of the line that goes through the endpoint of each vector i.e. a line that goes through the points (2, 1) & (0, 3)

From earlier vector subtraction we know that $\vec{b} - \vec{a}$ represents the vector $\vec{c}$ which is the different between the two vectors $\vec{a}$ and $\vec{b}$. If we then multiple $\vec{c}$ by a scalar $t$ we get any point in the line $\vec{c}$. But this only represents the set of points for $\vec{c}$ at the standard position i.e. through the origin. To offset $\vec{c}$ from the origin to pass through the endpoints of $\vec{a}$ and $\vec{b}$ we either need to additionally add either $\vec{a}$ or $\vec{b}$.

The set of Co-linear Position Vectors in the standard position (standard form) for $\vec{c}$ $ = \{t\vec{c} | t \in \Re \}$ which is equal to $t(\vec{b} - \vec{a})$

So:

$$ \vec{a} = \begin{bmatrix} 2 \\ 1 \end{bmatrix} , \vec{b} \begin{bmatrix} 0 \\ 3 \end{bmatrix} $$$$ \vec{c} = \vec{b} \begin{bmatrix} 2 \\ 1 \end{bmatrix} - \vec{a} \begin{bmatrix} 0 \\ 3 \end{bmatrix} = \begin{bmatrix} -2 \\ 2 \end{bmatrix} $$

But this vector $\vec{c}$ is the standard position as previously mentions. So by including an offset of $\vec{a}$ or $\vec{b}$ The line $L$ that represents the set of all colinear points on the vector $\vec{c}$ we get:

$$L = \{\vec{b} + t(\vec{b} - \vec{a} ) | t \in \Re\}$$

or

$$L = \{\vec{a} + t(\vec{a} - \vec{b} ) | t \in \Re\}$$

So taking the first definition:

$$ L = \Biggl\{ \begin{bmatrix} 0 \\ 3 \end{bmatrix} + t \begin{bmatrix} -2 \\ 2 \end{bmatrix} | t \in \Re \Biggl\} $$

Then to produce the parametric equation of the line that goes through the vector $\vec{b}$ - $\vec{a}$ we get:

$$x = 0 + (t\dot-2) $$

$$y = 3 + (t\dot2) $$

$$x = -2t $$

$$y = 2t + 3 $$

This then scales to $\Re^3$ and beyond (here we use the second definition of the set of points from above):

$$ \vec{P_1} = \begin{bmatrix} -1 \\ 2 \\ 7 \end{bmatrix} \vec{P_2} = \begin{bmatrix} 0 \\ 3 \\ 4 \end{bmatrix} $$$$ L = \Bigl\{ \vec{P_1} + t (\vec{P_1} - \vec{P_2} | t \in \Re \Bigl\} $$$$ L = \Biggl\{ \begin{bmatrix} -1 \\ 2 \\ 7 \end{bmatrix} + t \begin{bmatrix} -1 \\ -1 \\ 3 \end{bmatrix} | t \in \Re \Biggl\} $$

Which give our parametric equations:

$$x = -1 -t$$

$$y = 2 -t$$ $$y = 7 + 3t$$


Practice

Following the videos above there was a series of practice tests.

Scalar Multiplication

In [15]:
import numpy as np

# Question 1
w = np.array([-3, 5])

# what is 3w
print(f'{3 * w}')
[-9 15]
In [16]:
# Question 2
v = np.array([-2, -4])

# what is 2v
print(f'{2 * v}')
[-4 -8]
In [17]:
# Question 3
u = np.array([3, 2])

# what is -4u
print(f'{-4 * u}')
[-12  -8]
In [18]:
# Question 4
t = np.array([-3, 1])

# what is -3t
print(f'{-3 * t}')
[ 9 -3]

Unit vectors

Question 1:

Find the unit vector in the direction of $\vec{v} = (7, -3)$:

$$ \Bigg( \frac{7}{\sqrt{7^2 + (-3)^2}}, -\frac{3}{\sqrt{7^2 + (-3)^2}} \Bigg)$$$$ \Bigg( \frac{7}{\sqrt{58}}, -\frac{3}{\sqrt{58}} \Bigg)$$
Question 2:

Find the unit vector in the direction of $\vec{v} = (-7, 6)$:

$$ \Bigg( -\frac{7}{\sqrt{85}}, \frac{6}{\sqrt{85}} \Bigg)$$
Question 3:

Find the unit vector in the direction of $\vec{v} = (5, -2)$:

$$ \Bigg( \frac{5}{\sqrt{29}}, -\frac{2}{\sqrt{29}} \Bigg)$$
Question 4:

Find the unit vector in the direction of $\vec{v} = (-5, 5)$:

$$ \Bigg( -\frac{5}{\sqrt{50}}, \frac{5}{\sqrt{50}} \Bigg)$$

Adding and Subtracting Vectors

In [19]:
import numpy as np

# Question 1
u = np.array([-9, 10])
w = np.array([3, -5])

# what is u - w
print(f'{u - w}')
[-12  15]
In [20]:
# Question 2
u = np.array([1, -5])
w = np.array([8, 4])

# what is u + w
print(f'{u + w}')
[ 9 -1]
In [21]:
# Question 3
u = np.array([6, 5])
w = np.array([5, 4])

# what is w - u
print(f'{w - u}')
[-1 -1]
In [22]:
# Question 4
u = np.array([-1, -7])
w = np.array([7, 11])

# what is u + w
print(f'{u + w}')
[6 4]

Add vectors: magnitude & direction to component

Before completing this section of questions I heed to take a refresher on triganometry. This the the recommended set of videos from Khan Academy and for me they did the job:

This little video series is part of the wider Precalculus module that I may need to go back and take in whole if I continue to run into these types of problems.

Question 1:

image.png

So

  • The angle $\theta$ for $\vec{v}$ to the $x$ axis is $10^\circ$
    • $x$ component of $\vec{v}$ is negative
    • $y$ component of $\vec{v}$ is positive
  • The angle $\theta$ for $\vec{w}$ to the $y$ axis is $30^\circ$
    • $x$ component of $\vec{w}$ is positive
    • $y$ component of $\vec{w}$ is negative

import math import numpy as np

In [23]:
import math

adj_v = math.cos(math.radians(10)) * 5
opp_v = math.sin(math.radians(10)) * 5
v = np.array([-adj_v, opp_v])
v
Out[23]:
array([-4.92403877,  0.86824089])
In [24]:
opp_w = math.sin(math.radians(30)) * 8
adj_w = math.cos(math.radians(30)) * 8
w = np.array([opp_w, -adj_w])
w
Out[24]:
array([ 4.        , -6.92820323])
In [25]:
(v + w).round(2)
Out[25]:
array([-0.92, -6.06])
Question 2

The magnitude and direction angle of $\vec{v}$ and $\vec{w}$ are given below in the form $(r, \theta)$ where $r$ is the magnitude and $\theta$ is the direction angle.

  • $\vec{v}$ = $(8, 20^\circ)$
  • $\vec{w}$ = $(5, 240^\circ)$

Find the component form of $\vec{v}$ + $\vec{w}$.

So

  • The angle $\theta$ for $\vec{v}$ to the $x$ axis is $20^\circ$
    • $x$ component of $\vec{v}$ is positive
    • $y$ component of $\vec{v}$ is positive
  • The angle $\theta$ for $\vec{w}$ to the $y$ axis is $30^\circ$
    • $x$ component of $\vec{w}$ is negative
    • $y$ component of $\vec{w}$ is negative
In [26]:
adj_v = math.cos(math.radians(20)) * 8
opp_v = math.sin(math.radians(20)) * 8
v = np.array([adj_v, opp_v])
v
Out[26]:
array([7.51754097, 2.73616115])
In [27]:
opp_w = math.sin(math.radians(30)) * 5
adj_w = math.cos(math.radians(30)) * 5
w = np.array([-opp_w, -adj_w])
w
Out[27]:
array([-2.5       , -4.33012702])
In [28]:
(v + w).round(2)
Out[28]:
array([ 5.02, -1.59])
Question 3

The magnitude and direction angle of $\vec{v}$ and $\vec{w}$ are given below in the form $(r, \theta)$ where $r$ is the magnitude and $\theta$ is the direction angle.

  • $\vec{v}$ = $(7, 30^\circ)$
  • $\vec{w}$ = $(4, 140^\circ)$

Find the component form of $\vec{v}$ + $\vec{w}$.

So

  • The angle $\theta$ for $\vec{v}$ to the $x$ axis is $30^\circ$
    • $x$ component of $\vec{v}$ is positive
    • $y$ component of $\vec{v}$ is positive
  • The angle $\theta$ for $\vec{w}$ to the $y$ axis is $50^\circ$
    • $x$ component of $\vec{w}$ is negative
    • $y$ component of $\vec{w}$ is positive
In [29]:
adj_v = math.cos(math.radians(30)) * 7
opp_v = math.sin(math.radians(30)) * 7
v = np.array([adj_v, opp_v])
v
Out[29]:
array([6.06217783, 3.5       ])
In [30]:
opp_w = math.sin(math.radians(50)) * 4
adj_w = math.cos(math.radians(50)) * 4
w = np.array([-opp_w, adj_w])
w
Out[30]:
array([-3.06417777,  2.57115044])
In [31]:
(v + w).round(2)
Out[31]:
array([3.  , 6.07])

Worked solution for Question 3:

image.png

Question 4:

image.png

So

  • The angle $\theta$ for $\vec{v}$ to the $x$ axis is $10^\circ$
    • $x$ component of $\vec{v}$ is negative
    • $y$ component of $\vec{v}$ is positive
  • The angle $\theta$ for $\vec{w}$ to the $y$ axis is $10^\circ$
    • $x$ component of $\vec{w}$ is negative
    • $y$ component of $\vec{w}$ is negative
In [32]:
adj_v = math.cos(math.radians(10)) * 5
opp_v = math.sin(math.radians(10)) * 5
v = np.array([-adj_v, opp_v])
v
Out[32]:
array([-4.92403877,  0.86824089])
In [33]:
opp_w = math.sin(math.radians(10)) * 6
adj_w = math.cos(math.radians(10)) * 6
w = np.array([-opp_w, -adj_w])
w
Out[33]:
array([-1.04188907, -5.90884652])
In [34]:
(v + w).round(2)
Out[34]:
array([-5.97, -5.04])