Ordinary Squares project - image missing

Question

Hello everyone,

I'm working on the DS with python course and I am currently a bit confused about the project in the Linear Regression module.
In the task description it says
" Mathematically, the solution is given by the formula in the image, where the superscript T means the transpose of a matrix, and the superscript -1 means it is an inverse of a matrix. "
My problem is not the code or the maths, I simply can't find any image with the formula that is referred to here. It's just not there. Can someone provide the image for me?

Answer 1

I am also struggling to understand the language of these projects. I have no problem understanding the lessons and the exercises and answering the quiz questions, but I don't understand exactly what they want in the projects. When I come over the forums asking to explain the questions, I get someone handing me the code solution, which is not good for learning.

Answer 2

I couldn't find where this text is, by reading the text I imagine the formula it's referring to is the one in the link https://en.m.wikipedia.org/wiki/Ordinary_least_squares in Linear Models under Matrix/vector formulation.

Answer 3

You have this equation:

y = X*β, (1)

where X is a matrix (n, p), β and p are vectors with p and y components, respctivelly. Here '*' denotes multiplications.

The basic code gives you X and y. Your goal is to return β. You have to solve Eq. (1) for β, so you know what code to write.

First, multiply Eq. (1) by X^T, which is the transpose [1] of X:

(X^T)*y = (X^T)*X*β. (2)

Now, multiply by the inverse [2] of the (X^T)*X, which is ((X^T)*X)^(-1):

(((X^T)*X)^(-1))*(X^T)*y = (((X^T)*X)^(-1))*(X^T)*X*β, (3)

leading to

β = (((X^T)*X)^(-1))*(X^T)*y. (4)

Thus, you need to find:

- X^T;
You can write code to do it or one can simply use the function np.transpose(X) or np.array(X).T [3]:

X^T = np.array(X).T; (5)

- ((X^T)*X);
When multiplying matrices we cannot simply use the '*' operator as we use in numbers (scalars). Matrices multiply a little differently [4]. If you want you can do this by hand or you could use another wonderful numpy function, which is the np.matmul(x1, x2) [5], where x1 and x2 are the matrices you want to multiply:

((X^T)*X) = np.matmul(X^T, X) (6)

((X^T)*X)^(-1);
You already have the matrix (X^T)*X, all you have to do is simply calculate its inverse. Again, matrices are a bit different than scalars, so there is a rule to do that [5]. You can write a code to do it or you could use the numpy function np.linalg.inv() [6]:

((X^T)*X)^(-1) = np.linalg.inv((X^T)*X) (7)

After that you just have to multiply everything, i.e., multiply, in order, Eqs. (7) and (6) by the vecor y. Again, remember that when multiplying matrices the order matters, so multiply the factors as they appear in Eq. (4).

I hope this helps! :)

References
[1] https://3c5.com/SDJHH
[2] https://3c5.com/VDAIJ
[3] https://3c5.com/XISlQ
[4] https://3c5.com/BnYCl
[5] https://3c5.com/PiWXz
[6] https://3c5.com/cNEZy

P.s.: I had to use a URL shortener so it could fit everything in one comment.

Answer 4

I came here for the same problem.

Answer 5

n, p = [int(x) for x in input().split()]
X = []
for i in range(n):
X.append([float(x) for x in input().split()])

y = [float(x) for x in input().split()]

import numpy as np

X=np.array(X).reshape(n,p)
y=np.array(y)
b=np.linalg.pinv(X) @ y.transpose()
print(np.around(b,decimals=2))

Answer 6

this is the formula missing from the description

β = (((X^T)*X)^(-1))*(X^T)*y

Answer 7

Copy my code and like check