How are weights calculated for linear regression?


I have a confusion that how the weights (a and b) for equation (y= a+bx) are calculated in linear regression Machine learning Algorithm -

  1. by solving the linear equation a = mean (y) - b * mean(x) and b = correlation *(std dev of y /std dev of x)

  2. The weights are first arbitrarily taken and then cost function J(theta) is used to minimize the weights depending on the adjustment of the best fit line on the dataset.

1 Like

Hi @karan88,

I’ll put it in very simple steps to give a high-level overview of the process (and share the articles which explain in much more detail)

  1. You will have the coefficients (or weights) a, b, and a loss function J(theta).
  2. Randomly assign the values for a and b, and calculate the loss.
  3. Now, update the values of a and b, such that the loss value decreases.
  4. Repeat the process till you find the best combination of a and b, such that the loss is minimum.

This process of iterating continuously to reach the minimum loss is called gradient descent. Refer this article: