Simple linear regression problem: Gradient Descent implementation error

[tennis_geek]

For seasoned Python programmers, you can skip the below Background and directly look into attached screenshots.
Background: My code has a main function. AFrom this main, a function call is made to a function called Gradient descent where all iterations and updations of w and b happen with eventual Jcost function computations. Now there should be some stop criterion to be applied (This is where error happens as far as I understood) …now, when this gradient function has to return back the optimized w & b values to the main function…a prediction is to be made with set of new input values with help of the returned optimized w&b parameters.

Attaching screen shots of function definitions and the error I get upon executing the code.

[rmwkwok]

Hello SreeHarish @tennis_geek.

This error can raise when you ask whether a numpy array of more than 1 element is True or False, for example:

import numpy as np

ary = np.array([1, 2, 3])
if ary > 2:
    print('True')

running the above code will also have the same error as below

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [2], in <cell line: 4>()
      1 import numpy as np
      3 ary = np.array([1, 2, 3])
----> 4 if ary > 2:
      5     print('True')

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

It’s likely that your diff_costfunction being an array had caused the error. You may need to apply .any() or .all() to the array as the error message suggests to decide how you want to handle an array of boolean values in an if statement; or if it is not your intention for it to be an array then you might need to correct some lines such as the one you assign value to sumof_squared_error.

Lastly, you might share your own code in the way I did in above, so that others can just copy your code and run it in their machine to test it. You can wrap you code with three ` symbols before the start of your code and another three ` symbols after the end of your code to format it properly.

Raymond

[tennis_geek]

Thank you for the reply. I tried it this way (all those in bold)… and it is working now without errors and could get predictions too.

# Import libraries
import numpy as np; # Mostly used for array manipulation datatype numpy.ndarray
from sklearn.linear_model import LinearRegression # Used for implementing Linear regression
import matplotlib.pyplot as plt # Plotting in python3 using Matplotlib library

def grad_descent(x_array_reshaped,y_array_reshaped,trainingsamplesize,Jcostfunctionnew,max_iterations):

    # Initialize both w and b to 0
    w = 0;
    b = 0;
    term1 = (1/2*1/trainingsamplesize); # Part 1 of the J cost function definition involving sample size 
    learningrate = 0.000000000001;
    stop_threshold = 1e-10;

    error = 0; #bolded
    squared_error = 0; #bolded
    sumof_squared_error = 0; #bolded
    diff_costfunction = 0; #bolded
    
    # Iteration start
    counter = 0 ;
    for i in range(max_iterations):
        ypredictions=w*x_array_reshaped+b;
        print("Y responses:",ypredictions);
        error = (ypredictions-y_array_reshaped); # Part 2.1 of the cost function J definition
        #Below comments are 1-time intuitive checks
        #print("Expected Error size: a vector");
        #print("Size of Error:",len(error));
        squared_error = np.square(error); # Part 2.2 of the cost function J definition
        #Below comments are 1-time intuitive checks
        #print("Expected SE size: a vector");
        #print("Size of Squared Error:",len(squared_error));
        # Below LHS SumofSquaredError shall be a scalar
        sumof_squared_error = np.sum(squared_error); # Part 2.3 of the cost function J definition
        Jcostfunctionold = Jcostfunctionnew;
        Jcostfunctionnew = term1 * sumof_squared_error;
        #print("Cost function:",Jcostfunctionnew);
        diff_costfunction=**abs**(Jcostfunctionold-Jcostfunctionnew);
        #print("Costfunction difference:",diff_costfunction);
        #print(diff_costfunction);
        #print("diff_costfunction size",len(diff_costfunction));
        if diff_costfunction <= stop_threshold:
            break
    
     # Update w
        commonterm = (learningrate*1/trainingsamplesize);
        termforupdatingw = sum(error*ypredictions);
        w = w-commonterm*termforupdatingw;
        print("Updated w:",w);
    
    # Update b
        termforupdatingb = sum(error);
        b = b-commonterm*termforupdatingb;
        print("Updated b:",b);
        print("Iteration number:",counter);
        counter = counter+1;
    # Iteration end
    return w, b
    def main():
    
    # Input training data consisting of X inputs and corresponding Y outputs
    
    # input 
    housearea=np.array([1100, 1150, 1200, 1250, 1280, 1300, 1350, 1380, 1420, 1450, 1480, 1500, 1510, 1540, 1580, 1600, 1620, 1650, 1680, 1700, 1710, 1730, 1750, 1780, 1790, 1810, 1820, 1850, 1890, 1910, 1930, 1950, 1980, 2000, 2100, 2250, 2350, 2480, 2700, 2810, 2850, 2900, 3000]).reshape(-1,1);
    #output
    housecost=np.array([120000, 150000, 165000, 170000, 175000, 180000, 185000, 190000, 195000, 200000, 210000, 215000, 220000, 225000, 235000, 240000, 250000, 255000, 260000, 265000, 270000, 275000, 285000, 290000, 300000, 310000, 315000, 320000, 330000, 345000, 350000, 365000, 370000, 380000, 395000, 400000, 405000, 410000, 415000, 420000, 430000, 440000, 445000]).reshape(-1,1);
    #samplesize
    trainingsamplesize=len(housearea);
    #InitialJcostfunction = 0
    Jcostfunction=0;
    w, b=grad_descent(housearea,housecost,trainingsamplesize,Jcostfunction,max_iterations=50000000)
    
    print("Estimated w",w);
    print("Estimated b",b);
    
        
    # Making predictions using estimated parameters w & b
    newinputarray=np.array([1205,1281,1327,1391,1579,1631,1704,1881,1912,2005,2623,2781,2867]);
    newhousearea=newinputarray.reshape(-1,1);
    
    predicted_housecost=w*newhousearea + b # w slope and b y-intercept which are parameters coming out of grad descent algorithm
    # predicted house cost for any given housearea values for the settled w & b.
    
    # print("House cost predictions:",predicted_housecost);
    
    
    plt.plot(housearea,housecost,'bo'); 
    plt.xlabel('house sq.metres'); 
    plt.ylabel('house cost in USD');
    plt.xlim(800, 3500), plt.ylim(110000, 500000);
    plt.title('Response in Red amongst Blue training');
    plt.plot(newhousearea,predicted_housecost,'r*');
    plt.show();
       
       
    
if __name__ == "__main__":
    main()
    

[rmwkwok]

You are welcome @tennis_geek. I edited your post for formatting. You might click the edit button (in the lower right corner of the post) for your latest post to see my change.

Raymond