C2_W4_Decision_Tree_with_Markdown - Help in submitting the assignment

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

Hi everyone.

I’m working on the last assignment on Decision Trees, and I’m having a really difficult time with the Information Gain section toward the end. It gives me error messages like Weighted_Gain is being referenced without a value and such. I keep trying to submit the assignment since most of it is right, but it keeps giving me error messages. I really want to know what I’m doing wrong in the programming so I can complete the assignment and submit it, so I can pass the course. I’m stuck and I’m also trying to follow the code of ethics.

1 Like
2

Hello, Ben @hogansonb,

Welcome to our community! How this place works is that we will first need to see the full error traceback. You may either copy and paste the whole of it in your next reply, or you may take screenshot(s) to cover the whole of it and share them. Programming in Python demands accuracy, for example, “Weighted_Gain” is not required by any exercise and we usually name variables in lower case only, so description as such would not be very helpful for us to understand the situation. We need the full error traceback as the first step. :wink:

Cheers,
Raymond

3

Thank you! Here is the traceback. The errors I’ve received have been along the lines of variables were referenced that were not previously assigned, or zero division errors. In this case, the code runs, though all information gain values for each node are around 0.50, which is not the expected output, the highest being in the .2Xs.

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-107-2a50df79e115> in <module>
      9 
     10 # UNIT TESTS
---> 11 compute_information_gain_test(compute_information_gain)

~/work/public_tests.py in compute_information_gain_test(target)
     94     node_indexes = list(range(5))
     95 
---> 96     result1 = target(X, y, node_indexes, 0)
     97     result2 = target(X, y, node_indexes, 0)
     98 

<ipython-input-106-2aa6177752ab> in compute_information_gain(X, y, node_indices, feature)
     40     w_right = len(right_indices)/len(X)
     41     left_entropy = sum(y[left_indices])/len(left_indices)
---> 42     right_entropy = sum(y[right_indices])/len(right_indices)
     43 
     44     weighted_entropy = w_left * left_entropy + w_right * right_entropy

ZeroDivisionError: division by zero
4

Thanks. Let’s focus on just the error reported by Python, which is the division by zero.

sum(y[right_indices])/len(right_indices) is not the way to compute an entropy. The same goes to the line for left_entropy. You should have implemented the algorithm as a function in exercise 1. Do you know how to use a function?

1 Like
5

Besides, you may want to check this line again:

image

because it does not meet with the exercise’s description:

image

2 Likes
6

Thank you. The hints in the assignment say: node_entropy = compute_entropy(y_node). I know slightly how to use functions: first define, and then call them.

{edited by mentor}

print(entropy(0.5))

1 Like
7

Yes! That’s the idea, only you need to make sure the argument given to the function is correct. Please be noted that you will need to use it for both left_entropy and right_entropy, even though your last error traceback was only pointing at right_entropy.

8

Please don’t share your solution publicly as it’s not allowed here. I have thus removed it for you.

1 Like
9

I suggest you to make the changes to left_entropy and right_entropy first, and then you review the line for w_right (and perhaps also w_left) as per this post.

10

Suspecting a denominator other than X.

1 Like
11

That’s the right direction! X represents all data, but we are only considering a subset of it and that subset is defined by node_indices.

I have just drawn something to show you the relations between all those variables’ names. It’s terrible drawing but I hope it can deliver the message.

image
image1564×774 119 KB

Now, think again how to compute that proportion according to the exercise description.

12

Thank you for your help! I will work on this again tomorrow.

13

Okay, Ben @hogansonb . Please take your time. When you work on it again, just so you have something to cross check your work, I am sharing some printouts with you. You may not need them if you just finish the exercise correctly.

Based upon the code skeleton available to you in the hints section underneath the exercise’s code cell, I have added many prints like below:

image
image503×380 8.95 KB

image
image421×411 8.51 KB

The idea here is simple - we print all variables, so you can add them easily as well.

Then, with all the code filled in, I run the testing cell below, and get the following printouts, you may then crosscheck your numbers with mine. The idea is that, you find inconsistent value, you know where to double check.

This may serve as the first aid as I may not be around tomorrow. Note that the printout is long, because there are a couple of test cases, but they are separated by dash lines.

====================================================================================================
Input arguments:
X [[1 1 1]
 [1 0 1]
 [1 0 0]
 [1 0 0]
 [1 1 1]
 [0 1 1]
 [0 0 0]
 [1 0 1]
 [0 1 0]
 [1 0 0]]
y [1 1 0 0 1 0 0 1 1 0]
node_indices [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
feature 0
=====End of Input arguments=====
left_indices [0, 1, 2, 3, 4, 7, 9]
right_indices [5, 6, 8]
X_node [[1 1 1]
 [1 0 1]
 [1 0 0]
 [1 0 0]
 [1 1 1]
 [0 1 1]
 [0 0 0]
 [1 0 1]
 [0 1 0]
 [1 0 0]]
y_node [1 1 0 0 1 0 0 1 1 0]
X_left [[1 1 1]
 [1 0 1]
 [1 0 0]
 [1 0 0]
 [1 1 1]
 [1 0 1]
 [1 0 0]]
y_left [1 1 0 0 1 1 0]
X_right [[0 1 1]
 [0 0 0]
 [0 1 0]]
y_right [0 0 1]
=====End of given variables=====
node_entropy 1.0
left_entropy 0.9852281360342515
right_entropy 0.9182958340544896
w_left 0.7
w_right 0.3
weighted_entropy 0.965148445440323
information_gain 0.034851554559677034
Information Gain from splitting the root on brown cap:  0.034851554559677034
====================================================================================================
Input arguments:
X [[1 1 1]
 [1 0 1]
 [1 0 0]
 [1 0 0]
 [1 1 1]
 [0 1 1]
 [0 0 0]
 [1 0 1]
 [0 1 0]
 [1 0 0]]
y [1 1 0 0 1 0 0 1 1 0]
node_indices [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
feature 1
=====End of Input arguments=====
left_indices [0, 4, 5, 8]
right_indices [1, 2, 3, 6, 7, 9]
X_node [[1 1 1]
 [1 0 1]
 [1 0 0]
 [1 0 0]
 [1 1 1]
 [0 1 1]
 [0 0 0]
 [1 0 1]
 [0 1 0]
 [1 0 0]]
y_node [1 1 0 0 1 0 0 1 1 0]
X_left [[1 1 1]
 [1 1 1]
 [0 1 1]
 [0 1 0]]
y_left [1 1 0 1]
X_right [[1 0 1]
 [1 0 0]
 [1 0 0]
 [0 0 0]
 [1 0 1]
 [1 0 0]]
y_right [1 0 0 0 1 0]
=====End of given variables=====
node_entropy 1.0
left_entropy 0.8112781244591328
right_entropy 0.9182958340544896
w_left 0.4
w_right 0.6
weighted_entropy 0.8754887502163469
information_gain 0.12451124978365313
Information Gain from splitting the root on tapering stalk shape:  0.12451124978365313
====================================================================================================
Input arguments:
X [[1 1 1]
 [1 0 1]
 [1 0 0]
 [1 0 0]
 [1 1 1]
 [0 1 1]
 [0 0 0]
 [1 0 1]
 [0 1 0]
 [1 0 0]]
y [1 1 0 0 1 0 0 1 1 0]
node_indices [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
feature 2
=====End of Input arguments=====
left_indices [0, 1, 4, 5, 7]
right_indices [2, 3, 6, 8, 9]
X_node [[1 1 1]
 [1 0 1]
 [1 0 0]
 [1 0 0]
 [1 1 1]
 [0 1 1]
 [0 0 0]
 [1 0 1]
 [0 1 0]
 [1 0 0]]
y_node [1 1 0 0 1 0 0 1 1 0]
X_left [[1 1 1]
 [1 0 1]
 [1 1 1]
 [0 1 1]
 [1 0 1]]
y_left [1 1 1 0 1]
X_right [[1 0 0]
 [1 0 0]
 [0 0 0]
 [0 1 0]
 [1 0 0]]
y_right [0 0 0 1 0]
=====End of given variables=====
node_entropy 1.0
left_entropy 0.7219280948873623
right_entropy 0.7219280948873623
w_left 0.5
w_right 0.5
weighted_entropy 0.7219280948873623
information_gain 0.2780719051126377
Information Gain from splitting the root on solitary:  0.2780719051126377
====================================================================================================
Input arguments:
X [[1 0]
 [1 0]
 [1 0]
 [0 0]
 [0 1]]
y [[0]
 [0]
 [0]
 [0]
 [0]]
node_indices [0, 1, 2, 3, 4]
feature 0
=====End of Input arguments=====
left_indices [0, 1, 2]
right_indices [3, 4]
X_node [[1 0]
 [1 0]
 [1 0]
 [0 0]
 [0 1]]
y_node [[0]
 [0]
 [0]
 [0]
 [0]]
X_left [[1 0]
 [1 0]
 [1 0]]
y_left [[0]
 [0]
 [0]]
X_right [[0 0]
 [0 1]]
y_right [[0]
 [0]]
=====End of given variables=====
node_entropy 0.0
left_entropy 0.0
right_entropy 0.0
w_left 0.6
w_right 0.4
weighted_entropy 0.0
information_gain 0.0
====================================================================================================
Input arguments:
X [[1 0]
 [1 0]
 [1 0]
 [0 0]
 [0 1]]
y [[0]
 [0]
 [0]
 [0]
 [0]]
node_indices [0, 1, 2, 3, 4]
feature 0
=====End of Input arguments=====
left_indices [0, 1, 2]
right_indices [3, 4]
X_node [[1 0]
 [1 0]
 [1 0]
 [0 0]
 [0 1]]
y_node [[0]
 [0]
 [0]
 [0]
 [0]]
X_left [[1 0]
 [1 0]
 [1 0]]
y_left [[0]
 [0]
 [0]]
X_right [[0 0]
 [0 1]]
y_right [[0]
 [0]]
=====End of given variables=====
node_entropy 0.0
left_entropy 0.0
right_entropy 0.0
w_left 0.6
w_right 0.4
weighted_entropy 0.0
information_gain 0.0
====================================================================================================
Input arguments:
X [[1 0]
 [1 0]
 [1 0]
 [0 0]
 [0 1]]
y [[0]
 [1]
 [0]
 [1]
 [0]]
node_indices [0, 1, 2, 3, 4]
feature 0
=====End of Input arguments=====
left_indices [0, 1, 2]
right_indices [3, 4]
X_node [[1 0]
 [1 0]
 [1 0]
 [0 0]
 [0 1]]
y_node [[0]
 [1]
 [0]
 [1]
 [0]]
X_left [[1 0]
 [1 0]
 [1 0]]
y_left [[0]
 [1]
 [0]]
X_right [[0 0]
 [0 1]]
y_right [[1]
 [0]]
=====End of given variables=====
node_entropy 0.9709505944546686
left_entropy 0.9182958340544896
right_entropy 1.0
w_left 0.6
w_right 0.4
weighted_entropy 0.9509775004326937
information_gain 0.01997309402197489
====================================================================================================
Input arguments:
X [[1 0]
 [1 0]
 [1 0]
 [0 0]
 [0 1]]
y [[0]
 [1]
 [0]
 [1]
 [0]]
node_indices [0, 1, 2, 3, 4]
feature 1
=====End of Input arguments=====
left_indices [4]
right_indices [0, 1, 2, 3]
X_node [[1 0]
 [1 0]
 [1 0]
 [0 0]
 [0 1]]
y_node [[0]
 [1]
 [0]
 [1]
 [0]]
X_left [[0 1]]
y_left [[0]]
X_right [[1 0]
 [1 0]
 [1 0]
 [0 0]]
y_right [[0]
 [1]
 [0]
 [1]]
=====End of given variables=====
node_entropy 0.9709505944546686
left_entropy 0.0
right_entropy 1.0
w_left 0.2
w_right 0.8
weighted_entropy 0.8
information_gain 0.17095059445466854
====================================================================================================
Input arguments:
X [[1 0]
 [1 0]
 [1 0]
 [0 0]
 [0 1]]
y [[0]
 [1]
 [0]
 [1]
 [0]]
node_indices [0, 1, 2, 3]
feature 0
=====End of Input arguments=====
left_indices [0, 1, 2]
right_indices [3]
X_node [[1 0]
 [1 0]
 [1 0]
 [0 0]]
y_node [[0]
 [1]
 [0]
 [1]]
X_left [[1 0]
 [1 0]
 [1 0]]
y_left [[0]
 [1]
 [0]]
X_right [[0 0]]
y_right [[1]]
=====End of given variables=====
node_entropy 1.0
left_entropy 0.9182958340544896
right_entropy 0.0
w_left 0.75
w_right 0.25
weighted_entropy 0.6887218755408672
information_gain 0.31127812445913283
====================================================================================================
Input arguments:
X [[1 0]
 [1 0]
 [1 0]
 [0 0]
 [0 1]]
y [[0]
 [1]
 [0]
 [1]
 [0]]
node_indices [0, 1, 2, 3]
feature 1
=====End of Input arguments=====
left_indices []
right_indices [0, 1, 2, 3]
X_node [[1 0]
 [1 0]
 [1 0]
 [0 0]]
y_node [[0]
 [1]
 [0]
 [1]]
X_left []
y_left []
X_right [[1 0]
 [1 0]
 [1 0]
 [0 0]]
y_right [[0]
 [1]
 [0]
 [1]]
=====End of given variables=====
node_entropy 1.0
left_entropy 0.0
right_entropy 1.0
w_left 0.0
w_right 1.0
weighted_entropy 1.0
information_gain 0.0
 All tests passed.
1 Like
14

Three additional notes:

  1. If you add the prints like I did, remember to remove them after debugging is done, as they may interfere when the autograder processes your submission.
  2. If you get new errors, I suggest you to try the prints, locate the problem, and solve it yourself. If you need help, I recommend you to open a new topic with the latest full error traceback attached.
  3. For other exercises, you may add the prints as well, only this time you may need to work out the expected values yourself. Note that in my example, I am printing the inputs and that’s essential for your own calculation. The idea is, you see the inputs of all the public test cases, pick the one easier for you to calculate, work out the expected values based on the logic / algorithm explained to you in the exercise, check the printed values against the expected ones.
    Adding prints and checking numbers are useful debugging skills. :wink:

Good luck!
Raymond

15

Thank you! I went back and reviewed the videos for Measuring Purity, Choosing a Split, and Putting it all Together, and took lots of notes. Computing left_entropy and right_entropy are what I’m going to tackle next. node_entropy is already taken care of. I imagine the left and right entropies are similar to node entropy, but I need to make sure I’m referencing the right indices, so I’m not just recalculating the node entropy.

16

You are welcome!

It looks like you are making good progress!

Please take your time and do it your way. My last two replies might not be so relevant right now or even not ever, but they are really only as an immediate first aid you can reference on if your solution didn’t pass any public tests because, afterall, I do not want your momentum to be wasted on waiting for a response from me as I don’t station here all the time. In this sense, I hope my last two replies would never be useful :wink: :wink: because you can finish the exercises yourself which is really what we hope.

Good luck, Ben!

Raymond

17

Thank you for your help. I haven’t been successful at it, yet, and I need to step away for a few days and come back to it from a fresh perspective. Whenever I try submitting it, it fails to compile and gives me a 0 score. I’ve tried over a dozen times, so I need to take a break, maybe do the next class, and come back to this assignment.

1 Like
18

Sure, Ben! Please do it at your own pace. If it fails to compile, there must be an error and I suggest you to either share it with us on a new post or google for how others tackle it.

Cheers,
Raymond