While the first test of compute_cost function I wrote returns the expected output, 0.693, the subsequent one returns 2.759 instead of the expected output, 0.218, hence the assertionError. Why is this so?

Again the test for compute_cost_reg I wrote returns 0.821 instead of Expected output of 0.661. Please can anyone tell me what I am doing wrong?

The function Compute_cost_reg() calls compute_cost() to return cost without regulation. If your implementation of compute_cost() is incorrect, then it would produce the error as reported.
Please check the comput_cost() against the hints provided, and make sure there is no hard coded value used and that your code follows the formula as exactly.