C4W3 Lab2 : upstream connect error or disconnect/reset before headers. reset reason: connection terminationstudent_00_8847b1a44e8e@cloudshell

Hi mentors,

I have a problem when I want to run this command “curl -d @payloads/request-body.json -X POST http://$GATEWAY_URL/v1/models/image_classifier:predict” and I got this ERROR:
upstream connect error or disconnect/reset before headers. reset reason: connection terminationstudent_00_8847b1a44e8e@cloudshell

Could you help me how to solve it?
Thank you in advance

Hi Atefeh! Did you check if there is still time remaining on your lab? Also, are all your GCP windows logged in with the credentials Qwiklabs gave? Sometimes it switches to your personal account and that causes this type of issue. Please click on the profile icon on the right to check.

Screen Shot 2021-09-27 at 8.46.35 PM

One way to avoid this is to do the Qwiklabs in an Incognito or private browsing session to make sure that GCP does not see previously logged-in Gmail accounts.

When you switch accounts, also check if the project selected is also pointed to the Project ID given to you by Qwiklabs. It has the pattern qwiklabs-gcp-<identifier> as shown in the screenshot below. In some instances, we found that this can be unselected when you switch accounts.

Hope this helps!

Hi Chris,

Thank you for your response. I will try that.

Hello @chris.favila,

I am having the same problem, I made sure that the account is correct…
Is there any other solution we can think of?

Anyway @AtefehVhd, you can continue the assignment. When we find a solution, we can go back to this.

Thank you.

Hi Ammar and Atefeh! I’ll check this today and update you asap. Thanks!

1 Like

Hi sorry for the delay. I’m checking this now…

Hi Ammar and Atefeh! Unfortunately, I can not replicate this issue. The predictions returned as expected:

Some things I can think of that might need double checking:

  1. Is the deployment marked as READY when you do a kubectl get deployment? If not, please check the configmap-resnet50.yaml file and make sure that you’re replacing the bucket location with the output of echo $MODEL_BUCKET. Make sure that you’re prepending it with a gs:// and appending with /resnet_50 as well.
  2. Have you applied the gateway and virtualservice YAMLs before this command?
  3. Did you execute these commands and got a gateway URL? It should look something like 35.132.243.148:80 when you run echo $GATEWAY_URL.

  1. Are you executing the curl command from the /home/<student_id>/tfserving-canary directory? Please run pwd to verify.

If you answered Yes to all the above, then you might need to report it to the Qwiklabs support team for checking. You can find the chat support on the upper right:

But I hope this is just a matter of retrying the lab. That has happened to me before but with a different error message. I did not change anything in my approach and the error just disappeared on my 2nd attempt. I hope that is also the case here.

Hope this helps!

Hi Chris,

Thank you for your help.
I went through step by step and I was sure I did not make mistake. However, I got the Error again. But, when I checked the assignment, it was correct. I do not have a clue what happened. Fortunately, I could finish my assignment and got my score.

HI Atefeh! That is strange. Maybe it has something to do with your network settings? Are you behind a proxy server? Not really sure what might be the issue. Will monitor this and report to Qwiklabs as needed. But glad you got the score! For reference, that command will just show the probabilities as shown in the screenshot above. When you use resnet-101, then the results will be different. The rest of the lab shows how you can serve both models and setup canary testing. Hope you still got the core concepts even if you can’t access the output.

Glad to help!

I got the same connection issue. Not able to get the prediction as it always gives the connection error.
Error:
upstream connect error or disconnect/reset before headers. reset reason: connection terminationstudent_03_45c182be509f

However I was able to completed my exercise

Hi Chris,

I’m getting the same error as people have noted in this thread:
upstream connect error or disconnect/reset before headers. reset reason: connection terminationstudent_00_bd60f4e74f73@cloudshell

I have rechecked all the points you made above and I’m sure I havent made a mistake in any of the steps. Any suggestions as to why it is happenning?

Thanks,
Shresht