C4C2 Graded Lab Autoscaling TensorFlow Model Deployments with TF Serving and Kubernetes Exceptions

Course Q&A Machine Learning Engineering for Production(MLOps) Deploying Machine Learning Models in Production
1

When I am running locust I frequently get the following ERROR Message:

*[2023-09-23 13:42:29,178] cs-296442668656-default/ERROR/locust.user.task: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File “/usr/local/lib/python3.9/dist-packages/requests/models.py”, line 971, in json
return complexjson.loads(self.text, **kwargs)
File “/usr/lib/python3.9/json/init.py”, line 346, in loads
return _default_decoder.decode(s)
File “/usr/lib/python3.9/json/decoder.py”, line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “/usr/lib/python3.9/json/decoder.py”, line 355, in raw_decode
raise JSONDecodeError(“Expecting value”, s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/home/student_02_16d17326d933/.local/lib/python3.9/site-packages/locust/user/task.py”, line 290, in run
self.execute_next_task()
File “/home/student_02_16d17326d933/.local/lib/python3.9/site-packages/locust/user/task.py”, line 315, in execute_next_task
self.execute_task(self._task_queue.pop(0))
File “/home/student_02_16d17326d933/.local/lib/python3.9/site-packages/locust/user/task.py”, line 428, in execute_task
task(self.user)
File “/home/student_02_16d17326d933/tfserving-gke/locust/tasks.py”, line 69, in predict
if ‘error’ in response.json():
File “/usr/local/lib/python3.9/dist-packages/requests/models.py”, line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[2023-09-23 13:42:29,180] cs-296442668656-default/ERROR/locust.user.task: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File “/usr/local/lib/python3.9/dist-packages/requests/models.py”, line 971, in json
return complexjson.loads(self.text, *kwargs)
File “/usr/lib/python3.9/json/init.py”, line 346, in loads
return _default_decoder.decode(s)
File “/usr/lib/python3.9/json/decoder.py”, line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “/usr/lib/python3.9/json/decoder.py”, line 355, in raw_decode
raise JSONDecodeError(“Expecting value”, s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/home/student_02_16d17326d933/.local/lib/python3.9/site-packages/locust/user/task.py”, line 290, in run
self.execute_next_task()
File “/home/student_02_16d17326d933/.local/lib/python3.9/site-packages/locust/user/task.py”, line 315, in execute_next_task
self.execute_task(self._task_queue.pop(0))
File “/home/student_02_16d17326d933/.local/lib/python3.9/site-packages/locust/user/task.py”, line 428, in execute_task
task(self.user)
File “/home/student_02_16d17326d933/tfserving-gke/locust/tasks.py”, line 69, in predict
if ‘error’ in response.json():
File “/usr/local/lib/python3.9/dist-packages/requests/models.py”, line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I restarted the lab to make sure I was not mistaken anywhere, but the ERROR is the same again, and I don’t see a second pod getting started. I think it is coming of the ERROR in execute_next_task

2

Hello @Peter_Grabner
It appears something wrong with your server response (is either empty/non JSON)
Please try to print its response to confirm whether it’s giving a JSON format response

3

Good I will, just will take some time as my holidays are over and I am just starting in a new project so I will be quite busy till the weekend. But I will not forget to send you more information. Anyway as it is not a colab but a google cloud training, I might have to report it to them, I just didn’t find out were to do it.