BERT Deployment - Custom Inference

1. Can you provide more explanation why/when using a custom predictor class ?

class SentimentPredictor(Predictor):
    def __init__(self, endpoint_name, sagemaker_session):
        super().__init__(endpoint_name, 
                         sagemaker_session=sagemaker_session, 
                         serializer=JSONLinesSerializer(),
                         deserializer=JSONLinesDeserializer())

2. Why copy inference.py in the code folder in training scripts/container?

 os.makedirs(inference_path, exist_ok=True)
 os.system("cp inference.py {}".format(inference_path))
 os.system("cp requirements.txt {}".format(inference_path))
 os.system("cp config.json {}".format(inference_path))
1 Like

The topic has been closed.