I’m trying to save a model (in Colab) that I’ve quantized using QuantoConfig. It comes from the notebook referenced on the HuggingFace Quantization Guide (Quanto_integration.ipynb). The notebook states to save a model that’s been quantized, “you just need to call the save_pretrained method.” But when I try this I get an error:
The model is quantized with quanto and is not serializable - check out the warnings from the logger on the traceback to understand the reason why the quantized model is not serializable.
Is there any way of saving a model that’s been quantized? Thanks for any help.