This course helps us build models for different serving environments. Models are pushed as tensorflow js (node environment), as tensorflow lite (for mobile app developemnt), and tensorflow serving for “server farm”.
I want to push models for web development in Django environment.
My question: As Django developer, to which form of deployment should I push: tensorflow lite, tensorflow JS, tensorflow serving?
Hi @Getachew! In my opinion the one that seems to best suit your usecase is tf serving or just plain tf.
tf lite is meant to be used in cases were the hardware resources are minimal such as mobile devices or microcontrollers. tf JS is mostly used to easily load your model directly into the JS part of the app so you can load models directly into the front end, eliminating the need for a service in the backend.
Depending on what you need you might prefer tf serving or vanilla tf. With vanilla tf you will manage all of the load balancing stuff through your django app, you simply need to load the model in an endpoint and treat it as any other service.
The advantage of using tf serving is that it is already optimized to handle a lot of traffic and you could not link it to the django app but have a standalone service which is called from the front-end. You can also (I assume) map the endpoint to one inside your app if you really need the integration with django.
@a-zarta , thank you so much; this is helpfull. I think Tf serving is the right way. So, linking from front end to a different service requires me another domain name (for the ai app), right? From my reading of your response, it is just a the same as putting a link to an external web page. Is there a way to make my ml app as a django app within the same django project still using tf serving?
Not sure if I have got the last point (on mapping), however.
Thanks again for responfing.
I think it depends on the architecture of your backend. If you are using microservices you can simply spin a container using the tf serving Docker image (this is seen more in-depth in Course 4) and consume the service. If you have a monolith, you just need to know that tf serving basically spins a web server of its own (it can use HTTP or GRPC) and accommodate your VM (or wherever you are hosting your Django app) to handle this webserver alongside the Django server. I guess this will be possible using something like NGINX. If this is not possible I think then you need another domain name but I’m not really sure.
Thanks @a-zarta for the prompt response. Much clearer now. I will start from this and do my further readings. Yes, I will have a lot more info on course 4.
Happy to help! Let me know if some other questions pop up