What is generally the computation power required for the various neural networks for different tasks? is that optimizable?

The question is self explanatory.

You mean like how many GPUs?

There is no reliable answer to your question.

Different tasks will require different amounts of computation, based on many factors. Here are a few of them:

  • The size of the data set (the number of features and examples).
  • The complexity of the model.
  • The prediction accuracy you need to achieve.
  • The speed with which you need to get that accuracy (i.e. a real-time system with short deadlines will need higher computation rates than an off-line system that is updated at a lower rate).
  • How often you need to re-train the model to include new training data.