I just saw this graphic and thought it could be interesting to some of you:
GPT-4 is expected to come out within the next few months. The numbers are the parameters used by each one of these 2 models. Amazing.
Source: LinkedIn post.
1 Like
Hi, @Juan_Olano !
Sure! The performance of such big model will be amazing. I’m afraid that we won’t be able to run it in our modest nvidia gpus 
even if they release it, which they probably won’t
Wow. That’s crazy. So they are going from 175 billion to 100 trillion parameters. So if each parameter is even a puny 32 bit float, that’s 400 terabytes of storage just to store the parameters of the model. 
The mind boggles.
The performance had better be pretty amazing to justify the compute and storage cost.
It is really crazy. It seems it will be capable of creating much larger original content.