Image enhancement using deep learning

I have some x-ray images which are of (3072 x 3072) resolution. My task is to increase the spatial resolution (number of line pairs per mm). I came across several methods such as SWIN-IR transformer, ESRGAN, etc. but the problem is that since the input images are of very high resolution, it is impossible to carry out the task even on a system with 32 gb RAM. What are the possible alternatives?

1 Like

Have you considered preprocessing the image by splitting it in small patches of, say, 256x256, amplify each patch, and then concatenate at the end with some image manipulation software?

1 Like

Or get a bigger computer (“I think we’re gonna need a bigger boat!” :sweat_smile:). Of course the easiest way to do that is to use an online service like Colab or AWS. If your resource requirements are really large, you may have to pay for that service, but it will be a lot cheaper and more flexible than buying the new computer yourself. What if you buy a 64GB computer and then discover you later need even more? You can use Colab without paying and even have access to GPUs, it’s just that you may have to wait in line behind the paying customers when things are busy.

1 Like

Yes that is a good way to proceed. Thanks @Juan_Olano !

I’m already using colab pro but things are not as bright as I expected them to be :sweat_smile:
I guess what Juan suggested is a possible solution to this problem.

I have not tried solving any large problems on Colab. It would be useful information to know more about your experiences trying this even on the paid version of Colab. What problems did you encounter there that make it “not as bright” as you were expecting?

The problem I have had with Colab in the free mode is that everything is time limited and it will time out your session if it detects no interactive input over some relatively short period of time (single digit hours). That makes it hard to do long training runs. There are some “hacks” for faking interactive input, but the more general solution is to add logic to periodically checkpoint your results and make the training restartable. Of course that’s a good thing to do anyway, since you need the ability to save the trained model at any point in the training.

Of course Juan’s solution is a general method definitely worth pursuing, as it may have lots of applications even beyond your current specific case.

1 Like

Hi Paul!
I wish to share my experience using colab pro. I recently purchased it and I learnt many things about handling resources. Basically, we get 100 compute units per month with colab pro. We have access to SOTA GPUs here like the Nvidia Tesla A100 GPUs (This is the maximum access we will have with colab pro)

Just a comparison :

Free version :

Colab pro - But runtime with no accelerators and normal RAM

Colab pro - Max resources


GPU RAM is 40 GB here.

So let me explain my case for an example. I have this very large model - ResNet 50 and the image dataset is large too (around 75k) and I was feeding images of shape 256x256x3 to the model.
I was not able to run this in colab free or in my local jupyter notebook due to VRAM constraints. So I tried using Kaggle accelerators and it takes 13hrs to run a single epoch. But Kaggle comes with 2 GPUs so I tried using distributed strategy and it was taking 6 and a half hours approximately to run a single epoch. The sad part here is that Kaggle stops the background running after 7 hrs and saving checkpoints and accessing it back in Kaggle is quite a headache (At least it was difficult for me to do so). So I tried google colab pro.

I used max settings in my colab pro (figure 3) and it took around 1 hr only to run an epoch and you can easily save checkpoints and access it back. But the problem here is that it consumes compute units at the rate of 13.08 units per hour. So running a notebook for 7 hrs in total with max settings would exhaust all the 100 compute units and that’s the sad part. Even with no accelerators and standard RAM,colab pro consumes 0.08 compute units per hour and that’s why we have to plan carefully - I had to use my other account to access free colab for purposes which don’t need compute units. So it becomes a very expensive process (with respect to me).

Thanks for sharing your experience @Nithin_Skantha_M , very interesting.

So you are working images of 256x256x3 and it still takes a lot of power. Interesting.

1 Like

You could also open up a GCP account. You get 300 USD to spend within 3 months. Free! To get access to GPU you need to ask for a quota increase which is pretty straightforward.

You could also try TPU’s they have a pretty insane amount of RAM and it doesn’t take much to convert your code.

Hi Paul. My experience with Colab pro is pretty much the same what @Nithin_Skantha_M mentioned. Infact, my dataset is pretty huge (around 600 thousand images) and the input size is (256, 256, 3). When I trained with around 250 thousand images in colab pro (RAM - 25 gb, GPU RAM - 15 gb, disk space - 167 gb) it took around 8 hours to run a single epoch. I explored Vertex AI workbench of GCP but wasn’t able to find GPU resources in any region. So I’m on the lookout for further alternatives.

1 Like