Super Resolution: Adobe Photoshop versus Leading Deep Neural Networks
How effective is Adobe’s Super Resolution compared to the leading super resolution deep neural network models? This article attempts to evaluate that and the results of Adobe’s Super Resolution are very impressive.
Super Resolution
Super Resolution is the process of improving the quality of a image by enhancing its apparent resolution. Having an algorithm that effectively imagines the detail that would be present if the image was at a higher resolution.
There are many positive comments describing how good Adobe Photoshop’s Super Resolution is, such as “Made My Jaw Hit the Floor”. Having been researching and experimenting in super resolution for several years, I wanted to present a more empirical comparison of image resolution enhancement and actual image quality metrics. In this article I compare:
- Bicubic interpolation method upscaling
- Adobe Camera Raw’s Super Resolution
- Information Distillation Network Super Resolution
Bicubic interpolation
Bicubic interpolation is the most commonly used method for upscaling an image, usually resulting in a blurred appearance in the upscaled image. Almost every image editor allows for Bicubic interpolation, in fact most Internet browsers use bicubic interpolation to display if an image larger than its physical size.
Adobe Camera Raw’s Super Resolution
The Adobe Camera Raw Super Resolution, or equivalent Photoshop Camera Raw filter is a recent very fast and easy to use Super Resolution method literally possible by clicking “enhance” in Adobe’s products using Camera Raw.
Adobe state it is an advanced machine learning model trained on millions of photos, although little more detail is given. More information from Adobe’s blog.
It seems likely the algorithm is a deep neural network, Adobe have achieved remarkable results from an inference speed, performance and have the capability of processing very large resolution images.
Information Distillation Network (IDN) Super Resolution
Fast and Accurate Single Image Super-Resolution via Information Distillation Network demonstrates a model trained on a deep convolutional neural network architecture by researchers Zheng Hui, Xiumei Wang, Xinbo Gao.
The Information Distillation Network (IDN) was selected for a comparison as its Super Resolution performance is still among the state of the art and it generalises very well.
The researchers also evaluated their Super Resolution results on relatively high resolution images, rather than the typical ‘postage stamp’ size image commonly used in the earlier days of deep neural network based Super Resolution.
Quality (and loss) metrics
Which quality metric is best is a contentious subject that varies between researchers. I personally regard SSIM and MAE as the best indicators of quality, although PSNR is the metric most common in academic research evaluation of super resolution models and algorithms.
PSNR and MSE lack the ability to capture perceptual or feature differences, for instance high texture detail. The metrics are is very limited as they are defined based on pixel-wise image differences. The highest PSNR does not necessarily reflect the perceptually better Super Resolution improvement.
The metrics for the bicubic interpolation have been included for comparison and at these resolutions the bicubic enlargement results in a relatively high quality image.
Peak signal-to-noise ratio definition (PSNR)
Peak signal-to-noise ratio definition (PSNR) is very common for evaluating image enhancement techniques, such as Super Resolution where the signal is the ground truth/original image and the noise is the error not recovered by the model. Although PSNR is a logarithm based metric, it is based on the MSE. As a quality metric a higher value indicates a higher quality.
Structural Similarity Index (SSIM)
Structural Similarity Index (SSIM) is a perceptual metric. SSIM is based on visible structures in the image. The usage of SSIM for image enhancement evaluation came about as for some researchers PSNR is no longer regarded as a reliable indicator of image quality degradation. It is a perceptual metric that quantifies image quality degradation caused by processing. As a quality metric a higher value indicates a higher quality.
Mean Squared Error (MSE)
Mean Squared Error (MSE) is used to compare how far away the ground truth/original image’s pixels from the predicted/generated image’s pixels. The mean of each pixel’s difference is taken and then squared. As a loss or error metric, a lower value indicates a higher quality.
Mean Absolute Error (MAE)
Mean Absolute Error (MAE) is the sum of the absolute differences between the ground truth/original and the predicted/generated image. As a loss or error metric, a lower value indicates a higher quality.
Examples from Unsplash
These are a varied set of examples of Super Resolution performed on high quality and resolution images from the popular Unsplash royalty free photography site.
In each example the left image is bicubic interpolation upscaling, the centre image is Adobe’s Super Resolution and right image is the IDN deep neural network’s Super Resolution.
As mentioned above the metrics are comparing each upscaling/super resolution method’s output to the original ground truth image.
With these high resolution images the metrics from the bicubic interpolation are often very high as upscaling by effectively using a blur filter on its own can result in high quality metrics especially with MSE.
The visual improvement in resolution and quality with most of the images is very noticeable from Adobe’s Super Resolution, although artifacts are introduced or exaggerated that are not there in the IDN Deep Neural Network Super Resolution.
Example 1
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 816x1020 to 1632x2040 pixels
The visual improvement in resolution and quality with most of the images is noticeable from Adobe’s Super Resolution, although looking closely artifacts are introduced or exaggerated that are not there in the IDN Deep Neural Network Super Resolution.
Left: bicubic upscaling
PSNR bicubic: 38.3776
SSIM bicubic: 0.9331
MSE bicubic: 9.4475
MAE bicubic: 0.0122
Centre: Adobe’s Super Resolution
PSNR: 31.8368
SSIM: 0.8905
MSE: 42.5988
MAE: 0.0358
Right: IDN Super Resolution
PSNR: 38.9078
SSIM: 0.9400
MSE: 8.3619
MAE: 0.0122
An alternate comparison below also including the original (ground truth) image that the Super Resolution algorithms are attempting to recreate. Left top image: Bicubic upscaling. Right top image: Adobe’s Super Resolution. Left bottom: IDN Super Resolution. Right bottom: Original / Ground truth image.
Example 2
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 845x1020 to 1690x2040 pixels
Even though the metrics do not indicate it, from a human visual perspective, the improvement from Adobe’s Super Resolution is superior to that of bicubic interpolation. The improvement from the IDN model’s Super Resolution is greater.
Left: bicubic upscaling
PSNR: 28.0634
SSIM: 0.8128
MSE: 101.5632
MAE: 0.0278
Centre: Adobe’s Super Resolution
PSNR: 26.6016
SSIM: 0.7796
MSE: 142.2055
MAE: 0.0384
Right: IDN Super Resolution
PSNR: 29.0446
SSIM: 0.8541
MSE: 81.0245
MAE: 0.0254
An alternate comparison below also including the original (ground truth) image that the Super Resolution algorithms are attempting to recreate. Left top image: Bicubic upscaling. Right top image: Adobe’s Super Resolution. Left bottom: IDN Super Resolution. Right bottom: Original / Ground truth image.
Example 3
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 785x1020 to 1530x2040 pixels
With this image the improvements are very similar between Adobe’s Super Resolution and the IDN model. The fur and cat’s whiskers look slightly more defined and sharper with the IDN model.
Left: bicubic upscaling
PSNR: 31.5566
SSIM: 0.9564
MSE: 45.4386
MAE: 0.0091
Centre: Adobe’s Super Resolution
PSNR: 31.1871
SSIM: 0.9319
MSE: 49.4732
MAE: 0.0147
Right: IDN Super Resolution
PSNR: 34.1575
SSIM: 0.9712
MSE: 24.9650
MAE: 0.0077
An alternate comparison below also including the original (ground truth) image that the Super Resolution algorithms are attempting to recreate. Left top image: Bicubic upscaling. Right top image: Adobe’s Super Resolution. Left bottom: IDN Super Resolution. Right bottom: Original / Ground truth image.
Example 4
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 816x1020 to 1632x2040 pixels
Adobe’s Super Resolution enhances the resolution restoring some detail and features into the image although exaggerating many of the artifacts in the image and again the IDN model results in a much enhanced resolution image.
Left: bicubic upscaling
PSNR: 28.5939
SSIM: 0.7417
MSE: 89.8862
MAE: 0.0559
Centre: Adobe’s Super Resolution
PSNR: 26.5685
SSIM: 0.6350
MSE: 143.2957
MAE: 0.0747
Right: IDN Super Resolution
PSNR: 29.4697
SSIM: 0.8064
MSE: 73.4695
MAE: 0.0513
An alternate comparison below also including the original (ground truth) image that the Super Resolution algorithms are attempting to recreate. Left top image: Bicubic upscaling. Right top image: Adobe’s Super Resolution. Left bottom: IDN Super Resolution. Right bottom: Original / Ground truth image.
Example 5
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 680x1020 to 1360x2040 pixels
With this image the metrics are improved with both Adobe’s Super Resolution and the IDN model. Visually the enhancement in resolution appears more improved in the IDN model with the smoother detail.
Left: bicubic upscaling
PSNR: 31.8182
SSIM: 0.9673
MSE: 42.7823
MAE: 0.0062
Centre: Adobe’s Super Resolution
PSNR: 32.6134
SSIM: 0.9356
MSE: 35.6236
MAE: 0.0086
Right: IDN Super Resolution
PSNR: 39.0521
SSIM: 0.9861
MSE: 8.0886
MAE: 0.0038
An alternate comparison below also including the original (ground truth) image that the Super Resolution algorithms are attempting to recreate. Left top image: Bicubic upscaling. Right top image: Adobe’s Super Resolution. Left bottom: IDN Super Resolution. Right bottom: Original / Ground truth image.
Example 6
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 680x1020 to 1360x2040 pixels
This is a very complex image to enhance with interactive bill boards and different types of lighting. Adobe’s Super Resolution enhances the resolution restoring significant detail and features into the image and again the IDN model results in a much enhanced resolution image.
Left: bicubic upscaling
PSNR: 28.0320
SSIM: 0.8890
MSE: 102.3008
MAE: 0.0405
Centre: Adobe’s Super Resolution
PSNR: 27.4427
SSIM: 0.8495
MSE: 117.1681
MAE: 0.0591
Right: IDN Super Resolution
PSNR: 30.6234
SSIM: 0.9383
MSE: 56.3300
MAE: 0.0308
An alternate comparison below also including the original (ground truth) image that the Super Resolution algorithms are attempting to recreate. Left top image: Bicubic upscaling. Right top image: Adobe’s Super Resolution. Left bottom: IDN Super Resolution. Right bottom: Original / Ground truth image.
Example 7
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 680x1020 to 1360x2040 pixels
Adobe’s Super Resolution enhances the resolution restoring detail and features into the waves within the image although, again the IDN model results in a enhanced resolution image.
Left: bicubic upscaling
PSNR: 30.8357
SSIM: 0.8405
MSE: 53.6426
MAE: 0.0307
Centre: Adobe’s Super Resolution
PSNR: 28.0416
SSIM: 0.7269
MSE: 102.0751
MAE: 0.0449
Right: IDN Super Resolution
PSNR: 31.9297
SSIM: 0.8853
MSE: 41.6971
MAE: 0.0277
An alternate comparison below also including the original (ground truth) image that the Super Resolution algorithms are attempting to recreate. Left top image: Bicubic upscaling. Right top image: Adobe’s Super Resolution. Left bottom: IDN Super Resolution. Right bottom: Original / Ground truth image.
Example 8
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 1020x680 to 2040x1360 pixels
A complex image, Adobe’s Super Resolution enhances the resolution well with some details and features within the image especially the signs/placards although the IDN model results in a enhanced resolution across the entire image.
Left: bicubic upscaling
PSNR: 26.3684
SSIM: 0.8523
MSE: 150.0509
MAE: 0.0413
Centre: Adobe’s Super Resolution
PSNR: 26.2458
SSIM: 0.7969
MSE: 154.3489
MAE : 0.0503
Right: IDN Super Resolution
PSNR: 29.0718
SSIM: 0.9179
MSE: 80.5191
MAE: 0.0305
An alternate comparison below also including the original (ground truth) image that the Super Resolution algorithms are attempting to recreate. Left top image: Bicubic upscaling. Right top image: Adobe’s Super Resolution. Left bottom: IDN Super Resolution. Right bottom: Original / Ground truth image.
Example 9
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 1020x632 to 2040x1264 pixels
There are a few variation of images of Neuschwanstein Castle (Schwangau, Germany) in these image comparisons. It is a good example of how significant the visual improvement in resolution can be through these super resolution methods. Also, how the improvement in resolution can vary between differing images of the same subject matter in varying perspectives and quality.
Left: bicubic upscaling
PSNR: 30.0251
SSIM: 0.8722
MSE: 64.6507
MAE: 0.0181
Centre: Adobe’s Super Resolution
PSNR: 28.4533
SSIM: 0.8154
MSE: 92.8427
MAE: 0.0234
Right: IDN Super Resolution
PSNR: 31.3335
SSIM: 0.9158
MSE: 47.8331
MAE: 0.0159
An alternate comparison below also including the original (ground truth) image that the Super Resolution algorithms are attempting to recreate. Left top image: Bicubic upscaling. Right top image: Adobe’s Super Resolution. Left bottom: IDN Super Resolution. Right bottom: Original / Ground truth image.
Example 10
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 680x1020 to 1360x2040 pixels
A complex image with edges and reflections, Adobe’s Super Resolution enhances the resolution well with some details and features within the image although exaggerating some of the artifacts in the image. The IDN model results in a enhanced resolution across the entire image.
Left: bicubic upscaling
PSNR: 32.0612
SSIM: 0.9570
MSE: 40.4542
MAE: 0.0149
Centre: Adobe’s Super Resolution
PSNR: 31.7344
SSIM: 0.9239
MSE: 43.6156
MAE: 0.0338
Right: IDN Super Resolution
PSNR: 38.9284
SSIM: 0.9804
MSE: 8.3223
MAE: 0.0099
An alternate comparison below also including the original (ground truth) image that the Super Resolution algorithms are attempting to recreate. Left top image: Bicubic upscaling. Right top image: Adobe’s Super Resolution. Left bottom: IDN Super Resolution. Right bottom: Original / Ground truth image.
Example 11
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 680x1020 to 1360x2040 pixels
Again a complex image with edges and reflections, Adobe’s Super Resolution enhances the resolution well with some details and features within the image although exaggerating a few of the artifacts in the image. The IDN model results in a enhanced resolution across the entire image.
Left: bicubic upscaling
PSNR: 26.6965
SSIM: 0.9350
MSE: 139.1342
MAE: 0.0156
Centre: Adobe’s Super Resolution
PSNR: 28.8793
SSIM: 0.9115
MSE: 84.1695
MAE: 0.0161
Right: IDN Super Resolution
PSNR: 32.3578
SSIM: 0.9759
MSE: 37.7836
MAE: 0.0091
Example 12
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 785x1020 to 1530x2040 pixels
Even though the metrics do not indicate it, from a human visual perspective, the improvement from Adobe’s Super Resolution is superior to that of bicubic interpolation. The improvement from the IDN model’s Super Resolution is more greatly improved.
Left: bicubic upscaling
PSNR: 37.3767
SSIM: 0.9313
MSE: 11.8964
MAE: 0.0106
Centre: Adobe’s Super Resolution
PSNR: 30.3778
SSIM: 0.8495
MSE: 59.6077
MAE: 0.0302
Right: IDN Super Resolution
PSNR: 38.4426
SSIM: 0.9476
MSE: 9.3072
MAE: 0.0099
Example 13
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 1020x744 to 2040x1488 pixels
Another example of an image of Neuschwanstein Castle showing the improvement in resolution can vary between differing images of the same subject matter in varying perspectives and quality.
Left: bicubic upscaling
PSNR: 25.1374
SSIM: 0.8437
MSE: 199.2213
MAE: 0.0258
Centre: Adobe’s Super Resolution
PSNR: 24.8581
SSIM: 0.8193
MSE: 212.4563
MAE: 0.0285
Right: IDN Super Resolution
PSNR: 27.3212
SSIM: 0.9104
MSE: 120.4940
MAE: 0.0203
Example 14
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 1020x785 to 2040x1530 pixels
Another example of an image of Neuschwanstein Castle showing the improvement in resolution can vary between differing images of the same subject matter in varying perspectives and quality.
Left: bicubic upscaling
PSNR: 29.3615
SSIM: 0.8898
MSE: 75.3227
MAE: 0.0139
Centre: Adobe’s Super Resolution
PSNR: 29.0122
SSIM: 0.8530
MSE: 81.6319
MAE: 0.0170
Right: IDN Super Resolution
PSNR: 31.7219
SSIM: 0.9300
MSE: 43.7407
MAE: 0.0116
Example 15
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 1020x680 to 2040x1360 pixels
A complex image of Bangkok by night, Adobe’s Super Resolution enhances the resolution well with some details and features within the image although the IDN model results in a much enhanced resolution across the entire image.
Left: bicubic upscaling
PSNR: 25.2616
SSIM: 0.8842
MSE: 193.6065
MAE: 0.0452
Centre: Adobe’s Super Resolution
PSNR: 25.5575
SSIM: 0.8539
MSE: 180.8530
MAE: 0.0608
Right: IDN Super Resolution
PSNR: 29.0846
SSIM: 0.9545
MSE: 80.2820
MAE: 0.0304
Example 16
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 680x1020 to 1360x2040 pixels
Again a complex image with edges and reflections, Adobe’s Super Resolution enhances the resolution well with some details and features within the image although exaggerating a few of the artifacts in the image. The IDN model results in a enhanced resolution across the entire image.
Left: bicubic upscaling
PSNR: 27.7844
SSIM: 0.9332
MSE: 108.3041
MAE: 0.0244
Centre: Adobe’s Super Resolution
PSNR: 29.2190
SSIM: 0.8874
MSE: 77.8356
MAE: 0.0321
Right: IDN Super Resolution
PSNR: 34.6705
SSIM: 0.9767
MSE: 22.1834
MAE: 0.0126
Example 17
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 601x1020 to 1202x2040 pixels
Adobe’s Super Resolution enhances the resolution restoring some detail and features into the image although exaggerating the artifacts in the image and again the IDN model results in a much enhanced resolution image.
Left: bicubic upscaling
PSNR bicubic: 32.3341
SSIM bicubic: 0.8897
MSE bicubic: 37.9897
MAE bicubic: 0.0251
Centre: Adobe’s Super Resolution
PSNR: 29.5855
SSIM: 0.7945
MSE: 71.5363
MAE: 0.0438
Right: IDN Super Resolution
PSNR: 35.6721
SSIM: 0.9354
MSE: 17.6143
MAE: 0.0195
Example 18
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 1020x680 to 2040x1360 pixels
Adobe’s Super Resolution exaggerates many of the artifacts in the image, some features such as the eye is improved in resolution over the bicubic interpolation. The fur and dog’s whiskers look slightly more defined and sharper with the IDN model.
Left: bicubic upscaling
PSNR bicubic: 40.8572
SSIM bicubic: 0.9486
MSE bicubic: 5.3378
MAE bicubic: 0.0067
Centre: Adobe’s Super Resolution
PSNR: 35.5768
SSIM: 0.9001
MSE: 18.0053
MAE: 0.0140
Right: IDN Super Resolution
PSNR: 41.5297
SSIM: 0.9559
MSE: 4.5720
MAE: 0.0066
Example 19
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 823x1020 to 1646x2040 pixels
Adobe’s Super Resolution enhances the resolution restoring some detail and features into the image although exaggerating the artifacts in the image and again the IDN model results in a much enhanced resolution image.
Left: bicubic upscaling
PSNR: 37.0134
SSIM: 0.9717
MSE: 12.9344
MAE: 0.0056
Centre: Adobe’s Super Resolution
PSNR: 34.0506
SSIM: 0.9461
MSE: 25.5872
MAE : 0.0123
Right: IDN Super Resolution
PSNR: 39.1880
SSIM: 0.9792
MSE: 7.8393
MAE: 0.0052
Example 20
Left image: bicubic upscaling, Centre image: Adobe’s Super Resolution, Right image: IDN Super Resolution.
Resolution: Upscaled from 1020x680 to 2040x1360 pixels
A complex image of Toronto’s night skyline, Adobe’s Super Resolution enhances the resolution restoring detail and features into the image, again the IDN model results in a much enhanced resolution image.
Left: bicubic upscaling
PSNR: 29.9861
SSIM: 0.8897
MSE: 65.2330
MAE: 0.0269
Centre: Adobe’s Super Resolution
PSNR: 29.2766
SSIM: 0.8433
MSE: 76.8107
MAE: 0.0349
Right: IDN Super Resolution
PSNR: 32.5325
SSIM: 0.9231
MSE: 36.2939
MAE: 0.0213
Examples from my own photographs
A non-typical photograph from my kitchen to test the super resolution capabilities.
Here Adobe’s Super Resolution is considerably worse than the bicubic interpolation from a metric perspective, visually it does appear better than bicubic interpolation although only marginally.
Left: bicubic upscaling
PSNR: 37.2630
SSIM: 0.9419
MSE: 12.2117
MAE: 0.0101
Centre: Adobe’s Super Resolution
PSNR: 34.1691
SSIM: 0.8990
MSE: 24.8983
MAE: 0.0167
Right: IDN Super Resolution
PSNR: 39.3837
SSIM: 0.9528
MSE: 7.4939
MAE: 0.0089
As an additional comparison the Super Resolution of this photograph of cherry blossom trees in Oozells square (Birmingham, UK) is an example of performing Super Resolution on the source image, rather than a lower resolution downscaled version of the source image. This comparison is to take the bicubic downscaling used to create the lower resolution input away from any consideration.
Here the original input image is increased in resolution higher than the original, so there is no ground truth to compare metrics with. From a human visual perspective the enhancement in resolution appears greater in by the IDN model compared to Adobe’s Super Resolution. The brickwork, branches, window cladding for example are visibly sharper.
Note that the Adobe Super Resolution was carried out on the raw camera image to give adobe’s algorithm a potential advantage.
This example was included as there are researchers that believe downscaling an image to use as a comparison is not a real world test, that a bicubic downscale is not equivalent to an image captured originally at a lower resolution. For instance the ImagePairs: Realistic Super Resolution Dataset via Beam Splitter Camera Rig paper from Microsoft Research, where a dataset was used taken with a camera rig with both a lower and higher resolution camera sensor resulting in a true low and high resolution image pair.
Examples from the Set 5 dataset
These are images from the well known set 5 that’s very commonly used for Super Resolution evaluation:
From a metric perspective Adobe’s Super Resolution was the lowest enhancement in resolution. However from a human visual perspective, its improvement with the hat’s material makes it possibly appear the highest quality. Looking closely the IDN model does a better enhancement of some of the features such as the eye lashes and eyes. As I quite often raise, what is more important metrics or human perception.
Left: bicubic upscaling
PSNR: 37.4317
SSIM: 0.9554
MSE: 11.7465
MAE: 0.0075
Centre: Adobe’s Super Resolution
PSNR: 32.7958
SSIM: 0.9313
MSE: 34.1584
MAE: 0.0128
Right: IDN Super Resolution
PSNR: 38.4256
SSIM: 0.9658
MSE: 9.3438
MAE: 0.0069
With this small image Adobe’s Super Resolution does enhance the image’s resolution, although the IDN Super Resolution is enhances the resolution considerably further.
Left: bicubic upscaling
PSNR: 32.6560
SSIM: 0.9514
MSE: 35.2763
MAE: 0.0129
Centre: Adobe’s Super Resolution
PSNR 30.5751
SSIM: 0.9483
MSE: 56.9603
MAE: 0.0177
Right: IDN Super Resolution
PSNR: 35.3274
SSIM: 0.9707
MSE: 19.0694
MAE: 0.0101
Example from the Urban 100 dataset:
This is an image from the Urban 100 data set that’s regarded as quite complex for Super Resolution evaluation.
The IDN model is considerably superior to Adobe’s Super Resolution in its enhancement of the resolution of this complex image.
Left: bicubic upscaling
PSNR: 23.4326
SSIM: 0.7692
MSE: 294.9959
MAE: 0.0344
Centre: Adobe’s Super Resolution
PSNR: 24.8960
SSIM: 0.8612
MSE: 210.6105
MAE: 0.0305
Right: IDN Super Resolution
PSNR: 27.7919
SSIM: 0.9119
MSE: 108.1152
MAE: 0.0210
Conclusions
Adobe’s Super Resolution performance at enhancing resolution in images is exceptional for a feature built into an image processing and editing software suite.
With the higher resolution images the improvement in resolution for some images was close to a state of the art deep neural network, although the difference with the IDN model is significant. The variation in the improvement indicates the algorithm may not generalise well.
From a metric perspective Adobe’s Super Resolution was the lowest enhancement in resolution in many of the tests within this article. However from a human visual perspective, its improvement is far superior to that of bicubic interpolation. It is very possible its algorithm was trained with a more perceptual metric. As I quite often raise, what is more important: metrics or human perception of quality?
Adobe’s Super Resolution is much less tolerant of noise or artifacts in the images, in general performing worse on lower quality images.
Adobe’s Super Resolution can only perform 2X enhancement of image resolution, whereas many deep neural network models have been trained to perform 4X enhancement of image resolution.
Evaluations were also carried out on the DIV2K validation dataset, which was collated for super resolution competitions and research. As the DIV2K dataset is only licensed for academic use, it can’t be used here. Adobe’s Super Resolution performed exceptionally well on the DIV2K validation dataset. Compared to the results here, the reason for this could be that the training process of the Adobe’s Super Resolution included those images and it was not a fair evaluation.
Very high resolution
Adobe’s Super Resolution outstanding feature is the phenomenal size of image it can perform Super Resolution enhancement upon, a mind boggling 11,205×8,404 pixel image upscaled to 22,409×16,807 pixels. Most deep learning models would result in Out Of Memory issues at smaller resolutions.
How to use Adobe’s Super Resolution
This may seems hidden away, to use Adobe’s Super Resolution select an image from Adobe Bridge and open it in Camera Raw, then “enhance” is in the context menu when you right click on the image.
Super resolution becoming a common term
A footnote that my own Super Resolution article that used to be in the top two results in Google and had been the featured Google snippet for “Super Resolution” has now plummeted down the search engine rankings being surpassed by blogs and articles about Adobe Photoshop’s Super Resolution. That’s inevitable when a area of technology becomes more mainstream and accessible.
