noise_bayer.htm

Stephen Burch's Birding & Dragonfly Website

Home | Trip Reports | Gallery | UK index | Oxon pics | UK pics | Dragonflies | Other Nature | Links


Digital camera image noise levels before raw conversion

Now including the R5 Mk II!

Contents of this page

Introduction
As described in this page, my own measurements of digital camera image noise levels, made after raw conversion, showed marked variations depending which of the three raw converters (ACR, DPP4, DPP3) was being used. It seemed at least some of these raw converters had hidden parameters that varied from one camera model to another, which gave varying levels of noise reduction without any user adjustments being possible.

Given these results, I thought it would be interesting to see if I could access the raw camera sensor data, before processing (de-mosaicing) by a raw converter. This should allow me to make a direct comparisons between the different cameras. It should also provide values that could be compared directly with the measurements of signal-to-noise ratio given on the DXOMARK website, which are said to made on the raw sensor data.

To make measurements on the raw sensor data, prior to raw conversion, I could not find any software that would allow me to do exactly what I wanted. So I found it was necessary to devise a somewhat involved multi-staged processing method. This is described below, as it might be of interest to others.

The images from multiple cameras, including the new Canon R5 Mk II, that I processed using the method given below were generally of the "step wedge" test chart I have used for many years for measuring noise levels in digital camera images. See this page for more information.

I am very grateful to Mark Chivers for allowing me to use his R3 to take some images of the test chart, so I could include this camera in the analysis given below, in addition to the cameras I have (or had) more direct access to. 

Sensor data before raw conversion

It is important to appreciate that most camera sensors do not record full resolution colour images (i.e. 3 separate red, green and blue channels for every pixel on the sensor). Instead they use a set of RGB colour filters, arranged in front of the sensor elements, in 2x2 blocks of pixels, known as a Bayer mask or Bayer filter. Within each 2 x 2 block of sensor elements (or pixels) there are generally two green filters, one red and one blue. There are various possible arrangements for these Bayer filters as shown below. Surprisingly, in the six Canon cameras I have investigated, I have examples of all of these four types. It would seem easier for Canon to have standardised on one of them!

So the sensor output is an image of single channel (monochrome) pixels, with the colour information encoded within it using the colour filters in front of every pixel.


Different possible 2x2 pixel Bayer masks

The main function of any raw converter is to perform an interpolation using the 2 x 2 pixel colour information to generate separate R, G and B values for all 4 pixels in each block of 2 x 2, while maintaining the spatial resolution of the data as much as possible. The resultant full RGB image is then saved in standard image formats such as jpeg or tiff. This process is usually called de-mosaicing.

Raw converters also perform various other manipulations of the sensor data, one of which is to apply a non-linear transformation (inverse gamma correction) to the output values, so that the grey levels are no longer directly proportional to the incident light intensity. 

To access the raw sensor data before conversion it is necessary to go back to the original raw file from the sensor and read out the sensor data directly before any manipulation of the data takes place. This requires a specialised set of processing stages, as described below.

Processing principle and method

1. Aim of processing
My aim was to access the raw sensor data and then re-arrange the values into a standard image format (e.g. tiff) that could be analysed to measure the grey levels in the image using standard tools, and hence derive noise levels (from the standard deviation of the values in small areas) and signal-to-noise ratios (from the noise level divided by the mean grey level).

For each 2 x 2 block of Bayer filter values, I aimed to generate one pixel with the three channels of colour information. So the output RGB image from this process had a quarter of the number of total pixels in the original sensor data. Hence, for example, if the original sensor data was 6000 x 4000 single channel pixels, the output from this processing would be 3000 x 2000 RGB pixels. Note that this approach effectively preserves the original colour information in the Bayer Mask without any interpolation. In fact, there are 2 x 2 = 4 input values and only 3 output values, as there are two green pixels in the the Bayer Mask. The most obvious way of handling these was to average these two green pixels.

Note that my analysis of the resulting "half-size" images (quarter the number of pixels) is likely to give values for signal-to-noise ratio that will be higher than those of the full size image after de-mosiacing the image, possibly by as much as 6 dB (averaging over a 2 x 2 set of pixels will reduce the noise by the square root of 4 = 2, or 6dB). However this overestimate will be exactly the same for every image and hence the method will still allow a valid comparison to be made between the values obtained from the different camera sensors.

I have not been able to find any publicly available software that does the required re-arrangement of values in one step. Instead, I needed to devise a multi-stage method as described below, which also entailed me refreshing my rather rusty software development skills!

2. RawDigger
I first looked at the free RawTherapee software which appeared to have an option to access the raw sensor values before de-mosaicing. While this seemed to work OK for one or two cameras, it gave some very strange results indeed on others.

I then found that with the commercially available RawDigger I could access the required raw sensor values and output them as a single channel (monochrome) image, in 16-bit tiff format. This software is a free download but only for the first 30 days, after which a reasonably modest licence fee is needed. The RawDigger export options needed to achieve the required output are given below:


Raw Digger Export options

By looking at the resulting tiff image in Photoshop (or similar) it was possible to inspect the individual values to determine what type of Bayer Mask was being used. To do this, you need to realise that for an image of a monochrome subject (e.g. my steps test chart), the blue filter produces the lowest grey level, while the two greens are diagonally opposite one another and of closely similar grey level.

As example, the image below shows a small section of the top-left corner of an image from my Canon R5. From this, it can be seen that the Bayer mask arrangement is type 1, as shown above (i.e. red top left and blue bottom right).

 


Small extract of top-left corner of Canon R5 raw sensor data

By looking at these images from all the cameras, I was able to work out which type of Bayer mask was present on each one. I show my findings below:

Canon camera model Bayer Mask type
R7 1
R5 1
R3 1
7D Mk II 1
7D 2
50D 3
350D 4

The output from RawDigger was a single channel (monochrome) 16-bit tiff file containing the required raw sensor data that had 14-bit values for all the cameras apart from the 12-bit 350D. The maximum 14-bit number is 16383 whereas for 12-bits it is 4095. Note however that the zero value is offset substantially from zero for Canon cameras. RawDigger reads auto values for the zero level and then subtracts this from the stored sensor values. I presume any small negative values are then clipped to zero. This means the maximum possible values in the resulting files are less than the maximum possible values of 16383 (for 14-bit cameras) and 4095 (for 12-bit camera).

3. ImageJ
This free software is very handy for various image manipulation and analysis tasks. At this stage of my processing, I use it to convert the tiff files generated by Raw Digger, as described above, into an unformatted 16-bit binary file, which is somewhat confusingly termed "raw data" in this software. 

4. My program
The purpose of the program I wrote, which runs from the command prompt in Windows, is to read the unformatted 16-bit binary file produced by RawDigger/ImageJ and to then re-structure it into a 3 channel, half sized output file by performing the simple rearrangement of the RGB values described above. Hence for each 2 x 2 block of pixels in the monochrome input file, the output is a single colour pixel with red, green and blue values. This output file has half the dimensions (a quarter of the pixels) of the input file. As explained above, the two green values in the input are averaged to provide the single value needed for this file.

The output from my program is 3-channel 16-bit data written as unformatted binary numbers.

5. Import into Photoshop
I was pleased to find that the format of file written by my program could be read directly into Photoshop. I gave these files a *.raw extension, and opening them in Photoshop 2023 brings up a menu, as shown below.

By changing the defaults in this menu to those shown above (the width and height of the input data is needed here), Photoshop then reads the file in as a 16-bit 3-channel (RGB) colour image. This needed some processing of the colour information contained in this file, as described below.

6. Photoshop processing
The images processed as above and then opened in Photoshop showed a noticeable colour tint, usually a green cast. This was because the green filters in the Bayer mask transmit more light than the red and especially blue filters. This is normally corrected for in a raw converter but the whole purpose of the above processing was to avoid the need for raw conversion.

An example of the image initially open in Photoshop is shown below. Note the strong green cast.


Image of raw sensor values, shown strong green colour cast

It is isn't clear exactly what the best approach was to handling this colour cast, but I decided to handle it using the PS option Filter ---> Camera Raw Filter. Having opened the image in this option, I used the dropper tool to set the white balance anywhere in the white border of the test chart. This changed the colour balance of the image and gave an image of the monochrome test chart that looked more like it should:


Image of raw sensor values, after correction of white balance

To further ensure there was no significant spurious colour information in the image, I also applied the Photoshop option Image ---> Adjustments ---> Black & White using the default settings.

Important option to avoid: Initially I also applied the option Image ---> Mode ---> Grayscale to reduce the 3 channel RGB file into a single monochrome channel, hence saving storage space. However this turned out to be a time consuming mistake. This seemingly simple routine was actually quite involved and took it upon itself to modify the linearity of the grey levels and even reduced the noise preferentially in any large areas of mid-grey. Very bizarre and not described in any Adobe documentation I have seen, and definitely not what I wanted!

7. Measurements using ImageJ or Isee!
The final stage in the processing was to analyse the 16-bit 3-channel RGB image saved from Photoshop in tiff format to determine mean grey values and noise levels in the required areas of the image - see this page for more information.

Linearity of sensor grey levels before raw conversion

There is information on the web that suggests that the raw sensor data, before conversion, is linear with respect to incident light intensity, which is certainly not the case for the values after raw conversion. To investigate this, I took photos of my test chart (as shown above) over a wide range of exposures for each camera tested. To do this, I used the manual settings to keep the ISO and aperture constant, while varying the shutter time from the shortest possible (e.g. 1/8000) up to the value that started to give saturated values ("blown" highlights). To take these images, I had each camera mounted on a tripod so that there was no movement between exposures. I took these outside, and acquired the images with each camera as quickly as possible, one after the other, to minimise any changes the ambient light.

For these images I kept the ISO constant at 1600 for all cameras. Note however that I took the exposures on the different cameras on different days and hence under different overall levels of ambient light.

Analysis of the mean 14-bit grey level in an area of the white background surrounding the grey scale in the image gave values for all the different exposures taken. The area used was quite large, varying according to the pixel resolution of the sensor. For the highest resolution sensor, the area used was 400 x 400 pixels.

The results are plotted below for the five different cameras I still have available for these tests. Note that the oldest camera, the 350D, was only 12-bits, so I scaled its values up to match the 14-bit values from the other cameras. The factor I used was not exactly the expected 4, due to the different zero levels for the raw sensor values, and the values at which the data saturated.

Mean 14-bit grey levels from raw sensor data as a function of exposure time

The results shown above confirm that the various camera sensors gave values directly proportional to exposure for grey levels up to about 10,000. For higher grey levels they became highly non-linear and did not become fully saturated until much higher exposures are reached.

Noise level measurements and results at ISO 1600
Initially I used the multi-exposure ISO 1600 images, described above, to measure the noise level, without dividing by the mean grey level to get signal-to-noise ratio. For this, I used small regions (between about 30 x 30 and 75 x 75 pixels) depending on the pixel resolution of the sensor.

Important note: for any noise measurements, it is important to ensure all images are reasonably out of focus. This is to avoid any effects due to minor unevenness in the printing process used to produce the grey scale wedge.

I also tried measuring the different steps in the grey-scale image from one well illuminated R5 exposure to check for consistency with the multi-exposure approach above. I also used this for the 50D, which I no longer have available for new images. The results are shown below. For the R5, the values found from the grey scale steps agree quite well with the multi-exposure values, although there are small differences, mainly at the lower grey levels, for unknown reasons.


Noise levels as a function of 14-bit grey level for six different Canon camera before raw conversion

I found the results shown above quite surprising. The overall differences between the cameras were less than expected. For the R5, although it had the lowest overall noise levels, it was very close indeed to the 7D Mk II, especially at higher grey levels. The other four cameras, including the R7, showed somewhat higher noise levels, but the differences were quite small.

Signal-to-noise ratio (SNR) at 18% grey level results at ISO 1600

Derivation of 18% SNR values
Given the accurate linearity of the raw sensor values at lower grey levels, it means that valid measurements of signal-to-noise ratio can be made by simply dividing the mean grey level by the standard deviation of the values. As shown in the plot above, the noise level increased with grey level, so it is necessary to select a particular grey level for the calculation of SNR.

In order to try to reproduce the DXOMARK measurements as closely as possible, I tried using a measure of a grey level corresponding to 18% of the saturated value for each sensor. For the 14-bit sensors, this varied slightly between cameras but was around 14340, so 18% of this was 2581. I do not know if this is the method used by DXOMARK to define their 18% SNR values, as their website does not specify this detail.

To find the noise levels at 18% grey level, I fitted power law curves to the above measurements, which were straight lines on the above log-log plot. The fitted lines had gradients of close to the expected 0.5, and I then found the noise level of the fitted lines for the 18% grey level values. This was easier than performing some form of interpolation between adjacent measurements and should have reduced the "noise" from the minor variations about the fitted lines shown on the above plot. To derive the SNR, I simply took the 18% grey level value and divided by the corresponding noise level (standard deviations of the values in small regions). 

Comparison between 18% grey SNR and max noise levels after ACR

On another page, I show measurements of the maximum noise levels in images after raw conversion using Adobe Camera Raw (ACR) (and Digital Photo Professional). It is interesting to compare these values with those found as described above for the SNR at 18% grey. The scatter plot below shows the ACR maximum noise levels for eight Canon cameras against the values for SNR at 18% grey.

I find it surprising that the correlation between these two measurements of noise, before and after raw conversion, shows so much scatter. For example, before raw conversion, the SNR for the R5 is only 0.5 dB higher than for the 7D Mk II. But after ACR raw conversion, the maximum noise level for the R5 is appreciably lower than the 7D Mk II by 2.4 dB.

 These measurements also show that the R5 Mk II raw sensor data has a very similar 18% grey signal-to-noise ratio to that of the original R5, but that after raw conversion using ACR, the R5 Mk II image is somewhat more noisy than the original R5.

It seems that ACR is applying some camera specific noise reduction processing that affects some cameras more than others. How this works, I have no idea!

 

My measurements of 18% SNR from the sensor data before raw conversion, compared with my measurements of maximum noise level after raw conversion using ACR. All measurements at ISO 1600. Note that the R5 Mk II has a similar SNR to the R5 but using ACR the R5 Mk II image is noisier than the R5. 

DXOMARK 18% SNR values

DXOMARK is an organisation that performs independent assessments/tests on photographic equipment. Importantly, DXOMARK state that their measurements are made from the raw sensor data, i.e. before any de-mosaicing and other processing performed by a raw converter.

Of the various parameters DXOMARK give for camera sensors, the one with most relevance here is their measurements of what they call 18% SNR. This I take to mean the signal-to-noise ratio (SNR), as measured at a grey level of 18% brightness, which typically corresponds to mid-grey.

A description of their measurement procedure is given on their website, but some of details are not fully defined nor explained as far as I can tell. For example, it is not clear exactly how the grey level corresponding to 18% is determined for an image. In addition, it is not completely clear how their measurements of the individual noise levels in the red, green and blue channels are combined to provide a single value. I'd also be interested to know how their green channel is defined as typical Bayer masks on camera sensors have two green pixels. Are these averaged in some way to provide the green channel information?

It does however appear that the DXOMARK measurements are made very rigorously using specialised transmission test charts and analysis methods, so they should provide objective and accurate measurements of the parameters measured. But I am not aware of any other sources of this type of information, so there is nothing to cross check these measurements against.

In my most recent update of this page (8 October 2024) I find it highly notable that DXOMARK now show two different sets of 18% SNR values for each camera sensor, whereas previously I was only aware of one set. It seems that the previous values were what DXOMARK now refer to as their "Print" values and a whole new set of values referred to as "Screen" is now given.

I reproduce below the small text from their website that explains the meanings of these two terms below. Note that the explanation for the "print" values includes normalisation to an 8 Mb image, irrespective of the camera sensor resolution and various arbitrary values, completed unrelated to the camera sensor such 300dpi and 8"x12" format.

On the other hand, the DXOMARK definition of their new "screen" values looks like this is much more in line with what I am interested in measuring: the noise on the raw sensor values prior to de-mosaicing.

I find these new DXOMARK definitions and measurements quite surprising but they provide an explanation for previous discrepancies I was finding between my measurements and their values. 

Prior to this "clarification", the clear inference was that their 18% SNR values referred to the raw sensor data, and not a value related to some arbitrary sized image (8Mb), printed at a particular resolution (300dpi) on an 8" x 12" sized print! These new "screen" values are in better agreement with my measurements, than the previous "print" ones, albeit with a need to increase the DXOMARK "screen" values by a constant (see below for more details).

 
"Screen"   "Print"
DXOMARK's definition of their "screen" and "print" SNR values, taken from the DXOMARK website (October 2024)

Comparison with DXOMARK 18% SNR values

The results for the 18% SNR values, derived from my measurements at ISO 1600 as above, are shown below for direct comparison with the corresponding DXOMARK "screen" values, where available, for ISO 1600. All the SNR values are shown as dB, which is as DXOMARK shows them. Note that all the DXOMARK values were substantially lower than the values I have derived, for reasons no doubt related to the details of exactly how the measurements were made.

However by finding the average difference for all cameras (+4.7 dB), and adding this to the DXOMARK values, I then get very good relative agreement with the values I derived as described above.


18% "Screen" SNR values (in dB) at ISO 1600 as given by DXOMARK (where available) and the present analysis.

Having adjusted all the DXOMARK values by +4.7dB, the chart above shows very good agreement between my measurements and the "screen" DXOMARK values, over all the cameras assessed. The maximum difference between the two sets of measurements is only 0.6 dB - for the 7D Mk II.

As expected, the highest SNR was obtained with the Canon R3, which had the largest sensor element (pixel) size of all those shown above, with the exception of the very old Canon 350D. 

In the plot above, I also show SNR values for the new R5 Mk II when using the mechanical (mech) and electronic (el) shutter. These are both very similar to original R5 value (for the mechanical shutter only). Hence the electronic shutter does not appear to increase the image noise level.

After the R5 and R5 Mk II, the old 7D Mk II had an SNR only about 1 dB less. Rather surprisingly the next highest SNR was from the 50D (about 1.5 dB less than for the R5), with the lowest SNR from the oldest camera - the 350D, which was about 3 dB (just 1 stop) down on the R5.

It is also notable that the R7 gave a relatively low SNR value, which is no doubt due to its small pixel size (3.2 micron) - the smallest of the cameras shown above.

High ISO Noise of the R5 Mk II compared with the R5

It seems that there has been much discussion online about the high ISO noise performance of the R5 Mk II, with claims that the R5 Mk II is significantly worse than the original R5 at ISO values around 12,800 and 25,600. However these suggestions appear mainly to be based on subjective assessments of various different images obtained at different times with the two cameras.

The chart above, based on quantitative measurements of a standarised test chart. shows no evidence for poorer SNR in the R5 Mk II compared with the R5, but is based on measurements at the relatively low ISO of 1600.

Hence I have made quantitative measurements of signal-to-noise ratio (SNR) for higher ISO values up to 25,600 to try to find this reported large reduction in the R5 Mk II's performance.

My high ISO measurements are shown in the chart below which gives two sets of SNR values. The first is for the SNR of the images for mid-grey (18%). I also show values for 5% SNR which correspond to the dark grey/black areas in the images.

The chart above shows no evidence for a major reduction in noise performance for the R5 Mk II at high ISO. There is however a small effect for the electronic shutter images from the R5 Mk II which are slightly noisier than those for the R5. This effect is larger in the darker areas of the image but even then the 5% SNR value for the R5 Mk II is only about 0.8 db (or a quarter of one stop) lower than for the R5.

Hence these quantitative measurements of the raw sensor data from the R5 Mk II and the R5 show very similar noise levels all the way up to ISO 25,600. So the reason for the reports of noisier high ISO images from the R5 Mk II are unclear.

It is however possible that the raw converters (e.g. ACR, DPP etc) which are inevitably needed to view the images are in some way responsible for these reported differences between the R5 Mk II and the R5.

Conclusions

  • This page has detailed a method that allows access to the raw sensor data from digital cameras before demosaicing. This method has been used to measure the noise levels and signal-to-noise ratios from eight Canon cameras at ISO 1600.
  • The measurements described here show noticeable differences between these cameras, with the highest SNR (from the Canon R3) being some 6 dB higher than the lowest SNR (Canon 350D - an entry level camera released in 2005!).
  • The measurements of 18% SNR given here are higher (by an average of 4.7 dB) than the recently available "screen" values given by DXOMARK. When this systematic offset is allowed for, the measurements given here now agree well with those from DXOMARK, with a maximum difference of only 0.6 dB.
  • Use of the newly available "screen" SNR values from DXOMARK has removed a discrepancy found previously for the R5, based on DXOMARK values that are now clarified to represent "print" SNR values, unrelated to the properties of the camera sensors themselves.
  • The raw data (before de-mosaicing) from the new R5 Mk II, with its stacked, backside illuminated sensor gave an 18% grey SNR value very similar to that of the original R5. There was no measureable difference between the noise levels from the mechanical and electronic shutters on the R5 MK II.
  • After raw conversion using ACR, the R5 Mk II image was somewhat more noisy than that from the R5.
  • Results, not shown here, for ISO 6400 showed very similar relative results for the cameras, with the SNR values being about 6 dB less than those at ISO 1600, as expected.
  • High ISO measurements for the R5 Mk II and the R5 (ISO 12,800 and 25,600), show very similar SNR values. However, the electronic shutter images from the R5 Mk II are slightly noisier than those from the R5, but only by about a quarter of a stop (0.8dB) in the darker image areas.


Home | Trip Reports | Gallery | UK index | Oxon pics | UK pics | Dragonflies | Other Nature | Links

Fatbirder's Top 1000 Birding Websites