Difference between revisions of "RAW dynamic range extraction"

From PanoTools.org Wiki
Jump to navigation Jump to search
m (→‎Intro: clarified intention)
m (→‎Shooting: typo)
 
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
== RAW dynamic range extraction ==
 
== RAW dynamic range extraction ==
 +
[[Image:Camera-RAW-curve1.jpg|thumb|150px|Linear curve]]
 +
[[Image:Camera-RAW-camera.jpg|thumb|150px|Camera settings]]
 +
[[Image:Camera-RAW-basic.jpg|thumb|150px|Basic settings]]
 
=== Intro ===
 
=== Intro ===
 +
Sometimes it's not feasible or not recommended to shoot bracketed. Action scenes and large crowds f.e. or landscapes in windy weather, where lots of leafs and branches are moving.
  
−
This is a small tutorial on how to determine settings to extract the full [[dynamic range]] of a RAW camera file into a single 16 bit per channel document using Adobe camera RAW converter of photoshop CS2. This workflow might be possible with other RAW converters, too.
+
This is a small tutorial on how to extract the full [[dynamic range]] of a RAW camera file into a single 16 bit per channel document using Adobe camera RAW (ACR) converter of photoshop CS6 (same engine as Lightroom 5).  
  
−
For a general RAW workflow please see [[Working with RAW files in CS2]]
+
Extracting the whole dynamic range is a snap in recent ACR. No need to fiddle with low contrast anymore, like described in the [http://wiki.panotools.org/index.php?title=RAW_dynamic_range_extraction&oldid=10810 old article for photoshop CS2]. The Highlights, Shadows and other sliders provide tonemapping-like adjustment and recovery of almost all data that is in the RAW file, making [[virtual bracketing]] virtually unnecessary.
  
−
=== Preparation ===
+
=== Shooting ===
−
First we need an example with high dynamic range to find suitable parameters. Since it is very hard to shoot a controlled high dynamic range scene in one image it is advisable to use an exposure [[Bracketing|bracketed]] series for this purpose.  
+
Shooting for maximum dynamic range implies exposing such that no important highlights are clipped, since clipped data is lost forever. But it would be best to expose as high as possible in order to pull as much shadow details out of the noise floor as well. This is known as Expose-to-the-right or ETTR since 2003.
  
−
I've chosen a setup like the one proposed by Ben Kreunen on [http://www.path.unimelb.edu.au/~bernardk/tutorials/360/technical/hdri/] with a white, a grey and a black patch. This setup can be used to determine dynamic range of the camera, too. However, the use of a grey wedge would be better for our purpose.
+
The range of brightness values in relation to the sensor is usually displayed as a histogram. But that is a bit of a problem on most cameras. The histogram display and clipping indicators show the situation for jpegs, not for raw (only some Leicas use raw histograms when shooting raw). Jpeg highlight clipping is 1 or 2 EV lower than raw clipping. If you rely on those, you expose too low and details get lost in shadow noise. In camera inspection of raw images doesn't help either, since the display uses the same clipping as for jpegs.  
  
−
Shoot all images with fixed white balance and manual exposure set to different times. If the necessary exposure time exceeds the cameras possibilities you can vary the f-stop as well but be aware of vignetting if you shoot wide open. Best is to shoot at f/11 or f/16. Indirect lighting is preferable.
+
However, there is help:
 +
* The most recent builds (as of June 2013) of [http://www.magiclantern.fm/ Magic Lantern], a firmware extension for Canon EOS cameras, allows for raw histograms and other exposure helpers for raw shooting (namely raw zebras and ETTR helpers).
 +
* Unitary White Balance (UniWB) is a way to trick the jpeg histogram into showing correct values. See [http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm tutorial] by Guillermo Luijk.
 +
* UniWB can also be achieved with Magic Lantern stable release v2.3
  
−
Focusing is no issue - in contrary: if you shoot a bit blurred you can use the series to create a specialised noise reduction profile.
+
With raw zebras it is easiest to expose correctly, since all clipped highlights are marked with a zebra pattern. But raw histogram and clipping indicators are of incredible help, too. The result might look overexposed, but this is easily fixed during raw conversion and you will find incredible shadow details in the images.
  
−
I've used a Nikon D70 with a Nikkor 10.5mm fisheye for this test and shot a series of 17 images bracketed with one f-stop difference from 1/8000s at f/16 to 8s at f/16.
+
Load all the images the panorama consists of into ACR and press "Select All".
  
−
[[Image:bridge-dialog.jpg|thumb|150px|Adobe Bridge]]
+
=== Settings ===
 +
Choose a linear curve on the Tone Curve tab. All Contrast curves will make highlights and shadows details less distinguishable and worsen clipping.
  
−
=== Opening files ===
+
Choose ''Process: 2012'' from Camera Calibration tab. Older processes introduce much more clipping. In my tests ''Adobe Standard'' Camera Profile allowed for least highlight clipping.
  
−
You can open files via Adobe Bridge or with the file open dialog. In our case only files 1 to 13 are used since in 14 to 17 the grey patch is clipped to bright white. Images should show up in the left column of Adobe camera RAW.{{clr}}
+
Now you can start to adjust the ''Exposure'' slider such that your images start to look good. Don't keep too much attention to shadows and highlights in this step.
  
−
[[Image:Camera-RAW-curve.jpg|thumb|150px|Linear curve]]
+
On ''Basic'' tab press the O and U key in order to activate Shadow and Highlight clipping warning. The small indicators in the upper corners of the histogram need to show a white border. Clipped highlights are shown as red area in the image, clipped shadows blue.
−
=== Settings ===
 
  
−
Click the brightest image in left pane and then click ''Select all'' button top left. Choose ''Camera Raw Defaults'' in the ''Settings'' list. In the right pane select the ''Curve'' tab page and select ''Linear'' as curve type. ''Settings'' will change to ''Custom'' with this step.{{clr}}
+
Select the image with most highlights for display by Alt-clicking on it (in order to keep all images selected). Move the ''Highlights'' slider to the left until the indicator in the top right corner turns black. This is the exact point where there will be no highlight clipping any more. You might want to leave specular highlights (or the sun) clipped.
  
−
[[Image:Camera-RAW-01.jpg|thumb|150px|Camera Raw start settings]]
+
Select the image with most shadows for display by Alt-clicking on it. The ''Shadows'' slider needs to be moved to the right in order to lighten shadows. If the indicator in the top left corner turns black there is no shadow clipping any more. Colored indicators show which color channel is clipped.
−
In the top row check ''Preview'', ''Shadows'' and ''Highlights'' to see clipping on both ends. Then go to the ''Adjust'' tab page and set ''Shadows'', ''Brightness'' and ''Contrast'' to the lowest possible values (leftmost slider position). The bright red area in the preview images indicates clipped highlights.
 
−
{{clr}}
 
  
−
[[Image:Camera-RAW-02.jpg|thumb|150px|Camera Raw no highlight clipping]]
+
If this is not enough to recover all dynamic range and your highlights are grey to white (f.e. cloudy sky) you can play with the ''Whites'' slider. The ''Blacks'' slider helps if the Shadows are dark grey to black.  
−
This is perfectly ok but unfortunately there is no difference visible between clipping introduced by the camera and clipping introduced by RAW converter settings. To determine this point draw up and down the exposure slider. There is a point where lowering the exposure does not further decrease the size of the clipped area (a grey wedge would be better here). You can verify the setting with the next darker images. Don't forget to press ''Select All'' before you make any adjustments.{{clr}}
 
  
−
[[Image:Camera-RAW-03.jpg|thumb|150px|Camera Raw no shadow clipping]]
+
If you want more visible shadow details pull down (left) the ''Contrast'' slider. This will give you a dull image, but you have the possibility to do a local contrast enhancement later.
−
Now go to the darkest image. Since all images where selected it has the same settings as the brightest ones. Probably it will look perfectly black, but the RGB values above the histogram show some - albeit very low - values different from zero. As long as there are no bright blue areas there is no shadow clipping. (You can increase the ''Shadows'' slider to see how it might look like).
 
−
 
 
−
If there are bright blue areas try to increase the ''Exposure'' slider. If the area does not get smaller, this is clipping introduced by the camera. If it gets significantly smaller you have reached the limits of Adobe Camera RAW. Then only [[dcraw]] with linear 16 bit output might help.{{clr}}
 
  
 
=== Other settings ===
 
=== Other settings ===
 
Unfortunately some other settings influence the dynamic range, too.  
 
Unfortunately some other settings influence the dynamic range, too.  
  
−
On ''Adjust'' tab:
+
On ''Basic'' tab:
−
* ''White Balance'' This has a severe influence on dynamic range since white balance is performed by subtracting or adding values on one or more color channels. In my test images from the Nikon D70 clipping didn't change significantly between 4400K and 6500K. This range can be enlarged if you choose lower values for ''Exposure''. Perhaps it is a good idea to do the above steps with images shot under different color temperatures and save settings for typical white balance settings. The more the white balance will differ from the ideal color temperature (should be 5500K for most cameras) the lower will be the usable dynamic range.
+
* ''White Balance'' In recent versions of ACR this has very little influence on dynamic range.
−
 
+
* ''Clarity'' can be used to bring back some local contrast, but be aware: it might re-introduce clipping.
 +
* ''Vibrance'' has little influence on clipping, provided you don't overdo it.
 
* ''Saturation'' Speaks for itself. If you increase saturation you decrease dynamic range because the color channels are saturated earlier.
 
* ''Saturation'' Speaks for itself. If you increase saturation you decrease dynamic range because the color channels are saturated earlier.
  
−
On ''Details'' tab:
+
On ''Detail'' tab:
−
* ''Sharpness'' [[Sharpening]] might introduce some clipped highlight pixels since it increases contrast along edges. Better to sharpen the result image after contrast adjustment.
+
* ''[[Sharpening]]'' might introduce some clipped highlight pixels since it increases contrast along edges. Better to sharpen the result image after contrast adjustment.
−
 
+
* ''Noise Reduction'' settings try to reduce noise. They tend to eliminate single clipped pixels but don't impact over all dynamic range.
−
* ''Luminance Smoothing'' and ''Color Noise Reduction'' try to reduce noise. They tend to eliminate single clipped pixels but don't impact over all dynamic range.
 
  
−
On ''Lens'' tab:
+
On ''HSL / Grayscale'' and ''Split Toning'' tab:
−
* ''Chromatic Aberration'' has a very little influence on dynamic range, it might reveal single clipped pixels.
+
* Use the settings carefully if you need them.  
  
−
* ''Vignetting'' This might influences the dynamic range, since it changes pixel brightness in the corners. However, since vignetting normally is a darkening of the image corners you only limit the dynamic range in the corners to the same level as in the image center if you pull the slider up to an appropriate value. If you pull the slider down highlight clipped area gets smaller. Don't think you get higher dynamic range this way. There is simply a certain value subtracted from the pixel values which the clipping indicator does not see as clipped any more. But there is no additional information.
+
On ''Lens Corrections'' tab:
 +
* ''Vignetting'' (''Manual'' sub-tab) This might influences the dynamic range, since it changes pixel brightness in the corners. However, since vignetting normally is a darkening of the image corners you only limit the dynamic range in the corners to the same level as in the image center if you pull the slider up to an appropriate value. If you pull the slider down highlight clipped area gets smaller. Don't think you get higher dynamic range this way. There is simply a certain value subtracted from the pixel values which the clipping indicator does not see as clipped any more. But there is no additional information.
  
−
On ''Calibrate'' tab:
+
* ''Remove Chromatic Aberration'' and ''Defringe'' (''Profile'' sub-tab) can (and should) safely be used.
−
* All sliders on this page - especially the ''saturation'' sliders might influence dynamic range. If you need to work with other values than 0 for any slider here you should determine the dynamic range values with these settings and save them together with your settings subset. However, since the goal here is to extract full dynamic range and not to get a perfect image directly from RAW converter I suggest to do any color corrections in your final contrast adjusted result image.
 
  
−
[[Image:Camera-RAW-save-01.jpg|thumb|150px|Camera Raw Save Settings menu]]
+
On ''Effects'' tab:
−
[[Image:Camera-RAW-save-02.jpg|thumb|150px|Camera Raw Save Settings dialog]]
+
* You don't want to use those if you go for maximum dynamic range.
  
 
=== Save Settings ===
 
=== Save Settings ===
 +
You can save your settings to a preset. However, since dynamic range extraction highly depends from the image content, it's probably better to do it on an individual basis.
  
−
If anything is ok you can go back to the brightest image and save the relevant settings. Choose ''Save Settings Subset'' from the menu you get if you click the small triangle button right of the ''Settings'' list. From the dialog uncheck all settings but ''Exposure'', ''Shadows'', ''Brightness'', ''Contrast'' and ''Tone Curve'' (Check other settings if you want to save them together with the subset, of course). Press the ''Save'' button and give a speaking name. This name will appear in the camera RAW ''Settings'' list.<br style="clear:both;" />
+
Don't forget to switch to 16 bit per channel output and choose TIFF as file format, before you save your images.
−
 
 
−
[[Image:Camera-RAW-04.jpg|thumb|150px|Camera Raw Usage]]
 
−
=== Usage ===
 
−
 
 
−
The usage is straightforward. Open images with Camera Raw, ''Select All'', Choose settings subset from the ''Settings'' list, check ''Show Workflow Options'', choose your desired space and ''Depth'' ''16 Bits/Channel'', press ''Save'' or ''Open'' button.
 
−
<br style="clear:both;" />
 
−
 
 
−
[[Image:Camera-RAW-result.jpg|thumb|150px|Camera Raw Result]]
 
−
=== Result ===
 
−
 
 
−
To verify that the whole dynamic range the camera is capable of is captured in the result files we open the darkest and the brightest non-clipped image with the above determined settings. The dark image is 10 f-stops below the bright one (and almost 11 f-stops below clipping level) and hast to be lightened extremely in order to see anything. It shows heavy noise of course - noise is the limiting factor for digital camera dynamic range. You probably won't like to use it for anything.
 
  
 
=== How to proceed ===
 
=== How to proceed ===
−
What to do now with these files? You can use a simple s-shaped curve to simulate analog film characteristics, you can use photoshop "Shadows/Highlights" to reveal some of the shadows details. You can use my [[Contrast Masking Actions]] to have a more [[halo]] free result or you can use one of the [[HDR compression]] plugins.
+
What to do now with these files? There are tons of possibilities to enhance the result. You can increase local contrast by simple high radius, low amount unsharp masking or by HDR tonemapping (most tonemappers, including photoshop, work on 16 bit images as well).
−
 
 
−
For the latter there are some preparations necessary: Either convert the image to 32 Bit per channel by simply choosing from the menu or convert to a linear [[gamma]] color space. How to create a linear gamma color space can be found in the [[Dcraw#16_bit_files_with_Photoshop|dcraw article]].
 
  
 
=== Remark ===
 
=== Remark ===
 
It can not be guaranteed that you can extract the very last bit of information from your RAW this way since we don't exactly know what the RAW converter does. However, it could well be that you get even more dynamic range from the converter than your camera actually captured.
 
It can not be guaranteed that you can extract the very last bit of information from your RAW this way since we don't exactly know what the RAW converter does. However, it could well be that you get even more dynamic range from the converter than your camera actually captured.
  
−
How that? Color channels are not all saturated at the same level. If you f.e. use [[dcraw]] without the -n option you apparently get less dynamic range than from most other RAW converters. Dcraw clips the colors to the lowest clipped channel. If you use the -n option (and an appropriate -b value) you get anything the camera captured, including a single clipped channel. This might lead to very strange effects.  
+
How that? Color channels are not all saturated at the same level. If you f.e. use [[dcraw]] without the -n option you apparently get less dynamic range than from most other RAW converters. Dcraw clips the colors to the lowest clipped channel. If you use the -n option (and an appropriate -b value) you get anything the camera captured, including a single clipped channel. This might lead to very strange effects. However, ACR and other raw converters don't suffer from this shortcoming.
  
−
According to dave coffin (author of dcraw) most other RAW converters try to reconstruct the clipped channel from the other channels. This is why f.e. Adobe Camera RAW sometimes seems to deliver more dynamic range than dcraw in 16 bit mode does.
+
According to Dave Coffin (author of dcraw) most other RAW converters try to reconstruct the clipped channel from the other channels. This is why f.e. Adobe Camera RAW sometimes delivers more dynamic range than dcraw in 16 bit mode does.
  
 +
== External links ==
 +
=== ETTR ===
 +
* The original article: http://www.luminous-landscape.com/tutorials/expose-right.shtml
 +
* Signal to noise theory and exposure decisions: http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#ETTR
 +
* Good and quick explanation: https://en.wikipedia.org/wiki/Exposing_to_the_right
  
−
<small>--[[User:Erik Krause|Erik Krause]] 11:44, 11 Dec 2005 (EST)</small>
+
=== Unitary white balance ===
 +
* The original article: http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm
 +
* How to get UniWB faster on POTN: http://photography-on-the.net/forum/showthread.php?t=485349
  
 
[[Category:Tutorial:Specialised]]
 
[[Category:Tutorial:Specialised]]
 
[[Category:Software:Platform:Windows]][[Category:Software:Platform:Mac OS X]]
 
[[Category:Software:Platform:Windows]][[Category:Software:Platform:Mac OS X]]

Latest revision as of 22:57, 13 July 2013

RAW dynamic range extraction

Linear curve
Camera settings
Basic settings

Intro

Sometimes it's not feasible or not recommended to shoot bracketed. Action scenes and large crowds f.e. or landscapes in windy weather, where lots of leafs and branches are moving.

This is a small tutorial on how to extract the full dynamic range of a RAW camera file into a single 16 bit per channel document using Adobe camera RAW (ACR) converter of photoshop CS6 (same engine as Lightroom 5).

Extracting the whole dynamic range is a snap in recent ACR. No need to fiddle with low contrast anymore, like described in the old article for photoshop CS2. The Highlights, Shadows and other sliders provide tonemapping-like adjustment and recovery of almost all data that is in the RAW file, making virtual bracketing virtually unnecessary.

Shooting

Shooting for maximum dynamic range implies exposing such that no important highlights are clipped, since clipped data is lost forever. But it would be best to expose as high as possible in order to pull as much shadow details out of the noise floor as well. This is known as Expose-to-the-right or ETTR since 2003.

The range of brightness values in relation to the sensor is usually displayed as a histogram. But that is a bit of a problem on most cameras. The histogram display and clipping indicators show the situation for jpegs, not for raw (only some Leicas use raw histograms when shooting raw). Jpeg highlight clipping is 1 or 2 EV lower than raw clipping. If you rely on those, you expose too low and details get lost in shadow noise. In camera inspection of raw images doesn't help either, since the display uses the same clipping as for jpegs.

However, there is help:

  • The most recent builds (as of June 2013) of Magic Lantern, a firmware extension for Canon EOS cameras, allows for raw histograms and other exposure helpers for raw shooting (namely raw zebras and ETTR helpers).
  • Unitary White Balance (UniWB) is a way to trick the jpeg histogram into showing correct values. See tutorial by Guillermo Luijk.
  • UniWB can also be achieved with Magic Lantern stable release v2.3

With raw zebras it is easiest to expose correctly, since all clipped highlights are marked with a zebra pattern. But raw histogram and clipping indicators are of incredible help, too. The result might look overexposed, but this is easily fixed during raw conversion and you will find incredible shadow details in the images.

Load all the images the panorama consists of into ACR and press "Select All".

Settings

Choose a linear curve on the Tone Curve tab. All Contrast curves will make highlights and shadows details less distinguishable and worsen clipping.

Choose Process: 2012 from Camera Calibration tab. Older processes introduce much more clipping. In my tests Adobe Standard Camera Profile allowed for least highlight clipping.

Now you can start to adjust the Exposure slider such that your images start to look good. Don't keep too much attention to shadows and highlights in this step.

On Basic tab press the O and U key in order to activate Shadow and Highlight clipping warning. The small indicators in the upper corners of the histogram need to show a white border. Clipped highlights are shown as red area in the image, clipped shadows blue.

Select the image with most highlights for display by Alt-clicking on it (in order to keep all images selected). Move the Highlights slider to the left until the indicator in the top right corner turns black. This is the exact point where there will be no highlight clipping any more. You might want to leave specular highlights (or the sun) clipped.

Select the image with most shadows for display by Alt-clicking on it. The Shadows slider needs to be moved to the right in order to lighten shadows. If the indicator in the top left corner turns black there is no shadow clipping any more. Colored indicators show which color channel is clipped.

If this is not enough to recover all dynamic range and your highlights are grey to white (f.e. cloudy sky) you can play with the Whites slider. The Blacks slider helps if the Shadows are dark grey to black.

If you want more visible shadow details pull down (left) the Contrast slider. This will give you a dull image, but you have the possibility to do a local contrast enhancement later.

Other settings

Unfortunately some other settings influence the dynamic range, too.

On Basic tab:

  • White Balance In recent versions of ACR this has very little influence on dynamic range.
  • Clarity can be used to bring back some local contrast, but be aware: it might re-introduce clipping.
  • Vibrance has little influence on clipping, provided you don't overdo it.
  • Saturation Speaks for itself. If you increase saturation you decrease dynamic range because the color channels are saturated earlier.

On Detail tab:

  • Sharpening might introduce some clipped highlight pixels since it increases contrast along edges. Better to sharpen the result image after contrast adjustment.
  • Noise Reduction settings try to reduce noise. They tend to eliminate single clipped pixels but don't impact over all dynamic range.

On HSL / Grayscale and Split Toning tab:

  • Use the settings carefully if you need them.

On Lens Corrections tab:

  • Vignetting (Manual sub-tab) This might influences the dynamic range, since it changes pixel brightness in the corners. However, since vignetting normally is a darkening of the image corners you only limit the dynamic range in the corners to the same level as in the image center if you pull the slider up to an appropriate value. If you pull the slider down highlight clipped area gets smaller. Don't think you get higher dynamic range this way. There is simply a certain value subtracted from the pixel values which the clipping indicator does not see as clipped any more. But there is no additional information.
  • Remove Chromatic Aberration and Defringe (Profile sub-tab) can (and should) safely be used.

On Effects tab:

  • You don't want to use those if you go for maximum dynamic range.

Save Settings

You can save your settings to a preset. However, since dynamic range extraction highly depends from the image content, it's probably better to do it on an individual basis.

Don't forget to switch to 16 bit per channel output and choose TIFF as file format, before you save your images.

How to proceed

What to do now with these files? There are tons of possibilities to enhance the result. You can increase local contrast by simple high radius, low amount unsharp masking or by HDR tonemapping (most tonemappers, including photoshop, work on 16 bit images as well).

Remark

It can not be guaranteed that you can extract the very last bit of information from your RAW this way since we don't exactly know what the RAW converter does. However, it could well be that you get even more dynamic range from the converter than your camera actually captured.

How that? Color channels are not all saturated at the same level. If you f.e. use dcraw without the -n option you apparently get less dynamic range than from most other RAW converters. Dcraw clips the colors to the lowest clipped channel. If you use the -n option (and an appropriate -b value) you get anything the camera captured, including a single clipped channel. This might lead to very strange effects. However, ACR and other raw converters don't suffer from this shortcoming.

According to Dave Coffin (author of dcraw) most other RAW converters try to reconstruct the clipped channel from the other channels. This is why f.e. Adobe Camera RAW sometimes delivers more dynamic range than dcraw in 16 bit mode does.

External links

ETTR

Unitary white balance

Retrieved from "https://wiki.panotools.org/index.php?title=RAW_dynamic_range_extraction&oldid=15512"