Batch merge handheld images

Problem

Shooting for HDR requires the bracketed images to be perfectly aligned. Usually this is done using a sturdy tripod. But sometimes this isn't possible and images need to be aligned later. For not so wide rectilinear images this requires a simple image shift. PTGui and hugin allow for not linking bracketed shots to the same place in order to compensate for shifts and several HDR programs do a simple alignment as well.

Very wide angle and fisheye lenses are more difficult. Since handheld shooting means a possible displacement in all three spatial directions and fisheyes cover up to 180° or more a back or forth displacement effectively means a different viewpoint. Any attempts to align such images should take this into account. However, since handheld shooting always can cause parallax errors between the bracketed image a perfect alignment is not always possible.

Solution

Best results can be achieved if all possible image distortions are allowed. Panotools based stitchers allow for optimization of Yaw, Pitch and Roll, field of view, lens correction parameters and center shift. The most recent versions also allow for viewpoint correction (called translation in hugin). There should be plenty of control points throughout the whole image. All this possibilities together usually allow for a very good alignment.

hugin

align_image_stack

Hugin comes with a command line tool called align_image_stack. Since version 2011.5 it allows for optimization of camera position, too. For a fisheye lens you need to specify parameters -e (fisheye lens), -f HFOV (where HFOV is the approximate horizontal field of view in degrees), -m (Field of View), -d (distortion), -i (center shift) and -x -y -z (camera position, since version 2011.5 only).

The hugin windows bundle also includes a batch file called enfuse_align_droplet.bat which calls align_image_stack and passes the aligned images to enfuse for merging. You need to open the file with a text editor and put the correct parameters for your lens in the line starting with

set align_additional_parameters=

hugin GUI

A way to align images directly inside hugin GUI is outlined in Align a stack of photos.

PTGui

Aligning images in PTGui is a bit tricky, but it has the advantage of the Batch Builder and is relatively fast.

create a template

This outlines the creation of a template to align fullframe fisheye images with output dimensions exactly the same as input for use in the Batch Builder. For rectilinear images visit https://ptgui.com/support.html#7_8

Please follow the steps carefully (assuming advanced mode in PTGui pro):

1. open PTgui and load a set of images with the pixel size you will process.
2. on Lens settings tab:
   • choose Lens type Fullframe Fisheye
   • enter the approximate Focal length
   • enter 0 for Fisheye factor
   • click Reset in the Lens correction section.
   • Copy the value for Horizontal Field of View. Be sure to place the cursor in the field in order to get non-rounded values.
3. open Pano Editor (Ctrl+E) in the side-menu:
   • choose Projection->Fullframe fisheye and paste the copied Field of View value in the left field (hFoV).
   • calculate the other field (vertical Field of View) from the input image dimensions (as seen on Source Images tab) by this formula: ${\text{vFoV}}={{\text{pixel height}}/{\text{pixel width}}}\cdot {\text{hFoV}}$
4. on Create Panorama tab:
   • choose Megapixels from the fix at dropdown.
   • enter the correct width.
   • Now the correct height should be visible.

     All this is necessary because PTGui doesn't support Fit panorama for output projection Fullframe. Without those counter-intuitive steps it wouldn't be possible to have the exact same output dimensions and projection like input. The following steps prepare the template for the batch builder:

5. Your anchor image should be your best exposed one (depending on your bracketed sequence).
6. on Lens Settings tab check Lens and Shift for all but for your anchor image (double click the column header for all images).
7. on Optimizer tab Advanced:
   • set Viewpoint to Optimize and check FoV, a, b, c, HShift and VShift for all but your anchor image.
   • Uncheck anything under Optimize globally.
8. on Exposure/HDR tab:
   • use True HDR or Exposure fusion to your liking.
   • check Enable Exposure Fusion for single exposures or Tone map stitched panorama depending on your merge settings.
   • Click Reset for anything else.
9. on Project Settings tab:
   • in 'Align Images' behaviour section uncheck anything but Generate control points and Optimize project.
   • under HDR defaults choose Merge bracketed images to HDR, but align them individually and Apply the above without asking.
   • check anything in Batch Stitcher actions
   • under File naming, Project file save as choose <prefix first image filename> and enter _fused. Panorama save as Same as project name and save in Source image folder for both. Check Store relative path...
   • in Template behaviour section check anything but Image parameters and Image masks.
   • uncheck anything under Batch builder behaviour and Miscellaneous.
10. on Create Panorama tab choose TIFF, 16 bit, no compression, no alpha as LDR file format and your preferred output mode under Output.
11. choose File->Save as template from the menu and give it a reasonable name.

For later modification, it is advisable to save as a normal project, too.

Use the template

Your source images need to be in a folder with no supplemental images or files. In order to have enough control points you need to go to the PTGui menu Tools->Options->Control Point Generator and enter a high number (f.e. 2000) for Generate at most ... control points per image pair and an even higher one (f.e. 20000) for ... per project. Don't forget to reset these numbers to the previous values before you stitch normal panoramas again. Close PTGui in order to have this values saved.

To actually use the template

1. open PTGui
2. start the batch builder from Tools->Batch Builder menu and Click Generate new projects.
3. choose Method: Multiple panoramas per folder with a fixed number of images, choose source folder and enter number of images (maximum will be the number of images you created the template with). After confirmation the Batch Builder will show the projects with image thumbnails.
4. under the preview choose your template from Use template dropdown.
5. once you click Generate Projects PTGui creates a project file for each set of images and a batch list for the Batch Stitcher, which either can be processed immediately or later.

Use with SNS-HDR lite

It happens that SNS-HDR lite command line is partially compatible with PTGui plugin interface of version 9 and earlier. SNS-HDR lite can be used instead of enblend to process images. To do this go to PTGui menu Tools->Options->Plugins. In the Enblend section and browse to the SNS-HDR.exe you previously installed. Under Command line parameters enter

-da -natural -tiff16

-da disables SNS-HDR internal alignment, -natural is a style preset. Call SNS-HDR.exe directly (double click desktop shortcut) to get information about other command line parameters.

Remember to revert this if you actually want to use enblend for normal panoramas.

Modify the above template creation steps to create a template as follows:

1. on Exposure/HDR tab uncheck Enable HDR stitching
2. on Project Settings tab under HDR defauls choose Do not enable HDR mode and enter a different prefix (f.e. _SNS) for Project file: save as...
3. on Create Panorama tab choose TIFF, 16 bit, no compression, no alpha as File format and Blended panorama only for Layers. In Advanced section choose Blend using: Enblend plugin

Unfortunately this doesn't work with PTGui version 10 any more. However, you can use PTGui 9 alongside with higher versions if you install it into another folder. To get older versions follow the procedure outlined in the PTGui FAQ