Panorama scripting in a nutshell

This is a short document describing the various Open Source tools and techniques available for working with panoramas in a non-GUI environment. The examples are based on Linux experience, but most of them should work on OS X or Windows with minor modifications at most.

This document is also useful for understanding what happens 'under the hood' in hugin itself and recommended reading for anyone looking to work with the hugin sourcecode.

Why would you do that?

Creating and manipulating panoramas is something that should obviously be done in a graphical tool. This is true, but perhaps you find that you are spending more time operating a computer than being a photographer. This is an indication that something is wrong - The computer exists to do the boring work and scripting is one way to make this happen.

PanoTools 'Scripting'

The file format first used in Helmut Dersch's Panorama tools is often referred to as a 'script', but a 'PanoTools' script is really a data file describing a 'panorama stitching project'.

Although these original PanoTools are less used today, the file format is alive and well with several variations used as project files by PTGui, PTAssembler, hugin and related tools. These files are simple plain-text, sometimes it is useful to modify them directly with a text editor - Further on we will introduce various methods for manipulating them via the 'programming' kind of scripts.

Simple command-line stitching

Lets start with a basic but common use case: Photographers with quality panoramic tripod heads can reproduce identical sets of photos every time, in this situation there is no need to create a new stitching project for every panorama, it is enough to use a single project as a template.

This example takes a pre-existing project called template.pto, created with three photos. It uses nona for remapping, and enblend for blending the remapped photos into a finished TIFF image:

 nona -o out -m TIFF_m template.pto DSC_1234.JPG DSC_1235.JPG DSC_1236.JPG
 enblend -o finished.tif out0000.tif out0001.tif out0002.tif

The first nona command creates three remapped TIFF images called out0000.tif, out0001.tif etc... the list of input images can be substituted with any set of identically sized photos. The second enblend command uses these TIFF files as input and merges them to a file called finished.tif. template.pto

If the internal blender is used, the commands simplify to

 nona -o out -m TIFF template.pto DSC_1234.JPG DSC_1235.JPG DSC_1236.JPG

Creating hugin projects on the command-line

The entire panorama creation process can be performed on the command-line, and therefore scripted, doing this involves a series of steps similar to the workflow in a GUI tool.

Generating the project file

Most of hugin tools take a pto file as input and output. So the first step is to create this pto file. For this purpose use pto_gen.

  pto_gen *.jpg

will put all jpg files in the current directory into one project file with a default name. You can also specify single images and your own project name

  pto_gen -o project.pto image1.jpg image2.jpg image3.jpg

You can specify more options to the project (see pto_gen for details).

Generating control points

There are a number of tools available for generating control points from groups of photos, like cpfind. Here is an example using cpfind

  cpfind --multirow -o project.pto project.pto

Hint: Other control points generator, such as autopano-sift, autopano-sift-c, panomatic and match-n-shift take a list of images instead of a project. In this case the pto generation and control point generation are one step. Here is an example for autopano-sift-c:

 autopano-sift-c --projection 0,50 project.pto DSC_1234.JPG DSC_1235.JPG DSC_1236.JPG

This will create a .pto project with control points (if any) linking the three photos, note that the projection format (f0, rectilinear) of the input photos and approximate horizontal angle of view (v50, 50 degrees) have to be specified.

Pruning control points

This still isn't a finished panorama, at the very least this involves optimising the camera positions and then rendering the output.

You could go ahead and optimise this project file straight away, but this can be a bit hit and miss. First it is a good idea to clean up the control points. There are currently two useful tools for cleaning control points: celeste removes points from areas of sky and cpclean removes points with large error distances, first celeste_standalone:

 celeste_standalone -i project.pto -o project.pto

Then cpclean:

 cpclean -o project.pto project.pto

Note that in both cases we are overwriting the input project.pto file with the cleaned output. If you want you can create intermediate temporary files and keep each step in case you need to backtrack.

Other control point cleaners are ptscluster and ptoclean. cpclean has largely replaced ptoclean, but you could still run it like this:

 ptoclean -v --output project.pto project.pto

Optimising positions and geometry

Up to now, the project file simply contains an image list and control points, the images are not yet aligned, you can do this by optimising geometric parameters with the autooptimiser tool:

 autooptimiser -a -l -s -o project.pto project.pto

Now you could just skip ahead and render the output if photometric optimisation isn't required.

By adding the -m switch also photometric optimisation is done. So the output project is optimised for positions, geometry and photometric parameters in one run:

 autooptimiser -a -l -s -m -o project.pto project.pto

If you need finer control above the optimisation, set the parameter which should be optimized with pto_var and then run autooptimiser with switch -n:

 pto_var --opt y,p,r -o project.pto project.pto
 autooptimiser -n -o project.pto project.pto

(PToptimizer is an alternative geometrical optimiser)

Optimising photometric parameters

Photos have photometric exposure variables which can be optimised in much the same way as aligning positions. Doing this can remove brightness, white balance and vignetting differences between photos, giving better blending or allowing creation of HDR images.

For doing an automatic photometric optimisation like Hugin's assistant, run autooptimiser with -m switch:

 autooptimiser -m -o project.pto project.pto

For full manual control for photometric parameters use vig_optimize. This program does the equivalent job for photometric parameters as autooptimiser does for geometric parameters. You need to indicate in the .pto project which parameters are 'variables'. For this task you can edit the .pto file or use a tool such as pto_var or ptovariable:

 ptovariable --vignetting --response --exposure project.pto
 vig_optimize -o project.pto project.pto

 pto_var --opt Vb,Vx,Ra,EeV -o project.pto project.pto
 vig_optimize -o project.pto project.pto

Setting output options

For setting the output option (like on the stitcher tab) use pano_modify:

   pano_modify -o project.pto --center --straighten --canvas=AUTO --crop=AUTO project.pto

This command will straighten the pano, center it on the canvas, calculates the optimal size and calculates the optimal output region.

Example work flow similar to assistant in Hugin GUI

Hugins assistant can also be used from the command line

   hugin_executor --assistant project.pto

Or you can also put the task into PTBatcherGUI queue

   PTBatcherGUI --assistant project.pto

In the following the single steps of the assistant are described. We will use the same project file for all single steps. We always overwrite the same project file. You can also use another filename for every step.

1) Generate pto file

    pto_gen -o project.pto *.jpg

2) Find control points with cpfind, with celeste to ignore clouds

    cpfind -o project.pto --multirow --celeste project.pto

3) Control point cleaning

    cpclean -o project.pto project.pto

4) Find vertical lines

    linefind -o project.pto project.pto

5) Optimize position, do photometric optimization, straighten panorama and select suitable output projection

    autooptimiser -a -m -l -s -o project.pto project.pto

6) Calculate optimal crop and optimal size

    pano_modify --canvas=AUTO --crop=AUTO -o project.pto project.pto

7) Now create output

    hugin_executor --stitching --prefix=prefix project.pto

Stitching hugin projects on the command-line

The stitching option of hugin_executor can be broken apart into separate command lines.

The simplest case is remapping and blending.

Remapping (Rendering) images

The hugin tool for remapping and distorting the photos into the final panorama frame is nona, it uses the .pto project file as a set of instructions:

 nona -m TIFF_m -o project project.pto

This nona command creates one remapped TIFF image for each of the input photos, these will be named project0000.tif, project0001.tif, project0002.tif etc...

(PTblender is an alternative renderer to nona)

Blending

nona can do rudimentary assembly of the remapped images, but a much better tool for this is enblend, feed it the images, it will pick seam lines and blend the overlapping areas:

 enblend -o project.tif project0000.tif project0001.tif project0002.tif ...

Finally this produces the stitched panorama: project.tif

Holding it all together with panostart

The .pto.mk Makefile helps with the stitching at the end of the process, but doesn't help with the bits at the beginning such as: identifying which photos go in which panorama, generating control points, pruning and optimising.

panostart is a prototype tool that does it all. Given a list of photos in a folder, it identifies likely panoramas by comparing EXIF timestamps and writes a meta-Makefile containing: rules for generating .pto projects with match-n-shift, rules for generating corresponding .pto.mk Makefiles with pto2mk, rules for stitching the projects with make, plus some other rules for creating useful distributable output.

A typical panostart command for dealing with a folder full of photos taken with a pocket point-and-shoot camera on auto-exposure might look like this:

 panostart --output Makefile --projection 0 --fov 50 --nostacks --loquacious *.JPG

Processing this Makefile is simple:

 make

Some of the resulting panoramas will have position, cropping and stitching errors. Fix these by editing the relevant .pto projects in hugin or another tool, when you type make again on the command-line, only those projects that have changed will be restitched.

Another advantage of doing things this way is reproducibility: delete all the (enormous) TIFF output panoramas and all you need to keep permanently are the photos, the meta-Makefile and the associated .pto projects. If you ever need to recreate the TIFF panoramas in the future, just type make. If you know the name of the TIFF file you want, just type 'make project.tif' and only a single panorama will be processed.

Panotools::Script for manipulating project files

The name is a historical mistake, Panotools::Script is a perl library for working with hugin .pto projects, though it could be extended in the future to cover any of the related file formats. Many of the command-line tools described in this document are written with this library.

Here is a trivial script, all it does is open and save a .pto project, it also resets the output ('p' Panorama line) angle of view to 360 degrees (v360), the projection format to equirectangular (f2) and the output pixel size to 4096x2048 pixels (w4096 h2048):

 #!/usr/bin/perl
 use Panotools::Script;
 my $pto = new Panotools::Script;
 $pto->Read ($ARGV[0]);
 $pto->Panorama->Set (v => 360, f => 2, w => 4096, h = 2048);
 $pto->Write ($ARGV[0]);

Here's slightly more complex script, this iterates over the input images and increments the white balance making the panorama slightly 'warmer':

 #!/usr/bin/perl
 use Panotools::Script;
 my $pto = new Panotools::Script;
 $pto->Read ($ARGV[0]);
 for my $image (@{$pto->Image})
 {
     my $Er_new = $image->{Er} * 100 / 95; # red value
     my $Eb_new = $image->{Eb} * 95 / 100; # blue value
     $image->Set (Er => $Er_new, Eb => $Eb_new);
 }
 $pto->Write ($ARGV[0]);

Full list of command-line tools

Control point generation

• cpfind - Hugins own control point detector
• autopano-sift-c - All in one keypoint generator and matcher
• match-n-shift - All in one wrapper around align_image_stack and autopano-sift-c
• panomatic - All in one alternative to autopano-sift-c
• align_image_stack - Generates control points for stacked photos
• ptoanchor - add control points to a Hugin project
• ptochain - add control points to a Hugin project between consecutive photos
• ptobind - Join the ends of linked photos in a Hugin project
• ptofill - add control points to a Hugin project between likely overlapping photos
• generatekeys - Keypoint generator
• autopano - Keypoint matcher, part of autopano-sift-c and not to be confused with autopano

Project file modification

• pto_gen - Generates a new pto project from image files
• match-n-shift - (see above) Default usage is to initialise a new .pto project from EXIF data
• ptoset - Change global parameters and project options
• ptovariable - Set typical optimisation variables
• pto_var - Manipulate image variable, can also set optimisation variables
• pto_lensstack - Manipulate lenses and stacks
• pto_mask - Add mask to project file
• pto_template - Apply template to project file
• ptsed - Set any project parameter, convert .pto into PanoTools format
• ptosort - Sort control points or images, remove duplicate control points
• ptopath - Remove file paths in project files
• transform-pano - Roll, pitch and yaw rotation of .pto projects
• ptocentre - Rotate projects to put images in panorama centre
• pano_modify Change output parameters of project file

• ptomerge - Join two or more projects
• pto_merge Merges two or more Hugin project files (this is a different tool to ptomerge above)
• ptosplit - Extract subsets of images from a project into a new project

Control point pruning

• ptscluster - Clean 'bad' control points based on distances and position
• celeste_standalone - Clean sky control points
• ptoclean - Clean 'bad' control points based on standard deviation error
• cpclean - An improved version of ptoclean, part of the hugin project

Optimisation

• autooptimiser - Optimise geometry of .pto projects
• PToptimizer - Optimize geometry of PanoTools format projects
• vig_optimize - Optimise photometric parameters of .pto projects

Rendering

• nona - Default hugin rendering engine
• nona-mask - Wrapper around nona for using external bitmap masks
• PTmender - Render PanoTools format projects

• PTAInterpolate - Create intermediate images from PanoTools projects

Blending

• enblend - Merge partially overlapping images with multiresolution splines
• verdandi - Merge partially overlapping images with watershed algorithm
• enfuse - Merge overlapping images with exposure fusion
• PTroller - Merge partially overlapping images with no seaming
• PTmasker - Compute stitching masks for input by PTroller
• enblend-mask - Wrapper around enblend for using external bitmap masks
• enfuse-mask - Wrapper around enfuse for using external bitmap masks
• deghosting_mask - Creates mask for removing ghosting in images
• hugin_hdrmerge - Merge bracketed images for HDR generation
• enblend-svg - Wrapper around enblend for specifying input images via SVG files

• process-masks - Allow 'positive' masking in external masks for enblend
• tif2svg - Assemble multiple TIFF images into SVG files for enblend-svg
• PTblender - Correct colour and brightness of overlapping images

Misc

• panostart - All-in-one panorama workflow via Makefiles
• gigastart - All-in-one assembly of multi-row panoramas
• hugin_executor - Stitch project or run assistant from command line
• gigatile - Create a Makefile for stitching a project as multi-resolution tiles
• ptodummy - Generate missing input photos to debug .pto projects
• calibrate_lens - Automatic lens calibration using straight-line detection
• pafextract - Tool to use panoglview as a zenith and nadir editor

Chromatic aberration

• fulla - Correct lens distortion and chromatic aberration
• tca_correct - Calculate chromatic aberration in a photo

Querying

• ptograph - Draw undirected graphs of projects
• ptoget - Query any project parameter
• ptoinfo - Generate a report on a project file
• PTinfo - Display info about a panotools generated TIFF file
• panoinfo - Display information about installed libpano13 library
• pano_trafo - transform between image and panorama coordinates (and vice versa)

TIFF files

• PTtiff2psd - Join multiple TIFF images into a multilayer PSD file
• PTtiffdump - Compare two TIFF files
• PTcrop - Remove unwanted empty space in TIFF files and replace with offsets
• PTuncrop - Replace offsets in TIFF files with empty pixels

Output formats

• qtvr2img - Extract six cubefaces from a cubic QTVR
• cubic2erect - Render six cubefaces into a single equirectangular image
• qtvr2erect - Render a cubic QTVR into a single equirectangular image
• jpeg2qtvr - Assemble six JPEG cubefaces into a cubic QTVR
• erect2qtvr - Render a single equirectangular image into a cubic QTVR

• erect2cubic - Create a .pto project for extracting cube faces from an equirectangular image
• erect2planet - Create a .pto project for extracting a 'little planet' from an equirectangular image
• erect2mercator - Create a .pto project for extracting a mercator view from an equirectangular image

Older tools

Helmut Dersch's PTOptimizer and PTStitcher were the original command-line tools. These are not really recommendable any more as they require an older, buggier version of the libpano12 library, they are closed source so don't work on modern systems such as OS X or 64bit Linux, and are lacking in features compared to modern replacements such as autooptimiser and nona.

Useful Image processing tools

• ImageMagick - Many utilities for converting, manipulating and resizing images
• GraphicsMagick - A fork of ImageMagick with many improvements
• libtiff - Supplied with useful tools such as tiffcp for joining and recompression, and tifficc for adding ICC colour profiles
• exiftool - All purpose reading, setting and copying of EXIF data
• exiv2 - EXIF manipulation, useful for setting the file modification date from the EXIF date
• jpegtran - Lossless rotation of JPEG images
• ufraw-batch - Command-line RAW processing
• dcraw - Command-line RAW processing