Extract and insert rectilinear views

Intro

One of the most problematic points when shooting spherical panoramas is the view straight down also called the nadir view. Either the tripod is visible or there may be alignment or color mismatches if shooting handheld.

Unfortunately neither the nadir nor the zenith (straight above) area can be edited directly in the equirectangular image since it is very distorted. For this taske we need to extract a partial view - best in rectilinear projection.

Possibilities

There are basically three possibilities to extract partial rectilinear views:

• PTEditor
• Panotools Plugins
• PTStitcher used directly or indirectly by script or batch file, GUI or PanoCube Plus

PTEditor

PTeditor is a Java application provided with the standard Panotools distribution. It should run on any OS where Java and the panotools runtime (pano12) is installed. There is a nice tutorial on How to use PTEditor. This is the most convenient way to extract and insert any arbitrary rectilinear view. The disadvantage is that it doesn't save 16 bit images.

Panotools plugins

The Photoshop compatible Panorama Tools Plugins are part of the standard Panotools distribution as well. There is an upgraded version available from Thomas Niemann that corrects some bugs and works for 16 bit.

The plugins contain the Adjust filter, which is capable of extracting and inserting any view in almost any projection. Any data (pan/tilt angle, size of view etc.) has to be inserted manually. Yaw, roll and pitch value have to be calculated or estimated which makes the usage a bit inconvenient.

(More text of an experienced plugin user needed here)

PTStitcher

PTStitcher supports 16 bit images and can be used to extract and insert partial views in different ways.

• By directly scripting it.
• With PanoCube Plus
• By one of it's GUIs (PTGui, PTAssembler, Hugin, PTMac...)

Scripts

There are sevaral scripts or batch files available that extract 6 cube faces from an equirectangular image. One of the most convenient windows batch file is written by Eric Gerds and part of his DOS Utilities for Panoramas. A less convenient batch file is found on Erik Krause's page. PTStitcher scripts are found on Ben Kreunen's page

PanoCube Plus

PanoCube Plus can extract six cube faces (single, batch modes) for editing and then convert it directly into QuickTime cubic movie(s). Both 8/16 bits per channel are supported - dependend from pano12 library used the cube faces are the same bit depth as the source images.

To activate "equirectangular to 6 cube faces" mode copy the file Script.txt to the folder where your source image is and set "1" as first character in this line using a text editor (notepad!):

1  # create tiles for editing ( 0 = NO, 1 = YES). Default NO.

in the file Script.txt and put it into pano's folder. Next drag and drop your single pano or folder with panos onto the program icon. The source file name and path should contain no spaces. The source image must be exactly in 2:1 format (f.e. 2000x1000 or 6000x3000 pixels). Panocube calls PTStitcher 6 times and creates 6 files in the order front, right, back, left, top, bottom and replaces the last two characters of the name with 11, 22, 33, 44, 55, 66. All files ended with 55 are represent zenith view, ended with 66 - nadir view.

After editing drag again any one (not all!) tile or folder with tiles to get QuickTime cubic movie(s). In batch mode important to have first six letters differ in filenames, otherwice PanoCube can skips "similar" files.

This technique is good if the final product is a QT movie. If you need an equirectangular for PTViewer you can of course drag the .mov file on panocube plus to get it converted, but image quality suffers because of intermediate jpeg compression and decompression and another interpolation step.

Inserting cubefaces again

You can assemble the cube faces again using the PTGui template from http://www.erik-krause.de/ttt or the batch file from Eric Gerds: Dos for Panoramas.

You can even remap only one cube face to equirectangular and insert it as an additional layer into your layered photoshop document. This has several benefits:

• no additional interpolation for the rest of the panorama.
• possibility to patch nadir in a relative early stage of editing thus leaving open any possibility of correcting over all colors etc.
• preserving as much as possible of the original image data

To re-insert the patched cubeface into the original document implies that you have worked on a cube face with enough resolution, preferable the pixel width divided by 3.14

You can use the batch file or PTGui template mentioned above. You simply have to set output size equal to your original panorama size and create single image TIFFs as output. Any of those TIFF files can be added to the original panorama as a masked layer using the "Tiff to masked layers" action from http://www.erik-krause.de/ttt

To use any of the other GUIs you need to

• Add the cube face as input image
• specify rectilinear and FoV 90� for input images
• set all lens correction parameters to 0.0
• specify the correct yaw and pitch values for the particular image:

 tile    front   right   back    left    top     bottom
 Yaw     0       90      180     -90     0       0
 Pitch   0       0       0       0       90      -90

• specify the correct output size.
• specify Tiff without mask or feather (TIFF_m) as output format.
• don't set control points and don't optimize
• create the panorama

GUI

All of the GUIs should be capable of extracting an arbitrary rectilinear view from a spherical pano. Use the equirect image as input (don't forget to set the correct FoV) and specify a rectilinear image with 90� as output "panorama". Set all lens correction parameters to 0.0. Then change yaw and pitch of the input image until you get the desired view. Note that the values work the other way round than when inserting an image.

PTGui

PTGui offers an additional advantage: The panorama editor and numerical transform. To use it load your image as described above, open panorama editor and choose 'Numerical Transform' from the edit menu.

To pan along the horizon f.e. in 20� steps insert 20 into the 'Yaw' field and press 'Apply' repeatedly until you get the desired view. Positive values pan right, negatives left.

Do the corresponding for 'Pitch' and 'Roll' if needed. This way you can extract any partial view. Once your are done create the "Panorama" (which is the desired partial view) using output format TIFF. Note down or save the exact values for Yaw, Roll and Pitch as shown on 'Image Parameters" tab page.

Now edit the partial view to your need. Then add it to the project with your original panorama and on 'Panorama settings' tab change to equirectangular and 360�x180�. Choose individual lens parameters for the new image on 'Lens settings' tab and set 'Rectilinear' and 90� on 'Image parameters' tab. The image should fit now perfectly in the original panorama.

In order to move the panorama back to it's original position go to panorama editor again and choose 'Numerical transform'. Now Enter the exact values for Yaw, Roll and Pitch you previously saved or noted down. The panorama should move back to it's original position now.

You can create either the complete panorama or the partial view only. Choose the size of the original panorama for output.

--Erik Krause 18:30, 20 Mar 2005 (EST