Historical:SoC2007 project Batch Processing

See SoC 2007 overview for usage hints and a page list.

Foreword

Virtual reality together with panoramic images has gained popularity over the recent years with the development of supporting technologies. This novelty has rapidly grown and today it deeply affects our lives. Specially the World Wide Web is becoming increasingly interactive with the use of interactive panoramic images, interaction with mobile phones, flash movies,... On the other hand, the use of panoramic image in journalism, adveritising and marketing is gaining popularity.

Software industry has, in order to fulfill interactivity demands, responded with a number of open-source and commercial applications. One of them is also hugin, an open-source front end for panorama tools, and an application used for creating, stitching, and processing panoramic images.

Summary

In the past and present time, batch processing has been one of many ways to increase the speed of processing. Its common feature is grouping similar tasks together and then executing them. The main advantage of batch processing is the reduction of time needed to execute and organize a large amount of similar tasks.

In terms of panoramic images, batch processing can be applied to a common process of stitching together input images to an output panoramic image. This process can be repeated as long as there are input images and project files in the preselected directory. Photographers like to take panoramas in series and the result would be a large set of images, that need to be grouped together. Firstly, the batch processor needs to determine heuristically which ones belong to the same panorama, then create the project file, apply the right template(according to exif information), optimize and output images in the selected format. What is more, the batch processor also needs to check whether existing projects are also located in that folder and also process them.

PTButcher is project that carries out this idea. It implements features for batch project building, batch stitching and overall project management of hugin projects(or any other related project files). It is a part of hugin, and also a standalone script that can be invoked from the command line.

PTButcher's aim is to provide a solid framework for professional photographers and advanced users of hugin that produce panoramas almost every day. They urge to have a fast and effective workflow and also something less time consuming than the existing hugin GUI that does not offer any batch processing.

I am proposing this project in order that Google funds its development through the Summer of Code 2007.

Description and features

Here is a brief description of all features and possible ways of implementation.

hugin project manager

In order to make hugin 'de facto' application for panoramic images, hugin needs a good project organization and management. This includes: a list of all available projects, the ability to open, close and delete projects, in addition to that, users can copy or move the projects into another location. Users can optimize or stitch all the selected projects with a simple click. When this features will be implemented hugin will become an IDE for panoramic images.

Batch processing

Here is a timeline of PTButcher's batch processing execution:

• read the files in a directory
• determine heuristically which ones belong to the same project
• set up a project file
• run CP generator
• determine based on the CP's if the files really belong together
• apply template
• optimize
• output in different formats (layerd equirect photoshop; layered rectilieaar of the zenith, for example)

The batch process is fully customizable with the use of script parameters and templates(see #Command line interface and parameters).

Ability to stitch panovideos: Given a template with the (n) of images involved and (n) folders, containing (m) png images extracted from the (n) streams to stitch together. Invoke itself the softwares, and cycles the batch to build (m) images in a new folder.

Implementation

PTButcher will be implemented in c++ and python. The scripting language is intended to be the core part, which will then invoke the proper panorama tools. The c++(with Qt library) is intended to integrate PTButcher into hugin and provide a user friendly interface to access the scripting-language library.

More features

Code of PTButcher is divided into two parts:

• project manager in hugin(c++ and Qt)
• scripting library (perl or python)

Project manager:

• ability to translate projects between the various gui.
• to search projects in subfolders
• to keep history of all operations, with undos ability.

Scripting library:

• creates projects based on the subfolder of a main folders, creates CPs and optimizes.
• Default name schemes customizable for result images and projects. As well as project folder-locations
• Template application (selection in base of image number, or exif)
• Customizable CP finder and refinery
• Customizable optimizer
• report with result and-or creation of a little test image
• capable of hugin-ptgui-whatever panorama project format.

Support for HDR or ADR workflows: always template based. i.e. Given 3 set of 4 fisheye images and the corresponding template, it generates the main project, optimizes images together, and THEN, render 3images (a,b,c) involving images 0,3,6,9 (a) 1,4,7,10) (b) (2,5,8,11) (c)

Templates

One of many features PTButcher offers are also templates. Templates provide a common way of choosing the right settings(lens system, number of images, panorama type, output format) to a group of images or projects. Next to pre-defined templates, users can also create and save custom templates, based on the settings they applied to the project, The implementation plan of templates is to provide a xml file for each template.

Example of a template:

• spherical panorama
• 8 input pictures
• Canon 5D and 15mm Fisheye Lens
• output image format(jpeg)

Schedule

Development timeline

1. Analysis:

It is of great importance to carefully carry out this section, as wrong decisions at the beginning can cause great damage later on. Therefore, in this section I would first gather all the information about the project there is available, then reconsider all the ideas, and to put a lot of effort in clearing out all indistinct information. The main question here would be: "What is to be done?". '

    start date: May 1st 
    duration: 1 week  
    end date: May 8th

2. Planning:

The main question of this section would be:"How to carry out the work?". How do we store data? What data structures are required to hold the data? What algorithms are suitable for carrying out the work? Design of classes, interfaces, libraries, templates, ... (see the PTButcher_proposal#Discussion for some answers)

    start date: May 8th 
    duration: 1 week  
    end date: May 15th

3. Coding

    start date: June 15th 
    duration: 4 weeks  
    end date: July 9-16th

4. Testing:

Testing can be applied in many ways. In my oppinion it is best to get instant feedback of users as soon as the project becomes stable enough. The idea is to find a reliable group of testers(perhaps photographers and other developers on the mailing lists) that will test the suite with their own data and then improve the code according to their suggestions.

    start date: July 16th 
    duration: 4 weeks
    end date: August 20th

Implementation timeline

Please take under consideration that this values can change. Coding timeline:

• hugin project manager - 1 week expected:

- gui creation and filling it with data

- ability to move/copy projects – search and replace in project files

- interface for calling external libraries for batch processing

• scripting library - 2 weeks expected:

-heuristically determine which pictures belong to the same project

-batch processing

-template generator

• other features - 1 week

-support for HDR or ADR workflows

-ability to stitch panovideos

Discussion

One of the most intriguing questions is how to determine which files belong to the same panorama/project. This needs to be done heuristically. Several approaches so far has been discussed:

• EXIF information

The idea is that images that form one panorama have similar EXIF information(lens, focal distance, time of capture, ...), therefore we can group them together or assign some possibility value according to EXIF information. The problem of EXIF data is, that some images do not have this data included.

• possibility matrix:

If we have N input images we form a NxN matrix with input image names on both axes. Each cell with index then i,j represents the possibility that images i and j are a part of the same panorama(the diag of the matrix is 1 of course). To fill out values of this matrix, we need to compare images based on EXIF information or generated control points.

The algorithm would then search the possibility matrix for greatest values and then create a list of most probable panoramas. After this, it would invoke the panotools Control Points, and based on the results, found out whether the results of the possibility matrix are correct. If so, the panorama/project is ready for stitching, and if not, the values in the matrix need to be readjusted.

• one of the trivial ideas is, that there is a great possibility that panoramas are taken in series. This means that images that belong to the same panorama have a incrementing sequence in their filename. Example:

How to determine which pictures form panoramas, if we have a list of input images, read from the directory: DSCF6225.jpg, DSCF6226.jpg, DSCF6227.jpg, DSCF6228.jpg, DSCF6229.jpg , DSCF6230.jpg ?

We can quickly find out, that there is much greater possibility that images DSCF6226.jpg, DSCF6227.jpg and DSCF6228.jpg form one panorama, instead of that images DSCF6225.jpg, DSCF6228.jpg and DSCF6230.jpg belong to the same panorama. This small snippet of idea can optimize the final process.

• images in a single directory can form many types of panoramas(they vary on number on images, viewing angle, file formats...), therefore applying a single template for all is useless. Templates can be therefore created automatically(by the script) if a new type of panorama is found.

Graphical user interface

All PTButcher's features will be available in graphical user interface written using Qt library. The visual layout of the interface has not been designed.

Command line interface and parameters

./ptbutcher [-d path/to/direcory] [-pf] [-t /path/to/template]

-d path to the root directory. By default current directory is used

-p this tells PTButcher to only process project files(.pto). If this is not present, PTButcher tries to create project files for all images

-f output format(defult is jpeg)

-t template to be used for stitching. If this is not present, PTButcher tries to match the images with the predefined templates, according to exif information

Additional notes

Project idea

Luca Vascon

Mentor

Yuval Levy

Students

Zoran Mesec

Development process:

PTButcher will be hosted in a public CVS repository at [1]. Development progress and notes will be kept on the public wiki of panotools[2], where users can also post comments and suggestions. Development will take place fully in the open. Discussion about this project has already been set up on the panotools-devel mailing list. See the messages in the list archives [3].

Deliverables

The final package delivered at the end of the summer will include:

• PTButcher implementation in python, with all features described below
• PTButcher GUI implementation as part of hugin
• users manual
• documentation

About the author of this proposal

My programming skills and experience in open source community makes me a suitable candidate to carry out this project. I am an undergraduate student of Computer and Information science and also a photographer. I have also worked and co-managed an open-source application for instant messaging. Additional information can be provided upon request. I have announced my draft on the panotools-devel mailing lists and received a positive feedback. Yuval Levy, a panoramic photographer and project coordinator, has agreed to mentor me on this project.

--Zoran Mesec 02:36, 31 March 2007 (CEST)