Difference between revisions of "SoC2007 projects"

From PanoTools.org Wiki
Jump to: navigation, search
(split the feature matching projects into two subprojects.)
m (Automatic feature matching for panoramic images)
Line 70: Line 70:
  
 
A desired result of the projects would be:
 
A desired result of the projects would be:
* Standalone program for the feature matching part, which at the end should accept the features found by the automatic feature detection task.
+
* Standalone program for the feature matching part, which at the end should accept the features found by the automatic feature detection task. Preliminary studies can be done using the existing SIFT and SURF detector/descriptors.
Preliminary studies can be done using the existing SIFT and SURF detector/descriptors.
+
 
* Integration into [[hugin]] and a standalone executable similar to [[Autopano-sift]] or [[Autopano]]
 
* Integration into [[hugin]] and a standalone executable similar to [[Autopano-sift]] or [[Autopano]]
  
Line 81: Line 80:
 
Possibly useful resources and libraries:
 
Possibly useful resources and libraries:
 
* [http://hunch.net/~jl/projects/cover_tree/cover_tree.html Fast nearest neighbour matching using cover trees]
 
* [http://hunch.net/~jl/projects/cover_tree/cover_tree.html Fast nearest neighbour matching using cover trees]
* [http://kogs-www.informatik.uni-hamburg.de/~koethe/vigra/  VIGRA computer vision library]
 
  
 
Mentor: Pablo d'Angelo, ?
 
Mentor: Pablo d'Angelo, ?

Revision as of 06:29, 20 March 2007

Introduction

This is the work in progress list of possible projects for the Google_SoC_2007

Panoramic imaging is a very broad field and touches many different areas of expertise, such as photography, computer vision, art and programming. There is a thriving community with experience from arts to science that provides many interesting ideas and explores new territory in panoramic imaging. In addition to the mentors, this open community will provide good support and innovative ideas.

Generally, all development should done with multiple platforms in mind (at least Windows, OSX and Linux/Unix). We have an open communication culture via mailing-lists and mostly develop using C and C++.

The project below are just suggestions. If you are an interested student and have questions or new ideas, please let us know on the relevant Discussion_lists, for example panotools-devel.

Development style

Most of the projects below are related to Hugin, and some also relate to Panotools or tlalli. Hugin is mostly written in C++, and uses the VIGRA image processing library to support different types of images (for example, 8bit, 16bit and float (HDR) images). The core functionality is implemented in a platform independent C++ library, which is used by the GUI based on wxWidgets toolkit, and the command line programs (nona, fulla).

The development of the projects should take place in a separate branch of the projects CVS (or SVN) repository. Communication with the mentors should usually happen through the appropriate development mailing list. All code should work on the major platforms supported (Linux, OSX, Windows).

Possible Projects

Intuitive yet powerful GUI for panorama creation

Goal: Redesign/Reimplement the graphical user interface of the premiere open source panoramic imaging suite, Hugin, to increase ease of use, and provide better access to its unmatched capabilities. Currently the GUI is written using wxWidgets, which has proven to have slightly different behaviours on different platforms, especially with a complex GUI such as the one of Hugin. This is very annoying and a lot of time has been spent on minor issues such as making the GUI look good on all platforms with different font sizes etc. Therefore two steps are possible:

  1. Rewrite the whole GUI using QT (which is very consistent even across platforms, and has (IMHO) a much nicer API). This is a lot more work, and not all functionality of the current GUI can be recreated during the Summer of Code, but it will provide a better platform to build onto in the future. It is also possible to implement the GUI logic in a scripting language such as Ruby or Python. This project should only be tackled by a student who has experience in developing non-trivial GUI applications with QT. The GUI and core panorama code are already separated into different libraries.
  2. Extend the existing wxWidgets based GUI.

General goals for the improved GUI include:

  • Providing a simple, yet helpful user interface that suggests or highlights potentially useful next steps.
  • Enhancing and integrating manual and automated control point placement and management.
  • Improving lens parameter management.
  • Providing a batch processing interface.
  • Expert mode with access to all features and internals.

Recommended knowledge or interest in:

  • Workflow analysis and UI design skills
  • Experience with building cross platform GUI programs (Windows/Linux/OSX), either using wxWidgets or QT
  • Creative use of panoramic imaging

Mentor: Pablo d'Angelo, ?

License: GPL

Automatic feature detection for panoramic images

Goal: Robust extraction of local image features using a Hessian-based detector and a suitable descriptor. A detector and descriptor that takes into account the approximately known distortions will have a much higher matching rate, especially when fisheye or wide angle images are used.

Tasks:

  • Implementation of the feature detector and descriptor, and a suitable test suite to verify the correctness of the implementation.

Further details are being discussed in the separate SoC2007_project_Feature_Descriptor page.

A desired result of the projects would be:

  • C or C++ library for the matching step.
  • Standalone program to extract the matches from images.

Required knowledge or interest in:

  • signal or image processing background
  • C or C++ development skills.
  • Matlab or octave

Mentor: Pablo d'Angelo, Herbert Bay, ?

Automatic feature matching for panoramic images

Goal: Robust and efficient matching of local image features.

Tasks:

  • Implementation of the matching step, including geometry based outlier pruning (for example using RANSAC) and nearest neighbour matching, possibly using a fast algorithm such as cover trees. For the panoramic imaging use case, several heuristics could be used to improve the matching behaviour, including using the EXIF timestamps, or previously known approximate orientation of the images.

A desired result of the projects would be:

  • Standalone program for the feature matching part, which at the end should accept the features found by the automatic feature detection task. Preliminary studies can be done using the existing SIFT and SURF detector/descriptors.
  • Integration into hugin and a standalone executable similar to Autopano-sift or Autopano

Required knowledge or interest in:

  • signal or image processing background
  • C or C++ development skills.
  • Matlab or octave

Possibly useful resources and libraries:

Mentor: Pablo d'Angelo, ?

License: GPL

Interactive panoramic viewer

Goal: The Freepv panoramic viewer aims to provide a superior viewing experience for panoramas on all major platforms (Windows, Mac and Linux/Unix), based on exploiting powerful graphics hardware using OpenGL. Currently it provides basic but solid viewing capabilities for Quicktime VR, cylindrical, cubic and equi-rectangular panoramas. Plugins for Mozilla/Firefox and a standalone viewer are available. Several important features are still missing from the viewer include:

  • Support for hotspots
  • Optimisation for panoramas larger than the Video RAM
  • Display of high dynamic range panoramas with adaptive exposure
  • Support for reading a SPi-V compatible .xml file, for platforms where SPi-V is not available (Linux/Unix).
  • Fallback software renderer

Required knowledge or interest in:

  • OpenGL or other 3D programming experience.
  • Creating cool and nice looking interactive experiences.

Mentor: Pablo d'Angelo, ?

License: LGPL

Anti-ghosting HDR panorama blending and merging algorithm

Goal: Most HDR creation algorithms are designed to work only with very small variations in camera viewing direction. Assume that registration and response curve estimation has already happened. An improved blending method for HDR images together that have not been shot using the traditional exposure stack method. It should avoid ghosting and be insensitive to small misregistrations.

Interesting research papers:

Required knowledge or interest in:

  • Strong background signal/image processing
  • Creative mind with ideas beyond the state of the art in computer vision/graphics research.

Mentor: Pablo d'Angelo, ?

License: GPL

Processing of very large images

Goal: Allow the creation of arbitrary large panoramas. [1]

Currently panotools, as well as hugin/nona require memory to hold the complete input and the remapped output image in memory, which consumes a lot of RAM, especially for spherical panoramas. VIPS is a powerful and modular image processing library that supports very large images and multiple processors natively. By porting the core remapping routines of panotools/hugin to VIPS, panoramas of almost arbitrary size can be computed.

A desired result of the projects would be:

  • VIPS operations that support the geometric and photometric (vignetting correction) transformations.
  • Standalone command line program that remaps images using these routines.
  • Program/script to convert panotools scripts to nip2 projects.

Required knowledge or interest in:

  • C/C++ programming
  • image processing

Mentor: John Cupitt, Pablo d'Angelo

License: GPL


Architectural Overhaul of Panotools

Panotools is a very monolithic application. The goal of this project is to refactor the functionality of panotools into 4 main parts, as independent of each other as possible. These parts are (at least):

  • Calculation of position of images (optimization)
  • Mapping from input images to output images
  • Projection related computations
  • Parsing of input scripts/Generation of input scripts

Were this project successful the functionality of panotools will be available as a collection of routines that be called directly (as opposed to the current model that requires the creation and parsing of a script). It will make it easier to replace one component of panotools with another one; and it will improve the future maintenance of panotools.

Details to be discussed on the separate page.

A desired result of the projects would be:

  • Set of functions/classes that are available to hugin to perform optimization and remapping of images
  • Test suite to verify that the functionality before and after is equivalent.

Required knowledge or interest in:

  • C/C++ programming
  • Knowledge of Numerical methods
  • Basic knowledge of image processing

Mentor: D.M. German

License: GPL