Historical:SoC 2010 ideas

If you are a student willing to participate in The Google Summer of Code 2010:

• find out what ideas we have for SoC projects this year (read below);
• decide if you want to pick one of those tasks or if you have your own idea;
• join our community at hugin-ptx;
• introduce yourself and tell us about your plans and wishes; and
• add your proposal to the student proposal page - see examples from last year

Important: we are admitted to Google Summer of Code 2010 but can not guarantee you a place in the program. We recommend you start preparing your application early as the application process is very competitive.

Development style

Most of the projects below are related to Hugin, and some also relate to Panotools or tlalli. Hugin is mostly written in C++, and uses the VIGRA image processing library to support different types of images (for example, 8bit, 16bit and float (HDR) images). The core functionality is implemented in a platform independent C++ library, which is used by the GUI based on wxWidgets toolkit, and the command line programs (nona, fulla). We also very much welcome contributions to Enblend/Enfuse.

The development of the projects should take place in a separate branch of the project's version control system. Communication with the mentors should usually happen through the appropriate mailing list. All code should work on the major platforms supported (Linux, OSX, Windows).

Application Template

SoC2010 Application Template

Possible Projects

You are welcome to propose your own ideas.

Some of the SoC2007, SoC2008 and SoC2009 project proposals were not done in the past years and are potential projects for this year:

• SoC2007_projects#Interactive panoramic viewer (This was completed but there is further possible work to be done, particularly a joint project with VLC to integrate the viewer in their media player) (VLC integration was completed in 2009 GSoC)
• SoC2007_projects#Processing of very large images (using the VIPS framework, or even GEGL)
• SoC2007_projects#Architectural Overhaul of Panotools
• SoC_2008_ideas#munin_.E2.80.94_interactive_openGL_based_GUI
• SoC_2008_ideas#enblend-enfuse_zenith.2Fnadir_and_more
• SoC_2008_ideas#Lens_Database The library part, lensfun, is done, but there is still no system for updating the database. Important: Photoropter is being actively developed and any project should be complementary to that.
• SoC_2008_ideas#Extend_hugin.27s_output_options_for_stitching
• SoC_2008_ideas#Utility_for_creating_a_Philosphere Note this isn't enough of a project and probably is better done in mathmap
• SoC_2009_idea#Python_Bindings
• SoC_2009_idea#enblend/enfuse_gimp_plugin
• SoC_2009_idea#hugin_colour_balancing see #Colour below

Zooming for fast preview

It would be good if the user can zoom into Hugin's fast preview window. The the amount of approximation in the fast preview would have to reduce to display meaningful details. The areas off screen can be ignored, to keep up performance as more details need to be processed. It would be appropriate to dynamically load more image detail for the most visible images too.

Vetting exercise

Get the fast preview to scale using openGL transformations. The scale should be changeable, either by keypresses or a gui. Don't worry about scrolling when the panorama is bigger than the window, just show the centre of the panorama.

Threading for Hugin

note: this has now been implemented in Hugin so this is not available as a SoC project

Hugin currently becomes unresponsive while it loads images. It would be better to keep the interface responsive during image loading and scaling images in another thread. Also, something like this patch, which loads images when Hugin is otherwise idle, would be better in a background thread.

The user interface could display temporary placeholders while images are being loaded, and remain interactive.

To maximise the rate at which images are loaded, ideally we would have a thread that only reads files and waits for the filesystem, and another thread to uncompress image files and produce the small version of the image which waits only for CPU time. However doing both of these in a single thread, separate from the user interface thread, would provide a responsive interface.

Vetting exercise

When Hugin starts, start another thread that pops up a window every ten seconds showing a message. Extra bonus: the message should show relevant information (e.g. the number of processor cores, a thread id).

Patent free control point generator

note: the patent free control point generator is complete and is now part of Hugin as cpfind

We now have a patent free control point generator with libpanomatic, but this needs some integration:

• Ability to read and write .pto projects
• To classify features in a conformal space based on info in the .pto file
• Internal tonemapping of HDR images to log() space for feature classification
• To not classify features in masked areas
• Test suite
• integration of celeste at feature identification stage
• matching pairs of photos using heuristics (see gigastart)

A possible different project to the above would be to use GPU for feature classification as suggested on ptx, note however that patented techniques such as SIFT and SURF are not suitable for use in Hugin: SIFT GPU http://www.cs.unc.edu/~ccwu/siftgpu/ uses CUDA parallel processing to search for SIFT features in images. I'm not sure if it does the search to find nearest neighbor points. But there is also a GPU accelerated version of that algorithm too. It is a brute force version of the nearest-k points. Since it is done in parallel it is order of magnitudes faster than the ANN algorithm using by autopano-sift-c. http://www-sop.inria.fr/members/Vincent.Garcia/research_knn.php

UI testing

Update: Summer of Code is strictly coding, so this project isn't possible.

Everyone agrees that Hugin needs usability improvements, however the major usability issues are closely related to programming issues such as the quality and availability of control point generators. We do not want a programmer to dive-in and and try and fix 'usability' without a plan of action.

So Hugin could use a usability audit, i.e. write user profiles/personas, define tasks, collect real data from test subjects. This is a non-programming project for a student of interaction design

Colour

Hugin deals correctly with colour profiles in photos and passes them on to output, this doesn't need fixing, however there are some related tasks that could be tackled:

• Display of images in tabs and preview is not colour managed, integrate lcms and access system monitor colour profiles
• Hugin has a good backend for adjusting white-balance. Add a GUI grey picker and/or tools to adjust colour temperatures manually on a subset of photos to be able to do stuff like this: http://www.flickr.com/photos/sbprzd/4196026736/ already added to recent hugin
• EXIF metadata contains information on colour balance (WBRedLevel, WBGreenLevel & WBBlueLevel) which should be used to initialise the red/blue colour balance parameters in Hugin - Currently Hugin does something very similar for EV.
• tca correction in nona with support in GUI and .pto format, possible simple GUI to run tca_correct
• Vignetting of colour balance. For example Tokina 12-24 f4 exhibits this phenomenon (see right side of this image) in amounts that are easily seen, and according to Ken Rockwell (yeah) the effect plagues most ultra wide rectilinears. Could work similarily to current vignetting correction model, but working on red/blue colour channels? (do we really need this? Is the additional complication in the GUI worth it?)

Vetting exercise

Add support for reading EXIF colour-balance metadata, see feature request #2974841 on the Hugin tracker. Note that implementing the entire request in the tracker is probably too big a task, alternatively you can do something simpler with colour balance such as add a button to the Hugin Camera and Lens tab that increases the red levels in all selected images by 10% and decreases blue by 10%.

Makefile system and Detection of panoramas

note: this was implemented in GSoC 2010 and is available in the current Hugin release

Hugin uses gnu make to drive stitching, this involves writing makefiles and executing make as a sub-process. We have two issues with this;

• The code that handles makefiles is mixed up with stitching logic, the result is that this part of the codebase is quite hairy and difficult to extend
• 'make' places restrictions on characters in file paths but the Hugin GUI doesn't do anything to prevent users from using these characters

Write a C/C++ equivalent of Panotools::Makefile, write lots of tests, identify problem characters on each platform, port Hugin stitching to use this makefile library add filters to filename selection parts of Hugin GUI to prevent use of problem characters.

Further: Hugin already has 'Align' in the Hugin Assistant tab for creating panorama projects, but for bigger projects it takes some time. Otherwise there is PTBatcherGUI in the hugin package, which can run several 'stitching' tasks in a queue - i.e. we have a system for queueing 'stitching' but not 'aligning'.

• Extend PTBatcherGUI so that also the 'Align' functionality from the assistant (with control point detection, cpclean, celeste, optimisation for position and photometric optimisation) can be added to the queue using Makefiles. Note this has been prototyped with ptoanchor, but a C++ version of this code would need to be written.
• In the next step, allow the user to give a directory or list of photos, and search for all possible panoramas and pass these to PTBatcherGUI for 'Aligning' (maybe with a heuristic approach, based on the EXIF data like panostart in combination with match-n-shift).

Vetting exercise

Modify Hugin to read and use ignore EXIF Rotation tags in photos that have a pixel width smaller than the height (i.e. images that have been rotated in an image editor). feature request #2974831 on the Hugin tracker

Regression tests for pano13

note: this was completed in GSoC 2010

libpano13 is the panotools library containing panoramic math, if it was easier to test it might make life easier for those who want to hack it. A regression testing framework that can be configured for different related command-line tools would be even better.

Libpano is responsible for doing the image transformations required to project photos to any given output panorama. It is also responsible for the optimization step of Hugin (and PToptimizer).

libpano is a relatively mature and stable library, but its development has been slowed by the lack of regression testing. Currently there are only two sets of tests (tests directory of the libpano module of panotools).

This project consists in the creation of:

1. Infrastructure to test the main functionality of libpano as used by hugin (perhaps at the function level).

2. Infrastructure to test the command line applications in libpano (PTmender, PTroller, PTtiff2psd, etc).

3. Create sets of tests.

The main priority is to implement regression testing of PToptimizer and of the projections (input and output).

This project requires scripting skills (we are open to either using Python or Perl), knowledge of C and motivation to create tests sets.

Images will be provided by the maintainers of libpano, and some might be created by the student.

Your own project

We welcome students to choose their own project, instead of using the above ideas. Discus your project ideas on the hugin-ptx mailing list. Students that design their own projects tend to do well.

As a vetting exercise, we require you make a patch. Your patch should be relevant to your project. We don't require anything major. The patch should show us some relevant coding skills, and prove you can work with the version control system. Submit your patch to hugin-ptx for peer review.