PanoTools.org Wiki - User contributions [en]

SoC 2008 ideas

2008-03-10T21:51:57Z

Stereo sl: /* Utility for creating a Philosphere */

= Introduction =

If you are a student willing to participate in The Google Summer of Code 2008, do as suggested below:

* find out what ideas we have for SoC projects this year (read below);
* make up your mind, if you want to pick one of those tasks or if you have your own idea;
* join our community at [http://groups.google.com/group/hugin-ptx/ hugin-ptx];
* introduce yourself and tell us about your plans and wishes.

= Development style =

Most of the projects below are related to [[Hugin]], and some also relate to [[Panotools]] or [[tlalli]]. [[Hugin]] is mostly written in C++, and uses the [http://kogs-www.informatik.uni-hamburg.de/~koethe/vigra/ VIGRA image processing library] to support different types of images (for example, 8bit, 16bit and float (HDR) images). The core functionality is implemented in a platform independent C++ library, which is used by the GUI based on wxWidgets toolkit, and the command line programs ([[nona]], [[fulla]]). We also very much welcome contributions to [[Enblend]]/[[Enfuse]].

The development of the projects should take place in a separate branch of the projects CVS (or SVN) repository. Communication with the mentors should usually happen through the appropriate development mailing list. All code should work on the major platforms supported (Linux, OSX, Windows).

= Possible Projects =

Some of the [[SoC2007 projects]] proposals were not done last year and are potential projects for this year.

== munin — interactive openGL based GUI ==

'''Possible Mentors''':

* Pablo?

'''Description''':

Intuitive and interactive GUI, with priority in usability over available features and flexibility, based on what users should see — not on what software does internally.

'''Skills''':

* C++
* Qt4
* OpenGL

'''Discussion''':

how about relaxing the skills, and instead of fixing on Qt4 work on a combination of high level scripting language and widgets? Phythons + wxWidgets or Lua + wxWidgets come to my mind. Right now we have the option as both wxWidgets and Qt are in trunk, and if the student codes those widgets that require speed in OpenGL so that they integrate into such an API, any scripter can then leverage them for the fast and efficient design of GUI. [[User:Yuval|Yuval]] 12:50, 4 March 2008 (CET)

== enblend-enfuse zenith/nadir and more ==

'''Primary Mentor'''

Andrew Mihal

'''Description'''

* implementing blending of the zenith and nadir in Enblend and Enfuse
*:The best way I can think of to do this is to use a cubic projection and provide all cube faces to the tool simultaneously. All of the boundaries will be horizontal or vertical lines, which will fit in well with the SKIPSM-based pyramid algorithms currently in use. Enblend and Enfuse can share the same pyramid code.
* Enblend would additionally need a nearest feature transform algorithm that works on the cube surface for mask generation. I am not aware of a published algorithm to solve this.
* Improve the seam optimization algorithm in Enblend.
* GUI hooks: an API to a higher anstraction level language such as Python+wxWidgets or Lua+wxWidgets, with a real time interface into the enfuse functionality so that when changing weights with a sliders a preview can be generated in near real time.
* multi-step fusing and alternative, user controlled, weighting (sigma).

'''Skills'''

'''Admission Test'''

A potential applicant for this project should first try to add boundary conditions to the localVarianceIf function in enfuse.h.

== OpenGL accelerated hugin preview ==

'''Primary mentor'''

'''Discussion'''

There is a nice proof of concept for this idea created as a [http://vision.eng.shu.ac.uk/mmvlwiki/index.php/Panorama_Viewer student project at Hallam University].

'''Skills'''

== Ghost removal for enfuse ==

'''Primary mentor'''

'''Discussion'''

Thanks to [[SoC2007 project Anti Ghosting]], hugin has ghost removal when merging [[Bracketing|bracketed]] series to [[HDR]], but this isn't currently available for [[enfuse]].

'''Skills'''

== Masking in GUI ==

'''Primary mentor'''

'''Discussion'''

Currently masking to remove unwanted objects from panoramas has to be added to alpha channels which is laborious, this could be integrated into the hugin GUI via a simple vector/polygon editor. It would need to be simpler to use than the [http://hugin.sourceforge.net/tutorials/enblend-svg/ equivalent process with Inkscape]

'''Skills'''

== Batch processing ==

'''Primary mentor'''

'''Discussion'''

[[SoC2007 project Batch Processing]] was an unadopted project last year.

Now that hugin has a Makefile based stitching process, it is much clearer what a batch stitcher would look like. This would need to support 'plug-in' Makefile add-ons for additional views, QTVR generation, uploading, sharpening etc... as well as a queue/spool manager.

'''Skills'''

== Lens Database ==

'''Primary mentor'''

'''Discussion'''

Lens database of accumulated knowledge on distortion, CA, response curve.
The lens database aim : using stitching to feed an huge camera / lens database for distortion correction, vignetting correction, ca, etc. Then, it will be possible to reuse this database to correct single picture too.

'''Skills'''

== tCA Correction ==

'''Primary Mentor'''

'''Description'''

An utility to determining the Transverse Chromatic Aberration (tCA) of an image so it can be corrected.

CA is when the color channels in an image do not appear to line up well with each other. tCA is when colors are in focus but placed adjacent to each other. tCA can be corrected with Panorama Tool's Radial Shift. More info on [[Chromatic aberration]]

Correcting tCA using PanoTools' Radial Shift, requires knowing the radial correction coefficients a, b and c for your lens. PTOptimizer can be used to calculate the coefficients if known amounts of shift for each channel is known from the center to the corners of the image.

This utility will produce these coefficients to be used in the lens database.

Assume image is 180 deg diagonal filed of view. Regardless if it is rectangular or fisheye of any field of view. Map image to the nadir of a sphere, taking into account any offset from center of image.

The bottom of the remapped image is the center of the original. Moving up the remapped image is the same as moving away from the center of the original image.

Now we no longer have to deal with radial shift but vertical shift. The image can be divided into a number of strips going across the entire width of the image. Each strip represents a circle a different distance from the center. The strips are shifted up and down the same amount over the entire image width. Each strip can have the channel data adjusted up and down to find the best match to eliminate CAt at that position. One channel does not move while the other two are adjusted. The best result of the two channels are recored for every strip. Sub pixel accuracy can be achieved by increasing the height of the result when creating the remapped image.

This can be done manually in real-time giving visual feedback of the channels alignment. Or it can be done programmability doing a difference between the two channels and looking at the histogram. The blackest result has the best alignment. If accurate automated results are possible then there is no need for the manual method.

Current methods of correcting [http://hugin.sourceforge.net/tutorials/tca/en.shtml tCA] are fairly complicated. Though it can be [http://thread.gmane.org/gmane.comp.misc.ptx/8183/focus=8236 completely automated] using a patched version of [[align_image_stack]] from the [[hugin]] project, this technique works very well but could use some optimisation work to make it faster (and the align_image_stack patch needs integrating).

Also according to [http://lists.freedesktop.org/archives/create/2008-February/001093.html this post] by the author of ufraw, the tCA correction in the latest [[dcraw]] is done before bayer interpolation (note that my interpretation of this code was the opposite , so what do I know - [[User:Bruno|Bruno]]). This tool should also create parameters for use by dcraw.

== Extend hugin's output options for stitching ==

'''Primary Mentor'''

Zoran Mesec

'''Description'''

Currently hugin offers jpg, png, tiff and layered tiff output. Suggested are other formats like layered psd; swf , java and html generation for web publishing. Despite of all software projects efforts, stitched panoramas have some errors (like ghost objects) that can be easily removed in Photoshop or Gimp with layered output. A lot of open source graphical software already offers proposed file formats, but despite it is not an easy task that can be implemented in a week and therefore it is suitable to be a standalone SoC project

'''Skills'''

== Utility for creating a Philosphere ==

'''Primary Mentor'''

Zoran Mesec

'''Description'''

Goal: A utility integrated into hugin for creating images from panoramas, suitable for printing.

Tasks:
* Implementation of described process using modified PT scripts and them combining partial results to final image
* Integration into hugin

Discussion:
There are several possible outputs that can be used to print a panorama. Examples: polyhedron, "orange slices", rhombicuboctahedron...

Resources:
* http://www.philohome.com/rhombicuboctahedron/rhombicuboctahedron.htm
* [[PTStitcher]]

'''Skills'''

Required knowledge or interest in:
* panoramic imaging
* C++ and scripting languages development skills.

[[Category:Community:Project]]

SoC 2008 ideas

2008-03-10T21:48:29Z

Stereo sl: /* Utility for creating a Philosphere - added details obout the project */

= Introduction =

If you are a student willing to participate in The Google Summer of Code 2008, do as suggested below:

* find out what ideas we have for SoC projects this year (read below);
* make up your mind, if you want to pick one of those tasks or if you have your own idea;
* join our community at [http://groups.google.com/group/hugin-ptx/ hugin-ptx];
* introduce yourself and tell us about your plans and wishes.

= Development style =

Most of the projects below are related to [[Hugin]], and some also relate to [[Panotools]] or [[tlalli]]. [[Hugin]] is mostly written in C++, and uses the [http://kogs-www.informatik.uni-hamburg.de/~koethe/vigra/ VIGRA image processing library] to support different types of images (for example, 8bit, 16bit and float (HDR) images). The core functionality is implemented in a platform independent C++ library, which is used by the GUI based on wxWidgets toolkit, and the command line programs ([[nona]], [[fulla]]). We also very much welcome contributions to [[Enblend]]/[[Enfuse]].

The development of the projects should take place in a separate branch of the projects CVS (or SVN) repository. Communication with the mentors should usually happen through the appropriate development mailing list. All code should work on the major platforms supported (Linux, OSX, Windows).

= Possible Projects =

Some of the [[SoC2007 projects]] proposals were not done last year and are potential projects for this year.

== munin — interactive openGL based GUI ==

'''Possible Mentors''':

* Pablo?

'''Description''':

Intuitive and interactive GUI, with priority in usability over available features and flexibility, based on what users should see — not on what software does internally.

'''Skills''':

* C++
* Qt4
* OpenGL

'''Discussion''':

how about relaxing the skills, and instead of fixing on Qt4 work on a combination of high level scripting language and widgets? Phythons + wxWidgets or Lua + wxWidgets come to my mind. Right now we have the option as both wxWidgets and Qt are in trunk, and if the student codes those widgets that require speed in OpenGL so that they integrate into such an API, any scripter can then leverage them for the fast and efficient design of GUI. [[User:Yuval|Yuval]] 12:50, 4 March 2008 (CET)

== enblend-enfuse zenith/nadir and more ==

'''Primary Mentor'''

Andrew Mihal

'''Description'''

* implementing blending of the zenith and nadir in Enblend and Enfuse
*:The best way I can think of to do this is to use a cubic projection and provide all cube faces to the tool simultaneously. All of the boundaries will be horizontal or vertical lines, which will fit in well with the SKIPSM-based pyramid algorithms currently in use. Enblend and Enfuse can share the same pyramid code.
* Enblend would additionally need a nearest feature transform algorithm that works on the cube surface for mask generation. I am not aware of a published algorithm to solve this.
* Improve the seam optimization algorithm in Enblend.
* GUI hooks: an API to a higher anstraction level language such as Python+wxWidgets or Lua+wxWidgets, with a real time interface into the enfuse functionality so that when changing weights with a sliders a preview can be generated in near real time.
* multi-step fusing and alternative, user controlled, weighting (sigma).

'''Skills'''

'''Admission Test'''

A potential applicant for this project should first try to add boundary conditions to the localVarianceIf function in enfuse.h.

== OpenGL accelerated hugin preview ==

'''Primary mentor'''

'''Discussion'''

There is a nice proof of concept for this idea created as a [http://vision.eng.shu.ac.uk/mmvlwiki/index.php/Panorama_Viewer student project at Hallam University].

'''Skills'''

== Ghost removal for enfuse ==

'''Primary mentor'''

'''Discussion'''

Thanks to [[SoC2007 project Anti Ghosting]], hugin has ghost removal when merging [[Bracketing|bracketed]] series to [[HDR]], but this isn't currently available for [[enfuse]].

'''Skills'''

== Masking in GUI ==

'''Primary mentor'''

'''Discussion'''

Currently masking to remove unwanted objects from panoramas has to be added to alpha channels which is laborious, this could be integrated into the hugin GUI via a simple vector/polygon editor. It would need to be simpler to use than the [http://hugin.sourceforge.net/tutorials/enblend-svg/ equivalent process with Inkscape]

'''Skills'''

== Batch processing ==

'''Primary mentor'''

'''Discussion'''

[[SoC2007 project Batch Processing]] was an unadopted project last year.

Now that hugin has a Makefile based stitching process, it is much clearer what a batch stitcher would look like. This would need to support 'plug-in' Makefile add-ons for additional views, QTVR generation, uploading, sharpening etc... as well as a queue/spool manager.

'''Skills'''

== Lens Database ==

'''Primary mentor'''

'''Discussion'''

Lens database of accumulated knowledge on distortion, CA, response curve.
The lens database aim : using stitching to feed an huge camera / lens database for distortion correction, vignetting correction, ca, etc. Then, it will be possible to reuse this database to correct single picture too.

'''Skills'''

== tCA Correction ==

'''Primary Mentor'''

'''Description'''

An utility to determining the Transverse Chromatic Aberration (tCA) of an image so it can be corrected.

CA is when the color channels in an image do not appear to line up well with each other. tCA is when colors are in focus but placed adjacent to each other. tCA can be corrected with Panorama Tool's Radial Shift. More info on [[Chromatic aberration]]

Correcting tCA using PanoTools' Radial Shift, requires knowing the radial correction coefficients a, b and c for your lens. PTOptimizer can be used to calculate the coefficients if known amounts of shift for each channel is known from the center to the corners of the image.

This utility will produce these coefficients to be used in the lens database.

Assume image is 180 deg diagonal filed of view. Regardless if it is rectangular or fisheye of any field of view. Map image to the nadir of a sphere, taking into account any offset from center of image.

The bottom of the remapped image is the center of the original. Moving up the remapped image is the same as moving away from the center of the original image.

Now we no longer have to deal with radial shift but vertical shift. The image can be divided into a number of strips going across the entire width of the image. Each strip represents a circle a different distance from the center. The strips are shifted up and down the same amount over the entire image width. Each strip can have the channel data adjusted up and down to find the best match to eliminate CAt at that position. One channel does not move while the other two are adjusted. The best result of the two channels are recored for every strip. Sub pixel accuracy can be achieved by increasing the height of the result when creating the remapped image.

This can be done manually in real-time giving visual feedback of the channels alignment. Or it can be done programmability doing a difference between the two channels and looking at the histogram. The blackest result has the best alignment. If accurate automated results are possible then there is no need for the manual method.

Current methods of correcting [http://hugin.sourceforge.net/tutorials/tca/en.shtml tCA] are fairly complicated. Though it can be [http://thread.gmane.org/gmane.comp.misc.ptx/8183/focus=8236 completely automated] using a patched version of [[align_image_stack]] from the [[hugin]] project, this technique works very well but could use some optimisation work to make it faster (and the align_image_stack patch needs integrating).

Also according to [http://lists.freedesktop.org/archives/create/2008-February/001093.html this post] by the author of ufraw, the tCA correction in the latest [[dcraw]] is done before bayer interpolation (note that my interpretation of this code was the opposite , so what do I know - [[User:Bruno|Bruno]]). This tool should also create parameters for use by dcraw.

== Extend hugin's output options for stitching ==

'''Primary Mentor'''

Zoran Mesec

'''Description'''

Currently hugin offers jpg, png, tiff and layered tiff output. Suggested are other formats like layered psd; swf , java and html generation for web publishing. Despite of all software projects efforts, stitched panoramas have some errors (like ghost objects) that can be easily removed in Photoshop or Gimp with layered output. A lot of open source graphical software already offers proposed file formats, but despite it is not an easy task that can be implemented in a week and therefore it is suitable to be a standalone SoC project

'''Skills'''

== Utility for creating a Philosphere ==

'''Primary Mentor'''

Zoran Mesec

'''Description'''

Goal: A utility integrated into hugin for creating images from panoramas, suitable for printing.

Tasks:
* Implementation of described process using modified PT scripts and them combining partial results to final image
* Integration into hugin

Discussion:
There are several possible outputs that can be used to print a panorama. Examples: polyhedron, "orange slices", rhombicuboctahedron...

Resources:
* http://www.philohome.com/rhombicuboctahedron/rhombicuboctahedron.htm

'''Skills'''

Required knowledge or interest in:
* panoramic imaging
* C++ and scripting languages development skills.

[[Category:Community:Project]]

SoC 2008 application

2008-03-10T21:04:39Z

Stereo sl: Added a short bio for Zoran

=== Describe your organization ===
=== Why is your organization applying to participate in GSoC 2008? What do you hope to gain by participating? ===

http://www.flickr.com/search/groups/?q=hugin

http://www.flickr.com/photos/sbprzd/2126554118/

http://www.flickr.com/photos/sbprzd/2125768589/

=== Did your organization participate in past GSoCs? If so, please summarize your involvement and the successes and challenges of your participation. ===

hugin/panotools participated in GSoC 2007. We consider this participation successful for both organization and students. Our projects were:

* New extensible modular GUI framework for Panorama Photography. Ippei Ukai has refactored the hugin code, cleaning the interface and adding Qt support while keeping wxWidgets compatibility. The upcoming hugin 0.7.4 release will still feature wxWidgets directly bound from the C++ code, mainly because the development of specific Qt widgets is very time consuming. Future plans are to provide hooks for rapid GUI development with higher level languages such as Python (wxPython or PyQt).
* Anti-ghosting HDR panorama blending and merging algorithm. Jing Jin successfully finished the project and the code is now merged into hugin source code tree which we expect to release as 0.7.4 soon.
* Interactive Panoramic Viewer. This is a work on FreePV panoramic viewer. Since GSoC the work is still happening in the [http://freepv.svn.sourceforge.net/viewvc/freepv/freepv/branches/branch_leonox/ SoC branch of SVN]. There is no release featuring changes done by the student yet. It is quite possible to see FreePV coce being integrated into VLC which is a popular crossplatform media viewer.
* Feature matching for panoramic images. Zoran Mesec successfully created Matchpoint - a new automatic control points generator that isn't affected by patents. Zoran also volunteered to be mentor for further Matchpoint development this year in GSoC2008 which shows his constant involvement into the project.
* VIPS integration. This project was supposed to bring very large images support to hugin, but failed.

=== Who will your organization administrator be? Please include Google Account information. ===

Alexandre Prokoudine will be primary administrator. He currently resides in Moscow, Russia. He is involved into open source projects since early 2002 as technical writer, GUI translator (hugin, Inkscape, Scribus, Audacity, Rosegarden etc.) and functional specifications author.

Being interested in design and photography since 2005 he quickly started enjoying his role of communicator between developers of various open source projects. In 2006 Jon Philips, a Creative Commons advocate, and he co-founded a CREATE project where developers of creative applications (mostly graphics related ones as of now) can meet and work out standards, unified approaches to solving real life user issues etc.

In 2007 Alexandre served as backup administrator of hugin/panotools and Scribus organizations for Google Summer of Code.

=== What license(s) does your project use? ===

Both hugin, enblend/enfuse, panotools and matchpoint use GPL v2 or above.

=== What is the URL for your ideas page? ===

http://wiki.panotools.org/SoC_2008_ideas

=== What is the main development mailing list or forum for your organization? ===

http://lists.sourceforge.net/lists/listinfo/panotools-devel

=== What is the main IRC channel for your organization? ===

We do not use IRC for communication. During GSoC 2007 we used Skype, but this is unlikely to happen again.

=== Does your organization have an application template you would like to see students use? If so, please provide it now. ===

=== Who will be your backup organization administrator? Please include Google Account information. ===
Yuval Levy is chosen to be backup administrator. He did a perfect job as primary administrator during GSoC 2007.

=== Who will your mentors be? Please include Google Account information. ===

'''Pablo d'Angelo'''

Pablo d'Angelo is the initiator and main developer of the hugin project. He has studied computer engineering at the University of applied sciences Ulm, and is currently working at the DaimlerChrysler Research Center in Ulm, where he does research on advanced 3D reconstruction techniques for industrial quality inspection. He also has a PhD in the field of computer vision from the University of Bielefeld, which was granted in Summer 2007.

''' Andrew Mihal '''

Andrew Mihal is the project lead and main developer of the Enblend/Enfuse project. He received his Ph.D. in Electrical Engineering and Computer Science from the University of California, Berkeley. His specialization is in software for electronic design automation. He is also an amateur photographer, and enjoys turning academic research papers into useful open source projects as a hobby.

''' Jim Watters '''

Jim Watters is a Software Engineer at JFL Peripheral Solution, in Ottawa, Ontario, Canada, where he designs software for scanners. An avid user of PanoTools since 2000. A growing contributer to the source code of PanoTools since Aug 2003 and a current maintainer of PanoTools (http://panotools.sourceforge.net). Before receiving his degree in Computer Software in 1999, he received a diploma of Fine Art in Photography in 1990.

Most recently his attention has been directed to creating Immersive Panoramic Video.

''' Daniel M German '''

He is assistant professor at the Department of Computer Science at the University of Victoria, in British Columbia, in Canada. He received his PhD in Computer Science from the University of Waterloo in Canada.

One of his areas of research is open source software development. He is interested in understanding how globally distributed individuals are able to work together to create commercial strength software. He has also explored the use of historical artefacts (such as version control logs, emails, defect tracking) to understand how a system has evolved and how this information can be used to continue its development.

More recently he has been interested in the field of computational photography. More specifically on how to project extreme wide-angle images and spherical panoramas into acceptable flat representations.

During the last year he has been the maintainer of Panotools (http://panotools.sourceforge.net).

As a university professor one of his jobs is the supervision of students. He has graduated 6 Master's students, and currently supervising 3 Master's, and 1 Ph.D. Student. He has supervised more than a dozen honours projects of undergraduate students in computer science.

He is an avid photographer. His works have been exhibited in several galleries.

''' Alexandre Jenny '''

Alexandre Jenny is graduate from Ecole des mines de Nancy, in physics and computer sciences. He spent 5 years in the computer game industry before starting to interest in panoramic.

He's casual photographer and he started interest in panorama when he moved near Alps mountains. After having climbed during 5 hours and reached the top, it's really frustrating not beeing able to capture the whole panorama. So he started studying this field at this time. It was during the very early stage of Panotools. The main problem was not being able to create automatically controls points, so he wrote the first SIFT control point generator which is always used a lot today as a Panotools plugin : autopano v1.03.

After this first release and because he felt that some business could be build upon panoramic, he founded Kolor, a well know business which is the creator of [http://www.autopano.net Autopano Pro], a fully automatic stitcher. Autopano Pro uses an industrial strong implementation of the SIFT algorithm, but has also many other features in a single easy to use package.

'''Zoran Mesec'''

He is a student in the final year of undergraduate study of Computer and Information Science at University of Ljubljana. He has been a part of hugin's participation at GSoC 07 as a student where he successfully completed the project [[SoC2007_projects#Automatic_feature_detection_for_panoramic_images]] under the mentorship of dr. Bay. He continued to develop his work during the year and created MatchPoint, automatic control point suite. This year he is taking GSoC at a higher level, based on his past experience and knowledge in computer science and photography.

=== What criteria did you use to select these individuals as mentors? Please be as specific as possible. ===
=== Steering Committee ===

This year we also have a steering committee - a group of professionals in panorama making that will help us support our students.

'''G. Donald Bain'''

G. Donald Bain manages the Geography Computing Facility at the University of California Berkeley, where he also teaches cartography and field studies. The rest of the time he devotes to VR.

Hi is a board member of IVRPA.

He travels and take panoramas for his web site: [http://virtualguidebooks.com/ Don Bain's Virtual Guidebooks]. This project (his wife refers to it as his obsession) has taken him from Tahiti to the Arctic, from the prehistoric ruins of the Southwest to the glaciers of the Canadian Rockies. As a compulsive educator it has been a real treat, documenting and sharing his landscapes with the world. He has taken over 4000 panoramas, with about 3500 currently on the site.

He co-founded the World Wide Panorama, the largest international collaborative effort showcasing over 3000 panoramas from VR-artists all over the world taken over 11 editions.

'''Yuval Levy'''

Last year's GSoC admin for this team. [http://www.photopla.net/] [http://panospace.wordpress.com/]

'''Bruno Postle'''

Bruno Postle trained as an architect and now works designing and engineering lightweight/portable/temporary buildings, monumental sculptures and anything else that comes along. Bruno has been a long-time contributor to the [[hugin]] project, and maintains a number of CPAN modules including [http://search.cpan.org/dist/Panotools-Script/ Panotools::Script], a module for manipulating hugin project files. He also [http://www.flickr.com/photos/36383814@N00/ takes panoramic photographs].

'''Thomas Rauscher'''

'''Ken Turkowski'''

=== What is your plan for dealing with disappearing students? ===
=== What is your plan for dealing with disappearing mentors? ===
=== What steps will you take to encourage students to interact with your project's community before, during and after the program? ===
=== What will you do to ensure that your accepted students stick with the project after GSoC concludes? ===

[[Category:Community:Project]]

MatchPoint

2008-02-27T17:52:09Z

Stereo sl: /* Feature detection */

== Introduction ==
MatchPoint is a next generation CP generator. The result of a GSoC2007 project [[SoC2007_project_Feature_Descriptor]]. Currently it offers only keypoint detection but not matching(matching was part of other GSoC 07 project that was not carried out) and can currently be used only as a replacement for generatekeys from autopano.

Goal of this MatchPoint is to create a complete control point suite that will be used by hugin as a replacement(or at least an alternative) to existing autopano suites.

If you want to use MatchPoint in the process of creating panoramic images now(when not all features are implemented) you will have to use autopano-sift also.

Usage: 
// first extract features from the first image and output them to the image1.key: 
MatchPoint image1.jpg image1.key

// for second image: 
MatchPoint image2.jpg image2.key

// match features from the two generated files using autopano: 
autopano project.pto image1.key image2.key

// open the project file in hugin: 
hugin project.pto

== Command line usage ==
Usage: 
;MatchPoint [options] image1.jpg output.key: 
Parameters:
;-v: verbose output
;-t: generate keypoint file for matlab test suite(file name is generated using formula: image1.jpg.key)
Arguments:
;image1.jpg: Path to image to be analyzed.
;output.key: Output keypoint file.

== Algorithm description ==

=== Integral images ===

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image. This makes calculating sum of a region much faster. In addition, convolution at any scale has equal time complexity.
This part is neccessary only when using box filters for detection.

=== Feature detection ===

Points are detected with Hessian Detector using box filters. Here is a description of detection process over time. This may not be entirely compatible flow with code's flow, because some parts are simultaneous(e.q. first two steps).

''' Convolution of pixels ''' 
Convolution of a pixel at certain scale is carried out with approximation of Gauss 2D function - this is called box filter detection. Each kernel has 4 convolution filters(3 of them are unique - xy and yx filters are the same). The resulting value is then the determinant of Hessian matrix where elements represent convolution values of 4 filters.

Kernel sizes for convolution are: 
9,15,21,27, (1st octave) 
15,27,39,51, (2nd octave) 
21,45,69,93 (3rd octave) 

MatchPoint features also offers two ways of convolution:
* box filter: faster and preferable way
* sliding window(implemented for convinience but needs debug): slower, more accurate but also more sensitive to noise

''' Finding maxima ''' 
In order to achieve invariance to scaling, detection needs to find maxima of Hessian determinant across many scales. To handle this, octaves were introduced. Octaves are interpolation of determinants at various scales(usually two scales). MatchPoint offers detection at max 3 octaves(by setting a parameter). E.q. at first octave a point can be detected at scale 1.6(=((9+15/2)*1.2)/9 where 1.2 is initial scale and 9 is initial kernel size) or 3.2(=((21+27/2)*1.2)/9. Keypoint's main scale is then selected according to the highest value of Hessian determinant.

''' Selecting points ''' 

Next step is to select points with high values of their determinants(regardless of scale where points where detected) that represent invariant keypoints that will be used further processing. This is achieved using a fixed threshold which should be set low(because otherwise it may happen that no points would be detected).

Then non-maxima supression is applied(only points with local maxima of determinants are selected).

At this point we have a list of points that can vary in length, because threshold from previous step is hardcoded. This can result in over 200.000 points for larger images and that is too much(regardless of image size we need the same amount of control points for all images - min 3 control point/overlapping images), so we need to strip down the list below 10.000 points(this number is set intuitivelly and it is based on consumed time). (Note: this work is progress).

=== Feature description ===

''' Orientation assignment ''' 
Scale invariance is achieved by assigning main orientation of the interest point using special algorithm proposed by Herbert Bay. Efficiency of this algorithm has not been yet tested, therefore the executable of MatchPoint does not use any orientation assignment.

''' Shape Context descriptors ''' 
To each interest point a 36 element descriptor is assigned. Elements of this descriptors are organized according to Shape Context descriptor proposed by [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html S. Belongie, J. Malik and J. Puzicha] and adapted by [http://www.robots.ox.ac.uk/~vgg/research/affine/index.html K. Mikolajczyk] for the purpose of finding control points.

Steps in creating descriptors:
# detect edges in the region around the interest point(region size depends on the scale at which point was detected). MatchPoint uses vigra's API(Canny edge detection) for this.
# based on relative location of edge elements create a histogram with 36 values. Use log-polar values. Edge point contribution to the histogram is based on it's magnitude and orientation.
See reference papers for details.

==References==
* [http://www.vision.ee.ethz.ch/~surf/ Speeded Up Robust Features - SURF]
* [http://www.robots.ox.ac.uk/~vgg/research/affine Matlab suite for testing, papers for detection, descriptors and evaluation by K. Mikolajczyk]
* [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html Shape Context descriptors]
[[Category:Community:Project]]

MatchPoint

2008-02-27T17:50:18Z

Stereo sl: /* Feature description */

== Introduction ==
MatchPoint is a next generation CP generator. The result of a GSoC2007 project [[SoC2007_project_Feature_Descriptor]]. Currently it offers only keypoint detection but not matching(matching was part of other GSoC 07 project that was not carried out) and can currently be used only as a replacement for generatekeys from autopano.

Goal of this MatchPoint is to create a complete control point suite that will be used by hugin as a replacement(or at least an alternative) to existing autopano suites.

If you want to use MatchPoint in the process of creating panoramic images now(when not all features are implemented) you will have to use autopano-sift also.

Usage: 
// first extract features from the first image and output them to the image1.key: 
MatchPoint image1.jpg image1.key

// for second image: 
MatchPoint image2.jpg image2.key

// match features from the two generated files using autopano: 
autopano project.pto image1.key image2.key

// open the project file in hugin: 
hugin project.pto

== Command line usage ==
Usage: 
;MatchPoint [options] image1.jpg output.key: 
Parameters:
;-v: verbose output
;-t: generate keypoint file for matlab test suite(file name is generated using formula: image1.jpg.key)
Arguments:
;image1.jpg: Path to image to be analyzed.
;output.key: Output keypoint file.

== Algorithm description ==

=== Integral images ===

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image. This makes calculating sum of a region much faster. In addition, convolution at any scale has equal time complexity.
This part is neccessary only when using box filters for detection.

=== Feature detection ===

Points are detected with Hessian Detector using box filters. Here is a description of detection process over time. This may not be entirely compatible flow with the algorithm's flow, because some parts are simultaneous(e.q. first two steps).

''' Convolution of pixels ''' 
Convolution of a pixel at certain scale is carried out with approximation of Gauss 2D function - this is called box filter detection. Each kernel has 4 convolution filters(3 of them are unique - xy and yx filters are the same). The resulting value is then the determinant of Hessian matrix where elements represent convolution values of 4 filters.

Kernel sizes for convolution are: 
9,15,21,27, (1st octave) 
15,27,39,51, (2nd octave) 
21,45,69,93 (3rd octave) 

MatchPoint features also offers two ways of convolution:
* box filter: faster and preferable way
* sliding window(implemented for convinience but needs debug): slower, more accurate but also more sensitive to noise

''' Finding maxima ''' 
In order to achieve invariance to scaling, detection needs to find maxima of Hessian determinant across many scales. To handle this, octaves were introduced. Octaves are interpolation of determinants at various scales(usually two scales). MatchPoint offers detection at max 3 octaves(by setting a parameter). E.q. at first octave a point can be detected at scale 1.6(=((9+15/2)*1.2)/9 where 1.2 is initial scale and 9 is initial kernel size) or 3.2(=((21+27/2)*1.2)/9. Keypoint's main scale is then selected according to the highest value of Hessian determinant.

''' Selecting points ''' 

Next step is to select points with high values of their determinants(regardless of scale where points where detected) that represent invariant keypoints that will be used further processing. This is achieved using a fixed threshold which should be set low(because otherwise it may happen that no points would be detected).

Then non-maxima supression is applied(only points with local maxima of determinants are selected).

At this point we have a list of points that can vary in length, because threshold from previous step is hardcoded. This can result in over 200.000 points for larger images and that is too much(regardless of image size we need the same amount of control points for all images - min 3 control point/overlapping images), so we need to strip down the list below 10.000 points(this number is set intuitivelly and it is based on consumed time). (Note: this work is progress).

=== Feature description ===

''' Orientation assignment ''' 
Scale invariance is achieved by assigning main orientation of the interest point using special algorithm proposed by Herbert Bay. Efficiency of this algorithm has not been yet tested, therefore the executable of MatchPoint does not use any orientation assignment.

''' Shape Context descriptors ''' 
To each interest point a 36 element descriptor is assigned. Elements of this descriptors are organized according to Shape Context descriptor proposed by [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html S. Belongie, J. Malik and J. Puzicha] and adapted by [http://www.robots.ox.ac.uk/~vgg/research/affine/index.html K. Mikolajczyk] for the purpose of finding control points.

Steps in creating descriptors:
# detect edges in the region around the interest point(region size depends on the scale at which point was detected). MatchPoint uses vigra's API(Canny edge detection) for this.
# based on relative location of edge elements create a histogram with 36 values. Use log-polar values. Edge point contribution to the histogram is based on it's magnitude and orientation.
See reference papers for details.

==References==
* [http://www.vision.ee.ethz.ch/~surf/ Speeded Up Robust Features - SURF]
* [http://www.robots.ox.ac.uk/~vgg/research/affine Matlab suite for testing, papers for detection, descriptors and evaluation by K. Mikolajczyk]
* [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html Shape Context descriptors]
[[Category:Community:Project]]

MatchPoint

2008-02-27T17:49:25Z

Stereo sl: /* Feature description */

== Introduction ==
MatchPoint is a next generation CP generator. The result of a GSoC2007 project [[SoC2007_project_Feature_Descriptor]]. Currently it offers only keypoint detection but not matching(matching was part of other GSoC 07 project that was not carried out) and can currently be used only as a replacement for generatekeys from autopano.

Goal of this MatchPoint is to create a complete control point suite that will be used by hugin as a replacement(or at least an alternative) to existing autopano suites.

If you want to use MatchPoint in the process of creating panoramic images now(when not all features are implemented) you will have to use autopano-sift also.

Usage: 
// first extract features from the first image and output them to the image1.key: 
MatchPoint image1.jpg image1.key

// for second image: 
MatchPoint image2.jpg image2.key

// match features from the two generated files using autopano: 
autopano project.pto image1.key image2.key

// open the project file in hugin: 
hugin project.pto

== Command line usage ==
Usage: 
;MatchPoint [options] image1.jpg output.key: 
Parameters:
;-v: verbose output
;-t: generate keypoint file for matlab test suite(file name is generated using formula: image1.jpg.key)
Arguments:
;image1.jpg: Path to image to be analyzed.
;output.key: Output keypoint file.

== Algorithm description ==

=== Integral images ===

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image. This makes calculating sum of a region much faster. In addition, convolution at any scale has equal time complexity.
This part is neccessary only when using box filters for detection.

=== Feature detection ===

Points are detected with Hessian Detector using box filters. Here is a description of detection process over time. This may not be entirely compatible flow with the algorithm's flow, because some parts are simultaneous(e.q. first two steps).

''' Convolution of pixels ''' 
Convolution of a pixel at certain scale is carried out with approximation of Gauss 2D function - this is called box filter detection. Each kernel has 4 convolution filters(3 of them are unique - xy and yx filters are the same). The resulting value is then the determinant of Hessian matrix where elements represent convolution values of 4 filters.

Kernel sizes for convolution are: 
9,15,21,27, (1st octave) 
15,27,39,51, (2nd octave) 
21,45,69,93 (3rd octave) 

MatchPoint features also offers two ways of convolution:
* box filter: faster and preferable way
* sliding window(implemented for convinience but needs debug): slower, more accurate but also more sensitive to noise

''' Finding maxima ''' 
In order to achieve invariance to scaling, detection needs to find maxima of Hessian determinant across many scales. To handle this, octaves were introduced. Octaves are interpolation of determinants at various scales(usually two scales). MatchPoint offers detection at max 3 octaves(by setting a parameter). E.q. at first octave a point can be detected at scale 1.6(=((9+15/2)*1.2)/9 where 1.2 is initial scale and 9 is initial kernel size) or 3.2(=((21+27/2)*1.2)/9. Keypoint's main scale is then selected according to the highest value of Hessian determinant.

''' Selecting points ''' 

Next step is to select points with high values of their determinants(regardless of scale where points where detected) that represent invariant keypoints that will be used further processing. This is achieved using a fixed threshold which should be set low(because otherwise it may happen that no points would be detected).

Then non-maxima supression is applied(only points with local maxima of determinants are selected).

At this point we have a list of points that can vary in length, because threshold from previous step is hardcoded. This can result in over 200.000 points for larger images and that is too much(regardless of image size we need the same amount of control points for all images - min 3 control point/overlapping images), so we need to strip down the list below 10.000 points(this number is set intuitivelly and it is based on consumed time). (Note: this work is progress).

=== Feature description ===

''' Orientation assignment ''' 
Scale invariance is achieved by assigning main orientation of the interest point using special algorithm proposed by Herbert Bay. Efficiency of this algorithm has not been yet tested, therefore the executable of MatchPoint does not use any orientation assignment.

''' Shape Context descriptors ''' 
To each interest point a 36 element descriptor is assigned. Elements of this descriptors are organized according to Shape Context descriptor proposed by [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html S. Belongie, J. Malik and J. Puzicha] and adapted by [http://www.robots.ox.ac.uk/~vgg/research/affine/index.html K. Mikolajczyk] for the purpose of finding control points.

Steps in creating descriptors:
# detect edges in the region around the interest point. MatchPoint uses vigra's API(Canny edge detection) for this.
# based on relative location of edge elements create a histogram with 36 values. Use log-polar values. Edge point contribution to the histogram is based on it's magnitude and orientation.
See reference papers for details.

==References==
* [http://www.vision.ee.ethz.ch/~surf/ Speeded Up Robust Features - SURF]
* [http://www.robots.ox.ac.uk/~vgg/research/affine Matlab suite for testing, papers for detection, descriptors and evaluation by K. Mikolajczyk]
* [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html Shape Context descriptors]
[[Category:Community:Project]]

MatchPoint

2008-02-27T17:47:49Z

Stereo sl: /* Introduction */

== Introduction ==
MatchPoint is a next generation CP generator. The result of a GSoC2007 project [[SoC2007_project_Feature_Descriptor]]. Currently it offers only keypoint detection but not matching(matching was part of other GSoC 07 project that was not carried out) and can currently be used only as a replacement for generatekeys from autopano.

Goal of this MatchPoint is to create a complete control point suite that will be used by hugin as a replacement(or at least an alternative) to existing autopano suites.

If you want to use MatchPoint in the process of creating panoramic images now(when not all features are implemented) you will have to use autopano-sift also.

Usage: 
// first extract features from the first image and output them to the image1.key: 
MatchPoint image1.jpg image1.key

// for second image: 
MatchPoint image2.jpg image2.key

// match features from the two generated files using autopano: 
autopano project.pto image1.key image2.key

// open the project file in hugin: 
hugin project.pto

== Command line usage ==
Usage: 
;MatchPoint [options] image1.jpg output.key: 
Parameters:
;-v: verbose output
;-t: generate keypoint file for matlab test suite(file name is generated using formula: image1.jpg.key)
Arguments:
;image1.jpg: Path to image to be analyzed.
;output.key: Output keypoint file.

== Algorithm description ==

=== Integral images ===

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image. This makes calculating sum of a region much faster. In addition, convolution at any scale has equal time complexity.
This part is neccessary only when using box filters for detection.

=== Feature detection ===

Points are detected with Hessian Detector using box filters. Here is a description of detection process over time. This may not be entirely compatible flow with the algorithm's flow, because some parts are simultaneous(e.q. first two steps).

''' Convolution of pixels ''' 
Convolution of a pixel at certain scale is carried out with approximation of Gauss 2D function - this is called box filter detection. Each kernel has 4 convolution filters(3 of them are unique - xy and yx filters are the same). The resulting value is then the determinant of Hessian matrix where elements represent convolution values of 4 filters.

Kernel sizes for convolution are: 
9,15,21,27, (1st octave) 
15,27,39,51, (2nd octave) 
21,45,69,93 (3rd octave) 

MatchPoint features also offers two ways of convolution:
* box filter: faster and preferable way
* sliding window(implemented for convinience but needs debug): slower, more accurate but also more sensitive to noise

''' Finding maxima ''' 
In order to achieve invariance to scaling, detection needs to find maxima of Hessian determinant across many scales. To handle this, octaves were introduced. Octaves are interpolation of determinants at various scales(usually two scales). MatchPoint offers detection at max 3 octaves(by setting a parameter). E.q. at first octave a point can be detected at scale 1.6(=((9+15/2)*1.2)/9 where 1.2 is initial scale and 9 is initial kernel size) or 3.2(=((21+27/2)*1.2)/9. Keypoint's main scale is then selected according to the highest value of Hessian determinant.

''' Selecting points ''' 

Next step is to select points with high values of their determinants(regardless of scale where points where detected) that represent invariant keypoints that will be used further processing. This is achieved using a fixed threshold which should be set low(because otherwise it may happen that no points would be detected).

Then non-maxima supression is applied(only points with local maxima of determinants are selected).

At this point we have a list of points that can vary in length, because threshold from previous step is hardcoded. This can result in over 200.000 points for larger images and that is too much(regardless of image size we need the same amount of control points for all images - min 3 control point/overlapping images), so we need to strip down the list below 10.000 points(this number is set intuitivelly and it is based on consumed time). (Note: this work is progress).

=== Feature description ===

''' Orientation assignment ''' 
Scale invariance is achieved by assigning main orientation of the interest point using special algorithm proposed by Herbert Bay. Efficiency of this algorithm has not been yet tested, therefore the executable of MatchPoint does not use any orientation assignment.

''' Shape Context descriptors ''' 
To each interest point a 36 element descriptor is assigned. Elements of this descriptors are organized according to Shape Context descriptor proposed by [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html S. Belongie, J. Malik and J. Puzicha] and adapted by [http://www.robots.ox.ac.uk/~vgg/research/affine/index.html K. Mikolajczyk] for the purpose of finding control points.
Steps in creating point's descriptor:
# detect edges in the surrounding region. MatchPoint uses vigra's API(Canny edge detection) for this.
# based on relative location of edge elements create a histogram with 36 values. Use log-polar values. Edge point contribution to the histogram is based on it's magnitude and orientation.
See reference papers for details.

==References==
* [http://www.vision.ee.ethz.ch/~surf/ Speeded Up Robust Features - SURF]
* [http://www.robots.ox.ac.uk/~vgg/research/affine Matlab suite for testing, papers for detection, descriptors and evaluation by K. Mikolajczyk]
* [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html Shape Context descriptors]
[[Category:Community:Project]]

MatchPoint

2008-02-27T17:35:41Z

Stereo sl:

== Introduction ==
MatchPoint is a next generation CP generator. The result of a GSoC2007 project [[SoC2007_project_Feature_Descriptor]], it is still very experimental. Read more [http://groups.google.com/group/hugin-ptx/browse_thread/thread/cba2b2ce94dd9054 here]

== Command line usage ==
Usage: 
;MatchPoint [options] image1.jpg output.key: 
Parameters:
;-v: verbose output
;-t: generate keypoint file for matlab test suite(file name is generated using formula: image1.jpg.key)
Arguments:
;image1.jpg: Path to image to be analyzed.
;output.key: Output keypoint file.

== Algorithm description ==

=== Integral images ===

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image. This makes calculating sum of a region much faster. In addition, convolution at any scale has equal time complexity.
This part is neccessary only when using box filters for detection.

=== Feature detection ===

Points are detected with Hessian Detector using box filters. Here is a description of detection process over time. This may not be entirely compatible flow with the algorithm's flow, because some parts are simultaneous(e.q. first two steps).

''' Convolution of pixels ''' 
Convolution of a pixel at certain scale is carried out with approximation of Gauss 2D function - this is called box filter detection. Each kernel has 4 convolution filters(3 of them are unique - xy and yx filters are the same). The resulting value is then the determinant of Hessian matrix where elements represent convolution values of 4 filters.

Kernel sizes for convolution are: 
9,15,21,27, (1st octave) 
15,27,39,51, (2nd octave) 
21,45,69,93 (3rd octave) 

MatchPoint features also offers two ways of convolution:
* box filter: faster and preferable way
* sliding window(implemented for convinience but needs debug): slower, more accurate but also more sensitive to noise

''' Finding maxima ''' 
In order to achieve invariance to scaling, detection needs to find maxima of Hessian determinant across many scales. To handle this, octaves were introduced. Octaves are interpolation of determinants at various scales(usually two scales). MatchPoint offers detection at max 3 octaves(by setting a parameter). E.q. at first octave a point can be detected at scale 1.6(=((9+15/2)*1.2)/9 where 1.2 is initial scale and 9 is initial kernel size) or 3.2(=((21+27/2)*1.2)/9. Keypoint's main scale is then selected according to the highest value of Hessian determinant.

''' Selecting points ''' 

Next step is to select points with high values of their determinants(regardless of scale where points where detected) that represent invariant keypoints that will be used further processing. This is achieved using a fixed threshold which should be set low(because otherwise it may happen that no points would be detected).

Then non-maxima supression is applied(only points with local maxima of determinants are selected).

At this point we have a list of points that can vary in length, because threshold from previous step is hardcoded. This can result in over 200.000 points for larger images and that is too much(regardless of image size we need the same amount of control points for all images - min 3 control point/overlapping images), so we need to strip down the list below 10.000 points(this number is set intuitivelly and it is based on consumed time). (Note: this work is progress).

=== Feature description ===

''' Orientation assignment ''' 
Scale invariance is achieved by assigning main orientation of the interest point using special algorithm proposed by Herbert Bay. Efficiency of this algorithm has not been yet tested, therefore the executable of MatchPoint does not use any orientation assignment.

''' Shape Context descriptors ''' 
To each interest point a 36 element descriptor is assigned. Elements of this descriptors are organized according to Shape Context descriptor proposed by [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html S. Belongie, J. Malik and J. Puzicha] and adapted by [http://www.robots.ox.ac.uk/~vgg/research/affine/index.html K. Mikolajczyk] for the purpose of finding control points.
Steps in creating point's descriptor:
# detect edges in the surrounding region. MatchPoint uses vigra's API(Canny edge detection) for this.
# based on relative location of edge elements create a histogram with 36 values. Use log-polar values. Edge point contribution to the histogram is based on it's magnitude and orientation.
See reference papers for details.

==References==
* [http://www.vision.ee.ethz.ch/~surf/ Speeded Up Robust Features - SURF]
* [http://www.robots.ox.ac.uk/~vgg/research/affine Matlab suite for testing, papers for detection, descriptors and evaluation by K. Mikolajczyk]
* [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html Shape Context descriptors]
[[Category:Community:Project]]

MatchPoint

2008-02-27T17:34:50Z

Stereo sl: /* References */

MatchPoint

2008-02-27T17:33:29Z

Stereo sl: /* References */

== Introduction ==
MatchPoint is a next generation CP generator. The result of a GSoC2007 project [[SoC2007_project_Feature_Descriptor]], it is still very experimental. Read more [http://groups.google.com/group/hugin-ptx/browse_thread/thread/cba2b2ce94dd9054 here]

== Command line usage ==
Usage: 
;MatchPoint [options] image1.jpg output.key: 
Parameters:
;-v: verbose output
;-t: generate keypoint file for matlab test suite(file name is generated using formula: image1.jpg.key)
Arguments:
;image1.jpg: Path to image to be analyzed.
;output.key: Output keypoint file.

== Algorithm description ==

=== Integral images ===

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image. This makes calculating sum of a region much faster. In addition, convolution at any scale has equal time complexity.
This part is neccessary only when using box filters for detection.

=== Feature detection ===

Points are detected with Hessian Detector using box filters. Here is a description of detection process over time. This may not be entirely compatible flow with the algorithm's flow, because some parts are simultaneous(e.q. first two steps).

''' Convolution of pixels ''' 
Convolution of a pixel at certain scale is carried out with approximation of Gauss 2D function - this is called box filter detection. Each kernel has 4 convolution filters(3 of them are unique - xy and yx filters are the same). The resulting value is then the determinant of Hessian matrix where elements represent convolution values of 4 filters.

Kernel sizes for convolution are: 
9,15,21,27, (1st octave) 
15,27,39,51, (2nd octave) 
21,45,69,93 (3rd octave) 

MatchPoint features also offers two ways of convolution:
* box filter: faster and preferable way
* sliding window(implemented for convinience but needs debug): slower, more accurate but also more sensitive to noise

''' Finding maxima ''' 
In order to achieve invariance to scaling, detection needs to find maxima of Hessian determinant across many scales. To handle this, octaves were introduced. Octaves are interpolation of determinants at various scales(usually two scales). MatchPoint offers detection at max 3 octaves(by setting a parameter). E.q. at first octave a point can be detected at scale 1.6(=((9+15/2)*1.2)/9 where 1.2 is initial scale and 9 is initial kernel size) or 3.2(=((21+27/2)*1.2)/9. Keypoint's main scale is then selected according to the highest value of Hessian determinant.

''' Selecting points ''' 

Next step is to select points with high values of their determinants(regardless of scale where points where detected) that represent invariant keypoints that will be used further processing. This is achieved using a fixed threshold which should be set low(because otherwise it may happen that no points would be detected).

Then non-maxima supression is applied(only points with local maxima of determinants are selected).

At this point we have a list of points that can vary in length, because threshold from previous step is hardcoded. This can result in over 200.000 points for larger images and that is too much(regardless of image size we need the same amount of control points for all images - min 3 control point/overlapping images), so we need to strip down the list below 10.000 points(this number is set intuitivelly and it is based on consumed time). (Note: this work is progress).

=== Feature description ===

''' Orientation assignment ''' 
Scale invariance is achieved by assigning main orientation of the interest point using special algorithm proposed by Herbert Bay. Efficiency of this algorithm has not been yet tested, therefore the executable of MatchPoint does not use any orientation assignment.

''' Shape Context descriptors ''' 
To each interest point a 36 element descriptor is assigned. Elements of this descriptors are organized according to Shape Context descriptor proposed by [http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html S. Belongie, J. Malik and J. Puzicha] and adapted by [http://www.robots.ox.ac.uk/~vgg/research/affine/index.html K. Mikolajczyk] for the purpose of finding control points.
Steps in creating point's descriptor:
# detect edges in the surrounding region. MatchPoint uses vigra's API(Canny edge detection) for this.
# based on relative location of edge elements create a histogram with 36 values. Use log-polar values. Edge point contribution to the histogram is based on it's magnitude and orientation.
See reference papers for details.

==References==
* [http://www.vision.ee.ethz.ch/~surf/ Speeded Up Robust Features - SURF]
* [http://www.robots.ox.ac.uk/~vgg/research/affine Speeded Up Robust Features - SURF]

[[Category:Community:Project]]

MatchPoint

2008-02-27T17:22:52Z

Stereo sl: /* Feature description */

MatchPoint

2008-02-27T17:18:36Z

Stereo sl: /* Feature description */

== Introduction ==
MatchPoint is a next generation CP generator. The result of a GSoC2007 project [[SoC2007_project_Feature_Descriptor]], it is still very experimental. Read more [http://groups.google.com/group/hugin-ptx/browse_thread/thread/cba2b2ce94dd9054 here]

== Command line usage ==
Usage: 
;MatchPoint [options] image1.jpg output.key: 
Parameters:
;-v: verbose output
;-t: generate keypoint file for matlab test suite(file name is generated using formula: image1.jpg.key)
Arguments:
;image1.jpg: Path to image to be analyzed.
;output.key: Output keypoint file.

== Algorithm description ==

=== Integral images ===

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image. This makes calculating sum of a region much faster. In addition, convolution at any scale has equal time complexity.
This part is neccessary only when using box filters for detection.

=== Feature detection ===

Points are detected with Hessian Detector using box filters. Here is a description of detection process over time. This may not be entirely compatible flow with the algorithm's flow, because some parts are simultaneous(e.q. first two steps).

''' Convolution of pixels ''' 
Convolution of a pixel at certain scale is carried out with approximation of Gauss 2D function - this is called box filter detection. Each kernel has 4 convolution filters(3 of them are unique - xy and yx filters are the same). The resulting value is then the determinant of Hessian matrix where elements represent convolution values of 4 filters.

Kernel sizes for convolution are: 
9,15,21,27, (1st octave) 
15,27,39,51, (2nd octave) 
21,45,69,93 (3rd octave) 

MatchPoint features also offers two ways of convolution:
* box filter: faster and preferable way
* sliding window(implemented for convinience but needs debug): slower, more accurate but also more sensitive to noise

''' Finding maxima ''' 
In order to achieve invariance to scaling, detection needs to find maxima of Hessian determinant across many scales. To handle this, octaves were introduced. Octaves are interpolation of determinants at various scales(usually two scales). MatchPoint offers detection at max 3 octaves(by setting a parameter). E.q. at first octave a point can be detected at scale 1.6(=((9+15/2)*1.2)/9 where 1.2 is initial scale and 9 is initial kernel size) or 3.2(=((21+27/2)*1.2)/9. Keypoint's main scale is then selected according to the highest value of Hessian determinant.

''' Selecting points ''' 

Next step is to select points with high values of their determinants(regardless of scale where points where detected) that represent invariant keypoints that will be used further processing. This is achieved using a fixed threshold which should be set low(because otherwise it may happen that no points would be detected).

Then non-maxima supression is applied(only points with local maxima of determinants are selected).

At this point we have a list of points that can vary in length, because threshold from previous step is hardcoded. This can result in over 200.000 points for larger images and that is too much(regardless of image size we need the same amount of control points for all images - min 3 control point/overlapping images), so we need to strip down the list below 10.000 points(this number is set intuitivelly and it is based on consumed time). (Note: this work is progress).

=== Feature description ===

''' Orientation assignment ''' 
Scale invariance is achieved by assigning main orientation of the interest point using special algorithm proposed by Herbert Bay. Efficiency of this algorithm has not been yet tested, therefore the executable of MatchPoint does not use any orientation assignment.

''' Shape Context descriptors ''' 
To each interest point a 36 element descriptor is assigned. Elements of this descriptors are organized according to Shape Context descriptor proposed by ... and adapted by ... for the purpose of finding control points.
Steps in creating point's descriptor:
# detect edges in the surrounding region. MatchPoint uses vigra's API(Canny edge detection) for this.
# based on relative location of edge elements create a histogram with 36 values. Use log-polar values. Edge point contribution to the histogram is based on it's magnitude and orientation.
See reference papers for details.

==References==
<references />

[[Category:Community:Project]]

MatchPoint

2008-02-27T17:17:45Z

Stereo sl: /* Introduction */

== Introduction ==
MatchPoint is a next generation CP generator. The result of a GSoC2007 project [[SoC2007_project_Feature_Descriptor]], it is still very experimental. Read more [http://groups.google.com/group/hugin-ptx/browse_thread/thread/cba2b2ce94dd9054 here]

== Command line usage ==
Usage: 
;MatchPoint [options] image1.jpg output.key: 
Parameters:
;-v: verbose output
;-t: generate keypoint file for matlab test suite(file name is generated using formula: image1.jpg.key)
Arguments:
;image1.jpg: Path to image to be analyzed.
;output.key: Output keypoint file.

== Algorithm description ==

=== Integral images ===

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image. This makes calculating sum of a region much faster. In addition, convolution at any scale has equal time complexity.
This part is neccessary only when using box filters for detection.

=== Feature detection ===

Points are detected with Hessian Detector using box filters. Here is a description of detection process over time. This may not be entirely compatible flow with the algorithm's flow, because some parts are simultaneous(e.q. first two steps).

''' Convolution of pixels ''' 
Convolution of a pixel at certain scale is carried out with approximation of Gauss 2D function - this is called box filter detection. Each kernel has 4 convolution filters(3 of them are unique - xy and yx filters are the same). The resulting value is then the determinant of Hessian matrix where elements represent convolution values of 4 filters.

Kernel sizes for convolution are: 
9,15,21,27, (1st octave) 
15,27,39,51, (2nd octave) 
21,45,69,93 (3rd octave) 

MatchPoint features also offers two ways of convolution:
* box filter: faster and preferable way
* sliding window(implemented for convinience but needs debug): slower, more accurate but also more sensitive to noise

''' Finding maxima ''' 
In order to achieve invariance to scaling, detection needs to find maxima of Hessian determinant across many scales. To handle this, octaves were introduced. Octaves are interpolation of determinants at various scales(usually two scales). MatchPoint offers detection at max 3 octaves(by setting a parameter). E.q. at first octave a point can be detected at scale 1.6(=((9+15/2)*1.2)/9 where 1.2 is initial scale and 9 is initial kernel size) or 3.2(=((21+27/2)*1.2)/9. Keypoint's main scale is then selected according to the highest value of Hessian determinant.

''' Selecting points ''' 

Next step is to select points with high values of their determinants(regardless of scale where points where detected) that represent invariant keypoints that will be used further processing. This is achieved using a fixed threshold which should be set low(because otherwise it may happen that no points would be detected).

Then non-maxima supression is applied(only points with local maxima of determinants are selected).

At this point we have a list of points that can vary in length, because threshold from previous step is hardcoded. This can result in over 200.000 points for larger images and that is too much(regardless of image size we need the same amount of control points for all images - min 3 control point/overlapping images), so we need to strip down the list below 10.000 points(this number is set intuitivelly and it is based on consumed time). (Note: this work is progress).

=== Feature description ===

''' Orientation assignment ''' 
Scale invariance is achieved by assigning main orientation of the interest point using special algorithm proposed by Herbert Bay. Efficiency of this algorithm has not been yet tested, therefore the executable of MatchPoint does not use any orientation assignment.

''' Shape Context descriptors ''' 
To each interest point a 36 element descriptor is assigned. Elements of this descriptors are organized according to Shape Context descriptor proposed by ... and adapted by ... for the purpose of finding control points.
Steps in creating point's descriptor:
# detect edges in the surrounding region. MatchPoint uses vigra's API(Canny edge detection) for this.
# based on relative location of edge elements create a histogram with 36 values. Use log-polar values. Edge point contribution to the histogram is based on it's magnitude and orientation.
See reference papers for details.

==References==
<references />

[[Category:Community:Project]]

MatchPoint

2008-02-27T17:16:42Z

Stereo sl:

== Introduction ==
MatchPoint is a next generation CP generator. The result of a GSoC2007 project[[SoC2007_project_Feature_Descriptor]], it is still very experimental. Experience reports needed. Read more [http://groups.google.com/group/hugin-ptx/browse_thread/thread/cba2b2ce94dd9054 here]

== Command line usage ==
Usage: 
;MatchPoint [options] image1.jpg output.key: 
Parameters:
;-v: verbose output
;-t: generate keypoint file for matlab test suite(file name is generated using formula: image1.jpg.key)
Arguments:
;image1.jpg: Path to image to be analyzed.
;output.key: Output keypoint file.

== Algorithm description ==

=== Integral images ===

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image. This makes calculating sum of a region much faster. In addition, convolution at any scale has equal time complexity.
This part is neccessary only when using box filters for detection.

=== Feature detection ===

Points are detected with Hessian Detector using box filters. Here is a description of detection process over time. This may not be entirely compatible flow with the algorithm's flow, because some parts are simultaneous(e.q. first two steps).

''' Convolution of pixels ''' 
Convolution of a pixel at certain scale is carried out with approximation of Gauss 2D function - this is called box filter detection. Each kernel has 4 convolution filters(3 of them are unique - xy and yx filters are the same). The resulting value is then the determinant of Hessian matrix where elements represent convolution values of 4 filters.

Kernel sizes for convolution are: 
9,15,21,27, (1st octave) 
15,27,39,51, (2nd octave) 
21,45,69,93 (3rd octave) 

MatchPoint features also offers two ways of convolution:
* box filter: faster and preferable way
* sliding window(implemented for convinience but needs debug): slower, more accurate but also more sensitive to noise

''' Finding maxima ''' 
In order to achieve invariance to scaling, detection needs to find maxima of Hessian determinant across many scales. To handle this, octaves were introduced. Octaves are interpolation of determinants at various scales(usually two scales). MatchPoint offers detection at max 3 octaves(by setting a parameter). E.q. at first octave a point can be detected at scale 1.6(=((9+15/2)*1.2)/9 where 1.2 is initial scale and 9 is initial kernel size) or 3.2(=((21+27/2)*1.2)/9. Keypoint's main scale is then selected according to the highest value of Hessian determinant.

''' Selecting points ''' 

Next step is to select points with high values of their determinants(regardless of scale where points where detected) that represent invariant keypoints that will be used further processing. This is achieved using a fixed threshold which should be set low(because otherwise it may happen that no points would be detected).

Then non-maxima supression is applied(only points with local maxima of determinants are selected).

At this point we have a list of points that can vary in length, because threshold from previous step is hardcoded. This can result in over 200.000 points for larger images and that is too much(regardless of image size we need the same amount of control points for all images - min 3 control point/overlapping images), so we need to strip down the list below 10.000 points(this number is set intuitivelly and it is based on consumed time). (Note: this work is progress).

=== Feature description ===

''' Orientation assignment ''' 
Scale invariance is achieved by assigning main orientation of the interest point using special algorithm proposed by Herbert Bay. Efficiency of this algorithm has not been yet tested, therefore the executable of MatchPoint does not use any orientation assignment.

''' Shape Context descriptors ''' 
To each interest point a 36 element descriptor is assigned. Elements of this descriptors are organized according to Shape Context descriptor proposed by ... and adapted by ... for the purpose of finding control points.
Steps in creating point's descriptor:
# detect edges in the surrounding region. MatchPoint uses vigra's API(Canny edge detection) for this.
# based on relative location of edge elements create a histogram with 36 values. Use log-polar values. Edge point contribution to the histogram is based on it's magnitude and orientation.
See reference papers for details.

==References==
<references />

[[Category:Community:Project]]

MatchPoint

2008-02-27T16:54:44Z

Stereo sl:

MatchPoint is a next generation CP generator. The result of a GSoC2007 project[[SoC2007_project_Feature_Descriptor]], it is still very experimental. Experience reports needed. Read more [http://groups.google.com/group/hugin-ptx/browse_thread/thread/cba2b2ce94dd9054 here]

== Integral images ==

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image. This makes calculating sum of a region much faster. In addition, convolution at any scale has equal time complexity.
This part is neccessary only when using box filters for detection.

== Feature detection ==

Points are detected with Hessian Detector using box filters. Here is a description of detection process over time. This may not be entirely compatible flow with the algorithm's flow, because some parts are simultaneous(e.q. first two steps).

''' Convolution of pixels ''' 
Convolution of a pixel at certain scale is carried out with approximation of Gauss 2D function - this is called box filter detection. Each kernel has 4 convolution filters(3 of them are unique - xy and yx filters are the same). The resulting value is then the determinant of Hessian matrix where elements represent convolution values of 4 filters.

Kernel sizes for convolution are: 
9,15,21,27, (1st octave) 
15,27,39,51, (2nd octave) 
21,45,69,93 (3rd octave) 

MatchPoint features also offers two ways of convolution:
* box filter: faster and preferable way
* sliding window(implemented for convinience but needs debug): slower, more accurate but also more sensitive to noise

''' Finding maxima ''' 
In order to achieve invariance to scaling, detection needs to find maxima of Hessian determinant across many scales. To handle this, octaves were introduced. Octaves are interpolation of determinants at various scales(usually two scales). MatchPoint offers detection at max 3 octaves(by setting a parameter). E.q. at first octave a point can be detected at scale 1.6(=((9+15/2)*1.2)/9 where 1.2 is initial scale and 9 is initial kernel size) or 3.2(=((21+27/2)*1.2)/9. Keypoint's main scale is then selected according to the highest value of Hessian determinant.

''' Selecting points ''' 

Next step is to select points with high values of their determinants(regardless of scale where points where detected) that represent invariant keypoints that will be used further processing. This is achieved using a fixed threshold which should be set low(because otherwise it may happen that no points would be detected).

Then non-maxima supression is applied(only points with local maxima of determinants are selected).

At this point we have a list of points that can vary in length, because threshold from previous step is hardcoded. This can result in over 200.000 points for larger images and that is too much(regardless of image size we need the same amount of control points for all images - min 3 control point/overlapping images), so we need to strip down the list below 10.000 points(this number is set intuitivelly and it is based on consumed time). (Note: this work is progress).

== Feature description ==

''' Orientation assignment ''' 
Scale invariance is achieved by assigning main orientation of the interest point using special algorithm proposed by Herbert Bay. Efficiency of this algorithm has not been yet tested, therefore the executable of MatchPoint does not use any orientation assignment.

''' Shape Context descriptors ''' 
To each interest point a 36 element descriptor is assigned. Elements of this descriptors are organized according to Shape Context descriptor proposed by ... and adapted by ... for the purpose of finding control points.
Steps in creating point's descriptor:
# detect edges in the surrounding region. MatchPoint uses vigra's API(Canny edge detection) for this.
# based on relative location of edge elements create a histogram with 36 values. Use log-polar values. Edge point contribution to the histogram is based on it's magnitude and orientation.
See reference papers for details.

==References==
<references />

[[Category:Community:Project]]

MatchPoint

2008-02-27T16:39:46Z

Stereo sl:

MatchPoint

2008-02-27T16:39:35Z

Stereo sl: /* Feature description */

MatchPoint

2008-02-27T16:25:50Z

Stereo sl:

MatchPoint

2008-02-27T16:18:26Z

Stereo sl: /* Feature detection */

MatchPoint

2008-02-27T15:43:03Z

Stereo sl:

MatchPoint

2008-02-27T14:35:33Z

Stereo sl:

MatchPoint is a next generation CP generator. The result of a GSoC2007 project, it is still very experimental. Experience reports needed. Read more [http://groups.google.com/group/hugin-ptx/browse_thread/thread/cba2b2ce94dd9054 here]

== Integral images ==

Before detection process images are integrated. Each element-pixel (x,y) of integral image represents sum of pixels from (0,0) to (x,y) on initial image.

== Feature detection ==

MatchPoint

2008-02-27T14:27:50Z

Stereo sl: New page: MatchPoint is a next generation CP generator. The result of a GSoC2007 project, it is still very experimental. Experience reports needed. Read more [http://groups.google.com/group/hugin-pt...

SoC2007 project Batch Processing

2007-04-04T23:12:02Z

Stereo sl:

See [[SoC 2007 overview]] for usage hints and a page list.

= Foreword =

Virtual reality together with panoramic images has gained popularity over the recent years with the development of supporting technologies. This novelty has rapidly grown and today it deeply affects our lives. Specially the World Wide Web is becoming increasingly interactive with the use of interactive panoramic images, interaction with mobile phones, flash movies,... On the other hand, the use of panoramic image in journalism, adveritising and marketing is gaining popularity.

Software industry has, in order to fulfill interactivity demands, responded with a number of open-source and commercial applications. One of them is also hugin, an open-source front end for panorama tools, and an application used for creating, stitching, and processing panoramic images.

= Summary =

In the past and present time, batch processing has been one of many ways to increase the speed of processing. Its common feature is grouping similar tasks together and then executing them. The main advantage of batch processing is the reduction of time needed to execute and organize a large amount of similar tasks.

In terms of panoramic images, batch processing can be applied to a common process of stitching together input images to an output panoramic image. This process can be repeated as long as there are input images and project files in the preselected directory.
Photographers like to take panoramas in series and the result would be a large set of images, that need to be grouped together. Firstly, the batch processor needs to determine heuristically which ones belong to the same panorama, then create the project file, apply the right template(according to exif information), optimize and output images in the selected format. What is more, the batch processor also needs to check whether existing projects are also located in that folder and also process them.

PTButcher is project that carries out this idea. It implements features for batch project building, batch stitching and overall project management of hugin projects(or any other related project files). It is a part of hugin, and also a standalone script that can be invoked from the command line.

PTButcher's aim is to provide a solid framework for professional photographers and advanced users of hugin that produce panoramas almost every day. They urge to have a fast and effective workflow and also something less time consuming than the existing hugin GUI that does not offer any batch processing.

I am proposing this project in order that Google funds its development through the Summer of Code 2007.

= Description and features =

Here is a brief description of all features and possible ways of implementation.

== hugin project manager ==

In order to make hugin 'de facto' application for panoramic images, hugin needs a good project organization and management. This includes: a list of all available projects, the ability to open, close and delete projects, in addition to that, users can copy or move the projects into another location. Users can optimize or stitch all the selected projects with a simple click. When this features will be implemented hugin will become an IDE for panoramic images.

== Batch processing ==

Here is a timeline of PTButcher's batch processing execution:
* read the files in a directory
* determine heuristically which ones belong to the same project
* set up a project file
* run CP generator
* determine based on the CP's if the files really belong together
* apply template
* optimize
* output in different formats (layerd equirect photoshop; layered rectilieaar of the zenith, for example)

The batch process is fully customizable with the use of script parameters and templates(see [[#Command line interface and parameters]]).

Ability to stitch panovideos: Given a template with the (n) of images involved and (n) folders, containing (m) png images extracted from the (n) streams to stitch together. Invoke itself the softwares, and cycles the batch to build (m) images in a new folder.

== Implementation ==

PTButcher will be implemented in c++ and python. The scripting language is intended to be the core part, which will then invoke the proper panorama tools. The c++(with Qt library) is intended to integrate PTButcher into hugin and provide a user friendly interface to access the scripting-language library.

== More features ==

Code of PTButcher is divided into two parts:
* project manager in hugin(c++ and Qt)
* scripting library (perl or python)

Project manager:
* ability to translate projects between the various gui.
* to search projects in subfolders
* to keep history of all operations, with undos ability.

Scripting library:
* creates projects based on the subfolder of a main folders, creates CPs and optimizes.
* Default name schemes customizable for result images and projects. As well as project folder-locations
* Template application (selection in base of image number, or exif)
* Customizable CP finder and refinery
* Customizable optimizer
* report with result and-or creation of a little test image
* capable of hugin-ptgui-whatever panorama project format.

Support for HDR or ADR workflows:
always template based. i.e. Given 3 set of 4 fisheye images and the corresponding template, it generates the main project, optimizes images together, and THEN, render 3images (a,b,c) involving images 0,3,6,9 (a) 1,4,7,10) (b) (2,5,8,11) (c)

== Templates ==

One of many features PTButcher offers are also templates. Templates provide a common way of choosing the right settings(lens system, number of images, panorama type, output format) to a group of images or projects. Next to pre-defined templates, users can also create and save custom templates, based on the settings they applied to the project, The implementation plan of templates is to provide a xml file for each template.

Example of a template:
* spherical panorama
* 8 input pictures
* Canon 5D and 15mm Fisheye Lens
* output image format(jpeg)

= Schedule =
== Development timeline ==
1. Analysis:

It is of great importance to carefully carry out this section, as wrong decisions at the beginning can cause great damage later on. Therefore, in this section I would first gather all the information about the project there is available, then reconsider all the ideas, and to put a lot of effort in clearing out all indistinct information.
The main question here would be: "'''What is to be done?'''".
''''''
start date: May 1st
duration: 1 week
end date: May 8th

2. Planning:

The main question of this section would be:"'''How to carry out the work?'''".
How do we store data? What data structures are required to hold the data? What algorithms are suitable for carrying out the work? Design of classes, interfaces, libraries, templates, ... (see the [[PTButcher_proposal#Discussion]] for some answers)
''' '''
start date: May 8th
duration: 1 week
end date: May 15th

3. Coding
''' '''
start date: June 15th
duration: 4 weeks
end date: July 9-16th

4. Testing:

Testing can be applied in many ways. In my oppinion it is best to get instant feedback of users as soon as the project becomes stable enough. The idea is to find a reliable group of testers(perhaps photographers and other developers on the mailing lists) that will test the suite with their own data and then improve the code according to their suggestions.

''' '''
start date: July 16th
duration: 4 weeks
end date: August 20th

== Implementation timeline ==

Please take under consideration that this values can change. Coding timeline:
* hugin project manager - 1 week expected:
:- gui creation and filling it with data
:- ability to move/copy projects – search and replace in project files
:- interface for calling external libraries for batch processing
* scripting library - 2 weeks expected:
:-heuristically determine which pictures belong to the same project
:-batch processing
:-template generator
* other features - 1 week
:-support for HDR or ADR workflows
:-ability to stitch panovideos

= Discussion =
One of the most intriguing questions is how to determine which files belong to the same panorama/project. This needs to be done heuristically. Several approaches so far has been discussed:

* EXIF information
:The idea is that images that form one panorama have similar EXIF information(lens, focal distance, time of capture, ...), therefore we can group them together or assign some possibility value according to EXIF information. The problem of EXIF data is, that some images do not have this data included.
* possibility matrix:
:If we have N input images we form a NxN matrix with input image names on both axes. Each cell with index then i,j represents the possibility that images i and j are a part of the same panorama(the diag of the matrix is 1 of course). To fill out values of this matrix, we need to compare images based on EXIF information or generated control points.
:The algorithm would then search the possibility matrix for greatest values and then create a list of most probable panoramas. After this, it would invoke the panotools Control Points, and based on the results, found out whether the results of the possibility matrix are correct. If so, the panorama/project is ready for stitching, and if not, the values in the matrix need to be readjusted.

* one of the trivial ideas is, that there is a great possibility that panoramas are taken in series. This means that images that belong to the same panorama have a incrementing sequence in their filename. Example:
:How to determine which pictures form panoramas, if we have a list of input images, read from the directory: DSCF6225.jpg, DSCF6226.jpg, DSCF6227.jpg, DSCF6228.jpg, DSCF6229.jpg , DSCF6230.jpg ?
:We can quickly find out, that there is much greater possibility that images DSCF6226.jpg, DSCF6227.jpg and DSCF6228.jpg form one panorama, instead of that images DSCF6225.jpg, DSCF6228.jpg and DSCF6230.jpg belong to the same panorama. This small snippet of idea can optimize the final process.

* images in a single directory can form many types of panoramas(they vary on number on images, viewing angle, file formats...), therefore applying a single template for all is useless. Templates can be therefore created automatically(by the script) if a new type of panorama is found.

== Graphical user interface ==

All PTButcher's features will be available in graphical user interface written using Qt library. The visual layout of the interface has not been designed.

== Command line interface and parameters ==

./ptbutcher [-d path/to/direcory] [-pf] [-t /path/to/template]

-d path to the root directory. By default current directory is used

-p this tells PTButcher to only process project files(.pto). If this is not present, PTButcher tries to create project files for all images

-f output format(defult is jpeg)

-t template to be used for stitching. If this is not present, PTButcher tries to match the images with the predefined templates, according to exif information

= Additional notes =

'''Project idea'''

Luca Vascon

'''Mentor'''

Yuval Levy

'''Students'''

Zoran Mesec

== Development process: ==

PTButcher will be hosted in a public CVS repository at [http://hugin.cvs.sourceforge.net/hugin/]. Development progress and notes will be kept on the public wiki of panotools[http://wiki.panotools.org/], where users can also post comments and suggestions. Development will take place fully in the open.
Discussion about this project has already been set up on the panotools-devel mailing list. See the messages in the list archives
[http://sourceforge.net/mailarchive/forum.php?forum_name=panotools-devel].

== Deliverables ==

The final package delivered at the end of the summer will include:
* PTButcher implementation in python, with all features described below
* PTButcher GUI implementation as part of hugin
* users manual
* documentation

== About the author of this proposal ==

My programming skills and experience in open source community makes me a suitable candidate to carry out this project. I am an undergraduate student of Computer and Information science and also a photographer. I have also worked and co-managed an open-source application for instant messaging. Additional information can be provided upon request.
I have announced my draft on the panotools-devel mailing lists and received a positive feedback. Yuval Levy, a panoramic photographer and project coordinator, has agreed to mentor me on this project.

--[[User:Stereo sl|Zoran Mesec]] 02:36, 31 March 2007 (CEST)

[[Category:Community:Project]]

SoC2007 project Batch Processing

2007-04-04T23:01:26Z

Stereo sl: Project schedule and a part of discussion added.

See [[SoC 2007 overview]] for usage hints and a page list.

= Foreword =

Virtual reality together with panoramic images has gained popularity over the recent years with the development of supporting technologies. This novelty has rapidly grown and today it deeply affects our lives. Specially the World Wide Web is becoming increasingly interactive with the use of interactive panoramic images, interaction with mobile phones, flash movies,... On the other hand, the use of panoramic image in journalism, adveritising and marketing is gaining popularity.

Software industry has, in order to fulfill interactivity demands, responded with a number of open-source and commercial applications. One of them is also hugin, an open-source front end for panorama tools, and an application used for creating, stitching, and processing panoramic images.

= Summary =

In the past and present time, batch processing has been one of many ways to increase the speed of processing. Its common feature is grouping similar tasks together and then executing them. The main advantage of batch processing is the reduction of time needed to execute and organize a large amount of similar tasks.

In terms of panoramic images, batch processing can be applied to a common process of stitching together input images to an output panoramic image. This process can be repeated as long as there are input images and project files in the preselected directory.
Photographers like to take panoramas in series and the result would be a large set of images, that need to be grouped together. Firstly, the batch processor needs to determine heuristically which ones belong to the same panorama, then create the project file, apply the right template(according to exif information), optimize and output images in the selected format. What is more, the batch processor also needs to check whether existing projects are also located in that folder and also process them.

PTButcher is project that carries out this idea. It implements features for batch project building, batch stitching and overall project management of hugin projects(or any other related project files). It is a part of hugin, and also a standalone script that can be invoked from the command line.

PTButcher's aim is to provide a solid framework for professional photographers and advanced users of hugin that produce panoramas almost every day. They urge to have a fast and effective workflow and also something less time consuming than the existing hugin GUI that does not offer any batch processing.

I am proposing this project in order that Google funds its development through the Summer of Code 2007.

= Description and features =

Here is a brief description of all features and possible ways of implementation.

== hugin project manager ==

In order to make hugin 'de facto' application for panoramic images, hugin needs a good project organization and management. This includes: a list of all available projects, the ability to open, close and delete projects, in addition to that, users can copy or move the projects into another location. Users can optimize or stitch all the selected projects with a simple click. When this features will be implemented hugin will become an IDE for panoramic images.

== Batch processing ==

Here is a timeline of PTButcher's batch processing execution:
* read the files in a directory
* determine heuristically which ones belong to the same project
* set up a project file
* run CP generator
* determine based on the CP's if the files really belong together
* apply template
* optimize
* output in different formats (layerd equirect photoshop; layered rectilieaar of the zenith, for example)

The batch process is fully customizable with the use of script parameters and templates(see [[#Command line interface and parameters]]).

Ability to stitch panovideos: Given a template with the (n) of images involved and (n) folders, containing (m) png images extracted from the (n) streams to stitch together. Invoke itself the softwares, and cycles the batch to build (m) images in a new folder.

== Implementation ==

PTButcher will be implemented in c++ and python. The scripting language is intended to be the core part, which will then invoke the proper panorama tools. The c++(with Qt library) is intended to integrate PTButcher into hugin and provide a user friendly interface to access the scripting-language library.

== More features ==

Code of PTButcher is divided into two parts:
* project manager in hugin(c++ and Qt)
* scripting library (perl or python)

Project manager:
* ability to translate projects between the various gui.
* to search projects in subfolders
* to keep history of all operations, with undos ability.

Scripting library:
* creates projects based on the subfolder of a main folders, creates CPs and optimizes.
* Default name schemes customizable for result images and projects. As well as project folder-locations
* Template application (selection in base of image number, or exif)
* Customizable CP finder and refinery
* Customizable optimizer
* report with result and-or creation of a little test image
* capable of hugin-ptgui-whatever panorama project format.

Support for HDR or ADR workflows:
always template based. i.e. Given 3 set of 4 fisheye images and the corresponding template, it generates the main project, optimizes images together, and THEN, render 3images (a,b,c) involving images 0,3,6,9 (a) 1,4,7,10) (b) (2,5,8,11) (c)

== Templates ==

One of many features PTButcher offers are also templates. Templates provide a common way of choosing the right settings(lens system, number of images, panorama type, output format) to a group of images or projects. Next to pre-defined templates, users can also create and save custom templates, based on the settings they applied to the project, The implementation plan of templates is to provide a xml file for each template.

Example of a template:
* spherical panorama
* 8 input pictures
* Canon 5D and 15mm Fisheye Lens
* output image format(jpeg)

= Schedule =
== Development timeline ==
1. Analysis:

It is of great importance to carefully carry out this section, as wrong decisions at the beginning can cause great damage later on. Therefore, in this section I would first gather all the information about the project there is available, then reconsider all the ideas, and to put a lot of effort in clearing out all indistinct information.
The main question here would be: "'''What is to be done?'''".
''''''
start date: May 1st
duration: 1 week
end date: May 8th

2. Planning:

The main question of this section would be:"'''How to carry out the work?'''".
How do we store data? What data structures are required to hold the data? What algorithms are suitable for carrying out the work? Design of classes, interfaces, libraries, templates, ... (see the [[PTButcher_proposal#Discussion]] for some answers)
''' '''
start date: May 8th
duration: 1 week
end date: May 15th

3. Coding
''' '''
start date: June 15th
duration: 4 weeks
end date: July 9-16th

4. Testing:

Testing can be applied in many ways. In my oppinion it is best to get instant feedback of users as soon as the project becomes stable enough. The idea is to find a reliable group of testers(perhaps photographers and other developers on the mailing lists) that will test the suite with their own data and then improve the code according to their suggestions.

''' '''
start date: July 16th
duration: 4 weeks
end date: August 20th

== Implementation timeline ==

Please take under consideration that this values can change. Coding timeline:
* hugin project manager - 1 week expected:
:- gui creation and filling it with data
:- ability to move/copy projects – search and replace in project files
:- interface for calling external libraries for batch processing

* scripting library - 2 weeks expected:
:-heuristically determine which pictures belong to the same project
:-batch processing
:-template generator

* other features - 1 week
:-support for HDR or ADR workflows
:-ability to stitch panovideos

= Discussion =
One of the most intriguing questions is how to determine which files belong to the same panorama/project. This needs to be done heuristically. Several approaches so far has been discussed:

* EXIF information
:The idea is that images that form one panorama have similar EXIF information(lens, focal distance, time of capture, ...), therefore we can group them together or assign some possibility value according to EXIF information. The problem of EXIF data is, that some images do not have this data included.
* possibility matrix:
:If we have N input images we form a NxN matrix with input image names on both axes. Each cell with index then i,j represents the possibility that images i and j are a part of the same panorama(the diag of the matrix is 1 of course). To fill out values of this matrix, we need to compare images based on EXIF information or generated control points.
:The algorithm would then search the possibility matrix for greatest values and then create a list of most probable panoramas. After this, it would invoke the panotools Control Points, and based on the results, found out whether the results of the possibility matrix are correct. If so, the panorama/project is ready for stitching, and if not, the values in the matrix need to be readjusted.

* one of the trivial ideas is, that there is a great possibility that panoramas are taken in series. This means that images that belong to the same panorama have a incrementing sequence in their filename. Example:
:How to determine which pictures form panoramas, if we have a list of input images, read from the directory: DSCF6225.jpg, DSCF6226.jpg, DSCF6227.jpg, DSCF6228.jpg, DSCF6229.jpg , DSCF6230.jpg ?
:We can quickly find out, that there is much greater possibility that images DSCF6226.jpg, DSCF6227.jpg and DSCF6228.jpg form one panorama, instead of that images DSCF6225.jpg, DSCF6228.jpg and DSCF6230.jpg belong to the same panorama. This small snippet of idea can optimize the final process.

* images in a single directory can form many types of panoramas(they vary on number on images, viewing angle, file formats...), therefore applying a single template for all is useless. Templates can be therefore created automatically(by the script) if a new type of panorama is found.

== Graphical user interface ==

All PTButcher's features will be available in graphical user interface written using Qt library. The visual layout of the interface has not been designed.

== Command line interface and parameters ==

./ptbutcher [-d path/to/direcory] [-pf] [-t /path/to/template]

-d path to the root directory. By default current directory is used

-p this tells PTButcher to only process project files(.pto). If this is not present, PTButcher tries to create project files for all images

-f output format(defult is jpeg)

-t template to be used for stitching. If this is not present, PTButcher tries to match the images with the predefined templates, according to exif information

= Additional notes =

'''Project idea'''

Luca Vascon

'''Mentor'''

Yuval Levy

'''Students'''

Zoran Mesec

== Development process: ==

PTButcher will be hosted in a public CVS repository at [http://hugin.cvs.sourceforge.net/hugin/]. Development progress and notes will be kept on the public wiki of panotools[http://wiki.panotools.org/], where users can also post comments and suggestions. Development will take place fully in the open.
Discussion about this project has already been set up on the panotools-devel mailing list. See the messages in the list archives
[http://sourceforge.net/mailarchive/forum.php?forum_name=panotools-devel].

== Deliverables ==

The final package delivered at the end of the summer will include:
* PTButcher implementation in python, with all features described below
* PTButcher GUI implementation as part of hugin
* users manual
* documentation

== About the author of this proposal ==

My programming skills and experience in open source community makes me a suitable candidate to carry out this project. I am an undergraduate student of Computer and Information science and also a photographer. I have also worked and co-managed an open-source application for instant messaging. Additional information can be provided upon request.
I have announced my draft on the panotools-devel mailing lists and received a positive feedback. Yuval Levy, a panoramic photographer and project coordinator, has agreed to mentor me on this project.

--[[User:Stereo sl|Zoran Mesec]] 02:36, 31 March 2007 (CEST)

[[Category:Community:Project]]

SoC2007 project Batch Processing

2007-03-31T00:43:55Z

Stereo sl:

= Foreword =

Virtual reality together with panoramic images has gained popularity over the recent years with the development of supporting technologies. This novelty has rapidly grown and today it deeply affects our lives. Specially the World Wide Web is becoming increasingly interactive with the use of interactive panoramic images, interaction with mobile phones, flash movies,... On the other hand, the use of panoramic image in journalism, adveritising and marketing is gaining popularity.

Software industry has, in order to fulfill interactivity demands, responded with a number of open-source and commercial applications. One of them is also hugin, an open-source front end for panorama tools, and an application used for creating, stitching, and processing panoramic images.

= Summary =

In the past and present time, batch processing has been one of many ways to increase the speed of processing. Its common feature is grouping similar tasks together and then executing them. The main advantage of batch processing is the reduction of time needed to execute and organize a large amount of similar tasks.

In terms of panoramic images, batch processing can be applied to a common process of stitching together input images to an output panoramic image. This process can be repeated as long as there are input images and project files in the preselected directory.

Photographers like to take panoramas in series and the result would be a large set of images, that need to be grouped together. Firstly, the batch processor needs to determine heuristically which ones belong to the same panorama, then create the project file, apply the right template(according to exif information), optimize and output images in the selected format. What is more, the batch processor also needs to check whether existing projects are also located in that folder and also process them.

PTButcher is project that carries out this idea. It implements features for batch project building, batch stitching and overall project management of hugin projects(or any other related project files). It is a part of hugin, and also a standalone script that can be invoked from the command line.

PTButcher's aim is to provide a solid framework for professional photographers and advanced users of hugin that produce panoramas almost every day. They urge to have a fast and effective workflow and also something less time consuming than the existing hugin GUI that does not offer any batch processing.

I am proposing this project in order that Google funds its development through the Summer of Code 2007.

= Description and features =

Here is a brief description of all features and possible ways of implementation.

== hugin project manager ==

In order to make hugin 'de facto' application for panoramic images, hugin needs a good project organization and management. This includes: a list of all available projects, the ability to open, close and delete projects, in addition to that, users can copy or move the projects into another location. Users can optimize or stitch all the selected projects with a simple click. When this features will be implemented hugin will become an IDE for panoramic images.

== Batch processing ==

Here is a timeline of PTButcher's batch processing execution:
* read the files in a directory
* determine heuristically which ones belong to the same project
* set up a project file
* run CP generator
* determine based on the CP's if the files really belong together
* apply template
* optimize
* output in different formats (layerd equirect photoshop; layered rectilieaar of the zenith, for example)

The batch process is fully customizable with the use of script parameters and templates(see [[#Command line interface and parameters]]).

Ability to stitch panovideos: Given a template with the (n) of images involved and (n) folders, containing (m) png images extracted from the (n) streams to stitch together. Invoke itself the softwares, and cycles the batch to build (m) images in a new folder.

== Implementation ==

PTButcher will be implemented in c++ and python. The scripting language is intended to be the core part, which will then invoke the proper panorama tools. The c++(with Qt library) is intended to integrate PTButcher into hugin and provide a user friendly interface to access the scripting-language library.

== More features ==

Code of PTButcher is divided into two parts:
* project manager in hugin(c++ and Qt)
* scripting library (perl or python)

Project manager:
* ability to translate projects between the various gui.
* to search projects in subfolders
* to keep history of all operations, with undos ability.

Scripting library:
* creates projects based on the subfolder of a main folders, creates CPs and optimizes.
* Default name schemes customizable for result images and projects. As well as project folder-locations
* Template application (selection in base of image number, or exif)
* Customizable CP finder and refinery
* Customizable optimizer
* report with result and-or creation of a little test image
* capable of hugin-ptgui-whatever panorama project format.

Support for HDR or ADR workflows:
always template based. i.e. Given 3 set of 4 fisheye images and the corresponding template, it generates the main project, optimizes images together, and THEN, render 3images (a,b,c) involving images 0,3,6,9 (a) 1,4,7,10) (b) (2,5,8,11) (c)

== Templates ==

One of many features PTButcher offers are also templates. Templates provide a common way of choosing the right settings(lens system, number of images, panorama type, output format) to a group of images or projects. Next to pre-defined templates, users can also create and save custom templates, based on the settings they applied to the project, The implementation plan of templates is to provide a xml file for each template.

Example of a template:
* spherical panorama
* 8 input pictures
* Canon 5D and 15mm Fisheye Lens
* output image format(jpeg)

= Discussion =
== Graphical user interface ==

All PTButcher's features will be available in graphical user interface written using Qt library. The visual layout of the interface has not been designed.

== Command line interface and parameters ==

./ptbutcher [-d path/to/direcory] [-pf] [-t /path/to/template]

-d path to the root directory. By default current directory is used

-p this tells PTButcher to only process project files(.pto). If this is not present, PTButcher tries to create project files for all images

-f output format(defult is jpeg)

-t template to be used for stitching. If this is not present, PTButcher tries to match the images with the predefined templates, according to exif information

= Additional notes =

'''Project idea'''

Luca Vascon

'''Mentor'''

Yuval Levy

'''Students'''

Zoran Mesec

== Development process: ==

PTButcher will be hosted in a public CVS repository at [http://hugin.cvs.sourceforge.net/hugin/]. Development progress and notes will be kept on the public wiki of panotools[http://wiki.panotools.org/], where users can also post comments and suggestions. Development will take place fully in the open.
Discussion about this project has already been set up on the panotools-devel mailing list. See the messages in the list archives
[http://sourceforge.net/mailarchive/forum.php?forum_name=panotools-devel].

== Deliverables ==

The final package delivered at the end of the summer will include:
* PTButcher implementation in python, with all features described below
* PTButcher GUI implementation as part of hugin
* users manual
* documentation

== About the author of this proposal ==

My programming skills and experience in open source community makes me a suitable candidate to carry out this project. I am an undergraduate student of Computer and Information science and also a photographer. I have also worked and co-managed an open-source application for instant messaging. Additional information can be provided upon request.
I have announced my draft on the panotools-devel mailing lists and received a positive feedback. Yuval Levy, a panoramic photographer and project coordinator, has agreed to mentor me on this project.

--[[User:Stereo sl|Zoran Mesec]] 02:36, 31 March 2007 (CEST)

User:Stereo sl

2007-03-31T00:38:39Z

Stereo sl:

== About me ==

== Information ==

Please refer to [http://narcisse.selfip.com/zoranmesec] for my personal information & CV and to [http://narcisse.selfip.com/] for some of my photographic works.

== Contact Me ==

'''Gmail:''' zoran.mesec@gmail.com

'''Jabber:''' zoran.mesec@jabber.org

'''Skype:''' zoranmesec

== Google Summer of Code 2007 ==

I'm applying for the project:

[[SoC2007_projects#PTButcher]]

SoC2007 project Batch Processing

2007-03-31T00:36:22Z

Stereo sl:

= Foreword =

Virtual reality together with panoramic images has gained popularity over the recent years with the development of supporting technologies. This novelty has rapidly grown and today it deeply affects our lives. Specially the World Wide Web is becoming increasingly interactive with the use of interactive panoramic images, interaction with mobile phones, flash movies,... On the other hand, the use of panoramic image in journalism, adveritising and marketing is gaining popularity.

Software industry has, in order to fulfill interactivity demands, responded with a number of open-source and commercial applications. One of them is also hugin, an open-source front end for panorama tools, and an application used for creating, stitching, and processing panoramic images.

= Summary =

In the past and present time, batch processing has been one of many ways to increase the speed of processing. Its common feature is grouping similar tasks together and then executing them. The main advantage of batch processing is the reduction of time needed to execute and organize a large amount of similar tasks.

In terms of panoramic images, batch processing can be applied to a common process of stitching together input images to an output panoramic image. This process can be repeated as long as there are input images and project files in the preselected directory.

Photographers like to take panoramas in series and the result would be a large set of images, that need to be grouped together. Firstly, the batch processor needs to determine heuristically which ones belong to the same panorama, then create the project file, apply the right template(according to exif information), optimize and output images in the selected format. What is more, the batch processor also needs to check whether existing projects are also located in that folder and also process them.

PTButcher is project that carries out this idea. It implements features for batch project building, batch stitching and overall project management of hugin projects(or any other related project files). It is a part of hugin, and also a standalone script that can be invoked from the command line.

PTButcher's aim is to provide a solid framework for professional photographers and advanced users of hugin that produce panoramas almost every day. They urge to have a fast and effective workflow and also something less time consuming than the existing hugin GUI that does not offer any batch processing.

I am proposing this project in order that Google funds its development through the Summer of Code 2007.

= Description and features =

Here is a brief description of all features and possible ways of implementation.

== hugin project manager ==

In order to make hugin 'de facto' application for panoramic images, hugin needs a good project organization and management. This includes: a list of all available projects, the ability to open, close and delete projects, in addition to that, users can copy or move the projects into another location. Users can optimize or stitch all the selected projects with a simple click. When this features will be implemented hugin will become an IDE for panoramic images.

== Batch processing ==

Here is a timeline of PTButcher's batch processing execution:
* read the files in a directory
* determine heuristically which ones belong to the same project
* set up a project file
* run CP generator
* determine based on the CP's if the files really belong together
* apply template
* optimize
* output in different formats (layerd equirect photoshop; layered rectilieaar of the zenith, for example)

The batch process is fully customizable with the use of script parameters and templates(see [[#Command line interface and parameters]]).

Ability to stitch panovideos: Given a template with the (n) of images involved and (n) folders, containing (m) png images extracted from the (n) streams to stitch together. Invoke itself the softwares, and cycles the batch to build (m) images in a new folder.

== Implementation ==

PTButcher will be implemented in c++ and python. The scripting language is intended to be the core part, which will then invoke the proper panorama tools. The c++(with Qt library) is intended to integrate PTButcher into hugin and provide a user friendly interface to access the scripting-language library.

== More features ==

Code of PTButcher is divided into two parts:
* project manager in hugin(c++ and Qt)
* scripting library (perl or python)

Project manager:
* ability to translate projects between the various gui.
* to search projects in subfolders
* to keep history of all operations, with undos ability.

Scripting library:
* creates projects based on the subfolder of a main folders, creates CPs and optimizes.
* Default name schemes customizable for result images and projects. As well as project folder-locations
* Template application (selection in base of image number, or exif)
* Customizable CP finder and refinery
* Customizable optimizer
* report with result and-or creation of a little test image
* capable of hugin-ptgui-whatever panorama project format.

Support for HDR or ADR workflows:
always template based. i.e. Given 3 set of 4 fisheye images and the corresponding template, it generates the main project, optimizes images together, and THEN, render 3images (a,b,c) involving images 0,3,6,9 (a) 1,4,7,10) (b) (2,5,8,11) (c)

== Templates ==

One of many features PTButcher offers are also templates. Templates provide a common way of choosing the right settings(lens system, number of images, panorama type, output format) to a group of images or projects. Next to pre-defined templates, users can also create and save custom templates, based on the settings they applied to the project, The implementation plan of templates is to provide a xml file for each template.

Example of a template:
* spherical panorama
* 8 input pictures
* Canon 5D and 15mm Fisheye Lens
* output image format(jpeg)

= Discussion =
== Graphical user interface ==

All PTButcher's features will be available in graphical user interface written using Qt library. The visual layout of the interface has not been designed.

== Command line interface and parameters ==

./ptbutcher [-d path/to/direcory] [-pf] [-t /path/to/template]

-d path to the root directory. By default current directory is used

-p this tells PTButcher to only process project files(.pto). If this is not present, PTButcher tries to create project files for all images

-f output format(defult is jpeg)

-t template to be used for stitching. If this is not present, PTButcher tries to match the images with the predefined templates, according to exif information

= Additional notes =

'''Proposal'''

Luca Vascon

'''Mentor'''

Yuval Levy

'''Students'''

Zoran Mesec

== Development process: ==

PTButcher will be hosted in a public CVS repository at [http://hugin.cvs.sourceforge.net/hugin/]. Development progress and notes will be kept on the public wiki of panotools[http://wiki.panotools.org/], where users can also post comments and suggestions. Development will take place fully in the open.
Discussion about this project has already been set up on the panotools-devel mailing list. See the messages in the list archives
[http://sourceforge.net/mailarchive/forum.php?forum_name=panotools-devel].

== Deliverables ==

The final package delivered at the end of the summer will include:
* PTButcher implementation in python, with all features described below
* PTButcher GUI implementation as part of hugin
* users manual
* documentation

== About the author of this proposal ==

My programming skills and experience in open source community makes me a suitable candidate to carry out this project. I am an undergraduate student of Computer and Information science and also a photographer. I have also worked and co-managed an open-source application for instant messaging. Additional information can be provided upon request.
I have announced my draft on the panotools-devel mailing lists and received a positive feedback. Yuval Levy, a panoramic photographer and project coordinator, has agreed to mentor me on this project.

--[[User:Stereo sl|Zoran Mesec]] 02:36, 31 March 2007 (CEST)

User:Stereo sl

2007-03-31T00:31:51Z

Stereo sl:

User:Stereo sl

2007-03-31T00:31:06Z

Stereo sl:

== About me ==

== Information ==

Please refer to [http://narcisse.selfip.com/zoranmesec] for my personal information & CV and to [http://narcisse.selfip.com/zoranmesec] for some of my photographic works.

== Contact Me ==

'''Gmail:''' zoran.mesec@gmail.com

'''Jabber:''' zoran.mesec@jabber.org

'''Skype:''' zoranmesec

SoC2007 project Batch Processing

2007-03-31T00:25:45Z

Stereo sl:

= Foreword =

Virtual reality together with panoramic images has gained popularity over the recent years with the development of supporting technologies. This novelty has rapidly grown and today it deeply affects our lives. Specially the World Wide Web is becoming increasingly interactive with the use of interactive panoramic images, interaction with mobile phones, flash movies,... On the other hand, the use of panoramic image in journalism, adveritising and marketing is gaining popularity.

Software industry has, in order to fulfill interactivity demands, responded with a number of open-source and commercial applications. One of them is also hugin, an open-source front end for panorama tools, and an application used for creating, stitching, and processing panoramic images.

= Summary =

In the past and present time, batch processing has been one of many ways to increase the speed of processing. Its common feature is grouping similar tasks together and then executing them. The main advantage of batch processing is the reduction of time needed to execute and organize a large amount of similar tasks.

In terms of panoramic images, batch processing can be applied to a common process of stitching together input images to an output panoramic image. This process can be repeated as long as there are input images and project files in the preselected directory.

Photographers like to take panoramas in series and the result would be a large set of images, that need to be grouped together. Firstly, the batch processor needs to determine heuristically which ones belong to the same panorama, then create the project file, apply the right template(according to exif information), optimize and output images in the selected format. What is more, the batch processor also needs to check whether existing projects are also located in that folder and also process them.

PTButcher is project that carries out this idea. It implements features for batch project building, batch stitching and overall project management of hugin projects(or any other related project files). It is a part of hugin, and also a standalone script that can be invoked from the command line.

PTButcher's aim is to provide a solid framework for professional photographers and advanced users of hugin that produce panoramas almost every day. They urge to have a fast and effective workflow and also something less time consuming than the existing hugin GUI that does not offer any batch processing.

I am proposing this project in order that Google funds its development through the Summer of Code 2007.

= Description and features =

Here is a brief description of all features and possible ways of implementation.

== hugin project manager ==

In order to make hugin 'de facto' application for panoramic images, hugin needs a good project organization and management. This includes: a list of all available projects, the ability to open, close and delete projects, in addition to that, users can copy or move the projects into another location. Users can optimize or stitch all the selected projects with a simple click. When this features will be implemented hugin will become an IDE for panoramic images.

== Batch processing ==

Here is a timeline of PTButcher's batch processing execution:
* read the files in a directory
* determine heuristically which ones belong to the same project
* set up a project file
* run CP generator
* determine based on the CP's if the files really belong together
* apply template
* optimize
* output in different formats (layerd equirect photoshop; layered rectilieaar of the zenith, for example)

The batch process is fully customizable with the use of script parameters and templates(see [[#Command line interface and parameters]]).

Ability to stitch panovideos: Given a template with the (n) of images involved and (n) folders, containing (m) png images extracted from the (n) streams to stitch together. Invoke itself the softwares, and cycles the batch to build (m) images in a new folder.

== Implementation ==

PTButcher will be implemented in c++ and python. The scripting language is intended to be the core part, which will then invoke the proper panorama tools. The c++(with Qt library) is intended to integrate PTButcher into hugin and provide a user friendly interface to access the scripting-language library.

== More features ==

Code of PTButcher is divided into two parts:
* project manager in hugin(c++ and Qt)
* scripting library (perl or python)

Project manager:
* ability to translate projects between the various gui.
* to search projects in subfolders
* to keep history of all operations, with undos ability.

Scripting library:
* creates projects based on the subfolder of a main folders, creates CPs and optimizes.
* Default name schemes customizable for result images and projects. As well as project folder-locations
* Template application (selection in base of image number, or exif)
* Customizable CP finder and refinery
* Customizable optimizer
* report with result and-or creation of a little test image
* capable of hugin-ptgui-whatever panorama project format.

Support for HDR or ADR workflows:
always template based. i.e. Given 3 set of 4 fisheye images and the corresponding template, it generates the main project, optimizes images together, and THEN, render 3images (a,b,c) involving images 0,3,6,9 (a) 1,4,7,10) (b) (2,5,8,11) (c)

== Templates ==

One of many features PTButcher offers are also templates. Templates provide a common way of choosing the right settings(lens system, number of images, panorama type, output format) to a group of images or projects. Next to pre-defined templates, users can also create and save custom templates, based on the settings they applied to the project, The implementation plan of templates is to provide a xml file for each template.

Example of a template:
* spherical panorama
* 8 input pictures
* Canon 5D and 15mm Fisheye Lens
* output image format(jpeg)

= Discussion =
== Graphical user interface ==

All PTButcher's features will be available in graphical user interface written using Qt library. The visual layout of the interface has not been designed.

== Command line interface and parameters ==

./ptbutcher [-d path/to/direcory] [-pf] [-t /path/to/template]

-d path to the root directory. By default current directory is used

-p this tells PTButcher to only process project files(.pto). If this is not present, PTButcher tries to create project files for all images

-f output format(defult is jpeg)

-t template to be used for stitching. If this is not present, PTButcher tries to match the images with the predefined templates, according to exif information

= Additional notes =

'''Proposal'''

Luca Vascon

'''Mentor'''

Yuval Levy

'''Students'''

Zoran Mesec

== Development process: ==

PTButcher will be hosted in a public CVS repository at [http://hugin.cvs.sourceforge.net/hugin/]. Development progress and notes will be kept on the public wiki of panotools[http://wiki.panotools.org/], where users can also post comments and suggestions. Development will take place fully in the open.
Discussion about this project has already been set up on the panotools-devel mailing list. See the messages in the list archives
[http://sourceforge.net/mailarchive/forum.php?forum_name=panotools-devel].

== Deliverables ==

The final package delivered at the end of the summer will include:
* PTButcher implementation in python, with all features described below
* PTButcher GUI implementation as part of hugin
* users manual
* documentation

== About the author of this proposal ==

My programming skills and experience in open source community makes me a suitable candidate to carry out this project. I am an undergraduate student of Computer and Information science and also a photographer. I have also worked and co-managed an open-source application for instant messaging. Additional information can be provided upon request.
I have announced my draft on the panotools-devel mailing lists and received a positive feedback. Yuval Levy, a panoramic photographer and project coordinator, has agreed to mentor me on this project.

SoC2007 projects

2007-03-30T23:43:12Z

Stereo sl:

See [[SoC 2007 overview]] for usage hints and a page list.

= Introduction =

This is the work in progress list of possible projects for the [[Google_SoC_2007]]

Panoramic imaging is a very broad field and touches many different areas of expertise, such as photography, computer vision, art and programming.
There is a thriving community with experience from arts to science that provides many interesting ideas and explores new territory in panoramic imaging. In addition to the mentors, this open community will provide good support and innovative ideas.

Generally, all development should done with multiple platforms in mind (at least Windows, OSX and Linux/Unix). We have an open communication culture via mailing-lists and mostly develop using C and C++.

The project below are just suggestions. If you are an interested student and have questions or new ideas, please let us know on the relevant [[Discussion_lists]], for example [https://lists.sourceforge.net/lists/listinfo/panotools-devel panotools-devel].

= Development style =

Most of the projects below are related to [[Hugin]], and some also relate to [[Panotools]] or [[tlalli]]. [[Hugin]] is mostly written in C++, and uses the [http://kogs-www.informatik.uni-hamburg.de/~koethe/vigra/ VIGRA image processing library] to support different types of images (for example, 8bit, 16bit and float (HDR) images). The core functionality is implemented in a platform independent C++ library, which is used by the GUI based on wxWidgets toolkit, and the command line programs ([[nona]], [[fulla]]).

The development of the projects should take place in a separate branch of the projects CVS (or SVN) repository. Communication with the mentors should usually happen through the appropriate development mailing list. All code should work on the major platforms supported (Linux, OSX, Windows).

= Possible Projects =

== Intuitive yet powerful GUI for panorama creation ==

Goal: Redesign/Reimplement the graphical user interface of the premiere open source panoramic imaging suite, [[Hugin]], to increase ease of use, and provide better access to its unmatched capabilities. Currently the GUI is written using [http://www.wxwidgets.org wxWidgets], which has proven to have slightly different behaviours on different platforms, especially with a complex GUI such as the one of Hugin. This is very annoying and a lot of time has been spent on minor issues such as making the GUI look good on all platforms with different font sizes etc. Therefore two options are available:

# Rewrite the whole GUI using QT (which is very consistent even across platforms, and has (IMHO) a much nicer API). This is a lot more work, and not all functionality of the current GUI can be recreated during the Summer of Code, but it will provide a better platform to build onto in the future. It is also possible to implement the GUI logic in a scripting language such as Ruby or Python. This project should only be tackled by a student who has experience in developing non-trivial GUI applications with QT. The GUI and core panorama code are already separated into different libraries.
#* '''Details to be discussed on [[SoC_2007_project_New_GUI_Framework|the separate page]]'''.
# Extend and enhance the existing wxWidgets based GUI.

General goals for the improved GUI include:
* Providing a simple, yet helpful user interface that suggests or highlights potentially useful next steps.
* Enhancing and integrating manual and automated control point placement and management.
* Improving lens parameter management.
* Providing a batch processing interface.
* Expert mode with access to all features and internals.

Recommended knowledge or interest in:
* Workflow analysis and UI design skills
* Experience with building cross platform GUI programs (Windows/Linux/OSX), either using wxWidgets or QT
* Creative use of panoramic imaging

Mentor: Pablo d'Angelo, ?

License: GPL

== Automatic feature detection for panoramic images ==

Goal: Robust extraction of local image features using a Hessian-based detector and a suitable descriptor.
A detector and descriptor that takes into account the approximately known distortions will have a much higher matching rate, especially when fisheye or wide angle images are used.

Tasks:
* Implementation of the feature detector and descriptor, and a suitable test suite to verify the correctness of the implementation.

'''Further details are being discussed in the separate [[SoC2007_project_Feature_Descriptor]] page'''.

A desired result of the projects would be:
* C or C++ library for the matching step.
* Standalone program to extract the matches from images.

Required knowledge or interest in:
* signal or image processing background
* C or C++ development skills.
* Matlab or octave

Mentor: Pablo d'Angelo, Herbert Bay, ?

== Automatic feature matching for panoramic images ==
Goal: Robust and efficient matching of local image features.

Tasks:
* Implementation of the matching step, including geometry based outlier pruning (for example using RANSAC) and nearest neighbour matching, possibly using a fast algorithm such as cover trees. For the panoramic imaging use case, several heuristics could be used to improve the matching behaviour, including using the EXIF timestamps, or previously known approximate orientation of the images.

'''Further details are being discussed in the separate [[SoC2007_project_Feature_Matching| SoC Feature Matching project]] page'''.

A desired result of the projects would be:
* Standalone program for the feature matching part, which at the end should accept the features found by the automatic feature detection task. Preliminary studies can be done using the existing SIFT and SURF detector/descriptors.
* Integration into [[hugin]] and a standalone executable similar to [[Autopano-sift]] or [[Autopano]]

Required knowledge or interest in:
* signal or image processing background
* C or C++ development skills.

Possibly useful resources and libraries:
* [http://hunch.net/~jl/projects/cover_tree/cover_tree.html Fast nearest neighbour matching using cover trees]

Mentor: Pablo d'Angelo, ?

License: GPL

== Interactive panoramic viewer ==

Goal: The [[Freepv]] panoramic viewer aims to provide a superior viewing experience
for panoramas on all major platforms (Windows, Mac and Linux/Unix), based on
exploiting powerful graphics hardware using OpenGL. Currently it provides
basic but solid viewing capabilities for Quicktime VR, cylindrical, cubic and equi-rectangular panoramas. Plugins for Mozilla/Firefox and a standalone viewer are available. Several important features are still missing from the viewer include:

* Support for hotspots
* Optimisation for panoramas larger than the Video RAM
* Display of high dynamic range panoramas with adaptive exposure
* Support for reading a SPi-V compatible .xml file, for platforms where SPi-V is not available (Linux/Unix).
* Fallback software renderer

Required knowledge or interest in:
* OpenGL or other 3D programming experience.
* Creating cool and nice looking interactive experiences.

Mentor: Pablo d'Angelo, Fulvio Senore ?

Student: [[User:Leonox|Leon Moctezuma]]

[[Interactive_Panoramic_Viewer|Project detailed page]]

License: LGPL

== Anti-ghosting HDR panorama blending and merging algorithm ==

Goal: Most HDR creation algorithms are designed to work only with very small variations in camera viewing direction. Assume that registration and response curve estimation has already happened. An improved blending method for HDR images together that have not been shot using the traditional exposure stack method. It should avoid ghosting and be insensitive to small misregistrations.

'''Further details are being discussed in the separate [[SoC2007_project_Anti_Ghosting | SoC HDR Anti-ghosting]] project page.'''

Interesting research papers:
* [http://www.cs.berkeley.edu/~eden/737_eden_a.pdf Seamless Image Stitching of Scenes with Large Motions and Exposure Differences]
* [http://graphics.cs.ucf.edu/ekhan/project_ghost.htm Ghost Removal in High Dynamic Range Images]
* [http://research.microsoft.com/IVM/HDR/hdr_sg2003.pdf High Dynamic Range Video]

Required knowledge or interest in:
* Strong background signal/image processing
* Creative mind with ideas beyond the state of the art in computer vision/graphics research.

Mentor: Pablo d'Angelo, ?

License: GPL

== Processing of very large images ==

Goal: Allow the creation of arbitrary large panoramas.
[http://flickr.com/groups/83823859@N00/discuss/72157594574253488/]

Currently panotools, as well as hugin/nona require memory to hold the complete input and the remapped output image in memory, which consumes a lot of RAM, especially for spherical panoramas. [http://www.vips.ecs.soton.ac.uk/ VIPS] is a powerful and modular image processing library that supports very large images and multiple processors natively. By porting the core remapping routines of panotools/hugin to VIPS, panoramas of almost arbitrary size can be computed.

A desired result of the projects would be:
* VIPS operations that support the geometric and photometric (vignetting correction) transformations.
* Standalone command line program that remaps images using these routines.
* Program/script to convert panotools scripts to nip2 projects.

Required knowledge or interest in:
* C/C++ programming
* image processing

Mentor: John Cupitt, Pablo d'Angelo

License: GPL

== Architectural Overhaul of Panotools ==

Panotools is a very monolithic application. The goal of this project is to refactor the functionality of panotools into 4 main parts, as independent of each other as possible.
These parts are (at least):

* Calculation of position of images (optimization)
* Mapping from input images to output images
* Projection related computations
* Parsing of input scripts/Generation of input scripts

Were this project successful the functionality of panotools will be available as a collection of routines that be called directly (as opposed to the current model that requires
the creation and parsing of a script). It will make it easier to replace one component of panotools with another one; and it will improve the future maintenance of panotools.

'''Details to be discussed on [[SoC2007_project_Panotools_Architecture|the separate page]]'''.

A desired result of the projects would be:
* Set of functions/classes that are available to hugin to perform optimization and remapping of images
* Test suite to verify that the functionality before and after is equivalent.

Required knowledge or interest in:
* C/C++ programming
* Knowledge of Numerical methods
* Basic knowledge of image processing

Mentor: D.M. German

License: GPL

== PTButcher ==

A batch creator-project manager for batch creating projects, optimizing, managing groups of projects, names outputs, indipendently invoke the software parts and feed scripts.

All the software is based on advanced find-replace in text files, generation of script files, but needs an user-friendly GUI, the main investment in the coding I suppose.

General
* ability to translate projects between the various gui.
* To search projects in subfolders
* to keep history of all operations, with undos ability.

batch building projects
* creates projects basing on the subfolder of a main folders, creates CPs and optimizes.
* Default name schemes customizable for result images and projects. As well as project folder-locations
* Template application (selection in base of image number, or exif)
* Customizable CP finder and refinery
* Customizable optimizer
* report with result and-or creation of a little test image
* capable of hugin-ptgui-whatever panorama project format.

project manager,
you have a table of all your project, with ability to change, one by one or in group, and without opening them

* relative and absolute paths for images, changing the full path or a part of it, allowing HD migrations even if projects are foldered in creative ways.
* Path for output image
* output image type, resolution, layers.
* Involved software and modificators.
* 16 bit workflow warnings

Batch stitching
* ability of batch processing stitch operations, or to invoke the specific software and feed its batch

Ability to stitch panovideos:
Given a template with the (n) of images involved and (n) folders, containing (m) png images
extracted from the (n) streams to stitch together.
Invoke itself the softwares, and cycles the batch to build (m) images in a new folder.

Support for HDR or ADR workflows, always template based.
i.e.
Given 3 set of 4 fisheye images and the corresponding template, it generates the main project, optimizes images together, and THEN, render 3images (a,b,c) involving images 0,3,6,9 (a) 1,4,7,10) (b) (2,5,8,11) (c)

See [http://search.cpan.org/dist/Panotools-Script/ Panotools::Script] for a perl approach to this.
Note that the PanoTools script format is is a bit hard to understand, there is a proposal in the
[[SoC2007 project Panotools Architecture]] to replace it with a more workable XML file format, so work should concentrate on the workflow aspects instead of another panotools script parser.

Proposal: Luca Vascon

Mentor: Yuval Levy

Licence: GPL

Student: [[User:Stereo sl|Zoran Mesec]]

[[PTButcher proposal]]

== PtPatcher, module for Interactive panoramic viewer ==
Was: PTeditor2.

The basic need would be to have it integrated in photoshop and Gimp, with a simple “paint on this side” interface. 16Bit workflow needed, as multiple input-output filetype.
See also skypaint for inspiration interface and capabilities.

There is a lot of overlap here with [[SoC2007_projects#Interactive_panoramic_viewer|SoC2007 Interactive Panorama Viewer]]. Note
that the viewer simply needs to fork and pass the current filename, pitch, yaw and Field of View to an external script which can handle
the extraction, editing and reinsertion - The viewer itself doesn't need to be involved with any image processing.

: - If this should be a photoshop plugin it would be a possibility to interact with the Adjust (or PTAdjust) plugin, which can do the extraction and insertion within photoshop (might be possible for the Gimp, too. See [[Panorama Tools Plugins#Adjust]] for details.--[[User:Erik Krause|Erik Krause]] 23:55, 23 March 2007 (CET)

The viewer should allow to save a fileproject in order to allow:
* unlimited close and reopen of the viewer
* multiple interventions and extractions with only one final reinsertion, in order not to loose quality
* working in parallel with multiple images
* batchable alone or with PTButcher, in order to extraxct all nadirs and/or zeniths, with given view angle 'XxY' of panos in a folder and then reinsert them all.

Proposal: Luca Vascon

Mentor:

Do we incorporate this with the viewer?

Licence: GPL
[[Category:Community:Project]]

SoC2007 projects

2007-03-30T23:41:03Z

Stereo sl:

See [[SoC 2007 overview]] for usage hints and a page list.

= Introduction =

This is the work in progress list of possible projects for the [[Google_SoC_2007]]

Panoramic imaging is a very broad field and touches many different areas of expertise, such as photography, computer vision, art and programming.
There is a thriving community with experience from arts to science that provides many interesting ideas and explores new territory in panoramic imaging. In addition to the mentors, this open community will provide good support and innovative ideas.

Generally, all development should done with multiple platforms in mind (at least Windows, OSX and Linux/Unix). We have an open communication culture via mailing-lists and mostly develop using C and C++.

The project below are just suggestions. If you are an interested student and have questions or new ideas, please let us know on the relevant [[Discussion_lists]], for example [https://lists.sourceforge.net/lists/listinfo/panotools-devel panotools-devel].

= Development style =

Most of the projects below are related to [[Hugin]], and some also relate to [[Panotools]] or [[tlalli]]. [[Hugin]] is mostly written in C++, and uses the [http://kogs-www.informatik.uni-hamburg.de/~koethe/vigra/ VIGRA image processing library] to support different types of images (for example, 8bit, 16bit and float (HDR) images). The core functionality is implemented in a platform independent C++ library, which is used by the GUI based on wxWidgets toolkit, and the command line programs ([[nona]], [[fulla]]).

The development of the projects should take place in a separate branch of the projects CVS (or SVN) repository. Communication with the mentors should usually happen through the appropriate development mailing list. All code should work on the major platforms supported (Linux, OSX, Windows).

= Possible Projects =

== Intuitive yet powerful GUI for panorama creation ==

Goal: Redesign/Reimplement the graphical user interface of the premiere open source panoramic imaging suite, [[Hugin]], to increase ease of use, and provide better access to its unmatched capabilities. Currently the GUI is written using [http://www.wxwidgets.org wxWidgets], which has proven to have slightly different behaviours on different platforms, especially with a complex GUI such as the one of Hugin. This is very annoying and a lot of time has been spent on minor issues such as making the GUI look good on all platforms with different font sizes etc. Therefore two options are available:

# Rewrite the whole GUI using QT (which is very consistent even across platforms, and has (IMHO) a much nicer API). This is a lot more work, and not all functionality of the current GUI can be recreated during the Summer of Code, but it will provide a better platform to build onto in the future. It is also possible to implement the GUI logic in a scripting language such as Ruby or Python. This project should only be tackled by a student who has experience in developing non-trivial GUI applications with QT. The GUI and core panorama code are already separated into different libraries.
#* '''Details to be discussed on [[SoC_2007_project_New_GUI_Framework|the separate page]]'''.
# Extend and enhance the existing wxWidgets based GUI.

General goals for the improved GUI include:
* Providing a simple, yet helpful user interface that suggests or highlights potentially useful next steps.
* Enhancing and integrating manual and automated control point placement and management.
* Improving lens parameter management.
* Providing a batch processing interface.
* Expert mode with access to all features and internals.

Recommended knowledge or interest in:
* Workflow analysis and UI design skills
* Experience with building cross platform GUI programs (Windows/Linux/OSX), either using wxWidgets or QT
* Creative use of panoramic imaging

Mentor: Pablo d'Angelo, ?

License: GPL

== Automatic feature detection for panoramic images ==

Goal: Robust extraction of local image features using a Hessian-based detector and a suitable descriptor.
A detector and descriptor that takes into account the approximately known distortions will have a much higher matching rate, especially when fisheye or wide angle images are used.

Tasks:
* Implementation of the feature detector and descriptor, and a suitable test suite to verify the correctness of the implementation.

'''Further details are being discussed in the separate [[SoC2007_project_Feature_Descriptor]] page'''.

A desired result of the projects would be:
* C or C++ library for the matching step.
* Standalone program to extract the matches from images.

Required knowledge or interest in:
* signal or image processing background
* C or C++ development skills.
* Matlab or octave

Mentor: Pablo d'Angelo, Herbert Bay, ?

== Automatic feature matching for panoramic images ==
Goal: Robust and efficient matching of local image features.

Tasks:
* Implementation of the matching step, including geometry based outlier pruning (for example using RANSAC) and nearest neighbour matching, possibly using a fast algorithm such as cover trees. For the panoramic imaging use case, several heuristics could be used to improve the matching behaviour, including using the EXIF timestamps, or previously known approximate orientation of the images.

'''Further details are being discussed in the separate [[SoC2007_project_Feature_Matching| SoC Feature Matching project]] page'''.

A desired result of the projects would be:
* Standalone program for the feature matching part, which at the end should accept the features found by the automatic feature detection task. Preliminary studies can be done using the existing SIFT and SURF detector/descriptors.
* Integration into [[hugin]] and a standalone executable similar to [[Autopano-sift]] or [[Autopano]]

Required knowledge or interest in:
* signal or image processing background
* C or C++ development skills.

Possibly useful resources and libraries:
* [http://hunch.net/~jl/projects/cover_tree/cover_tree.html Fast nearest neighbour matching using cover trees]

Mentor: Pablo d'Angelo, ?

License: GPL

== Interactive panoramic viewer ==

Goal: The [[Freepv]] panoramic viewer aims to provide a superior viewing experience
for panoramas on all major platforms (Windows, Mac and Linux/Unix), based on
exploiting powerful graphics hardware using OpenGL. Currently it provides
basic but solid viewing capabilities for Quicktime VR, cylindrical, cubic and equi-rectangular panoramas. Plugins for Mozilla/Firefox and a standalone viewer are available. Several important features are still missing from the viewer include:

* Support for hotspots
* Optimisation for panoramas larger than the Video RAM
* Display of high dynamic range panoramas with adaptive exposure
* Support for reading a SPi-V compatible .xml file, for platforms where SPi-V is not available (Linux/Unix).
* Fallback software renderer

Required knowledge or interest in:
* OpenGL or other 3D programming experience.
* Creating cool and nice looking interactive experiences.

Mentor: Pablo d'Angelo, Fulvio Senore ?

Student: [[User:Leonox|Leon Moctezuma]]

[[Interactive_Panoramic_Viewer|Project detailed page]]

License: LGPL

== Anti-ghosting HDR panorama blending and merging algorithm ==

Goal: Most HDR creation algorithms are designed to work only with very small variations in camera viewing direction. Assume that registration and response curve estimation has already happened. An improved blending method for HDR images together that have not been shot using the traditional exposure stack method. It should avoid ghosting and be insensitive to small misregistrations.

'''Further details are being discussed in the separate [[SoC2007_project_Anti_Ghosting | SoC HDR Anti-ghosting]] project page.'''

Interesting research papers:
* [http://www.cs.berkeley.edu/~eden/737_eden_a.pdf Seamless Image Stitching of Scenes with Large Motions and Exposure Differences]
* [http://graphics.cs.ucf.edu/ekhan/project_ghost.htm Ghost Removal in High Dynamic Range Images]
* [http://research.microsoft.com/IVM/HDR/hdr_sg2003.pdf High Dynamic Range Video]

Required knowledge or interest in:
* Strong background signal/image processing
* Creative mind with ideas beyond the state of the art in computer vision/graphics research.

Mentor: Pablo d'Angelo, ?

License: GPL

== Processing of very large images ==

Goal: Allow the creation of arbitrary large panoramas.
[http://flickr.com/groups/83823859@N00/discuss/72157594574253488/]

Currently panotools, as well as hugin/nona require memory to hold the complete input and the remapped output image in memory, which consumes a lot of RAM, especially for spherical panoramas. [http://www.vips.ecs.soton.ac.uk/ VIPS] is a powerful and modular image processing library that supports very large images and multiple processors natively. By porting the core remapping routines of panotools/hugin to VIPS, panoramas of almost arbitrary size can be computed.

A desired result of the projects would be:
* VIPS operations that support the geometric and photometric (vignetting correction) transformations.
* Standalone command line program that remaps images using these routines.
* Program/script to convert panotools scripts to nip2 projects.

Required knowledge or interest in:
* C/C++ programming
* image processing

Mentor: John Cupitt, Pablo d'Angelo

License: GPL

== Architectural Overhaul of Panotools ==

Panotools is a very monolithic application. The goal of this project is to refactor the functionality of panotools into 4 main parts, as independent of each other as possible.
These parts are (at least):

* Calculation of position of images (optimization)
* Mapping from input images to output images
* Projection related computations
* Parsing of input scripts/Generation of input scripts

Were this project successful the functionality of panotools will be available as a collection of routines that be called directly (as opposed to the current model that requires
the creation and parsing of a script). It will make it easier to replace one component of panotools with another one; and it will improve the future maintenance of panotools.

'''Details to be discussed on [[SoC2007_project_Panotools_Architecture|the separate page]]'''.

A desired result of the projects would be:
* Set of functions/classes that are available to hugin to perform optimization and remapping of images
* Test suite to verify that the functionality before and after is equivalent.

Required knowledge or interest in:
* C/C++ programming
* Knowledge of Numerical methods
* Basic knowledge of image processing

Mentor: D.M. German

License: GPL

== PTButcher ==

A batch creator-project manager for batch creating projects, optimizing, managing groups of projects, names outputs, indipendently invoke the software parts and feed scripts.

All the software is based on advanced find-replace in text files, generation of script files, but needs an user-friendly GUI, the main investment in the coding I suppose.

General
* ability to translate projects between the various gui.
* To search projects in subfolders
* to keep history of all operations, with undos ability.

batch building projects
* creates projects basing on the subfolder of a main folders, creates CPs and optimizes.
* Default name schemes customizable for result images and projects. As well as project folder-locations
* Template application (selection in base of image number, or exif)
* Customizable CP finder and refinery
* Customizable optimizer
* report with result and-or creation of a little test image
* capable of hugin-ptgui-whatever panorama project format.

project manager,
you have a table of all your project, with ability to change, one by one or in group, and without opening them

* relative and absolute paths for images, changing the full path or a part of it, allowing HD migrations even if projects are foldered in creative ways.
* Path for output image
* output image type, resolution, layers.
* Involved software and modificators.
* 16 bit workflow warnings

Batch stitching
* ability of batch processing stitch operations, or to invoke the specific software and feed its batch

Ability to stitch panovideos:
Given a template with the (n) of images involved and (n) folders, containing (m) png images
extracted from the (n) streams to stitch together.
Invoke itself the softwares, and cycles the batch to build (m) images in a new folder.

Support for HDR or ADR workflows, always template based.
i.e.
Given 3 set of 4 fisheye images and the corresponding template, it generates the main project, optimizes images together, and THEN, render 3images (a,b,c) involving images 0,3,6,9 (a) 1,4,7,10) (b) (2,5,8,11) (c)

See [http://search.cpan.org/dist/Panotools-Script/ Panotools::Script] for a perl approach to this.
Note that the PanoTools script format is is a bit hard to understand, there is a proposal in the
[[SoC2007 project Panotools Architecture]] to replace it with a more workable XML file format, so work should concentrate on the workflow aspects instead of another panotools script parser.

Proposal: Luca Vascon

Mentor: Yuval Levy

Licence: GPL

[[PTButcher proposal]]

== PtPatcher, module for Interactive panoramic viewer ==
Was: PTeditor2.

The basic need would be to have it integrated in photoshop and Gimp, with a simple “paint on this side” interface. 16Bit workflow needed, as multiple input-output filetype.
See also skypaint for inspiration interface and capabilities.

There is a lot of overlap here with [[SoC2007_projects#Interactive_panoramic_viewer|SoC2007 Interactive Panorama Viewer]]. Note
that the viewer simply needs to fork and pass the current filename, pitch, yaw and Field of View to an external script which can handle
the extraction, editing and reinsertion - The viewer itself doesn't need to be involved with any image processing.

: - If this should be a photoshop plugin it would be a possibility to interact with the Adjust (or PTAdjust) plugin, which can do the extraction and insertion within photoshop (might be possible for the Gimp, too. See [[Panorama Tools Plugins#Adjust]] for details.--[[User:Erik Krause|Erik Krause]] 23:55, 23 March 2007 (CET)

The viewer should allow to save a fileproject in order to allow:
* unlimited close and reopen of the viewer
* multiple interventions and extractions with only one final reinsertion, in order not to loose quality
* working in parallel with multiple images
* batchable alone or with PTButcher, in order to extraxct all nadirs and/or zeniths, with given view angle 'XxY' of panos in a folder and then reinsert them all.

Proposal: Luca Vascon

Mentor:

Do we incorporate this with the viewer?

Licence: GPL
[[Category:Community:Project]]