PanoTools.org Wiki - User contributions [en]

Nona GPU stitching reports

2010-10-31T21:37:02Z

James Legg: /* Other video cards */ I fixed the hang with Intel 965, so update its entry.

The purpose of this page is to collect individual experience reports and learn to know what hardware / driver combination work with nona-GPU.

== Individual Reports ==

* '''IMPORTANT: before reporting, make sure you were using nona-gpu. When stitching from Hugin, set it in the Preferences -> Stitching tab -> activate the checkbox "Use GPU for remapping". When stitching from the command line, add the -g switch to the nona command line.
* identify your video card. http://www.techarp.com/showarticle.aspx?artno=88 should help. select your GPU manufacturer's full list and note the exact name. The more precise you are, the better it is for everybody. Note that this is not your video card manufacturer / brand. Today most video card manufacturers get their GPU from either nVidia or ATI.
* identify the exact driver you were using. Do so '''before''' upgrading driver, collecting reports for older drivers is useful. If you do upgrade later, make a second report for the upgraded driver.
* copy the template below.
* based on your video card and operating system, select the appropriate section to edit below.
* Edit the section and append your report at the '''bottom''' of the list (but above the lines terminating the table).
* The developers may contact you to follow up on your report.

Thank you!

<pre>

|-valign="top"
! style="border-top: thin dotted #333333;" |
[ENTER YOUR GPU DETAILS]
! style="border-top: thin dotted #333333;" |
[ENTER DRIVER VERSION]
| style="border-top: thin dotted #333333;" |
[ENTER SYSTEM DETAILS]
| style="border-top: thin dotted #333333;" |
[ENTER COMMENTS]

</pre>

== nVidia ==

This section is for nVidia powered video cards only.

=== Linux ===


{| style="margin: 1em auto 1em 1em;background:#FFFFDD;text-align:left;border-top: thin dotted #333333;"

|-valign="top"
! width="200" |
GPU Detail
! width="100" |
Driver Version
! width="200" |
System Detail
! width="400" |
Comments



|-valign="top"
! style="border-top: thin dotted #333333;" |
Riva TNT2 Pro 32GB
! style="border-top: thin dotted #333333;" |
71.86.11
| style="border-top: thin dotted #333333;" |
Ubuntu 9.04 32bit AMD Athlon XP 3200+ 2GB
| style="border-top: thin dotted #333333;" |
this system is obviously outdated for the task



|-valign="top"
! style="border-top: thin dotted #333333;" |
GeForce 9600GT
! style="border-top: thin dotted #333333;" |
xorg-x11-drv-nvidia-185.18.36
| style="border-top: thin dotted #333333;" |
Fedora 11 x86_64, Phenom 9550, 8GB
| style="border-top: thin dotted #333333;" |
This combination works OK



|-valign="top"
! style="border-top: thin dotted #333333;" |
GeForce 9600GT
! style="border-top: thin dotted #333333;" |
vesa
| style="border-top: thin dotted #333333;" |
Fedora 11 x86_64, Phenom 9550, 8GB
| style="border-top: thin dotted #333333;" |
This combination does not work



|-valign="top"
! style="border-top: thin dotted #333333;" |
GeForce 9600GT
! style="border-top: thin dotted #333333;" |
nv
| style="border-top: thin dotted #333333;" |
Fedora 11 x86_64, Phenom 9550, 8GB
| style="border-top: thin dotted #333333;" |
This combination does not work



|-valign="top"
! style="border-top: thin dotted #333333;" |
GeForce 9600GT
! style="border-top: thin dotted #333333;" |
nouveau
| style="border-top: thin dotted #333333;" |
Fedora 11 x86_64, Phenom 9550, 8GB
| style="border-top: thin dotted #333333;" |
This combination does not work



|-valign="top"
! style="border-top: thin dotted #333333;" |
GeForce 8400 GS
! style="border-top: thin dotted #333333;" |
nvidia 256.53-1
| style="border-top: thin dotted #333333;" |
Archlinux x64, C2D T5450 @1.66GHz, 2Go RAM
| style="border-top: thin dotted #333333;" |
1min 7s for a 6x10MP images equirectangular panorama, 1min 47s without GPU



|}

=== Windows ===


{| style="margin: 1em auto 1em 1em;background:#FFFFDD;text-align:left;border-top: thin dotted #333333;"

|-valign="top"
! width="200" |
GPU Detail
! width="100" |
Driver Version
! width="200" |
System Detail
! width="400" |
Comments



|-valign="top"
! style="border-top: thin dotted #333333;" |
Nvidia 8800 GT (Factory overclocked)
! style="border-top: thin dotted #333333;" |
190.62
| style="border-top: thin dotted #333333;" |
Win 7 RTM x64, 4 GB RAM, 2.7 GHz C2D E6400 (overclocked)
| style="border-top: thin dotted #333333;" |
Stitched 360x180 equirect. from fisheye images. No problems.



|-valign="top"
! style="border-top: thin dotted #333333;" |
Nvidia 8800 GT (default clocks)
! style="border-top: thin dotted #333333;" |
185.81
| style="border-top: thin dotted #333333;" |
Win 7 RTM x64, 4 GB RAM, 2.7 GHz C2D E6400 (overclocked)
| style="border-top: thin dotted #333333;" |
Remapped some images. No problems.



|-valign="top"
! style="border-top: thin dotted #333333;" |
GTX 260 Core 216 (stock frequencies)
! style="border-top: thin dotted #333333;" |
190.62
| style="border-top: thin dotted #333333;" |
Win XP SP3, Sempron 3400+ @ 2.5 GHz (socket 754), 2 GB RAM
| style="border-top: thin dotted #333333;" |
A regular equirectangular panorama that took Nona 257 seconds to process with the CPU only took 22 seconds using the GPU!



|-valign="top"
! style="border-top: thin dotted #333333;" |
GTX 260 Core 216 (stock frequencies)
! style="border-top: thin dotted #333333;" |
195.62
| style="border-top: thin dotted #333333;" |
Win 7 x64, Core i5 750 @ 2.67 GHz (LGA 1154), 4 GB RAM
| style="border-top: thin dotted #333333;" |
GPU is 10-50% faster then CPU, depends on kind of interpolation used.
Remapping took 20 sec with GPU and 22 sec with CPU, bicubic interpolation used.



|}

=== OSX ===


{| style="margin: 1em auto 1em 1em;background:#FFFFDD;text-align:left;border-top: thin dotted #333333;"

|-valign="top"
! width="200" |
GPU Detail
! width="100" |
Driver Version
! width="200" |
System Detail
! width="400" |
Comments

|-valign="top"
! style="border-top: thin dotted #333333;" |
NVIDIA GeForce 7300 GT
! style="border-top: thin dotted #333333;" |
Unknown
| style="border-top: thin dotted #333333;" |
MacOSX 10.6.1 Snow Leopard
| style="border-top: thin dotted #333333;" |
Crashes immediately. KERN_PROTECTION_FAILURE at 0x0000000000000000
|-valign="top"
! style="border-top: thin dotted #333333;" |
NVIDIA GeForce 7300 GT
! style="border-top: thin dotted #333333;" |
Unknown
| style="border-top: thin dotted #333333;" |
MacOSX 10.6.3 Snow Leopard
| style="border-top: thin dotted #333333;" |
Dies with "GL error: Framebuffer incomplete, incomplete attachment" Hugin 2010.1.0-svn-5156

|}

=== other ===


{| style="margin: 1em auto 1em 1em;background:#FFFFDD;text-align:left;border-top: thin dotted #333333;"

|-valign="top"
! width="200" |
GPU Detail
! width="100" |
Driver Version
! width="200" |
System Detail
! width="400" |
Comments



|}

== AMD/ATI ==

This section is for AMD/ATI video cards only

=== Linux ===


{| style="margin: 1em auto 1em 1em;background:#FFFFDD;text-align:left;border-top: thin dotted #333333;"

|-valign="top"
! width="200" |
GPU Detail
! width="100" |
Driver Version
! width="200" |
System Detail
! width="400" |
Comments


|-valign="top"
! style="border-top: thin dotted #333333;" |
Radeon X550
! style="border-top: thin dotted #333333;" |
xserver-xorg-video-radeon 6.12.6
| style="border-top: thin dotted #333333;" |
Debian lenny 64 bit on Athlon64 x2 3800+, 6GB
| style="border-top: thin dotted #333333;" |
"nona: this graphics card lacks the necessary extensions for -g." ([http://www.deegan.id.au/temp/hugin-nona-gpu-fail.txt Hugin stitch output])


|}

=== Windows ===


{| style="margin: 1em auto 1em 1em;background:#FFFFDD;text-align:left;border-top: thin dotted #333333;"

|-valign="top"
! width="200" |
GPU Detail
! width="100" |
Driver Version
! width="200" |
System Detail
! width="400" |
Comments


|-valign="top"
! style="border-top: thin dotted #333333;" |
Radeon 4850
! style="border-top: thin dotted #333333;" |
8.64-090714a1-085212C-ATI
| style="border-top: thin dotted #333333;" |
WIN XP-sp3, C2D-E8400
| style="border-top: thin dotted #333333;" |
Worked ok, small performance increase (74s CPU, 68s GPU)


|-valign="top"
! style="border-top: thin dotted #333333;" |
Radeon 4870 1GB (RV770)
! style="border-top: thin dotted #333333;" |
atiumdag 8.753.0.0 (Catalyst 10.7)
| style="border-top: thin dotted #333333;" |
Win 7 (6.1, Build 7600), Intel Q6600 ~3.5GHz, 4GB RAM
| style="border-top: thin dotted #333333;" |
Nona GPU remapping functional. No significant improvement in performance, maybe even slightly slower than without GPU acceleration. Average GPU usage close to 0%, max ~5%. Using latest stable driver for GPU as of this posting.



|}

=== OSX ===


{| style="margin: 1em auto 1em 1em;background:#FFFFDD;text-align:left;border-top: thin dotted #333333;"

|-valign="top"
! width="200" |
GPU Detail
! width="100" |
Driver Version
! width="200" |
System Detail
! width="400" |
Comments



|}

=== other ===


{| style="margin: 1em auto 1em 1em;background:#FFFFDD;text-align:left;border-top: thin dotted #333333;"

|-valign="top"
! width="200" |
GPU Detail
! width="100" |
Driver Version
! width="200" |
System Detail
! width="400" |
Comments



|-valign="top"
! style="border-top: thin dotted #333333;" |
ATI Radeon X 1600
! style="border-top: thin dotted #333333;" |
01.00.158
| style="border-top: thin dotted #333333;" |
OS X Snow Leopard (10.6.1)
| style="border-top: thin dotted #333333;" |
nona does quit unexpectedly, crash log is [http://habi.gna.ch/tmp/nona-crash.txt here]



|}

== Other video cards ==

our current knowledge is that nona-gpu only works with nVidia and AMD/ATI video cards. If you find out something different, please let us know here.


{| style="margin: 1em auto 1em 1em;background:#FFFFDD;text-align:left;border-top: thin dotted #333333;"

|-valign="top"
! width="200" |
GPU Detail
! width="100" |
Driver Version
! width="200" |
System Detail
! width="400" |
Comments



|-valign="top"
! style="border-top: thin dotted #333333;" |
Intel 965
! style="border-top: thin dotted #333333;" |
unknown
| style="border-top: thin dotted #333333;" |
Ubuntu 9.04
| style="border-top: thin dotted #333333;" |
Does not support the OpenGL extensions nona GPU uses, so it always does CPU stitching.



|}

[[Category:Software:Survey]]
[[Category:Software:Hugin]]

Hugin Preferences

2010-07-23T03:09:03Z

James Legg: /* Nona */ grammar corrections

= General =

== Resource usage ==

To speed things up [[Hugin]] keeps a copy in memory of as many input photos as possible. With very large projects, this would use all your system memory, so set '''Image cache memory''' to a value below your available free RAM. The default of 256MB should be ok for a system with 512MB of RAM, however this is very conservative, for large projects you will want to set this to a high proportion of your available system memory.

The [[Hugin Preview window]] is multi-threaded so can use more than one CPU/core if required. Set '''Number of CPUs''' to how many CPUs you wish to use.

== User interface ==

Usually, [[Hugin]] will use the current locale to determine the language of buttons, menus etc...
Set the '''Language''' if you need to switch languages temporarily or if you are using a platform
such as Windows95 that doesn't support localised software. Hugin won't change language
immediately, you will need to stop and restart it.

== File options ==

Some [[Hugin]] actions generate large temporary files, change the '''Temporary dir''' to specify an alternative location for writing these files. One reason for setting this independently to the operating system default would be to use a RAM disk to speed up stitching.

Note that intermediate stitching files are created in the output folder and not in this '''Temporary dir'''.

= Assistant =

The [[Hugin Assistant tab]] automates the entire panorama creation process, these
settings allow you to customise the assistant.

== Image loading ==

Select '''Automatically align images after loading''' to run the second '''Align...'''
step immediately after loading the images.

== Automatic control point checking after detecting control points==

Select '''Remove cloud-like control points (Celeste)''' to run [[Celeste standalone|celeste]] after detecting control points. Celeste will remove [[Control points]] set to clouds, this is useful because clouds will move several pixels between shots and are therefore bad scene objects to use for alignment.

Select '''Remove outlying control points by statistical method''' to run [[cpclean]], this will try to remove control points with positions that are not credible under pairwise optimisation.

== Auto align ==

Auto align uses [[autopano-sift]] or [[autopano]] to generate [[control points]]
between pairs of images, set '''Number of Ctrl Points per overlap''' to control
the number of control points. Note that although most pictures can be stitched
with just three or four control points, automatically generated points tend not
to be very evenly distributed, so this number should be set to ten or more

The size of the output '''Panorama Image Size''' is usually set in the
[[Hugin Stitcher tab]] where it is also possible to '''Calculate Optimal Size'''
based on the sizes of the input images. The '''Auto align''' process
does something similar, though here you can set a smaller output as a percentage.
Generally setting a percentage of 70% leads to no great loss of quality due to
the way a camera [[CCD]] samples data.

== Show preview ==
After completing '''Align...''', the [[Hugin Assistant tab]] will usually display the result in a preview window, here you can change this to '''Nothing''' for no preview at all, [[Hugin Fast Preview window|Fast Preview Window]] or [[Hugin Preview window|Preview Window]].

{{clr}}

= Control Points Editor =

== HDR and 16bit display ==

[[Hugin]] supports both [[HDR]] and [[16bit]] imaging. These image formats
contain a lot more brightness and colour information than can be displayed
on a standard computer monitor, so Hugin only shows a rough representation
of these pictures.

16bit data can have ''linear'' or ''corrected'' [[gamma]]. Linear images
appear very dark on many monitors, so set the '''Curve''' to '''gamma 2.2'''.

For HDR data, try setting the '''Curve'''
to '''logarithmic'''.

Changes to the '''HDR and 16bit display mode''' require restarting Hugin to
take effect.

== Fine-tune ==

[[Hugin]] helps position [[control points]] to within a fraction of a pixel distance automatically:

* When '''auto fine-tune''' is selected in the [[Hugin Control Points tab]] while picking control points.
* When clicking '''Fine-tune''' in the [[Hugin Control Points tab]]
* When picking '''Fine-tune all Points''' in the [[Hugin Main window]] '''Edit''' menu.

* '''Patch width''', the size of the square of pixels taken from the left photo to match with the right photo when picking [[control points]], reduce if this is taking a long time on your system.
* '''Search area width''', the percentage area of the right photo that is searched when picking '''control points''', reduce if this is taking a long time on your system.
* '''Local search area width''', the region of the right photo searched when you click '''Fine-tune''' in the [[Hugin Control Points tab]] or '''Fine-tune all Points''' in the [[Hugin Main window]] '''Edit''' menu.
* '''Correlation Threshold'''. For each '''Fine-tune''', [[Hugin]] calculates the quality of the '''control points''' match, raise this threshold to reject dubious matches.
* '''Peak Curvature Threshold''', Currently unused.

== Rotation search ==

Enable this if your photos:

* have a very wide angle [[Field of View]] or [[fisheye Projection]].
* are tilted up or down, [[control points]] near the [[zenith]] or [[nadir]] may need to have full 360 degree rotation search

{{clr}}
= Control Point Detectors =
[[Hugin]] uses an external tool for automatically creating [[control points]] for a set of images either when
* clicking the '''2. Align...''' button in the [[Hugin Assistant tab]] or
* clicking the '''Create control points''' button in the [[Hugin Images tab]].

In the '''Control Point Detector Programs''' list box you can choose between several presets such as
* [[autopano-sift-C]] - part of Hugin suite
* [[Panomatic]] (by Anael Orlinski)
* [[Align image stack]] - part of Hugin suite. Note that align_image_stack is not a general purpose control point detector, but is very effective for aligning images within stacks.
* [[Match-n-shift]]
* [[Autopano]] (by A. Jenny), closed source, available for Linux i386 and Windows 32bit.
* [[autopano-sift|Autopano-SIFT]] (by S. Nowozin), open source, available for Linux, Windows and OS X

Parameters for these tools can be customized in the [[Hugin Parameters for Control Point Detectors dialog]] which you can access by clicking one of the buttons '''Edit...''' or '''New...'''.

These [[Hugin Parameters for Control Point Detectors dialog|parameters]] are also helpful if you want to use a similar command line tool that isn't already listed. Click the '''New...''' button to configure a new preset to use in the [[Hugin Assistant tab|Assistant]] or the [[Hugin Images tab|Images]] tabs.

The '''Set default''' button will mark the preset selected in this list box to be used automatically in the [[Hugin Assistant tab]] when clicking the '''2. Align...''' button.

= Stitching =

In the final stitching process [[nona]] reprojects and distorts images to fit, [[enblend]] takes these
images as individual [[TIFF]] files and merges them using sophisticated seam positioning and blending and/or [[Exposure fusion]] into
a single finished image file.

'''Important note:''' The settings here are the defaults for ''new projects'', change settings for the ''current project'' in the [[Hugin Stitcher tab]].

== Nona ==

Here you can set the '''Default interpolator''' used during stitching. [[Interpolation]] is a quality setting, but the default of '''Poly3 (Bicubic)''' is good for most purposes. You are unlikely to notice any difference between interpolators other than that '''Nearest neighbor''' is fast but very low quality.

You can '''Create cropped images by default''', these [[Cropped TIFF]] images will speed up stitching, but some image editors do not process the offsets correctly.

'''Use GPU for remapping''' will activate experimental [[nona]] code to remap images using the shading language of the ''Graphics Processing Unit'' in modern video hardware. However some projections and the translation parameters are not yet supported by this experimental code. In this case Nona will automatically switch back to CPU calculation.

== Enblend ==

The '''Use alternative Enblend program''' option allows you to use other tools with a similar interface
such as [[smartblend]] or [[enblend-mask]].

enblend supports a range of '''Additional arguments''', for example you may want to set:

* '''-a''' Pre-assemble non-overlapping images to speed up blending. This is generally useful, but will slow blending in rare cases.
* '''-l number''' Number of levels to use (1 to 29), larger numbers result in wider seams. E.g. setting 1 will result in a 2 pixel wide blend, 8 will result in a 256 pixel wide blend and you are extremely unlikely to want a blend level as high as 16.

* '''-b kilobytes''' Image cache block size (default=2MiB)
* '''-c''' Use CIECAM02 to blend colors. Your input images need to have embedded [[Colour profile|colour profiles]] for this to work.
* '''-m megabytes''' Use this much memory before going to disk (default=1GiB). Increase if you have a lot of memory on your system.
* '''--fine-mask''' Enables detailed mask generation.
* '''--no-optimize''' Turn off mask optimization.

Note that setting '''Additional arguments''' here will only effect new projects, to change [[enblend]] and [[enfuse]] settings for the current project use the [[Hugin Stitcher tab]].

== Enfuse ==

If one of '''Exposure fusion''' output options is selected in the [[Hugin Stitcher tab]] then [[enfuse]] will be used to merge bracketed exposures during stitching.

= Celeste =

Often a project has many control points attached to clouds in the sky, this is usually unwanted as clouds move between photos. [[Using Celeste with hugin|celeste]] will attempt to identify 'sky' control points and delete them.

{{clr}}

[[Category:Software:Hugin]]

Hugin Images tab

2010-07-22T18:57:25Z

James Legg: add stack documentation

The '''Images tab''' is used to manage the images in a [[hugin]] project. Additionally the positions of the images in the final panorama can be edited here.

= Adding images =

Images can either be added with the '''Add individual images...''' and '''Add time series of images...''' button or via drag and drop. '''Add time series of images...''' adds all images with a similar file modification time as the selected image; if the the project is empty then a file dialogue opens to allow you to pick this initial file.

= Control points =

Individual [[control points]] can be created and edited in the [[Hugin Control Points tab]], here in the '''images tab''' they can be manipulated together.

Automatic creation of [[control points]] can be done by pressing the '''Create control Points''' button (if you select just some images, then control points will only be found for those selected). [[Hugin]] will then launch whichever control point generator is selected in '''Settings''' to add detected control points to the project. The [[Hugin Preferences]] determine the list of these ''autopano'' programs and can be used and set further options and add new tools to the list.

'''Remove Points''' does exactly what its name suggests, it removes control points between the selected images, or all control points if no image is selected.

Often a project has many control points attached to clouds in the sky, this is usually unwanted as clouds move between photos. Clicking '''Run Celeste''' will attempt to identify 'sky' control points using the [[Using Celeste with hugin|celeste]] tool and delete them.

Using '''Clean control points''' button, hugin will try to find and remove outlying control points using statistical methods.

= Image orientation =

In the '''Image Orientation''' section, the position of the selected images in the final panorama can be specified by [[yaw]], [[pitch]] and [[roll]] angle (in degrees); and the X, Y, and Z translation parameters. The '''Reset''' button will reset all angles and translations to zero. This is useful if the optimizer could not determine the image orientation well and got stuck with a suboptimal result. It is possible to select multiple images at the same time. Changes in orientation will be applied to all selected images.

Note that it is also possible to reset '''Image Orientation''' along with other parameters using the '''Reset...''' button on the [[Hugin Camera and Lens tab]].

Select '''Anchor this image for position''' to indicate that a particular image shouldn't move when
optimising with the [[hugin Optimizer tab]]. Only one image can be the ''anchor'', and by default this
is the first image in the project.

Image orientation can be linked between a group of images if they are already aligned. For example, if they form a bracketed set shot on a sturdy tripod. To do this, select all the images in the stack, and press the '''New stack''' button. You can also move images into an existing stack without selecting all the other images in the stack: press '''Change stack...''' and enter the stack number you want the selected images to join. The stack number of each image is given in the table.

If your images form approximately aligned stacks, you can create stacks as usual, select all the images and then uncheck '''Link''' by the image orientation. Approximately aligned stacks are common when you shoot bracketed sets handheld, or in windy conditions with a light tripod. Hugin will remember that the images form a stack, but does not force the image orientation and translation to be the same across the stack. This is useful for specifying stacks when you have a Panorama with Stacks [[Hugin Preferences#Control_Point_Detectors|control point detector configuration]] before control point generation. You do not have to manually mark approximately aligned stacks in other circumstances, however.

= Exposure =

Selecting '''Anchor this image for exposure''' will indicate that a particular image should be used as
an unchanging reference ''anchor'' when optimising '''Exposure''' or '''White balance''' in the
[[hugin Exposure tab]]. Usually this should be the image with the least under or over-exposure or
the image with the most typical '''White balance'''.

[[Category:Software:Hugin]]

Hugin Optimizer tab

2010-07-21T20:12:10Z

James Legg: Describe Translation parameters and optimise presets

[[hugin]] uses a photo alignment scheme where it adjusts image orientation and lens settings of source photos
until the [[control points]] line-up, this process is called ''optimisation'' and the '''hugin Optimizer tab''' is where it is controlled. You actually ''create'' individual '''control points''' in the [[hugin Control Points tab]], and ''manage'' them in the [[hugin Images tab]] and [[hugin Control Points table]].

So to align photos you need some control points, a general rule is that optimising more parameters requires more control points.

== Quick Optimizer ==

Use the '''Optimize''' combo box to pick one of several pre-set optimisation schemes, then click the '''Optimize now!''' button to calculate the best available fit. For a spherical panorama, which is where each image is taken from the same position, you should not optimise translation. For a [[linear panorama]], where you take pictures from different locations of the same flat surface, you must optimise translation.

One valid technique is to try each optimisation scheme in turn, starting at the top, and skipping the ones with translation if you are making a spherical panorama, until you are satisfied with the results.

The '''Optimisation result''' tells you how good the alignment is, large control point error distances indicate one of several things:

* Some '''control points''' are in the wrong place, look at the list in the [[hugin Control Points table]] and identify points with large distances and check they are set properly.
* If points are set correctly, there may be [[parallax]] errors caused by camera movement between shots. If so it may be necessary to decide which objects in the scene are important to have aligned and delete unrelated control points in the [[hugin Control Points tab]].
* Otherwise you may not be optimising enough parameters. For example, the [[EXIF]] data retrieved from a [[JPEG]] file may not give a very accurate starting [[Field of View]], in this case you should optimise '''view (v)'''.

'''Use only control points between images selected in preview window.''' allows you to work on just a few
of the images in the current project rather than all of them. Use the buttons along the top of the
[[Hugin Preview window]] to enable and disable source photos. When optimising with the
[[Hugin Optimizer tab]], all the hidden images will be ignored.

The following pre-set optimisation schemes are provided:

=== Positions (incremental, starting from anchor) ===

This is the simplest setting, and is probably sufficient for a lot of purposes. Only the relative orientation of images are optimised, lens parameters are left untouched, this works best if either of the following is true:

* The lens has minimal [[barrel distortion]] and the photo [[EXIF]] information supplies an accurate [[Field of View]].
or
* [[Lens calibration]] has already been performed, saved to a file and loaded into the current project via the [[hugin Camera and Lens tab]].

Note that to align any pair of photos, there should be at least two pairs of [[control points]] connecting them.

=== Positions (y,p,r) ===

This is exactly the same as the ''incremental'' setting above except that the parameters are optimised at once, this may confuse the optimiser if the images are not already roughly in the right place. Don't use this setting.

=== Positions and Translations (y,p,r,x,y,z) ===

This will optimise image orientation and the translation you get when moving the camera for a linear panorama. This is great for stitching images of the same flat surface taken from different places, for example when you couldn't fit a painting or the front of a building into one picture. Again it isn't ideal if the lens information such as the field of view is wrong.

=== Positions and View (y,p,r,v) ===

This is the same as optimising '''Positions''' except that the lens [[Field of View]] is optimised too - Use this if you don't trust the Field of View calculated from the photo's [[EXIF]] data.

Note that for this too work you need at least three well-spaced pairs of [[control points]] between any pair of photos. With a 360 degree panorama it is usually beneficial to optimise the '''Field of View''', even if you have already calibrated this beforehand.

=== Positions, Translation and View (y,p,r,x,y,z,v) ===

This is similar to '''Positions and Translations''', except you can use it when you don't trust the [[Field of View]] from the [[EXIF]] data.

=== Positions and Barrel Distortion (y,p,r,b) ===

This is the same as optimising '''Positions''' except that an attempt is made to discover the lens [[barrel distortion]] at the same time. Only the '''b''' parameter of the full [[lens correction model]] is varied by this optimisation scheme, as this is a reasonable approximation of the distortion of a typical lens. You should trust the field of view if you use this.

Again you need at least three well-placed pairs of [[control points]] between any pair of photos.

=== Positions, View and Barrel (y,p,r,v,b) ===

As the name suggests this optimises positions, [[Field of View]] and [[barrel distortion]] all at once.

=== Positions, Translation, View and Barrel (y,p,r,x,y,z,b,b) ===

This is useful when you are optimising a linear panorama, don't trust the field of view, and want to correct barrel distortion.

=== Everything without translation ===

This optimises image orientation and all geometric parameters in the full [[lens correction model]]. It includes more lens distortion parameters. The x shift and y shift (d and e) parameters account for the centre of the projection not being in the centre of the image. This is quite common, and gets very bad if an image is the cropped corner of another image.

You will need many control points, the more the better, and preferably a full spherical panorama (360 by 180 degrees) to get the best correction. You should also use a calibrated [[Heads|panoramic head]]. If the control points are bad (either there are not enough or some are in the wrong place), or your images were not taken around the [[no-parallax point]], this could produce bizarre results.

However, if you do this well, you will accurately perform [[lens calibration]]. You can save and reuse this lens information on the [[Camera and Lens tab]].

=== Everything ===

Selecting '''Everything''' really optimises everything, including the translation parameters. You will need a huge number of control points to do this successfully.

=== The Custom parameters below ===

The pre-set optimisation options are useful for most situations, but often it is necessary to switch to '''Custom parameters'''.

For example, when shooting hand-held panoramas, some of the position variation between shots can be resolved by using different '''d''' and '''e''' '''Image Center Shift''' parameters for each shot. Select '''The Custom parameters below''' and pick the '''d''' and '''e''' parameters for optimisation. Note that by default, lens settings are shared for all photos in the project, go to the [[hugin Camera and Lens tab]] and ''uncheck'' the '''link''' box for any you want to have different values in each photo.

Similarly, the translation parameters could be used to correct a wonky shot. However, they were meant for linear panoramas which expand to infinite distance at 180 degrees field of view, so you must make sure any images with non-zero translation (X, Y, or Z parameters) are in the middle of the panorama (y and p should all be around zero, and the field of view shouldn't be large enough to make them expand more than 180 degrees). This can be used to patch in the floor after you have removed the tripod that was obscuring it. You can take a shot of the floor where the tripod was at an angle (therefore you can avoid casting a shadow on the image in most cases). You can then optimise X,Y,Z on only this image. However, the floor must be flat for this to work, and this shot must be in the middle of the panorama. You can make down the middle by rotating the panorama on either of the previews.

==== Image Orientation ====

The '''Image Orientation''' section shows the photo number and [[roll]], [[pitch]] and [[yaw]] rotation/orientation values for all input photos (in parenthesis), and the X, Y, and Z translation parameters. The ''check'' mark indicates parameters that will be optimised. The translation parameters should all be 0 on a spherical panorama.

==== Lens Parameters ====

The '''Lens Parameters''' section shows the lens number and the [[lens correction model]] parameters (in parenthesis) for each of the distinct lenses in the project. Only parameters that are '''linked''' in the [[hugin Camera and Lens tab]] are shown in parenthesis.

Note that normally all the photos in a project have the same lens, so there is usually only one row here.

==== edit script before optimizing ====

[[hugin]] actually creates a [[PTOptimizer]] script file which it passes to the panorama tools library for optimisation, select '''edit script before optimizing''' and you can tweak this file for each optimisation run.

This is something you would only do in obscure cases such as:

* Linking the positions of input photos. For example photos in a multi-row panorama taken with tripod can all be assumed to have the same [[roll]], similarly photos in each row can be assumed to have the same [[pitch]] - This will help when there are few details available for setting [[control points]].
* Optimising the '''g''' or '''t''' shear parameters of the [[lens correction model]] for scanned images is only possible by editing the script file.

__NOTOC__
[[Category:Software:Hugin]]

Hugin Fast Preview window

2010-04-19T18:15:56Z

James Legg: /* Image:Show_Control_Points_Button.svg‎ Show control points */ Update for new behaviour: colour varies with error and crosses are drawn

Like the more accurate [[Hugin Preview window]], the fast preview shows something similar to the final stitched output, but with a few important differences:
* Reduced resolution input images are used, so some areas can appear blurred that will be sharp in the final output.
* Seams are not created, images are simply overlaid with the first image at the bottom of the stack and the last at the top.
* Blending by a tool such as [[enblend]] isn't shown.
* The brightness display of [[HDR]] and 16bit images is controlled by settings in the hugin Preferences, these settings are not used when stitching. The colouring of these images will also be inaccurate when using exposure or white balance correction. For HDR panoramas, the [[Hugin Preview window]] is recommended instead.
* Photometric correction only includes white balance and exposure, unless full photometric correction is enabled with the the Photometrics button.
* The remappings are approximate, the output by a tool such as [[nona]] is more accurate. If this concerns you more than speed, use the [[Hugin Preview window]] instead.
* It's much faster ;-)

The window features a row of tabs at the top, clicking on a tab switches between modes that allow you to interact with the panorama in different ways:

== Preview tab ==

===[[Image:Fast_preview_icon_identify.svg]] Identify===

Using this tool you can find where your images are, and match them to their number. You can also edit control points.

When this tool is turned on, move the mouse over the visibility buttons for the images (the numbers at the top of the preview). The image with the number on the button under the mouse lights up red in the preview. Moving the mouse over the panorama highlights all the images under the mouse in different colours. The buttons for those images lights up in matching colours.

When the mouse is on the overlap of two images, click to edit the control points between those images.

===[[Image:Fast_preview_icon_photometric.svg]] Photometrics===

Enables full photometric correction. When turned on, this will cause significant delay when changing photometric parameters. It also takes a while to turn on and off. However, with it enabled you get much better representation of the colours in the output. With it turned off, you get correction only for exposure and white balance. With it turned on, you also get vignetting and colour response correction. The [[Hugin Preview window]] does all these things by default, so you may wish to use that instead.

===[[Image:Show_Control_Points_Button.svg‎]] Show control points===

When this tool is turned on, all [[Control points]] are drawn as lines with crosses at each end. Green, yellow, orange, and red lines and their crosses indicate 'normal' control points. A red control point is misaligned (the ends of the line are father apart), and a green control point is well aligned (the ends of the line are almost in the same position). Blue indicates [[horizontal control points]], [[vertical control points]] or [[straight line control points]].

=== Blend mode ===

The ''normal'' blend mode will draw the images as a stack. The ''difference'' blend mode will do the same, except the image under the mouse pointer will be subtracted from the rest of the stack. Use this to determine if the alignment went well: where you can see edges in the subtracted image, these edges are misaligned. Be warned that this isn't fully accurate, the other preview has a better difference mode.

=== EV ===

'''EV''' stands for ''Exposure Value'', clicking the '''Reset''' button will set it to the average of all
the input image exposures or setting it to '''0''' (zero)
will result in no exposure change being applied to the panorama.

'''EV''' is a standard photographic scale, each increase or decrease by one unit will change the exposure by
the equivalent of one ''f-stop'' (ie. halving or doubling the exposure). It is worth adjusting the exposure
here in [[hugin]] rather than later in an external image editor, since '''hugin''' uses the [[camera response curve]]
calculated in the [[hugin Exposure tab]] to perform the correction in a ''linear'' colour space.

The average value isn't always wanted. If you see colour artefacts in bright sky areas, set this
to the negative of the darkest input image - This has a side-effect of clipping brighter images.

== Projection tab ==

This tab is for adjusting [[Projections|projection]] of the panorama, some projections have adjustable parameters which will appear when selected.

=== [[Image:Hugin_fit_pano.png]] Fit ===

This doesn't change any input image parameters, it just readjusts the output [[Field of View]] such that all the input images are visible. If the images are all off-centre, then there will be a lot of black space.

=== Field of View ===

This is the horizontal and vertical angle of view of the output image.

=== Projection ===

Use the drop down list to change the output projection of the panorama, the list is exactly the same as that
in the [[hugin Stitcher tab]]. Note that for some projections, the scroll-bar sliders to change the
[[Field of View]] are disabled. If you are having trouble, switch to [[Equirectangular Projection]], adjust
the field of view and switch back.

== [[Image:Fast_preview_icon_drag.svg]] Move/Drag tab ==

Using this tool you can recentre the panorama interactively. With it turned on, try the following:
* Drag the panorama with the left mouse button to rotate the panorama's images. The centre of rotation is the point where you pushed the mouse button down.
* Hold shift when doing the above to constrain movement to yaw or pitch. Note pitch is affected by the centre of rotation.
* Drag the panorama with the right mouse button or hold control and drag with the left to roll the panorama (rotate around the middle)

If the panorama contains unconnected components, they will move individually.

=== [[Image:Hugin_center_pano.png]] Center ===

This button horizontally ''pans'' the output, changing the [[yaw]] of the remapped images so they fit to the centre of the output frame. This is useful if there is a lot of black space on the left or right of the output. This also performs a '''Fit''', equivalent to the next button.

Note that centering a [[Rectilinear Projection]] or [[Fisheye Projection]] panorama will change the perspective, this may be unwanted.

=== [[Image:Hugin_fit_pano.png]] Fit ===

This doesn't change any input image parameters, it just readjusts the output [[Field of View]] such that all the input images are visible. If the images are all off-centre, then there will be a lot of black space.

=== [[Image:Hugin_straighten_pano.png]] Straighten ===

Straightening the panorama optimises the [[roll]] and [[pitch]] of the input images without changing their relative positions, levelling the panorama in the process. This normally produces good results, if you need more accurate positioning, try adding [[vertical control points]] in the [[hugin Control Points tab]] and reoptimise.

=== Numeric Transform ===

Enter a '''numerical transform''' to rotate the input images without changing their relative positions - Effectively this rotates the entire panorama. Enter [[roll]], [[pitch]] and [[yaw]] values in degrees.

==[[Image:Fast_preview_icon_crop.svg]] Crop tab ==

Using this tool you can set the output cropping region interactively. To do this precisely instead, use the [[Hugin Stitcher tab| Stitcher tab]]. Initially, the entire panorama is in the output region (i.e. nothing is cropped).

To change the cropping at each edge, move the mouse towards that edge until a white box appears along it, then drag with the left mouse button until the edge is where you want it. The darker areas represent the region that is cropped off. You can move two edges at once by moving the mouse towards the corner shared by the edges until both white boxes appear. If you wish to move the whole region at once, move the mouse into the middle so that all four edges have boxes along them and drag.

=== [[Image:Fast_preview_icon_autocrop.svg‎]] Autocrop ===

The autocrop button adjusts the crop rectangle so that it is entirely within the image area, i.e. there will be no 'black' borders in the final stitched image. It does this by maximising the area of the rectangle rather than the width or height.

== Displayed images ==

Every input image in the preview has ''toggle'' button where display can be disabled or enabled.

In addition, this display also controls the behaviour of the [[hugin Optimizer tab]] and the [[hugin Stitcher tab]] - Any photos disabled here are not used in optimisation or stitching.

=== [[Image:Hugin_preview_show_all.png]] All ===

By default all input images are shown in the preview, however individual images can be enabled and disabled in the
'''Displayed images''' section. Use the '''All''' button to return to the default and display all the images.

=== [[Image:Hugin_preview_show_none.png]] None ===

Similarly, hide all images with the '''None''' button, use this if you want to enable preview images one by one.

== Preview canvas ==

The image window itself shows a representation of the final stitched output panorama, use the scroll bars
to change the horizontal and vertical [[Field of View]].

==In practice==
[[Image:Fast_preview_guide_start.png | thumb | A panorama with unconnected image groups after optimisation.]]
Let's try using this preview to help with a panorama where automatic alignment failed. This panorama was taken where a lot of things were blowing around in the wind, and the clouds were changing quickly, so it is not surprising that it aligning it is a struggle. The [[Hugin Assistant tab|Assistant tab]] tells us there are multiple unconnected image groups. We can optimise the panorama and end up with a images correctly positioned amongst the group they are in, but the groups themselves are not aligned. Try this first.

[[Image:Fast_preview_guide_drag.png | thumb | Dragging images with the drag tool.]]
Using the Drag tool, we can roughly align the groups together:
# Turn on the tool with the ''Drag'' button.
# Drag each component so the horizon is in the middle, using the left mouse button.
# Drag with the right mouse button (or hold control and drag with the left) to level the horizon.[[Image:Fast_preview_guide_rotate_drag.png | thumb | Rotating images with the drag tool.]]
# Hold shift and drag with the left mouse button sideways to approximately line up the image with the other groups.

[[Image:Fast_preview_guide_identify.png | thumb | Two images highlighted in the preview window.]]
When we have the images in approximately the right position, we can begin placing control points to guide the optimiser. The Identify tool lends a hand here. Firstly, turn on the identify tool. Move the mouse on an overlap that was recently created between two image groups. The images in the overlap light up. Move the mouse to a place where there is only two images in that overlap, and click. The two images are opened in the [[Hugin Control Points tab|control point editor]] (there may be a short pause while the images are loaded). Once you have placed control points manually, you can return to the preview to find some more image pairs.

[[Image:Fast_preview_guide_crop.png | thumb | Adjusting the cropping region.]]
When you are happy that your panorama contains sufficient control points, optimise it again. The panorama will likely have the horizon at the wrong angle, in this case press ''Straighten'' on the preview window. You can then frame the panorama using the drag tool (hold shift so you don't make it wonky again!). Use your artistic judgement here. If you want to crop your panorama, click ''Crop'' and drag the edges of the cropping rectangle.

[[Category:Software:Hugin]]
[[Category:Tutorial:Nice to know]]

SoC 2010 ideas

2010-03-26T13:55:26Z

James Legg: /* Vetting exercise */

If you are a student willing to participate in The Google Summer of Code 2010:
* find out what ideas we have for SoC projects this year (read below);
* decide if you want to pick one of those tasks or if you have your own idea;
* join our community at [http://groups.google.com/group/hugin-ptx/ hugin-ptx];
* introduce yourself and tell us about your plans and wishes; and
* add your proposal to the [[SoC_2010_student_proposals | student proposal page]] - see examples from [[SoC_2009_student_proposals | last year]]

'''Important:''' we are admitted to Google Summer of Code 2010 but can not guarantee you a place in the program. We recommend you start preparing your application early as the application process is very competitive.

== Development style ==

Most of the projects below are related to [[Hugin]], and some also relate to [[Panotools]] or [[tlalli]]. [[Hugin]] is mostly written in C++, and uses the [http://kogs-www.informatik.uni-hamburg.de/~koethe/vigra/ VIGRA image processing library] to support different types of images (for example, 8bit, 16bit and float (HDR) images). The core functionality is implemented in a platform independent C++ library, which is used by the GUI based on wxWidgets toolkit, and the command line programs ([[nona]], [[fulla]]). We also very much welcome contributions to [[Enblend]]/[[Enfuse]].

The development of the projects should take place in a separate branch of the project's version control system. Communication with the mentors should usually happen through the appropriate mailing list. All code should work on the major platforms supported (Linux, OSX, Windows).

== Application Template ==

[http://socghop.appspot.com/gsoc/org/show/google/gsoc2010/hugin SoC2010 Application Template]

== Possible Projects ==

You are welcome to propose your own ideas.

Some of the [[SoC2007 projects | SoC2007]], [[SoC_2008_ideas | SoC2008]] and [[SoC_2009_idea | SoC2009]] project proposals were not done in the past years and are potential projects for this year:

* <del>[[SoC2007_projects#Interactive panoramic viewer]] (This was completed but there is further possible work to be done, particularly a joint project with VLC to integrate the viewer in their media player)</del> (VLC integration was completed in 2009 GSoC)
* [[SoC2007_projects#Processing of very large images]] (using the VIPS framework, or even GEGL)
* [[SoC2007_projects#Architectural Overhaul of Panotools]]
* [[SoC_2008_ideas#munin_.E2.80.94_interactive_openGL_based_GUI]]
* [[SoC_2008_ideas#enblend-enfuse_zenith.2Fnadir_and_more]]
* [[SoC_2008_ideas#Lens_Database]] The library part, [[lensfun]], is done, but there is still no system for updating the database. ''Important:'' [http://photoropter.berlios.de Photoropter] is being actively developed and any project should be complementary to that.
* [[SoC_2008_ideas#Extend_hugin.27s_output_options_for_stitching]]
* <del>[[SoC_2008_ideas#Utility_for_creating_a_Philosphere]]</del> Note this isn't enough of a project and probably is better done in mathmap
* [[SoC_2009_idea#Python_Bindings]]
* [[SoC_2009_idea#enblend/enfuse_gimp_plugin]]
* <del>[[SoC_2009_idea#hugin_colour_balancing]]</del> see [[#Colour]] below

=== Zooming for fast preview ===
It would be good if the user can zoom into Hugin's fast preview window. The the amount of approximation in the fast preview would have to reduce to display meaningful details. The areas off screen can be ignored, to keep up performance as more details need to be processed. It would be appropriate to dynamically load more image detail for the most visible images too.

==== Vetting exercise ====
Get the fast preview to scale using openGL transformations. The scale should be changeable, either by keypresses or a gui. Don't worry about scrolling when the panorama is bigger than the window, just show the centre of the panorama.

===Threading for Hugin===
Hugin currently becomes unresponsive while it loads images. It would be better to keep the interface responsive during image loading and scaling images in another thread. Also, something like [http://sourceforge.net/tracker/?func=detail&aid=2872111&group_id=77506&atid=550443 this patch], which loads images when Hugin is otherwise idle, would be better in a background thread.

The user interface could display temporary placeholders while images are being loaded, and remain interactive.

To maximise the rate at which images are loaded, ideally we would have a thread that only reads files and waits for the filesystem, and another thread to uncompress image files and produce the small version of the image which waits only for CPU time. However doing both of these in a single thread, separate from the user interface thread, would provide a responsive interface.

==== Vetting exercise ====
When Hugin starts, start another thread that pops up a window every ten seconds showing a
message. Extra bonus: the message should show relevant information
(e.g. the number of processor cores, a thread id).

=== Patent free control point generator ===

We now have a patent free control point generator with libpanomatic, but this needs some integration:

* Ability to read and write .pto projects
* To classify features in a conformal space based on info in the .pto file
* Internal tonemapping of [[HDR]] images to log() space for feature classification
* To not classify features in masked areas
* Test suite
* integration of celeste at feature identification stage
* matching pairs of photos using heuristics (see gigastart)

A possible different project to the above would be to use GPU for feature classification as suggested on ptx, note however that patented techniques such as SIFT and SURF are not suitable for use in Hugin: SIFT GPU http://www.cs.unc.edu/~ccwu/siftgpu/ uses CUDA parallel processing to search for SIFT features in images. I'm not sure if it does the search to find nearest neighbor points. But there is also a GPU accelerated version of that algorithm too. It is a brute force version of the nearest-k points. Since it is done in parallel it is order of magnitudes faster than the ANN algorithm using by autopano-sift-c. http://www-sop.inria.fr/members/Vincent.Garcia/research_knn.php

=== UI testing ===

Update: Summer of Code is strictly coding, so this project isn't possible.

<del>Everyone agrees that Hugin needs usability improvements, however the major usability issues are closely related to programming issues such as the quality and availability of control point generators. We do not want a programmer to dive-in and and try and fix 'usability' without a plan of action.

So Hugin could use a usability audit, i.e. write user profiles/personas, define tasks, collect real data from test subjects. This is a non-programming project for a student of interaction design</del>

=== Colour ===

Hugin deals correctly with colour profiles in photos and passes them on to output, this doesn't need fixing, however there are some related tasks that could be tackled:

* Display of images in tabs and preview is not colour managed, integrate [http://www.littlecms.com/ lcms] and access system monitor colour profiles
* Hugin has a good backend for adjusting white-balance. Add a GUI grey picker and/or tools to adjust colour temperatures manually on a subset of photos to be able to do stuff like this: http://www.flickr.com/photos/sbprzd/4196026736/
* [[EXIF]] metadata contains information on colour balance (WBRedLevel, WBGreenLevel & WBBlueLevel) which should be used to initialise the red/blue colour balance parameters in Hugin - Currently Hugin does something very similar for EV.
* [[Chromatic aberration|tca]] correction in [[nona]] with support in GUI and .pto format, possible simple GUI to run [[tca_correct]]
* [[Vignetting]] of colour balance. For example Tokina 12-24 f4 exhibits this phenomenon (see right side of this [http://farm5.static.flickr.com/4043/4385932566_4e3204a12d_o.jpg image]) in amounts that are easily seen, and according to Ken Rockwell (yeah) the effect plagues most ultra wide rectilinears. Could work similarily to current vignetting correction model, but working on red/blue colour channels? (do we really need this? Is the additional complication in the GUI worth it?)

==== Vetting exercise ====

Add support for reading EXIF colour-balance metadata, see [https://sourceforge.net/tracker/?func=detail&aid=2974841&group_id=77506&atid=550444 feature request #2974841 on the Hugin tracker]

=== Makefile system and Detection of panoramas ===

Hugin uses gnu ''make'' to drive stitching, this involves writing makefiles and executing ''make'' as a sub-process. We have two issues with this;

* The code that handles makefiles is mixed up with stitching logic, the result is that this part of the codebase is quite hairy and difficult to extend
* <del>'make' places restrictions on characters in file paths but the Hugin GUI doesn't do anything to prevent users from using these characters</del>

Write a C/C++ equivalent of [http://search.cpan.org/dist/Panotools-Script/lib/Panotools/Makefile.pm Panotools::Makefile], write lots of tests, identify problem characters on each platform, port Hugin stitching to use this makefile library <del>add filters to filename selection parts of Hugin GUI to prevent use of problem characters</del>.

Further: Hugin already has 'Align' in the [[Hugin Assistant tab]] for creating panorama projects, but for bigger projects it takes some time. Otherwise there is [[Hugin Batch Processor|PTBatcherGUI]] in the hugin package, which can run several 'stitching' tasks in a queue - i.e. we have a system for queueing 'stitching' but not 'aligning'.

* Extend [[Hugin Batch Processor|PTBatcherGUI]] so that also the 'Align' functionality from the assistant (with control point detection, cpclean, celeste, optimisation for position and photometric optimisation) can be added to the queue using Makefiles. Note this has been prototyped with [[ptoanchor]], but a C++ version of this code would need to be written.
* In the next step, allow the user to give a directory or list of photos, and search for all possible panoramas and pass these to PTBatcherGUI for 'Aligning' (maybe with a heuristic approach, based on the EXIF data like [[ panostart]] in combination with [[match-n-shift]]).

==== Vetting exercise ====

Modify Hugin to read and use EXIF Rotation tags in photos [http://sourceforge.net/tracker/?func=detail&aid=2974831&group_id=77506&atid=550444 feature request #2974831 on the Hugin tracker]

=== Regression tests for pano13 ===

libpano13 is the panotools library containing panoramic math, if it was easier to test it might make life easier for those who want to hack it. A regression testing framework that can be configured for different related command-line tools would be even better.

=== Your own project ===
We welcome students to choose their own project, instead of using the above ideas. Discus your project ideas on the [http://groups.google.com/group/hugin-ptx/ hugin-ptx mailing list]. Students that design their own projects tend to do well.

As a vetting exercise, we require you make a patch. Your patch should be relevant to your project. We don't require anything major. The patch should show us some relevant coding skills, and prove you can work with the version control system. Submit your patch to hugin-ptx for peer review.

[[Category:Community:Project]]

SoC 2010 student proposals

2010-03-23T01:38:01Z

James Legg: template proposal

Student info and a short project synopsis, for each project application for Google summer of code 2010. See the template below. Use your user page to describe your project in detail.

== Student name: Project Title ==
* Your name, course, and institution
* A Short description of your coding experience. We mostly want to know about C++ and C. An example of some code you wrote would be nice.
* Mention any other open source, Hugin, or Panotools contibution you have made.
* Do you photograph panoramas? You can link to some examples if they are on the web. Other things you do with Hugin, panotools and friends are interesting too.
* Your main development platform (operating system).
* A Short description of your project.
* A Link to your user page: type <nowiki>[[User:Username]]</nowiki> replacing 'Username' with your username. Put a full description of your project on your user page.
* (When it is available) link to your proposal on the GSoC web app: Type <nowiki>[http://socghop.appspot.com/full_url_here Google Proposal]</nowiki>. The space after the URL is important.

[[Category:Community:Project]]

SoC 2010 ideas

2010-03-22T20:56:59Z

James Legg: /* Vetting exercise */ correct heading level

If you are a student willing to participate in The Google Summer of Code 2010:
* find out what ideas we have for SoC projects this year (read below);
* decide if you want to pick one of those tasks or if you have your own idea;
* join our community at [http://groups.google.com/group/hugin-ptx/ hugin-ptx];
* introduce yourself and tell us about your plans and wishes; and
* add your proposal to the [[SoC_2010_student_proposals | student proposal page]] - see examples from [[SoC_2009_student_proposals | last year]]

'''Important:''' we are admitted to Google Summer of Code 2010 but can not guarantee you a place in the program. We recommend you start preparing your application early as the application process is very competitive.

== Development style ==

Most of the projects below are related to [[Hugin]], and some also relate to [[Panotools]] or [[tlalli]]. [[Hugin]] is mostly written in C++, and uses the [http://kogs-www.informatik.uni-hamburg.de/~koethe/vigra/ VIGRA image processing library] to support different types of images (for example, 8bit, 16bit and float (HDR) images). The core functionality is implemented in a platform independent C++ library, which is used by the GUI based on wxWidgets toolkit, and the command line programs ([[nona]], [[fulla]]). We also very much welcome contributions to [[Enblend]]/[[Enfuse]].

The development of the projects should take place in a separate branch of the project's version control system. Communication with the mentors should usually happen through the appropriate mailing list. All code should work on the major platforms supported (Linux, OSX, Windows).

== Application Template ==

[http://socghop.appspot.com/gsoc/org/show/google/gsoc2010/hugin SoC2010 Application Template]

== Possible Projects ==

You are welcome to propose your own ideas.

Some of the [[SoC2007 projects | SoC2007]], [[SoC_2008_ideas | SoC2008]] and [[SoC_2009_idea | SoC2009]] project proposals were not done in the past years and are potential projects for this year:

* <del>[[SoC2007_projects#Interactive panoramic viewer]] (This was completed but there is further possible work to be done, particularly a joint project with VLC to integrate the viewer in their media player)</del> (VLC integration was completed in 2009 GSoC)
* [[SoC2007_projects#Processing of very large images]] (using the VIPS framework, or even GEGL)
* [[SoC2007_projects#Architectural Overhaul of Panotools]]
* [[SoC_2008_ideas#munin_.E2.80.94_interactive_openGL_based_GUI]]
* [[SoC_2008_ideas#enblend-enfuse_zenith.2Fnadir_and_more]]
* [[SoC_2008_ideas#Lens_Database]] The library part, [[lensfun]], is done, but there is still no system for updating the database. ''Important:'' [http://photoropter.berlios.de Photoropter] is being actively developed and any project should be complementary to that.
* [[SoC_2008_ideas#Extend_hugin.27s_output_options_for_stitching]]
* <del>[[SoC_2008_ideas#Utility_for_creating_a_Philosphere]]</del> Note this isn't enough of a project and probably is better done in mathmap
* [[SoC_2009_idea#Python_Bindings]]
* [[SoC_2009_idea#enblend/enfuse_gimp_plugin]]
* <del>[[SoC_2009_idea#hugin_colour_balancing]]</del> see [[#Colour]] below

=== Zooming for fast preview ===
It would be good if the user can zoom into Hugin's fast preview window. The the amount of approximation in the fast preview would have to reduce to display meaningful details. The areas off screen can be ignored, to keep up performance as more details need to be processed. It would be appropriate to dynamically load more image detail for the most visible images too.

==== Vetting exercise ====
Get the fast preview to scale using openGL transformations. The scale should be changeable, either by keypresses or a gui. Don't worry about scrolling when the panorama is bigger than the window, just show the centre of the panorama.

===Threading for Hugin===
Hugin currently becomes unresponsive while it loads images. It would be better to keep the interface responsive during image loading and scaling images in another thread. Also, something like [http://sourceforge.net/tracker/?func=detail&aid=2872111&group_id=77506&atid=550443 this patch], which loads images when Hugin is otherwise idle, would be better in a background thread.

The user interface could display temporary placeholders while images are being loaded, and remain interactive.

To maximise the rate at which images are loaded, ideally we would have a thread that only reads files and waits for the filesystem, and another thread to uncompress image files and produce the small version of the image which waits only for CPU time. However doing both of these in a single thread, separate from the user interface thread, would provide a responsive interface.

==== Vetting exercise ====
Start a thread that pops up a window every ten seconds showing a
message. Extra bonus: the message should show relevant information
(e.g. the number of processor cores, a thread id).

=== Patent free control point generator ===

We now have a patent free control point generator with libpanomatic, but this needs some integration:

* Ability to read and write .pto projects
* To classify features in a conformal space based on info in the .pto file
* Internal tonemapping of [[HDR]] images to log() space for feature classification
* To not classify features in masked areas
* Test suite
* integration of celeste at feature identification stage
* matching pairs of photos using heuristics (see gigastart)

A possible different project to the above would be to use GPU for feature classification as suggested on ptx, note however that patented techniques such as SIFT and SURF are not suitable for use in Hugin: SIFT GPU http://www.cs.unc.edu/~ccwu/siftgpu/ uses CUDA parallel processing to search for SIFT features in images. I'm not sure if it does the search to find nearest neighbor points. But there is also a GPU accelerated version of that algorithm too. It is a brute force version of the nearest-k points. Since it is done in parallel it is order of magnitudes faster than the ANN algorithm using by autopano-sift-c. http://www-sop.inria.fr/members/Vincent.Garcia/research_knn.php

=== UI testing ===

Update: Summer of Code is strictly coding, so this project isn't possible.

<del>Everyone agrees that Hugin needs usability improvements, however the major usability issues are closely related to programming issues such as the quality and availability of control point generators. We do not want a programmer to dive-in and and try and fix 'usability' without a plan of action.

So Hugin could use a usability audit, i.e. write user profiles/personas, define tasks, collect real data from test subjects. This is a non-programming project for a student of interaction design</del>

=== Colour ===

Hugin deals correctly with colour profiles in photos and passes them on to output, this doesn't need fixing, however there are some related tasks that could be tackled:

* Display of images in tabs and preview is not colour managed, integrate [http://www.littlecms.com/ lcms] and access system monitor colour profiles
* Hugin has a good backend for adjusting white-balance. Add a GUI grey picker and/or tools to adjust colour temperatures manually on a subset of photos to be able to do stuff like this: http://www.flickr.com/photos/sbprzd/4196026736/
* [[EXIF]] metadata contains information on colour balance (WBRedLevel, WBGreenLevel & WBBlueLevel) which should be used to initialise the red/blue colour balance parameters in Hugin - Currently Hugin does something very similar for EV.
* [[Chromatic aberration|tca]] correction in [[nona]] with support in GUI and .pto format, possible simple GUI to run [[tca_correct]]
* [[Vignetting]] of colour balance. For example Tokina 12-24 f4 exhibits this phenomenon (see right side of this [http://farm5.static.flickr.com/4043/4385932566_4e3204a12d_o.jpg image]) in amounts that are easily seen, and according to Ken Rockwell (yeah) the effect plagues most ultra wide rectilinears. Could work similarily to current vignetting correction model, but working on red/blue colour channels? (do we really need this? Is the additional complication in the GUI worth it?)

=== Makefile system and Detection of panoramas ===

Hugin uses gnu ''make'' to drive stitching, this involves writing makefiles and executing ''make'' as a sub-process. We have two issues with this;

* The code that handles makefiles is mixed up with stitching logic, the result is that this part of the codebase is quite hairy and difficult to extend
* <del>'make' places restrictions on characters in file paths but the Hugin GUI doesn't do anything to prevent users from using these characters</del>

Write a C/C++ equivalent of [http://search.cpan.org/dist/Panotools-Script/lib/Panotools/Makefile.pm Panotools::Makefile], write lots of tests, identify problem characters on each platform, port Hugin stitching to use this makefile library <del>add filters to filename selection parts of Hugin GUI to prevent use of problem characters</del>.

Further: Hugin already has 'Align' in the [[Hugin Assistant tab]] for creating panorama projects, but for bigger projects it takes some time. Otherwise there is [[Hugin Batch Processor|PTBatcherGUI]] in the hugin package, which can run several 'stitching' tasks in a queue - i.e. we have a system for queueing 'stitching' but not 'aligning'.

* Extend [[Hugin Batch Processor|PTBatcherGUI]] so that also the 'Align' functionality from the assistant (with control point detection, cpclean, celeste, optimisation for position and photometric optimisation) can be added to the queue using Makefiles. Note this has been prototyped with [[ptoanchor]], but a C++ version of this code would need to be written.
* In the next step, allow the user to give a directory or list of photos, and search for all possible panoramas and pass these to PTBatcherGUI for 'Aligning' (maybe with a heuristic approach, based on the EXIF data like [[ panostart]] in combination with [[match-n-shift]]).

==== Vetting exercise ====

Modify Hugin to read and use EXIF Rotation tags in photos [http://sourceforge.net/tracker/?func=detail&aid=2974831&group_id=77506&atid=550444 feature request #2974831 on the Hugin tracker]

=== Your own project ===
We welcome students to choose their own project, instead of using the above ideas. Discus your project ideas on the [http://groups.google.com/group/hugin-ptx/ hugin-ptx mailing list]. Students that design their own projects tend to do well.

As a vetting exercise, we require you make a patch. Your patch should be relevant to your project. We don't require anything major. The patch should show us some relevant coding skills, and prove you can work with the version control system. Submit your patch to hugin-ptx for peer review.

[[Category:Community:Project]]

SoC 2010 ideas

2010-03-22T16:25:54Z

James Legg: /* Possible Projects */ Add vetting exercises for Zooming for fast preview, Threading for Hugin, and own ideas.

If you are a student willing to participate in The Google Summer of Code 2010:
* find out what ideas we have for SoC projects this year (read below);
* decide if you want to pick one of those tasks or if you have your own idea;
* join our community at [http://groups.google.com/group/hugin-ptx/ hugin-ptx];
* introduce yourself and tell us about your plans and wishes; and
* add your proposal to the [[SoC_2010_student_proposals | student proposal page]] - see examples from [[SoC_2009_student_proposals | last year]]

'''Important:''' we are admitted to Google Summer of Code 2010 but can not guarantee you a place in the program. We recommend you start preparing your application early as the application process is very competitive.

== Development style ==

Most of the projects below are related to [[Hugin]], and some also relate to [[Panotools]] or [[tlalli]]. [[Hugin]] is mostly written in C++, and uses the [http://kogs-www.informatik.uni-hamburg.de/~koethe/vigra/ VIGRA image processing library] to support different types of images (for example, 8bit, 16bit and float (HDR) images). The core functionality is implemented in a platform independent C++ library, which is used by the GUI based on wxWidgets toolkit, and the command line programs ([[nona]], [[fulla]]). We also very much welcome contributions to [[Enblend]]/[[Enfuse]].

The development of the projects should take place in a separate branch of the project's version control system. Communication with the mentors should usually happen through the appropriate mailing list. All code should work on the major platforms supported (Linux, OSX, Windows).

== Application Template ==

[http://socghop.appspot.com/gsoc/org/show/google/gsoc2010/hugin SoC2010 Application Template]

== Possible Projects ==

You are welcome to propose your own ideas.

Some of the [[SoC2007 projects | SoC2007]], [[SoC_2008_ideas | SoC2008]] and [[SoC_2009_idea | SoC2009]] project proposals were not done in the past years and are potential projects for this year:

* <del>[[SoC2007_projects#Interactive panoramic viewer]] (This was completed but there is further possible work to be done, particularly a joint project with VLC to integrate the viewer in their media player)</del> (VLC integration was completed in 2009 GSoC)
* [[SoC2007_projects#Processing of very large images]] (using the VIPS framework, or even GEGL)
* [[SoC2007_projects#Architectural Overhaul of Panotools]]
* [[SoC_2008_ideas#munin_.E2.80.94_interactive_openGL_based_GUI]]
* [[SoC_2008_ideas#enblend-enfuse_zenith.2Fnadir_and_more]]
* [[SoC_2008_ideas#Lens_Database]] The library part, [[lensfun]], is done, but there is still no system for updating the database. ''Important:'' [http://photoropter.berlios.de Photoropter] is being actively developed and any project should be complementary to that.
* [[SoC_2008_ideas#Extend_hugin.27s_output_options_for_stitching]]
* <del>[[SoC_2008_ideas#Utility_for_creating_a_Philosphere]]</del> Note this isn't enough of a project and probably is better done in mathmap
* [[SoC_2009_idea#Python_Bindings]]
* [[SoC_2009_idea#enblend/enfuse_gimp_plugin]]
* <del>[[SoC_2009_idea#hugin_colour_balancing]]</del> see [[#Colour]] below

=== Zooming for fast preview ===
It would be good if the user can zoom into Hugin's fast preview window. The the amount of approximation in the fast preview would have to reduce to display meaningful details. The areas off screen can be ignored, to keep up performance as more details need to be processed. It would be appropriate to dynamically load more image detail for the most visible images too.

==== Vetting exercise ====
Get the fast preview to scale using openGL transformations. The scale should be changeable, either by keypresses or a gui. Don't worry about scrolling when the panorama is bigger than the window, just show the centre of the panorama.

===Threading for Hugin===
Hugin currently becomes unresponsive while it loads images. It would be better to keep the interface responsive during image loading and scaling images in another thread. Also, something like [http://sourceforge.net/tracker/?func=detail&aid=2872111&group_id=77506&atid=550443 this patch], which loads images when Hugin is otherwise idle, would be better in a background thread.

The user interface could display temporary placeholders while images are being loaded, and remain interactive.

To maximise the rate at which images are loaded, ideally we would have a thread that only reads files and waits for the filesystem, and another thread to uncompress image files and produce the small version of the image which waits only for CPU time. However doing both of these in a single thread, separate from the user interface thread, would provide a responsive interface.

==== Vetting exercise ====
Start a thread that pops up a window every ten seconds showing a
message. Extra bonus: the message should show relevant information
(e.g. the number of processor cores, a thread id).

=== Patent free control point generator ===

We now have a patent free control point generator with libpanomatic, but this needs some integration:

* Ability to read and write .pto projects
* To classify features in a conformal space based on info in the .pto file
* Internal tonemapping of [[HDR]] images to log() space for feature classification
* To not classify features in masked areas
* Test suite
* integration of celeste at feature identification stage
* matching pairs of photos using heuristics (see gigastart)

A possible different project to the above would be to use GPU for feature classification as suggested on ptx, note however that patented techniques such as SIFT and SURF are not suitable for use in Hugin: SIFT GPU http://www.cs.unc.edu/~ccwu/siftgpu/ uses CUDA parallel processing to search for SIFT features in images. I'm not sure if it does the search to find nearest neighbor points. But there is also a GPU accelerated version of that algorithm too. It is a brute force version of the nearest-k points. Since it is done in parallel it is order of magnitudes faster than the ANN algorithm using by autopano-sift-c. http://www-sop.inria.fr/members/Vincent.Garcia/research_knn.php

=== UI testing ===

Update: Summer of Code is strictly coding, so this project isn't possible.

<del>Everyone agrees that Hugin needs usability improvements, however the major usability issues are closely related to programming issues such as the quality and availability of control point generators. We do not want a programmer to dive-in and and try and fix 'usability' without a plan of action.

So Hugin could use a usability audit, i.e. write user profiles/personas, define tasks, collect real data from test subjects. This is a non-programming project for a student of interaction design</del>

=== Colour ===

Hugin deals correctly with colour profiles in photos and passes them on to output, this doesn't need fixing, however there are some related tasks that could be tackled:

* Display of images in tabs and preview is not colour managed, integrate [http://www.littlecms.com/ lcms] and access system monitor colour profiles
* Hugin has a good backend for adjusting white-balance. Add a GUI grey picker and/or tools to adjust colour temperatures manually on a subset of photos to be able to do stuff like this: http://www.flickr.com/photos/sbprzd/4196026736/
* [[EXIF]] metadata contains information on colour balance (WBRedLevel, WBGreenLevel & WBBlueLevel) which should be used to initialise the red/blue colour balance parameters in Hugin - Currently Hugin does something very similar for EV.
* [[Chromatic aberration|tca]] correction in [[nona]] with support in GUI and .pto format, possible simple GUI to run [[tca_correct]]
* [[Vignetting]] of colour balance. For example Tokina 12-24 f4 exhibits this phenomenon (see right side of this [http://farm5.static.flickr.com/4043/4385932566_4e3204a12d_o.jpg image]) in amounts that are easily seen, and according to Ken Rockwell (yeah) the effect plagues most ultra wide rectilinears. Could work similarily to current vignetting correction model, but working on red/blue colour channels? (do we really need this? Is the additional complication in the GUI worth it?)

=== Makefile system and Detection of panoramas ===

Hugin uses gnu ''make'' to drive stitching, this involves writing makefiles and executing ''make'' as a sub-process. We have two issues with this;

* The code that handles makefiles is mixed up with stitching logic, the result is that this part of the codebase is quite hairy and difficult to extend
* <del>'make' places restrictions on characters in file paths but the Hugin GUI doesn't do anything to prevent users from using these characters</del>

Write a C/C++ equivalent of [http://search.cpan.org/dist/Panotools-Script/lib/Panotools/Makefile.pm Panotools::Makefile], write lots of tests, identify problem characters on each platform, port Hugin stitching to use this makefile library <del>add filters to filename selection parts of Hugin GUI to prevent use of problem characters</del>.

Further: Hugin already has 'Align' in the [[Hugin Assistant tab]] for creating panorama projects, but for bigger projects it takes some time. Otherwise there is [[Hugin Batch Processor|PTBatcherGUI]] in the hugin package, which can run several 'stitching' tasks in a queue - i.e. we have a system for queueing 'stitching' but not 'aligning'.

* Extend [[Hugin Batch Processor|PTBatcherGUI]] so that also the 'Align' functionality from the assistant (with control point detection, cpclean, celeste, optimisation for position and photometric optimisation) can be added to the queue using Makefiles. Note this has been prototyped with [[ptoanchor]], but a C++ version of this code would need to be written.
* In the next step, allow the user to give a directory or list of photos, and search for all possible panoramas and pass these to PTBatcherGUI for 'Aligning' (maybe with a heuristic approach, based on the EXIF data like [[ panostart]] in combination with [[match-n-shift]]).

=== Your own project ===
We welcome students to choose their own project, instead of using the above ideas. Discus your project ideas on the [http://groups.google.com/group/hugin-ptx/ hugin-ptx mailing list]. Students that design their own projects tend to do well.

As a vetting exercise, we require you make a patch. Your patch should be relevant to your project. We don't require anything major. The patch should show us some relevant coding skills, and prove you can work with the version control system. Submit your patch to hugin-ptx for peer review.

[[Category:Community:Project]]

SoC 2010 ideas

2010-02-24T18:42:26Z

James Legg: /* Possible Projects */ Fix links

SoC 2010 ideas

2010-02-24T18:01:17Z

James Legg: Copy some things over from last year, and suggest a couple of projects.

Hugin Compiling Ubuntu

2010-01-02T20:07:06Z

James Legg: /* Build and Install */ add ldconfig to instructions

These instructions are work in progress and updated when newer versions of Ubuntu are released or when Hugin introduces new dependencies. They have worked, at the time of release or soon thereafter, with the following Ubuntu/Kubuntu versions:
* Ubuntu 9.10: on AMD64 computer
* Ubuntu 9.04: on AMD64 computer
* Ubuntu 8.10: on AMD64 computer
* Ubuntu 8.04: on an Intel processor
* Ubuntu 7.10: on AMD Athlon XP
* Ubuntu 7.04: on AMD64 computer
* Ubuntu 6.06: on AMD64 computer

They are likely to work only for the latest one, but you can check this page's history at the time that the older Ubuntu version was the most recent one to find the specifics for that version.

Apart from the odd change in package name, nothing should be substantially different (and if does not work, please leave a comment on the [http://groups.google.com/group/hugin-ptx hugin-ptx mailing list]). Don't worry if the same package appears twice in an apt-get install line - apt-get will update existing packages if there is a newer version, and ignore duplicates if the latest version is already installed. On the other hand, if apt-get says that it can't find a package, it might be the odd change in package name. You can find a replacement package by using apt-cache search with a substring of the package required, e.g.
<pre>apt-cache search wxW
</pre>

The goal is to build hugin and the whole set of helper applications required.

== Building environment ==
Since we are going to build hugin, libpano13 and enblend we need to download and install all the development packages. This is very easy with apt-get.
In a terminal window (K menu -> System -> Konsole or Applications -> Accessories -> Terminal (in Kubuntu), Applications -> Accessories -> Terminal (in Ubuntu))

<pre>sudo apt-get install build-essential autoconf automake1.9 libtool flex bison gdb
</pre>

Unless you have an amd64 system prior to 7.10, add the following packages.
<pre>sudo apt-get install libc6-dev libgcc1
</pre>

For older amd64 environments, the packages have a slightly different name:
<pre>sudo apt-get install libc6-dev-amd64 lib64gcc1
</pre>

To get the bleeding edge we'll need access to the SVN and Mercurial repositories, and for this we need the correct tools:

<pre>
sudo apt-get install subversion mercurial
</pre>

Not necessary, but useful if you want to move to the next level and build packages for distribution (list is incomplete):

<pre>
sudo apt-get install subversion-tools
</pre>

== Building Enblend ==

Get the dependencies. If you are working with large images (300 megapixels and up), you should have a libtiff-devel compiled with large file support and libstdc++6.

<pre>sudo apt-get install pkg-config libtiff4-dev libboost-graph-dev libboost-thread-dev \
liblcms1-dev libglew1.5-dev libplot-dev libglut3-dev libopenexr-dev libxi-dev libxmu-dev \
help2man texi2html texinfo fig2ps tidy
</pre>

For Ubuntu systems prior to 8.10, you will also need

<pre>sudo apt-get install libopenexr2ldbl
</pre>

Once all dependencies are in place, get the code. The official repository has migrated from CVS to Mercurial:

<pre>
hg clone http://enblend.hg.sourceforge.net:8000/hgroot/enblend/enblend enblend
cd enblend
</pre>

If you already have checked out enblend, to update the code to the latest version you need to run:
<pre>
hg pull
hg update
</pre>

We are then ready to compile. The CXXFLAGS <tt>--param inline-unit-growth=60</tt> is a workaround for x86_64 systems, Feb 2009, and apparently no longer needed (set by default). If you have an SMP machine and you're building a recent version, <tt>--disable-image-cache</tt> and <tt>--enable-openmp</tt> will speed up enblend/enfuse massively but may cause segmentation faults; in that case you will be better off with <tt>--enable-image-cache</tt> and <tt>--disable-openmp.</tt> <tt>-march=native</tt> is also beneficial and -O2 is better than -O3.

<pre>
make --makefile=Makefile.scm
mkdir BUILD
cd BUILD
CXXFLAGS="--param inline-unit-growth=60 -march=native -O2" ../configure --disable-image-cache --enable-openmp
make
</pre>

The second make step can be very long and memory consuming. Make sure you have enough swap space, and go get a healthy snack while the computer is compiling.

If you encounter problems at any of these stages, please report back to the hugin-ptx mailing list. Report what command in the sequence you were executing, what machine/operating system, the revision checked out from CVS, and all other relevant information.

You are then ready to install with
<pre>
sudo make install
</pre>

== Building Libpano13 ==

libpano13 is the new version of the PanoTools libraries. This is a necessary component for hugin, and we need to build it first.
To build libpano13 we need some libraries and particularly their <code>dev</code> package:

<pre>sudo apt-get install zlib1g zlib1g-dev libpng12-dev libjpeg62-dev libtiff4-dev</pre>

On older distributions, zlib1g and zlib1g-dev may not be found. In that case, the same library may be available with the names lib64z1 and lib64z1-dev.

We then need to download the source code from SVN:

<pre>
svn co https://panotools.svn.sourceforge.net/svnroot/panotools/trunk/libpano libpano13
cd libpano13
</pre>

In the future there is no need to get the whole source again. Just issue the following commands to bring your source up to date:
<pre>
cd libpano13
svn up
</pre>

Now start the building process.

<pre>
cd ..
mkdir build.libpano
cd build.libpano
cmake ../libpano13 -DCMAKE_INSTALL_PREFIX=/usr/local -DCPACK_BINARY_DEB:BOOL=ON -DCPACK_BINARY_NSIS:BOOL=OFF \
-DCPACK_BINARY_RPM:BOOL=OFF -DCPACK_BINARY_STGZ:BOOL=OFF -DCPACK_BINARY_TBZ2:BOOL=OFF \
-DCPACK_BINARY_TGZ:BOOL=OFF -DCPACK_BINARY_TZ:BOOL=OFF
make package
sudo dpkg -i libpano13-*-Linux.deb
</pre>

=== autotools ===

The above described building process uses CMake to build libpano which has many advantages. Amongst others it creates a clean package to install / deinstall and decreases the likelihood of wrecking the system. However CMake support is relatively new in libpano. The old autotools way is documented below for completness. You should not need it. However if you do, make sure to run either the CMake build or the autotools build on fresh SVN checkouts to avoid interferences.

<pre>
./bootstrap
</pre>

If any libraries are missing, the script will complain (or at least, let you know that some library hasn't been found). In that case you probably need to install the library. To find in what package is that library, a general rule is to run the command <code>apt-cache search ''missingfile''</code>, find the relevant library and install both the library and the related <code>-dev</code> package.
Run the <code>./configure</code> script and repeat this process until you have met all the dependencies.

Then we are ready to launch the make process with
<pre>make</pre>

If the library successfully compiles, you have to install it with
<pre>sudo make install
sudo ldconfig</pre>
The last part is for the OS to be aware of the new library (that has been installed in <code>/usr/local/lib</code>).
We can now go back up one folder level and get ready for hugin.
<pre>cd ..</pre>

== Building Hugin ==

=== Dependencies ===

First we need to activate the <code>universe</code> repository (in adept package manager, or by editing the /etc/apt/sources.lst file for example) and get the dependencies:

<pre>
sudo apt-get install cmake libopenexr-dev libboost-dev boost-build libboost-thread-dev libboost-graph-dev \
gettext libwxgtk2.8-dev libexiv2-dev libimage-exiftool-perl libglew-dev liblapack-dev
</pre>

=== Fetch the Source Code from SVN ===

Next we download Hugin from SVN. But which Hugin? There are many [http://svnbook.red-bean.com/en/1.1/ch04.html#svn-ch-4-sect-1 branches] in SVN. Also SVN keeps track of every single change (revision) in time, so we can download a branch in its state at a specific point in time. For example <code>-r 4008 https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/trunk/</code> will download the trunk branch at the point in time when revision [http://hugin.svn.sourceforge.net/viewvc/hugin?view=rev&revision=4008 4008] was committed, which happens to be Tue Jul 7 22:07:54 2009 UTC. It also happens to be the same as the 0.8.0 release. For a list of branches and revisions that build well, refer to these [http://wiki.panotools.org/Development_of_Open_Source_tools#Specific_revisions lists]. There is no guarantee that a particular branch and/or revision builds and/or performs as expected.

<pre>
svn co -r 4061 https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/tags/hugin-2009.07.1/ hugin
</pre>

If you know what you are doing, you can replace /hugin-2009.07.1/ with another branch; and you can omit the -r and get the last revision of that branch.

In the future there is no need to get the whole source again. Just issue the following commands to bring your source up to date:
<pre>
cd hugin
svn up
</pre>

=== Set the Build Environment ===

Next we set up the build environment using cmake.

If you compiled and installed libpano to a non standard location set the variables CMAKE_INCLUDE_PATH and CMAKE_LIBRARY_PATH to point to your install location's include and lib directories.

If you are building for distribution, you want to set CMAKE_INSTALL_PREFIX=/usr

<pre>
mkdir hugin-build
cd hugin-build
cmake ../hugin -DENABLE_LAPACK=YES -DCPACK_BINARY_DEB:BOOL=ON -DCPACK_BINARY_NSIS:BOOL=OFF \
-DCPACK_BINARY_RPM:BOOL=OFF -DCPACK_BINARY_STGZ:BOOL=OFF -DCPACK_BINARY_TBZ2:BOOL=OFF \
-DCPACK_BINARY_TGZ:BOOL=OFF -DCPACK_BINARY_TZ:BOOL=OFF
</pre>

=== Build and Install ===

Finally, we use make to build the code and package it, and dpkg to install it. Look for the version number in the .deb file created and edit the lines below accordingly:

<pre>
make package
sudo dpkg -i hugin-*-Linux.deb
sudo ldconfig
</pre>

'''Important:''' the package does not track dependencies yet, so it likely to fail on machines others than yours.

Reference: [http://theseblog.free.fr/2007/10/building-hugin-in-ubuntu-feisty-fawn.php]

== Autopano-sift-C ==

Pre-Requisites:
<pre>
sudo apt-get install libxml2-dev
</pre>

To build
<pre>
svn co https://hugin.svn.sourceforge.net/svnroot/hugin/autopano-sift-C/trunk/ autopano-sift-C
mkdir autopano-sift-C.build
cd autopano-sift-C.build
cmake ../autopano-sift-C -DCMAKE_INSTALL_PREFIX=/usr/local -DCPACK_BINARY_DEB:BOOL=ON -DCPACK_BINARY_NSIS:BOOL=OFF \
-DCPACK_BINARY_RPM:BOOL=OFF -DCPACK_BINARY_STGZ:BOOL=OFF -DCPACK_BINARY_TBZ2:BOOL=OFF \
-DCPACK_BINARY_TGZ:BOOL=OFF -DCPACK_BINARY_TZ:BOOL=OFF
make package
sudo dpkg -i autopano-sift-C-*-Linux.deb
</pre>

== Match-n-shift ==

Match-n-shift is yet an other Autopano-SIFT replacement. It comes in a bundle with pto file manipulation perl libraries and a rich selection of other tools that use nona, enblend, ImageMagick among other things. To use match-n-shift, you need to install at least the [http://search.cpan.org/dist/Panotools-Script/ Panotools::Script] library and some other perl modules. It requires:

* Image::Size 2.9
* Storable 2.0
* Image::ExifTool 6

Chances are that you do not need to update Storable which is a standard perl module. To check the version of installed versions of these modules, write these lines to the command prompt:

perl -MStorable -le 'print Storable->VERSION;'
perl -MImage::Size -le 'print Image::Size->VERSION;'
perl -MImage::ExifTool -le 'print Image::ExifTool->VERSION;'

There are various ways to install or upgrade these modules. Ubuntu has binary packages for many perl modules, you can skip the CPAN installation below and type:

sudo apt-get install libimage-size-perl libimage-exiftool-perl

Alternatively, perl is an interpreted language and these modules are pure perl, so installing from source is an easy thing.

The best place to install perl modules is directly from a CPAN archive. CPAN is short for Comprehensive Perl Archive Network. The program to interact with CPAN comes with all perl installations. Start it by running:

sudo cpan

If this is your first time running the program you will be asked a number of questions (you can safely accept the suggested values). Last question lets you select a number of CPAN mirrors nearest to you.

When all is done, you can enter the install commands after the 'cpan>' prompt:

install Image::Size Storable Image::ExifTool

Answer 'y' to any question of dependencies and wait for install to complete. 'exit' leaves the cpan shell.

Next, you need to retrieve and install the Panotools-Script collection
of perl libraries (Panotools::Script) and programs that use them -
including match-n-shift:

svn co https://panotools.svn.sourceforge.net/svnroot/panotools/trunk/Panotools-Script Panotools-Script
cd Panotools-Script
perl Makefile.PL
make
make test
sudo make install

Just to confuse you, you can also ignore everything above and just install Panotools::Script and all its dependencies from CPAN. From command line, type:

sudo cpan Panotools::Script

You can run match-n-shift from command line using the same parameters as autopano-c-complete.sh or you can modify hugin to run it for you:

* start Hugin
* navigate the menu File to Preferences
* In the Preferences window, open the Autopano tab
* Select Autopano -> Autopano-SIFT
* tick the checkbox for Use alternative Autopano-SIFT program
* enter the full path to match-n-shift (/usr/local/bin/match-n-shift) in the Autopano-SIFT field
* enter the following string in the Arguments field:
-f %f -v %v -c -p %p -o %o %i

== MatchPoint ==

MatchPoint is a next generation CP generator. The result of a GSoC2007 project, it is still very experimental. Experience reports needed. Read more [http://groups.google.com/group/hugin-ptx/browse_thread/thread/cba2b2ce94dd9054 here]

Matchpoint is now located inside the main hugin source tree, no need to checkout https://hugin.svn.sourceforge.net/svnroot/hugin/gsoc07_featuredetection anymore. It is compiled together with hugin, but not installed by default.

* copy the MatchPoint executable into /usr/local/bin manually
<pre>
sudo cp src/matchpoint/matchpoint /usr/local/bin/
</pre>
* edit /usr/local/bin/autopano-c-complete.sh (do it with sudo to have the necessary permission)
** line 79: replace the .key.gz extension with .key (not sure if matchpoint supports compression)
** lines 83 and 88: replace '''generatekeys "$arg" $FILENAME $SIZE''' with '''matchpoint "$arg" $FILENAME''' (not sure if the size option or any other options of the original generatekeys are applicable)

== Pan-o-matic ==

Yet another control point generator [http://aorlinsk2.free.fr/panomatic/?p=home Home Page]

<pre>sudo apt-get install libboost-dev</pre>

Download [http://aorlinsk2.free.fr/panomatic/download.php?d=7&v=0.9.4 Panomatic 0.9.4 bz2]

<pre>
tar xvfj panomatic-0.9.4-src.tar.bz2
cd panomatic-0.9.4
./configure
make
sudo make install
</pre>

To use Pano-o-matic, open Hugin

* In File > Preferences > Autopano :
* Select Autopano-SIFT
* Check "Use alternative autopaon-SIFT program"
* Choose the path to the binary ( /usr/local/bin )
* In Arguments, put : -o %o %i

== Exiftool ==

ExifTool is a platform-independent Perl library plus a command-line application for reading, writing and editing meta information in image, audio and video files.

<pre>
sudo apt-get purge libimage-exiftool-perl
</pre>

<pre>
wget http://www.sno.phy.queensu.ca/~phil/exiftool/Image-ExifTool-7.94.tar.gz
tar -xzf Image-ExifTool-7.94.tar.gz
cd Image-ExifTool-7.94
perl Makefile.PL
make test
sudo make install
</pre>

== Panoglview ==

PanoGLView is an OpenGL hardware accelerated interactive immersive viewer for equirectangular images.

Pre-Requisites:
<pre>sudo apt-get install wx-common</pre>

To build
<pre>
svn co https://hugin.svn.sourceforge.net/svnroot/hugin/panoglview/trunk panoglview
cd panoglview
./bootstrap
./configure
make
sudo make install
</pre>

== FreePV ==

Pre-Requisites:
<pre>
sudo apt-get install cmake make pkg-config g++ mozilla-dev freeglut-dev zlib1g-dev libjpeg-dev libxext-dev libxmu-dev \
x11proto-xf86vidmode-dev libxxf86vm-dev libnspr4-dev libxml2-dev libpng12-dev
</pre>
To build
<pre>
svn co https://freepv.svn.sourceforge.net/svnroot/freepv/freepv/trunk/ freepv
cd freepv
cmake .
make
sudo make install
</pre>

== Notes for Packagers ==
* before releasing a tarball, upgrade the Changelog with svn2cl
<pre>
$ svn up
$ svn2cl
$ svn ci
</pre>

[[Category:Software:Hugin]]
[[Category:Software:Platform:Linux]]

SoC 2009 project Layout Model

2009-04-27T02:22:52Z

James Legg: Added page to project category

=Google Summer of Code 2009: Layout Model project=
This project will make Hugin aware of what sort of layout the input images of a panorama take.
The layout model would consist of information about bracketing, the number of images that make up each row, what direction the camera the moves between pictures, and so on.

There will be:
# an automatic process to guess the layout model after control point generation and automatic alignment.
# a user interface to define the layout model.
# features that use knowledge of the layout to improve user experience:
#* Optionally image alignment keeps the stacks intact, treating each stack as a set of images with the same position.
#* The output options will reflect the recommended stitching processes for the given images.
#* The makefiles will be able to process stacks, and rows of images more correctly.
#* The images used to make the previews can be filtered by bracket.
#* Places where control points are present and shouldn't be, or not present and should be, can be pointed out to the user. Information about pairs of images can be represented in a table, and also graphically.

[[Category:Community:Project]]

SoC 2009 project Layout Model

2009-04-27T02:17:38Z

James Legg: Explain project goals

User:James Legg

2009-04-02T22:33:04Z

James Legg: Added alternative projects.

This is the Google Summer of Code 2009 project proposal page for James Legg.
In the 2008 Summer of Code I wrote the [[Hugin Fast Preview window]]

==Enfuse / Enblend Gimp Plugins==

I would like to create Enfuse / Enblend Gimp plugins so that people can blend image layers with minimal visible seams directly in the GNU Image manipulation program.

Previously, a user would have to export each layer as a separate file and blend them on the command line. After the project's completion, a user should be able to have the same functionality directly in the Gimp. This would have the advantages of speed of use, as it removes several steps for the user; the ability to use Enblend and Enfuse in Gimp scripts; and the ability to consider blending like any other operation in the Gimp, with features such as undo, redo, and repeat last effect. A user interface will be created that allows the user to pick the options that are currently command line arguments.
To demonstrate the plugins, I will create scripts that:
#'enfuses' exposure stacks
#'enfuses' focus stacks
#makes any texture a seamless image, similar the "Make seamless" plugin, however it will use Enblend to reduce visible seams.

If time allows, I could also create a Gimp plugins to do the job of nona and align_image_stack (with layered, unblended output, as the Gimp does not support high dynamic range images). Then I would create scripts for aligning and blending focus stacks or exposure stacks directly in the Gimp.

This project is based on [[SoC 2009 idea#enblend.2Fenfuse_gimp_plugin|this project suggestion]]

I have picked this idea as it would be useful for panoramic photography and seamless texture creation, both of which I do myself. I use Hugin and the Gimp, and such a plugin will be very useful for me as well as others.

I will work full time for the majority of Summer of Code. However, my final exams are at the beginning of June, so I intend to be revising for them, instead of coding, for the first three weeks. I will also need to take a couple of days off for my graduation and a family event.

This is a rough estimation as to how I spend the time:

* Weeks 1-3: revision (not coding).
* Week 4: Create the user interface for the Enblend and Enfuse plugins.
* Week 5: Write the part of the plugins that gets the required image information (e.g. image format, sizes of layers) from the Gimp, rather than a file.
* Weeks 6-10: Modify Enblend so that the source of the image data can be from the Gimp, and not just a file, and the result can be written back to the Gimp rather than written to a file.
* Weeks 11-13: Testing, bug fixing, and enhancements.

==Alternative suggestions==
Here are other projects I would be happy to work on:
===Bracketing Panorama Model===
I will add row, column, and bracket fields to the images.

They should be set automatically after alignment. However, if there is any confusion, the user may set them manually: either individually on the images tab, or in a new layout tab. This tab will allow the user to specify something like "I was shooting with 3 bracketed images per direction, my rows span a full 360 degrees, I have 7 rows spanning 180 degrees. They are 3 rows of 14 positions, 2 rows of 6, and a single angle zenith and nadir row. Oh, and I shot in that direction twice since someone walked across the shot, so don't count those!". There is a possibility of taking the intended positions as they are given and showing a fast preview so the user is aware their settings are working quickly.

Alignment can then be done optionally per stack, or with the stacks intact as required. Also the rows can grouped together if wanted.

The previews will be modified so that there is an option to show only images in one bracket without manually picking the numbers.

The makefiles will be modified to merge the brackets based on this information, and to enblend each row together, then enblend the rows as a separate step if wanted. The interface to control what is given as output can be altered depending on the bracketing information. For example, if only focal distance changes between images in the stacks, "Create focus blended panorama" will be the most prominent option. If exposure varies greatly between images in the stacks "Create high dynamic range panorama" and "Create exposure blended panorama" will be the most prominent options. If there is only 1 image per stack and similar settings across the panorama, "Create low dynamic range panorama" will be the most prominent.

===GLSL shaders in the Fast Preview===
For a computer to support the OpenGL Shading Language (GLSL), the graphics hardware must conform to OpenGL 2.0 or greater, or support the GL_ARB_fragment_program and GL_ARB_vertex_program extensions. Shaders can be written which are compiled by the graphics driver, and run on graphics hardware. They are highly parallisable, as the code is run independently on each vertex in an object, or each pixel in the output depending on the type of shader. We can specify values in Hugin to be passed to shader per vertex, or for the entire scene.

This would be ideal for photometric correction, which is currently poor in the fast preview. We can also support high dynamic range images, and write code to map hdr information to a ldr image for display.

Hugin could check if the OpenGL shading language is supported, and if it is offer more features and replace some functionality.

I propose to enable HDR display, an entire panorama difference mode, and real-time full photometric correction in the fast preview for hardware supporting the OpenGL shading language. I will also write a remapper that uses a GLSL vertex program to perform the remapping, and compare it with the non-GLSL originals. I do not have particularly "high end" graphics hardware to test it on, though I have a new machine and it supports GLSL, so the performance may vary widely to systems other than my own. (My graphics chipset is an Intel 965GM.) An option will be present in the preferences to use this remapper if supported.

Unlike the nona GPU branch, the shaders that remap positions should not be compiled for every image, as this would be slow during updates. Instead, modifiable parameters will be passed to the shader from Hugin. Once a shader is compiled, it can be reused with different parameters.

User:James Legg

2009-04-02T20:58:20Z

James Legg: Wrote Gimp plugins project proposal

SoC 2009 student proposals

2009-04-02T12:53:38Z

James Legg: Add James Legg's project

Student Proposals: Student info and short project synopsis, with link to a new Wiki page where the project is expanded in full detail. See template below:

== Tom Templeton: The Template Project ==

* Enrolled second year master of Sample at Example University in Nowhereland.
* Coding Platform: Ubuntu 8.10, Pentium 4 3GHz, 1GB RAM
* The [[SoC2009_Tom_Templeton | Template Project]] is all about a short description of an example project that does what it does according to specifications.

== Lukáš Jirkovský ==

* first year undergrad of University of West Bohemia
* Platform used for coding: Arch Linux, Pentium 4 Northwood 2GHz, 1.5 GB RAM
* Coding skills: C++, PHP, Java (Java only because I have to ;-) )
* [[SoC2009_Lukas_Jirkovsky | Ghost removal for enfuse]]
* http://socghop.appspot.com/student_proposal/show/google/gsoc2009/stativ/t123797116006 Project proposal in Google's webapp

== [[User:Leonox|León Moctezuma]]: QuickTimeVR Playback in VLC ==

* Enrolled last Semester Bachelor of Computer Science at [www.buap.mx Benemérita Universidad Autónoma de Puebla].
* Coding Platform: Ubuntu 8.10
* [http://wiki.videolan.org/SoC_2009/QuickTimeVR_Playback Project proposal] on the VideoLAN Wiki
* Mentor (VLC): Most likely Antoine Cellier.
* Comentor (FreePV): Yuval Levy. Note: we're discussing this with VLC / Jean-Baptiste Kempf, including Wiimote navigation.
* http://socghop.appspot.com/student_proposal/review/google/gsoc2009/leonox/t123831053541 Google webapp proposal

== Dev Ghosh ==

* PhD candidate, Electrical Engineering and Computer Science, Northwestern University, Evanston, Illinois
* Coding Platform: Ubuntu 8.10, AMD Athlon 64 3500+, 2 GB RAM, Eclipse
* Coding skills: C/C++, MATLAB, Python
* Possible Mentor: Daniel German
* [[SoC2009_Dev_Ghosh | mosaic mode for Hugin]]
* Google webapp Proposal Link: Coming soon!

== [[User:Albiorix|Yulia Kotseruba]]: Straight-line detection for automated lens calibration ==

* 4th year Computer Science student at University of Toronto studying Artificial Intelligence
* Coding Platform: Mac OS 10.5.6, 2.53 GHz Intel Core 2 Duo, 4 GB RAM, XCode, Eclipse
* Coding skills: C/C++, Java, Python, Matlab, OpenGL, Prolog
* [[SoC2009_Yulia_Kotseruba | Straight-line detection for automated lens calibration]]
* http://socghop.appspot.com/student_proposal/show/google/gsoc2009/albiorix/t123854228105 Google proposal

== Mokhtar M. Khorshid: Accounting for Camera Movements ==

* Holder of B.S. in Computer Science & Mathematics from AUC and starting my master studies in Computer Science.
* Coding Platform: Vista 64-bit, Quad Core 2.4 GHz, 6GB RAM, 8800 GTS graphics card, Visual Studio 2005/2008 .NET.
* Coding skills: Expert in C++ (10+ years), seasoned algorithm designer, I have experience with Computer graphics, OpenGL, and wxWidgets.
* [[SoC2009_Mokhtar_Khorshid | Accounting for Camera Movements]]

== Tim Nugent ==

* 3rd Year PhD student in Bioinformatics at University College London, UK
* Ubuntu and Centos Linux, with XP imprisoned in Vmware
* Coding skills: C/C++, Perl
* [http://socghop.appspot.com/student_proposal/show/google/gsoc2009/ultrawide/t123859124187 Straight-line detection for automated lens calibration]
* [http://socghop.appspot.com/student_proposal/show/google/gsoc2009/ultrawide/t123859431227 Bracketing Panorama Model]

== Seth Berrier ==
(Sorry for the last minute addition. Hope I can find room among this qualified bunch!)

* 6th Year PhD student & Doctoral Candidate in Computer Science at University of Minnestoa, USA
* Main Coding Platform: Mac OS X 10.5 w/ Apple GNU Toolchain & Eclipse IDE, 15" MacBook Pro, 2.4 GHz Core 2 Duo, 4G mem, NVIDIA GeForce 8600M GT
* Other Platforms:
** Win XP Pro w/ Visual C++ & VS .net, 2k5 or 2k8 OR MinGW & Eclipse IDE
** Other *nix platforms (Ubuntu & Solaris in particular) using basic GNU toolchain from shell
* Computer Science Knowledge:
** Veteran in C++ (with a bit of C#, Java, Perl, LISP, StandardML, Pascal, Fortran, Basic, VB for Apps, HTML, Javascript)
** Very experienced in OpenGL, GLUT, Cg and general graphics algorithms
** Basic knowledge of Qt GUI programming & Qt Eclipse integration
** Experienced with Matlab and some LabVIEW
** Good Numerical Methods foundation with some practical experience
** Basic, non-theoretical experience in real-time image processing for computer vision (Hough transform, Canny edge detection, etc)
* [[SoC2009_Seth_Berrier | Proposal Coming Soon]]

== James Legg: Enblend / Enfuse Gimp plugin ==

* I'm in the third and final year of a Computer Science and Mathmatics BSc at the University of York, in the United Kingdom.
* Coding Platform: Ubuntu 8.10, Pentium dual-core 2GHz, 2GB RAM
* The [[SoC2009_James_Legg | Enfuse / Enblend Gimp plugin]] will allow a user of the Gimp to use the Enblend or Enfuse algorithms to merge layers of an image through the Gimp's menu or using a Gimp script.

Hugin Fast Preview window

2008-08-23T18:02:59Z

James Legg: Began article, Includes explanation of the buttons and a guide

This page refers to features found in a branch of [[Hugin]], and is not in any stable branches or the main trunk. For more information on this project, see [[SoC 2008 Project OpenGL Preview]].

Like the more accurate [[Hugin Preview window]], the fast preview shows something similar to the final stitched output, but with a few important differences:
* Reduced resolution input images are used, so some areas can appear blurred that will be sharp in the final output.
* Seams are not created, images are simply overlaid with the first image at the bottom of the stack and the last at the top.
* Blending by a tool such as [[enblend]] isn't shown.
* The brightness display of [[HDR]] and 16bit images is controlled by settings in the hugin Preferences, these settings are not used when stitching. The colouring of these images will also be inaccurate when using exposure or white balance correction. For HDR panoramas, the [[Hugin Preview window]] is recommended instead.
* Photometric correction only includes white balance and exposure, unless full photometric correction is enabled with the the Photometrics button.
* The remappings are approximate, the output by a tool such as [[nona]] is more accurate. If this concerns you more than speed, use the [[Hugin Preview window]] instead.
* It's much faster ;-)

==Buttons==
Any button appearing in both preview windows works the same, see [[Hugin Preview window]] for how to use them. The buttons specific to the fast preview are explained below.

===[[Image:Fast_preview_icon_photometric.svg]] Photometrics===
Enables full photometric correction. When turned on, this will cause significant delay when changing photometric parameters. It also takes a while to turn on and off. However, with it enabled you get much better representation of the colours in the output. With it turned off, you get correction only for exposure and white balance. With it turned on, you also get vignetting and colour response correction. The [[Hugin Preview window]] does all these things by default, so you may wish to use that instead.

===[[Image:Fast_preview_icon_drag.svg]] Drag===
Using this tool you can recentre the panorama interactively. With it turned on, try the following:
* Drag the panorama with the left mouse button to rotate the panorama's images. The centre of rotation is the point where you pushed the mouse button down.
* Hold shift when doing the above to constrain movement to yaw or pitch. Note pitch is affected by the centre of rotation.
* Drag the panorama with the right mouse button or hold control and drag with the left to roll the panorama (rotate around the middle)

If the panorama contains unconnected components, they will move individually.

Press the button again to turn it off.

===[[Image:Fast_preview_icon_crop.svg]] Crop===
Using this tool you can set the output cropping region interactively. To do this precisely instead, use the [[Hugin Stitcher tab| Stitcher tab]]. Initially, the entire panorama is in the output region (i.e. nothing is cropped).

To change the cropping at each edge, move the mouse towards that edge until a white box appears along it, then drag with the left mouse button until the edge is where you want it. The darker areas represent the region that is cropped off. You can move two edges at once by moving the mouse towards the corner shared by the edges until both white boxes appear. If you wish to move the whole region at once, move the mouse into the middle so that all four edges have boxes along them and drag.

Press the button again to turn it off.

===[[Image:Fast_preview_icon_identify.svg]] Identify===
Using this tool you can find where your images are, and match them to their number. You can also edit control points.

When this tool is turned on, move the mouse over the visibility buttons for the images (the numbers at the top of the preview). The image with the number on the button under the mouse lights up red in the preview. Moving the mouse over the panorama highlights all the images under the mouse in different colours. The buttons for those images lights up in matching colours.

When the mouse is on the overlap of two images, click to edit the control points between those images.

==Blend modes==
The ''normal'' blend mode will draw the images as a stack. The ''difference'' blend mode will do the same, except the image under the mouse pointer will be subtracted from the rest of the stack. Use this to determine if the alignment went well: where you can see edges in the subtracted image, these edges are misaligned. Be warned that this isn't fully accurate, the other preview has a better difference mode.

==In practice==
[[Image:Fast_preview_guide_start.png | thumb | A panorama with unconnected image groups after optimisation.]]
Let's try using this preview to help with a panorama where automatic alignment failed. This panorama was taken where a lot of things were blowing around in the wind, and the clouds were changing quickly, so it is not surprising that it aligning it is a struggle. The [[Hugin Assistant tab|Assistant tab]] tells us there are multiple unconnected image groups. We can optimise the panorama and end up with a images correctly positioned amongst the group they are in, but the groups themselves are not aligned. Try this first.

[[Image:Fast_preview_guide_drag.png | thumb | Dragging images with the drag tool.]]
Using the Drag tool, we can roughly align the groups together:
# Turn on the tool with the ''Drag'' button.
# Drag each component so the horizon is in the middle, using the left mouse button.
# Drag with the right mouse button (or hold control and drag with the left) to level the horizon.[[Image:Fast_preview_guide_rotate_drag.png | thumb | Rotating images with the drag tool.]]
# Hold shift and drag with the left mouse button sideways to approximately line up the image with the other groups.

[[Image:Fast_preview_guide_identify.png | thumb | Two images highlighted in the preview window.]]
When we have the images in approximately the right position, we can begin placing control points to guide the optimiser. The Identify tool lends a hand here. Firstly, turn on the identify tool. Move the mouse on an overlap that was recently created between two image groups. The images in the overlap light up. Move the mouse to a place where there is only two images in that overlap, and click. The two images are opened in the [[Hugin Control Points tab|control point editor]] (there may be a short pause while the images are loaded). Once you have placed control points manually, you can return to the preview to find some more image pairs.

[[Image:Fast_preview_guide_crop.png | thumb | Adjusting the cropping region.]]
When you are happy that your panorama contains sufficient control points, optimise it again. The panorama will likely have the horizon at the wrong angle, in this case press ''Straighten'' on the preview window. You can then frame the panorama using the drag tool (hold shift so you don't make it wonky again!). Use your artistic judgement here. If you want to crop your panorama, click ''Crop'' and drag the edges of the cropping rectangle.

[[Category:Software:Hugin]]
[[Category:Tutorial:Nice to know]]

File:Fast preview icon drag.svg

2008-08-23T16:20:09Z

James Legg: The icon used by Hugin's fast preview to enable dragging images across the panorama.

The icon used by Hugin's fast preview to enable dragging images across the panorama.

File:Fast preview icon crop.svg

2008-08-23T16:18:48Z

James Legg: The icon used by Hugin's fast preview to enable panorama cropping.

The icon used by Hugin's fast preview to enable panorama cropping.

File:Fast preview icon identify.svg

2008-08-23T16:17:20Z

James Legg: The icon used by Hugin's fast preview for turning on image identification.

The icon used by Hugin's fast preview for turning on image identification.

File:Fast preview icon photometric.svg

2008-08-23T16:14:59Z

James Legg: The icon used by Hugin's fast preview for enabling full photometric correction.

The icon used by Hugin's fast preview for enabling full photometric correction.

File:Fast preview guide start.png

2008-08-23T16:12:37Z

James Legg: Hugin's fast preview, the result of James Legg's Google Summer of Code 2008 project, mentored by Pablo d'Angelo. It is is displaying a panorama where automatic alignment has failed, leaving several misaligned components.

Hugin's fast preview, the result of James Legg's Google Summer of Code 2008 project, mentored by Pablo d'Angelo. It is is displaying a panorama where automatic alignment has failed, leaving several misaligned components.

File:Fast preview guide rotate drag.png

2008-08-23T16:10:16Z

James Legg: Hugin's fast preview, the result of James Legg's Google Summer of Code 2008 project, mentored by Pablo d'Angelo, being used to rotate an image component in a panorama.

Hugin's fast preview, the result of James Legg's Google Summer of Code 2008 project, mentored by Pablo d'Angelo, being used to rotate an image component in a panorama.

File:Fast preview guide identify.png

2008-08-23T16:07:10Z

James Legg: Hugin's fast preview, the result of James Legg's Google Summer of Code 2008 project, mentored by Pablo d'Angelo, being used to identify two overlapping images.

Hugin's fast preview, the result of James Legg's Google Summer of Code 2008 project, mentored by Pablo d'Angelo, being used to identify two overlapping images.

File:Fast preview guide drag.png

2008-08-23T16:04:16Z

James Legg: Hugin's fast preview, the result of James Legg's Google Summer of Code 2008 project, mentored by Pablo d'Angelo, being used to move an image component across a panorama.

Hugin's fast preview, the result of James Legg's Google Summer of Code 2008 project, mentored by Pablo d'Angelo, being used to move an image component across a panorama.

File:Fast preview guide crop.png

2008-08-23T16:00:52Z

James Legg: Hugin's fast preview, the result of James Legg's Google Summer of Code 2008 project, mentored by Pablo d'Angelo, being used to crop a panorama.

Hugin's fast preview, the result of James Legg's Google Summer of Code 2008 project, mentored by Pablo d'Angelo, being used to crop a panorama.

Writing Hugin PreviewTools

2008-08-23T15:42:44Z

James Legg: Began article

For the purpose of [[Hugin]]'s [[SoC 2008 Project OpenGL Preview|OpenGL approximate preview]], a tool is something that does something from the following list:
* Draws something, using OpenGL calls, under or above the images in the preview.
* Changes how images are drawn in the preview.
* Responds to the user clicking or dragging in the preview.
This page introduces writing these tools.

==The <tt>PreviewTool</tt> class==
To make a new tool, first derive an object from this class. There are a few virtual functions to pay attention to, so read the following:
===Constructor===
This is a base class from which all tools must derive from. You must call the constructor in the base class, <tt>PreviewTool::PreviewTool(PreviewToolHelper *helper)</tt>, but you can do anything else you want in your constructor. If you want to create OpenGL Display lists or textures you may do it here.

===<tt>Activate</tt>===
You must override the <tt>Activate</tt> function. This is called when your tool is turned on by the <tt>GLPreviewFrame</tt>. Here you must use the <tt>PreviewToolHelper</tt> object your tool is linked to to request notifications of any events (your tool cannot do anything unless an event occurs). If you need to initialise any values, do it here. If your tool is turned off by the user your tool will get no notification until Activate is called again. Register events by using something like
<source lang="cpp">
helper->NotifyMe(PreviewToolHelper::MOUSE_MOVE, this);
</source>

Have a look at PreviewToolHelper.h for more information about events. For an example, look at one of the existing <tt>PreviewTool</tt>s, such as <tt>PreviewCropTool</tt>.

===The Event Functions===
Each event has a corresponding virtual function in <tt>PreviewTool</tt>. When an event occurs, this function is called. So override any of these functions for events that you request notification of. You can request more event notifications in these functions, but be aware that if they thieve something from another tool (e.g. mouse clicks) the other tool will be disabled, but no notification to the <tt>GLPreviewFrame</tt> will be made, so the button for it on the user interface may remain in the on position. For the drawing functions you can draw what you want, the ViewState object can provide you with access the textures and meshes used to draw the images (see [[Hugin's OpenGL preview system overview]] to see how it works). Use <tt>helper->GetViewStatePtr()</tt> to get a pointer to the VeiwState used for the preview. Read ViewState.h to see how to use it.

==OpenGL State==
If you use OpenGL functions in your tool, here are some simple rules to follow to ensure everything works:
* When you are done, set the colour to (1.0, 1.0, 1.0, 1.0).
* You can leave the blending function set to whatever you want, but leave blending turned off.
* 2D Texturing will be turned on when your functions are called, please leave it on when you are done.
* Use glGenLists and glGenTextures to get a new display list number or texture name, as any of them may already be used.
* Generally the rest of the system expects something near the default OpenGL state.

==A few notes on the <tt>ViewState</tt> object==
The view state handles what has changed in the preview's representation of the panorama. It redraws the preview automatically when the panorama has changed. However you probably don't want to change the panorama during interactive things, as each change must have a corresponding undo level. You can therefore change the information used to draw the preview using the <tt>ViewState</tt> directly. When you have changed everything you need, call the <tt>ViewState</tt>'s <tt>Redraw()</tt> function. At this point the <tt>ViewState</tt> will redraw the preview if it notices something has changed. However, if the representation of the panorama doesn't change, no drawing will take place. Therefore if you want to display something different yourself, you must call <tt>ForceRequireRedraw()</tt> before <tt>Redraw()</tt>.

==Integrating with <tt>GLPreviewFrame</tt>==
Firstly you will want to make a pointer to the type of your tool in the declaration of <tt>GLPreviewFrame</tt>. Leave it with the others for clarity. The class type should be forward declared at the top of the file to save on the <tt>#include</tt>s.

In GLPreviewFrame.cpp you will want to add a <tt>#include</tt> to the header file your tool is defined in. Next you will want to create an instance of it in <tt>GLPreviewFrame::MakeTools</tt>, using your pointer. The creation of the tools is delayed so they can expect an OpenGL context in their constructor (the context is made by the <tt>GLViewer</tt>, which is inside the <tt>GLPreviewFrame</tt>). You will then want to free your tool in <tt>GLPreviewFrame::~GLPreviewFrame</tt>, providing <tt>MakeTools</tt> was called. It isn't called if the user closes Hugin without opening the OpenGL preview frame, no OpenGL context is made until the window becomes visible.

The next step is getting your tool activated when you want.
If your tool provides some useful service continuously, you can activate it when <tt>MakeTools</tt> is called. However make sure it can never be automatically turned off in this case: it cannot draw under or over individual images or respond to mouse clicks, but it can use every other event including drawing under or over all of the panorama. You can add buttons in a similar style to the buttons already there. Remember to disable the buttons for any tools your tool may switch off on activation, use something like
<source lang="cpp">
TurnOffTools(helper->ActivateTool(my_tool));
</source>
Also add your tool to <tt>TurnOffTools</tt>, in case another tool does the same to yours. You may need to redraw the preview when your tool is activated or turned off, if so use <tt>m_GLViewer->Refresh();</tt> afterwards.

Finally, add any new source files you made in the corresponding CMakeLists.txt.
After that, you can compile it, give it some testing and off you go. If you've made a useful tool, be sure to show it off on the [http://groups.google.com/group/hugin-ptx Hugin mailing list].

==See Also==
[[Hugin's OpenGL preview system overview]]
[[Category:Software:Hugin]]
[[Category:Tutorial:Other]]

Hugin's OpenGL preview system overview

2008-08-23T14:49:31Z

James Legg: Began article

This page describes features in an experimental branch of [[Hugin]], it is not in the stable versions, or the main trunk (yet). The information on this page may be outdated quickly. This page gives an overview of how the [[SoC 2008 Project OpenGL Preview]] works, for interested developers.

The OpenGL preview's window is a GLPreviewFrame class, similar to the PreviewFrame that the [[Hugin Preview window|accurate preview]] uses. This class contains all the GUI controls in the window. The preview itself is a GLViewer widget, which is derived from wxWidget's wxGLCanvas.

There is a class called ViewState that is handles finding what has changed in the panorama, so that it can perform the necessary steps to update the preview. It updates automatically when the panorama changes, but changes to the preview can be made directly using the ViewState object, so that the preview is updated while the panorama is not. This is used for interactive updates, such as when adjusting the field of view using the sliders.

==Displaying remapped images==
The images in the preview are drawn using two components: the image, with or without photometric correction; and a set of faces that approximate the remapping performed.
===Images===
The actual image content is managed by a TextureManager. It creates an OpenGL texture containing a scaled down version of the original image, in 8 bits per colour channel per pixel. The texture is then either a 24 bit RGB, or a 32 bit RGBA texture depending on the presence of a mask. With real time photometric correction, the original image has the same colours as the texture (unless it is an HDR image). With full photometric correction, photometric correction is performed on the texture before it is passed to OpenGL.

===Remapping===
The remapping is approximated by a set of quadrilaterals. Each vertex of each quadrilateral has vertex coordinates (where it appears in the output) and texture coordinates (where in the texture that represents the image it should be map to). Using this information, the OpenGL renderer can display faces that show parts if the input image. The mapping of remapped images to image numbers is handled by a MeshManager object, but the transformations are performed by child classes of MeshRemapper. The MeshManager creates an OpenGL display list that contains the instructions for drawing the faces.

The MeshManager only uses ChoosyRemapper, a child of MeshRemapper. This picks between the other two MeshRemappers: VertexCoordRemapper and TexCoordRemapper.
The VertexCoordRemapper transforms texture coordinates to the corresponding vertex coordinates. It uses an adaptive subdivision method: first the outer corners are transformed, then the single face that is produced is divided into four smaller faces, by splitting each edge exactly in half. The extra coordinates are then mapped to their correct positions. The process repeats on these smaller faces until the geometric difference created when splitting the face becomes negligible, or the face is too small to be worth doing anything with, or the face is significantly outside the panorama. To avoid certain situations where subdividing a face produces little extra detail but subdividing it more shows some, there is a minimum amount of subdivisions that occur. There is also a maximum subdivision level to limit memory usage. With the cylindrical projection types, an image that crosses the +/- 180° seam is not approximated well by this remapping. There will be faces that stretch across the width of the panorama, where the image actually goes off one end and comes on the other. To correct for this discontinuity in the projection, the remapper tries to find the stretched faces, and changes them into two faces that go off the edges instead of one that goes across the middle.

The TexCoordRemapper works the other way around, it specifies vertex coordinates in a grid over the panorama, and then projects these into the image coordinates which it uses for the texture coordinates. The faces are then clipped so they only contain the desired part of the image, and nothing outside of it. This projection is preferable where a point is stretched into a line or disk, such as the poles of an equirectangular projection, or the outside ring of a disk-like projection. In these circumstances, the VertexCoordRemapper would only see one point of the ring or line, so it does not perform so well. However the TexCoordRemapper does not in general handle curved images as neatly as the VertexCoordRemapper, and the TexCoordRemapper samples much of the panorama outside of the image, so the TexCoordRemapper sees limited use.

The ChoosyRemapper gets VertexCoordRemapper or a TexCoordRemapper to do all the remapping work, but it decides which one to use by detecting when an image crosses a pole or similar.

The textures and images are brought together to draw the preview. The actual drawing is managed by a GLRenderer object.

==Interactive Tools==
The interactive tools in the preview derive from the PreviewTool class. PreviewTools can change how the preview is drawn and respond to the user. There is a PreviewHelper object that each tool has access to, which provides common information to the tools and manages the events they use. An instance of each tool is created by the GLPreviewPanel, after GLViewer has created an OpenGL context. This is important as tools may wish to create OpenGL textures when they are created. The GLPreviewPanel panel then uses the PreviewToolHelper to "Activate" a tool when the user hits the button. The helper calls the Activate function of the tool, the tool requests any events it needs notification of, and the helper disables any other tool that needs notification only one tool should have (such as mouse clicks) and returns the list of disabled tools back to the GLPreviewPanel, which updates the preview window's buttons to reflect this. The GLViewer tells the helper when some wxWidgets events occur, it processes these as necessary (for example, converting mouse coordinates to panorama coordinates, finding if the images under the mouse have changed) and tells any tool that has registered for notification of any occurring events by calling the corresponding virtual function of PreviewTool.

==See also==
[[SoC 2008 Project OpenGL Preview]]

[[Writing Hugin PreviewTools]]

[[Category:Software:Hugin]]
[[Category:How it works]]

SoC 2008 Project OpenGL Preview

2008-08-21T17:46:27Z

James Legg: Update since GSOC is complete. Links to soon-to-be new pages added.

[[Hugin]]'s OpenGL Preview is a fast but approximate previewer, which is more interactive than the previous one. It was James Legg's [[SoC 2008 overview|Google Summer of Code 2008]] project, mentored by Pablo d'Angelo.

==Image Transformations==
The new previewer will have to approximate the [[remapping]] of the images to fit on low resolution meshes. There are two methods it uses for this:

===Remapping Vertices===
[[Image:Blend_project_mesh.jpg|thumb|Example of using vertices of a mesh to remap an image]]

Taking a point in the input image, we can find where the point is transformed to on the final panorama. Therefore, we can take a uniform grid over an input image, and create a mesh that shows remaps this grid into the output projection.

'''Disadvantages''':
*This method suffers when the input image crosses over the +/-180° boundary in projections such as [[Equirectangular Projection]], as there will be faces with vertices on either side of the seam. In this case the previewer replaces the face with two faces, one for each side.
*The poles of an equirectangular image will also cause similar problems. At a pole, the image should cover the whole width of the panorama, but this is only one point in the input image. Vertices in the mesh near the pole are define a face that does not cover the whole row. Also the outer rings of disk like projections are poorly done.
*The detail in the faces might not match up well with the area of the panorama. The faces are all of different sizes and some mappings may put the details in parts that can't be seen as easily as the lower resolution parts.

'''Advantages''':
*The mesh resembles the output projection well, and covers up roughly the same area as the correct projection. The edges of the mesh lies along the edges of the image, so we don't need to worry about what happens outside of it (except with circular cropping). Most warped images can be displayed reasonably well using this method.

'''Implementation''':
This is implemented in the VertexCoordRemapper object. It is used for most images.

===Remapping Texture Coordinates===
[[Image:Blend_project_UV.jpg|thumb|Example of using the texture coordinates of a mesh to remap an image. The internal repetitions are caused by wrapping the texture in the vertical direction; there should be hole there, but the mesh goes through regardless.]]
Alternatively, given a point on the final panorama, we can calculate what part of an input image belongs there. Therefore, we can take a uniform grid over the panorama, and map the input image across it.

'''Disadvantages''':
*This mapping breaks down at the borders of the input image. The borders of the input image may cross the output mesh at any position. We clip the faces to the input image to avoid this problem.
*This is inefficient for small images, as they do not cover much of the panorama. Many faces are considered outside of the input image and dropped during clipping.
*The quality of the shape of curved images is a little lower than with the vertex coordinate remapping.

'''Advantages''':
*There are no problems at the poles and edges of the panorama.
*The output is of uniform quality throughout, as the faces always have the same density.

'''Implementation''':
This is implemented in the TexCoordRemapper object. It is used for images that cross the poles in many cylinderical type projections, or the outer ring of disk like projections (e.g. fisheye) also uses this. It is also used in Alber's equal area conic projection since the +/- 180° boundary correction is difficult to implement, and the poles need it anyway.

===Distortions Comparison===
To check that these transformation methods do not distort the image too much, I devised a test.
Using [[Nona]] to produce a coordinate transformation map from input image coordinates to output image coordinates, and similarly creating a projection that goes the other way, I tried these transformations in Blender.

I projected a [[Cylindrical Projection|cylindrical]] panorama, 360° wide, and with a pixel resolution 4 times as wide as it is tall, into a [[Fisheye Projection|fisheye]] image pointing at the lower pole of the input image, 270° wide, and square.

{| border="1"
|+'''Comparison of Image Transformation Approximations'''
|-valign="top"
|[[Image:Blender_mesh_transform_output.jpg|left|256px|Mesh transformation]]
|[[Image:Blender_transform_Nona_comparison.jpg|left|256px|Transformation using Nona]]
|[[Image:Blender_UV_transform_output.jpg|left|256px|Texture Coordinate transformation]]
|-valign="top"
|'''Vertex mapping Approximate Transformation'''

This mesh has 32 columns of faces, and 8 rows, across the input image. Since the image is 4 times as wide as it is tall, this means there are 256 faces which cover an equal amount of the input image.
|'''Transformation with Nona'''

This is the remapping that the other images are approximating.
|'''Texture Coordinate mapping Approximate Transformation'''

This mesh has 16 rows of faces and 16 columns, spread equally over the output panorama, so there are 256 faces which cover an equal amount of the output.

Note that for these images I have not attempted to correct the texture coordinates around the boundary of the input image.
|}

Another test performed was mapping a rectilinear image to the zenith of an equirectangular image.
{| border="1"
|[[Image:Blender_zenith_mesh_transform.png]]

'''Vertex mapping Approximate Transformation'''

This is the transformation performed by arranging vertices that cover a uniform grid along the input image. The mesh was 32 faces wide and 16 faces tall, and covered an input image that was 4:3. I removed a row of faces that went right across the image, connecting the left edge to the right edge. This was there since the left and right edge represent the same line through the input image, and the mapping ignores the discontinuity across that edge. If I use this method, I will have to draw the faces of that row twice, once for each side, using some extra vertices somewhere off the sides of the output, so that the mapping is continuous and the corners are not missing.

|-
|[[Image:Blender_zenith_transform_nona_output.png]]

'''Transformation with Nona'''

This is the mapping the other images are approximating.

|-
|[[Image:Blender_zenith_tex_coord_transform.png]]

'''Texture Coordinate mapping Approximate Transformation'''

This is the transformation performed by finding the points on the input image that are at the points on a grid over the output image. The grid was 64 faces wide and 8 tall. Two of the edges are bad because I couldn't get the negative coordinates (they were clamped to 0), the other two are reasonable because the grid points mapped to the correct places, even though some of them were outside of the input image. If I use this method, I should be able to get the negative coordinates easily and therefore the poles will work without much effort.

|}

==Modules==
See main article: [[Hugin's OpenGL preview system overview]]

==Building and Usage==
The OpenGL preview is not in the main Hugin branch yet. If you wish to experiment, and don't mind compiling it yourself, you can check out the latest version using svn with:
<source lang="bash">
svn co https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/branches/gsoc2008_opengl_preview hugin
</source>
For help compiling, you might want to read [[Hugin Compiling OSX]] or [[Hugin Compiling Windows]] depending on you operating system. The preview adds extra dependencies, firstly you'll need [http://glew.sourceforge.net glew], and also the wxGLCanvas object in the [http://www.wxwidgets.org wxWidgets] library (which is not compiled by default on Windows).

See [[Hugin Fast Preview window]] for usage instructions.

[[Category:Community:Project]]

SoC 2008 Project OpenGL Preview

2008-06-30T00:11:27Z

James Legg: /* Modules */ Updated to reflect current state.

The OpenGL Preview project aims to create a fast previewer for [[Hugin]], reducing the time taken to redraw the preview to real time rates. After that, the new previewer can be made more interactive than the current one. The project is James Legg's [[SoC 2008 overview|Google Summer of Code 2008]] project, mentored by Pablo d'Angelo.

==Image Transformations==
The new previewer will have to approximate the [[remapping]] of the images to fit on low resolution meshes. There are two methods for this:

===Remapping Vertices===
[[Image:Blend_project_mesh.jpg|thumb|Example of using vertices of a mesh to remap an image]]

Taking a point in the input image, we can find where the point is transformed to on the final panorama. Therefore, we can take a uniform grid over an input image, and create a mesh that shows remaps this grid into the output projection.

'''Disadvantages''':
*This method suffers when the input image crosses over the +/-180° boundary in projections such as [[Equirectangular Projection]], as there will be faces connecting one end to the other, and nothing between the edges of those faces and the actual boundary. We must split the mesh so that it each part is continuous, which means some faces will have to be defined off the edge of the panorama.
*The poles of an equirectangular image will also cause similar problems. At a pole, the image should cover the whole width of the panorama, but this is only one point in the input image. Vertices in the mesh near the pole are define a face that does not cover the whole row.
*The detail in the faces might not match up well with the area of the panorama. The faces are all of different sizes and some mappings may put the details in parts that can't be seen as easily as the lower resolution parts.

'''Advantages''':
*The mesh resembles the output projection well, and covers up roughly the same area as the correct projection. The edges of the mesh lies along the edges of the image, so we don't need to worry about what happens outside of it.

===Remapping Texture Coordinates===
[[Image:Blend_project_UV.jpg|thumb|Example of using the texture coordinates of a mesh to remap an image. The internal repetitions are caused by wrapping the texture in the vertical direction; there should be hole there, but the mesh goes through regardless.]]
Alternatively, given a point on the final panorama, we can calculate what part of an input image belongs there. Therefore, we can take a uniform grid over the panorama, and map the input image across it.

'''Disadvantages''':
*This mapping breaks down at the borders of the input image. The borders of the input image may cross the output mesh at any position, so we need to draw the image with a transparent border.
*We must also calculate the extent of the images in the result so the mesh only covers the minimal area.
*Some faces of the minimal bounding rectangle still don't contain any of the input image at all, processing them would be a waste of time.

'''Advantages''':
*There are no problems at the poles and edges of the panorama.
*The output is of uniform quality throughout, as the faces always have the same density.

===Distortions Comparison===
To check that these transformation methods do not distort the image too much, I devised a test.
Using [[Nona]] to produce a coordinate transformation map from input image coordinates to output image coordinates, and similarly creating a projection that goes the other way, I tried these transformations in Blender.

I projected a [[Cylindrical Projection|cylindrical]] panorama, 360° wide, and with a pixel resolution 4 times as wide as it is tall, into a [[Fisheye Projection|fisheye]] image pointing at the lower pole of the input image, 270° wide, and square.

{| border="1"
|+'''Comparison of Image Transformation Approximations'''
|-valign="top"
|[[Image:Blender_mesh_transform_output.jpg|left|256px|Mesh transformation]]
|[[Image:Blender_transform_Nona_comparison.jpg|left|256px|Transformation using Nona]]
|[[Image:Blender_UV_transform_output.jpg|left|256px|Texture Coordinate transformation]]
|-valign="top"
|'''Vertex mapping Approximate Transformation'''

This mesh has 32 columns of faces, and 8 rows, across the input image. Since the image is 4 times as wide as it is tall, this means there are 256 faces which cover an equal amount of the input image.
|'''Transformation with Nona'''

This is the remapping that the other images are approximating.
|'''Texture Coordinate mapping Approximate Transformation'''

This mesh has 16 rows of faces and 16 columns, spread equally over the output panorama, so there are 256 faces which cover an equal amount of the output.

Note that at this point I have not attempted to correct the texture coordinates around the boundary of the input image.
|}

Another test performed was mapping a rectilinear image to the zenith of an equirectangular image.
{| border="1"
|[[Image:Blender_zenith_mesh_transform.png]]

'''Vertex mapping Approximate Transformation'''

This is the transformation performed by arranging vertices that cover a uniform grid along the input image. The mesh was 32 faces wide and 16 faces tall, and covered an input image that was 4:3. I removed a row of faces that went right across the image, connecting the left edge to the right edge. This was there since the left and right edge represent the same line through the input image, and the mapping ignores the discontinuity across that edge. If I use this method, I will have to draw the faces of that row twice, once for each side, using some extra vertices somewhere off the sides of the output, so that the mapping is continuous and the corners are not missing.

|-
|[[Image:Blender_zenith_transform_nona_output.png]]

'''Transformation with Nona'''

This is the mapping the other images are approximating.

|-
|[[Image:Blender_zenith_tex_coord_transform.png]]

'''Texture Coordinate mapping Approximate Transformation'''

This is the transformation performed by finding the points on the input image that are at the points on a grid over the output image. The grid was 64 faces wide and 8 tall. Two of the edges are bad because I couldn't get the negative coordinates (they were clamped to 0), the other two are reasonable because the grid points mapped to the correct places, even though some of them were outside of the input image. If I use this method, I should be able to get the negative coordinates easily and therefore the poles will work without much effort.

|}

==Modules==
As there are many good ideas for the future of a graphically-accelerated Hugin after this project is complete (See the [http://groups.google.com/group/hugin-ptx/browse_thread/thread/7c838b0cda383922 discussion on the mailing list]), it is clear that a lot of development can occur using OpenGL and the new preview after I have completed this project. Therefore it is especially important that I provide clean, highly modular and extensible code for future use. Here is how the modules and their interfaces look:

; Texture Manager
: The texture manager is built on top of Hugin's ImageCache, and will perform a similar function but using graphics memory and textures. It is able to set the active texture to one that represents an image. It adjusts the size of the textures it is storing so they fill a sensible amount of memory and provide a even level of detail for the images. This is partially complete, it works fine with real time photometric adjustment (only corrects exposure and white balance) and, photometric correction is in progress, and a logarithmic display for high dynamic range images may be yet to come.

; Mesh Manager
: The mesh manager sets up lists of instructions for the graphics system to draw a given image, remapped how it should be in the panorama's output. It uses a mesh remapper for every image to find what faces it needs. This has been implemented.

; Mesh Remapper
: This provides the coordinates of points in a mesh. It splits the image into faces that approximate the transformation using the remapping with vertices method above. However, the rectangles of the input image that have their corners transformed to make the face vertices are sized adaptively: in the areas where the remapping is almost linear there are less faces. Also the size of the faces is limited when they become too small to be noticeable. This is partially complete.

; Renderer
: This is responsible for drawing the panorama, although it only really compiles a list of render layers and draws them when asked to. Currently it just draws all the active images, as the render layers are not implemented yet.

; Render Layers
: Everything the renderer draws would have been put there by a render layer. As the name suggests, they are ordered and can cover each other up. At the end of the project there will obviously be a layer to draw the selected images. Other layers provide indication of image outlines, or perhaps redraw an image in a specific way. Not implemented yet.

; Image Render Hooks
: When drawing, we may want to do something different for a selection of the images. A render hook can be set up that changes how an image is rendered. It would provide callbacks for before and after drawing, a condition to drop an image from rendering altogether, or a replacement function for drawing it. An example would be to provide different blending modes. We could subtract an image from the ones behind it by dropping it from the images render layer, and then drawing it again in a different render layer, but subtracting it from the layers beneath. Alternatively, if we wanted something drawn in place but semitransparent, we could turn on blending before it is drawn and turn it off afterwards. These are not implemented yet.

; Viewer
: The viewer combines together a renderer and a some basic interface for the tools. It tells the renderer when to render, but just before it does so it lets the mesh and texture managers update. It also stores the view state object. It will be able convert between a mouse position and a position in the panorama. It is implemented as a wxWidget.

; Input Manager
: An input manager gets all keyboard and mouse events the viewer sees and passes them to the correct tool. Tools register events they want. When tools are disabled they should give up their events, which can then be taken by a newly activated tool. Not implemented yet

; Tools
: Tools can be enabled or disabled. When switching states they must set up or turn off events in the input manager, render layers, and image render hooks. They get access to the panorama data and viewer data. None are implemented yet. A image drag / recentre tool, and a cropping tool is planned. Other tools could be built to work with control points and selection of images for example.

; GLPreviewPanel
: This is a replacement PreviewPanel. It holds the viewer and gives it some screen space. It shows controls for the viewer and tools. It will keep an input manager to decide what to do with user input events. Some tools may be mutually exclusive, for example there might be a few that want to handle mouse events. The preview panel should turn off one tool before starting another in this case.

; View State
: As the user changes properties of the panorama, we want something to accept the modifications in real time to control what is drawn. This state should differ from the state of the panorama so the user can cancel their changes and make visible changes during dragging things with the mouse (without creating an undo step for each mouse motion). The ViewState object is what is used to hold the temporary state. It also updates to reflect the other conventional modifications. Finally, this can report what type of changes have been made to the panorama and therefore deduce what expensive calculations need to be done to get the preview up to date, and which things we previously calculated are still valid. It is implemented now.

[[Category:Community:Project]]

SoC 2008 Project OpenGL Preview

2008-05-20T23:48:53Z

James Legg: Added modules suggestion

The OpenGL Preview project aims to create a fast previewer for [[Hugin]], reducing the time taken to redraw the preview to real time rates. After that, the new previewer can be made more interactive than the current one. The project is James Legg's [[SoC 2008 overview|Google Summer of Code 2008]] project, mentored by Pablo d'Angelo.

==Image Transformations==
The new previewer will have to approximate the [[remapping]] of the images to fit on low resolution meshes. There are two methods for this:

===Remapping Vertices===
[[Image:Blend_project_mesh.jpg|thumb|Example of using vertices of a mesh to remap an image]]

Taking a point in the input image, we can find where the point is transformed to on the final panorama. Therefore, we can take a uniform grid over an input image, and create a mesh that shows remaps this grid into the output projection.

'''Disadvantages''':
*This method suffers when the input image crosses over the +/-180° boundary in projections such as [[Equirectangular Projection]], as there will be faces connecting one end to the other, and nothing between the edges of those faces and the actual boundary. We must split the mesh so that it each part is continuous, which means some faces will have to be defined off the edge of the panorama.
*The poles of an equirectangular image will also cause similar problems. At a pole, the image should cover the whole width of the panorama, but this is only one point in the input image. Vertices in the mesh near the pole are define a face that does not cover the whole row.
*The detail in the faces might not match up well with the area of the panorama. The faces are all of different sizes and some mappings may put the details in parts that can't be seen as easily as the lower resolution parts.

'''Advantages''':
*The mesh resembles the output projection well, and covers up roughly the same area as the correct projection. The edges of the mesh lies along the edges of the image, so we don't need to worry about what happens outside of it.

===Remapping Texture Coordinates===
[[Image:Blend_project_UV.jpg|thumb|Example of using the texture coordinates of a mesh to remap an image. The internal repetitions are caused by wrapping the texture in the vertical direction; there should be hole there, but the mesh goes through regardless.]]
Alternatively, given a point on the final panorama, we can calculate what part of an input image belongs there. Therefore, we can take a uniform grid over the panorama, and map the input image across it.

'''Disadvantages''':
*This mapping breaks down at the borders of the input image. The borders of the input image may cross the output mesh at any position, so we need to draw the image with a transparent border.
*We must also calculate the extent of the images in the result so the mesh only covers the minimal area.
*Some faces of the minimal bounding rectangle still don't contain any of the input image at all, processing them would be a waste of time.

'''Advantages''':
*There are no problems at the poles and edges of the panorama.
*The output is of uniform quality throughout, as the faces always have the same density.

===Distortions Comparison===
To check that these transformation methods do not distort the image too much, I devised a test.
Using [[Nona]] to produce a coordinate transformation map from input image coordinates to output image coordinates, and similarly creating a projection that goes the other way, I tried these transformations in Blender.

I projected a [[Cylindrical Projection|cylindrical]] panorama, 360° wide, and with a pixel resolution 4 times as wide as it is tall, into a [[Fisheye Projection|fisheye]] image pointing at the lower pole of the input image, 270° wide, and square.

{| border="1"
|+'''Comparison of Image Transformation Approximations'''
|-valign="top"
|[[Image:Blender_mesh_transform_output.jpg|left|256px|Mesh transformation]]
|[[Image:Blender_transform_Nona_comparison.jpg|left|256px|Transformation using Nona]]
|[[Image:Blender_UV_transform_output.jpg|left|256px|Texture Coordinate transformation]]
|-valign="top"
|'''Vertex mapping Approximate Transformation'''

This mesh has 32 columns of faces, and 8 rows, across the input image. Since the image is 4 times as wide as it is tall, this means there are 256 faces which cover an equal amount of the input image.
|'''Transformation with Nona'''

This is the remapping that the other images are approximating.
|'''Texture Coordinate mapping Approximate Transformation'''

This mesh has 16 rows of faces and 16 columns, spread equally over the output panorama, so there are 256 faces which cover an equal amount of the output.

Note that at this point I have not attempted to correct the texture coordinates around the boundary of the input image.
|}

Another test performed was mapping a rectilinear image to the zenith of an equirectangular image.
{| border="1"
|[[Image:Blender_zenith_mesh_transform.png]]

'''Vertex mapping Approximate Transformation'''

This is the transformation performed by arranging vertices that cover a uniform grid along the input image. The mesh was 32 faces wide and 16 faces tall, and covered an input image that was 4:3. I removed a row of faces that went right across the image, connecting the left edge to the right edge. This was there since the left and right edge represent the same line through the input image, and the mapping ignores the discontinuity across that edge. If I use this method, I will have to draw the faces of that row twice, once for each side, using some extra vertices somewhere off the sides of the output, so that the mapping is continuous and the corners are not missing.

|-
|[[Image:Blender_zenith_transform_nona_output.png]]

'''Transformation with Nona'''

This is the mapping the other images are approximating.

|-
|[[Image:Blender_zenith_tex_coord_transform.png]]

'''Texture Coordinate mapping Approximate Transformation'''

This is the transformation performed by finding the points on the input image that are at the points on a grid over the output image. The grid was 64 faces wide and 8 tall. Two of the edges are bad because I couldn't get the negative coordinates (they were clamped to 0), the other two are reasonable because the grid points mapped to the correct places, even though some of them were outside of the input image. If I use this method, I should be able to get the negative coordinates easily and therefore the poles will work without much effort.

|}

==Modules==
As there are many good ideas for the future of a graphically-accelerated Hugin after this project is complete (See the [http://groups.google.com/group/hugin-ptx/browse_thread/thread/7c838b0cda383922 discussion on the mailing list]), it is clear that a lot of development can occur using OpenGL and the new preview after I have completed this project. Therefore it is especially important that I provide clean, highly modular and extensible code for future use. Here is a proposal for some modules and interfaces:

; Texture Manager
: The texture manager will be built on top of Hugin's ImageCache, and will perform a similar function but using graphics memory and textures. It will be able to set the active texture to one that represents an image. It should be guaranteed to have a texture for each image when you want to use one, but it won't necessarily give you one that is a good size. It will adjust the size of the textures it is storing so they fill a sensible amount of memory and provide a even level of detail for the images.

; Mesh Manager
: The mesh manager will set up lists of instructions for the graphics system to draw a given image, remapped how it should be in the panorama's output.

; Mesh Transformation Stack
: This will provide the coordinates of points in a mesh. It is a stack so that when only moving the image, we can just recalculate the top of the stack and not have to worry about how distortion parameters are affecting the image. The mesh manager uses the top of this transformation stack. This will also be where coverage tests are performed, so we can identify images under a point.

; Renderer
: This is responsible for drawing the panorama, although it only really compiles a list of render layers and draws them when asked to.

; Render Layers
: Everything the renderer draws would have been put there by a render layer. As the name suggests, they are ordered and can cover each other up. At the end of the project there will obviously be a layer to draw the selected images. Other layers provide indication of image outlines, or perhaps redraw an image in a specific way.

; Image Render Hooks
: When drawing, we may want to do something different for a selection of the images. A render hook can be set up that changes how an image is rendered. It can provide callbacks for before and after drawing, a condition to drop an image from rendering altogether, or a replacement function for drawing it. An example would be to provide different blending modes. We could subtract an image from the ones behind it by dropping it from the images render layer, and then drawing it again in a different render layer, but subtracting it from the layers beneath. Alternatively, if we wanted something drawn in place but semitransparent, we could turn on blending before it is drawn and turn it off afterwards.

; Viewer
: The viewer combines together a renderer and a some basic interface for the tools. It tells the renderer to redraw when the window needs redrawing. Also it stores where in the panorama the user is looking at and how far they are zoomed in. It can convert between a mouse position and a position in the panorama.

; Input Manager
: An input manager gets all keyboard and mouse events the viewer sees and passes them to the correct tool. Tools register events they want. When tools are disabled they should give up their events, which can then be taken by a newly activated tool.

; Tools
: Tools can be enabled or disabled. When switching states they must set up or turn off events in the input manager, render layers, and image render hooks. They get access to the panorama data and viewer data.

; GLPreviewPanel
: This is a replacement PreviewPanel. It holds the viewer and gives it some screen space. It shows controls for the viewer and tools. It will keep an input manager to decide what to do with user input events. Some tools may be mutually exclusive, for example there might be a few that want to handle mouse events. The preview panel should turn off one tool before starting another in this case.

The PreviewFrame will then have an option to switch between the existing PreviewPanel and the new GLPreviewPanel.

===Events===
Currently there is a PanoramaObserver class, children of this class are notified when the panorama changes. However the viewer is somewhat independent of the panorama, it even makes sense to have multiple viewers of the same panorama. Still, some objects would need to be notified of when the view changes. Therefore we will leave the viewing conditions to the viewer, and provide some more event notifications for when they change. This will be helpful for the texture and mesh managers for instance, since they can change the level of detail as the user zooms on an image. Also any tool that requires this information can also use it.

Additionally, a texture detail change event would be nice, so that when a higher detail texture is ready for use, we can immediately redraw with the higher resolution. This will allow adaptive detail, since we can delay getting the more detailed texture and keep the user interface interactive for a while, and switch when we are ready.
Similarly a mesh detail event should be created, this will allow the window to be redrawn after a mesh has been regenerated. We may also need an idle event, so that when we are done processing important things we can consider updating the textures or meshes.

Perhaps it should be possible to delay these events, so that when the preview window is not visible we don't try to update anything. This would mean you can make many minor alterations to the panorama without waiting for any part of the preview to be recalculated, if it was out of sight.

[[Category:Community:Project]]

SoC 2008 Project OpenGL Preview

2008-05-13T09:26:42Z

James Legg: /* Distortions Comparison */ Added a comparison for a rectilinear image at the zenith -> equirectangular

File:Blender zenith mesh transform.png

2008-05-13T08:40:45Z

James Legg: This is an approximated remapping performed by mapping a 32 by 16 grid over the input image to the correct place in the output image, and mapping linearly between grid points on that. A row of faces that went across the image (since the left and right ed

This is an approximated remapping performed by mapping a 32 by 16 grid over the input image to the correct place in the output image, and mapping linearly between grid points on that. A row of faces that went across the image (since the left and right edge of the output image represent the same place in the input one) was removed. The remapping was from a rectilinear image placed at the pole into a cropped equirectangular image.

File:Blender zenith tex coord transform.png

2008-05-13T08:36:26Z

James Legg: This is image is an approximated remapping performed by mapping points on a 64 by 8 grid to the correct place on the input image and mapping linearly between those points.

This is image is an approximated remapping performed by mapping points on a 64 by 8 grid to the correct place on the input image and mapping linearly between those points.

File:Blender zenith transform nona output.png

2008-05-13T08:32:31Z

James Legg: This image is a remapping of a single rectilinear image rotated to the zenith, to an equirectangular image, which has been cropped. The remapping was performed by Nona.

This image is a remapping of a single rectilinear image rotated to the zenith, to an equirectangular image, which has been cropped. The remapping was performed by Nona.

SoC 2008 Project OpenGL Preview

2008-05-06T22:10:22Z

James Legg: Added to Community:Project category

SoC 2008 Project OpenGL Preview

2008-05-06T20:05:10Z

James Legg: Distortion comparison of approximate image transformations, and added somelinks

File:Blender transform Nona comparison.jpg

2008-05-06T19:08:30Z

James Legg: This is a cylindrical image remapped by Nona to a fish-eye projection. This is the 'correct' version of the Blender approximations, used for comparison.

This is a cylindrical image remapped by Nona to a fish-eye projection. This is the 'correct' version of the Blender approximations, used for comparison.

File:Blender UV transform output.jpg

2008-05-06T19:04:44Z

James Legg: A remapping performed by Blender, using a uniform mesh with texture coordinates defined as given by nona when performing the same remapping. Before the remapping, this was a cylindrical image.

A remapping performed by Blender, using a uniform mesh with texture coordinates defined as given by nona when performing the same remapping. Before the remapping, this was a cylindrical image.

File:Blender mesh transform output.jpg

2008-05-06T19:02:09Z

James Legg: A remapping performed by blender, using a mesh with vertices defined from coordinates given by nona when performing the reverse of the remapping. Before the remapping, this was a cylindrical image.

A remapping performed by blender, using a mesh with vertices defined from coordinates given by nona when performing the reverse of the remapping. Before the remapping, this was a cylindrical image.

SoC 2008 Project OpenGL Preview

2008-05-06T18:48:25Z

James Legg: blender diagrams for image transformations, more on image transformations

The OpenGL Preview project aims to create a fast previewer for Hugin, reducing the time taken to redraw the preview to real time rates. After that, the new previewer can be made more interactive than the current one.

==Image Transformations==
The new previewer will have to approximate the transformations of the images to fit on low resolution meshes. There are two methods for this:

===Remapping Vertices===
[[Image:Blend_project_mesh.jpg|thumb|Example of using vertices of a mesh to remap an image]]

Taking a point in the input image, we can find where the point is transformed to on the final panorama. Therefore, we can take a uniform grid over an input image, and create a mesh that shows remaps this grid into the output projection.

'''Disadvantages''':
*This method suffers when the input image crosses over the +/-180&deg boundary in projections such as equirectangular, as there will be faces connecting one end to the other, and nothing between the edges of those faces and the actual boundary. We must split the mesh so that it each part is continuous, which means some faces will have to be defined off the edge of the panorama.
*The poles of an equirectangular image will cause similar problems. At a pole, the image should cover the whole width of the panorama, but this is only one point in the input image. Vertices in the mesh near the pole are define a face that does not cover the whole row.
*The detail in the faces may not match up well with the area of the panorama. The faces are all of different sizes and some mappings may put the details in parts that can't be seen so easily.

'''Advantages''':
*The mesh resembles the output projection well, and covers up roughly the same area as the correct projection. The edges of the mesh lies along the edges of the image, so we don't need to worry about what happens outside of it.

===Remapping Texture Coordinates===
[[Image:Blend_project_UV.jpg|thumb|Example of using the texture coordinates of a mesh to remap an image. The internal repetitions are caused by wrapping the texture in the vertical direction; there should be hole there, but the mesh goes through regardless.]]
Alternatively, given a point on the final panorama, we can calculate what part of an input image belongs there. Therefore, we can take a uniform grid over the panorama, and map the input image across it.

'''Disadvantages''':
*This mapping breaks down at the borders of the input image. The borders of the input image may cross the output mesh at any position, so we need to draw the image with a transparent border.
*We must also calculate the extent of the images in the result so the mesh only covers the minimal area.
*Some faces of the minimal bounding rectangle still don't contain any of the input image at all, processing them would be a waste of time.

'''Advantages''':
*There are no problems at the poles and edges of the panorama.
*The output is of uniform quality throughout, as the faces always have the same density.

===Distortions Comparison===
To check that these transformation methods do not distort the image too much, I devised a test.
Using nona to produce a coordinate transformation map from input image coordinates to output image coordinates, and similarly creating a projection that goes the other way, I tried these transformations in Blender.

File:Blend project UV.jpg

2008-05-06T18:07:31Z

James Legg: This is a projection of a cylindrical panorama to a fish eye one, created with a uniform mesh representing the output panorama mapped to the input image a texture map. The texture coordinates do the transformation.

This is a projection of a cylindrical panorama to a fish eye one, created with a uniform mesh representing the output panorama mapped to the input image a texture map. The texture coordinates do the transformation.

File:Blend project mesh.jpg

2008-05-06T18:03:27Z

James Legg: The results of a transformation from a cylindrical input image to a fish eye image, using a specifically shaped mesh to remap a uniform grid over the input image.

The results of a transformation from a cylindrical input image to a fish eye image, using a specifically shaped mesh to remap a uniform grid over the input image.

SoC 2008 Project OpenGL Preview

2008-05-06T17:54:29Z

James Legg: Intro, started explaning image transformations

SoC 2008 student proposals

2008-05-06T17:12:15Z

James Legg: /* James Legg, OpenGL Accelerated Hugin Preview */ added link to project page.

== Template ==

* name / university / current enrollment
* short bio / overview of your educational background
* did you ever code in C or C++, yes/no? please provide examples of code.
* do you photograph panoramas? please provide examples.
* do you make other use of hugin/panotools than for stitching panoramas? please describe and show examples.
* were you involved in hugin/panotools development in the past? what was your contribution?
* were you involved in other OpenSource development projects in the past? which, when and in what role?
* why have you chosen your development idea and what do you expect from your implementation?
* how much time you plan to invest in the project? (we expected full time 40h/week but better make this explicit)
* please provide a schedule of how this time will be spent on subtasks of the project. While this is only preliminary, be aware that at the beginning of the project you will be required to provide a detailed plan, and during the project you will issue weekly progress reports against that plan.

== James Legg, OpenGL Accelerated Hugin Preview==
This project was accepted, see [[SoC 2008 Project OpenGL Preview]] for details.

* I am James Legg, a 2<sup>nd</sup> year undergraduate in Computer Science and Mathematics at The University of York (UK).
* At university I have taken modules in human computer interaction, artificial intelligence, computer graphics and visualisation, theory of computation, algorithms and data structures, group & ring theory, linear algebra, vector calculus, and analysis among others. I have A-Levels in Physics, Maths, and Further Maths, and a Critical Thinking AS-Level.
* I have experience in C++ and OpenGL, for example I created a program that displays a procedural random terrain, including multiple layers of textures, using OpenGL, SDL (for platform independent event handling), and libnoise (for coherent noise generation). The source is available from http://lankyleggy.members.beeb.net (Download megamap.tar.gz)
* I do photograph panoramas and stitch them with Hugin, there are a few examples at: http://lankyleggy.members.beeb.net/panoramas/
* I also use Hugin for aligning bracketed photos to create HDR images, and I use enblend (and the Gimp) to create seamless textures from photographs. An HDR example and a tonemap are on the same page as linked above.
* I have only recently started contributing to Hugin, see http://groups.google.com/group/hugin-ptx/browse_thread/thread/f94c584095fcb9d0.
* I have not been involved in other open source development projects beyond use, evangelising and bug reporting.
* After completing this project, I expect to have a new optional preview display, acting largely like the existing one. This should instead use OpenGL, and a grid-like mesh to remap the images. It will only be approximate, but still functional. It will not require OpenGL shaders, as the hardware for this is not available to most users, although I would like to make it easy for someone to add this functionality at a later date. The main purpose is to speed up the preview process. Unfortunately, it will not have the accuracy and logarithmic HDR display of the current preview.
* I would like to do this project since I have an interest in computer graphics and human interface design.
* I will invest at least 40 hours a week after the first 3 weeks of the project. Unfortunately the 3rd week of Coding Phase 1 is an exam week, However I will attempt to make up the time before the official coding start date.
* I plan to split the project into the following stages:
*# Beginnings: I will derive a new panorama preview from Hugin's existing one, set up an OpenGL context for it, and add extra preferences into Hugin to control selection of preview types, and the amount of texture memory and mesh resolution to use. I plan to have this done before the official "Coding starts" date.
*# Texture creation: After this each image would be scaled appropriately and placed in texture memory.
*# Photometric Alignment: OpenGL (without shaders) only directly supports linear colour transformations, however after this stage, white balance correction and (approximate) exposure correction will be used to display the images.
*# Mesh creation: A grid-like mesh will be created for each displayed image. This will then be transformed under distortion correction, input lens projection and output projection, to get the coordinates used to make the display on the preview window. This will require using and extending the current remapping code. I will find a way to track duplicates (e.g. fisheye projections with a FOV > 360 degrees), and to split the mesh across discontinuities in the projection. The final output would then be a linear approximation between the mesh points (along continuous mappings), which is then handled by OpenGL. The results of the mesh transformations will be stored in memory so that changing parameters does not cause unnecessary costly calculations. I expect to have started this by the end of coding phase 1, but I expect this will be the most time consuming stage.
*# Usability improvements: This will add extra features, e.g. a feature show the outline of an image in the preview, which will hopefully provide quicker identification of images.
*# Performance enhancing and testing: I will try to improve the performance of my code, and fix any bugs I may have. I will call for testers on the mailing list during this stage, and I may add additional requested features if I have the time. I will also make sure that the documentation is completely up-to-date.

If this feature is not wanted, or mentors are not available, I have an alternative proposal: I would like to make a cube map projection, since cube maps are the most common way of handling static reflections in real-time computer graphics, they should work well with interactive panorama viewers such as FreePV, and it has been [http://groups.google.com/group/hugin-ptx/browse_thread/thread/dd4cc95b0f5422b5/ requested before].

I would keep the definition of the faces that make the projection generic enough (perhaps specified in a script) to allow shapes other than cubes to be defined (I see there is interest in foldable panoramas and other shapes, including [[SoC_2008_ideas#Utility_for_creating_a_Philosphere| ceating a Pilosphere]]). This will require work on making enblend and enfuse blend across distinct faces. I don't know any algorithms that can do this well, but I would be willing to research it and experiment.

== Marko Kuder, Utility for Creating A Philosphere ==

Here is the application I intend to submit to Google, combined with the information I gave on the mailing list it should answer the questions in the template.

=== ABSTRACT ===
Hugin is a software tool for simple design of panoramic images. It's main function is to aid in combining multiple photographs, taken from a point in space in different directions, by finding common control points in neighbouring views and applying necessary transformations so that the images can be joined into a single, bigger picture. This can of course become an even tougher problem as photographs usually suffer from irregular lighting and Hugin is able to compensate for that also by color and light balancing.

The ultimate application of these functions is the creation of 360 degree panoramas. But such photographs have an innate problem - how to represent them in a natural and more attractive way than long and sometimes oddly curved pictures. The function that is yet missing from Hugin and would solve this problem is the ability to transform an image in such a way that it can be applied to a spherical body or, in practice, the automatic creation of models which can be printed and then folded into a polyhedron or sphere. The development and addition of this feature to Hugin would be my assignment on this project.

The original idea description can be found at:
http://wiki.panotools.org/SoC_2008_ideas#Utility_for_creating_a_Philosphere

=== INTRODUCTION ===
Printing panoramas as foldable cut-out models has been somewhat popularized by Philippe Hurbain, a French electronics engineer and a panoramic photography enthusiast (his homepage is accessible at http://www.philohome.com). The rhombicuboctahedron, which he often uses for displaying his panoramic photographs, was nicknamed »Philosphere«, after Hurbain's first name. Even though printing panorama models has little technical potential, it is an idea that could bring panoramic photography closer to the general public, as it displays the photographs in a more interesting way. For this reason and because image transformations are a problem domain which can be handled with my education without much practical work experience (which I lack), I decided to apply for this specific idea. This way I could get some much needed programming experience, which is often expected when applying for a job, familiarize myself with working on a bigger project, which I would have to get to know without being involved in it's previous development, and work on something that is interesting to me. Other projects which are part of Google Summer of Code mostly require special skills or their project ideas are a very integrated part of the software they are based on. Since I plan to attend classes and pass my exams in June, I had to be realistic about my abilities and choose a project that I could handle in the time I would have. I was also a bit late in finding out about GSoC, so I have chosen only one project, this one, and spent the free time I had recently getting to know Hugin, the mentoring organization's project (which I haven't used before) and making sure I was fit for the job.

=== THE PROBLEM ===
While the basic idea of projecting an image on a sphere seems simple, it can get quite complicated if we want the projection to be as realistic as possible. The problem is that we have many types of cameras with different lenses and consequently multiple types of projections e.g. rectilinear, equirectangular, cylindrical and fisheye. Each of these should be handled differently and considering the fact that what we are working with are panoramas constructed out of multiple photographs, the simple problem can be expanded quite far.
I do not intend to pretend that the original problem isn't trivial. If we had to map multiple non-stiched overlapping images to a single body, it would be difficult, but Hugin already contains the necessary code to join these images into one large panorama, so our task becomes easier. We can map an image to a sphere simply by converting it to so called »orange slices« (as shown on http://www.philohome.com/rhombicuboctahedron/rhombicuboctahedron.htm), which can be done by transforming each horizontal line of a slice into a shorter one, so it fits into the set form. There is a problem with this approach though. The final result depends greatly on the projection of the original image and the number of slices. The more slices we choose, the more equirectangular should the original photo be (as opposed to a fisheye projection), but such a model can be very hard to fold when printed. The alternative to orange slices which can be easier to assemble is the use of polyhedra like a rhombicubioctahedron or icosahedron, although model images of these could prove to be harder to construct, as multiple different transformations would have to be applied to separate parts of the original image. Some software which offers 3D panorama model construction already exists, like PTGui (http://www.ptgui.com) and IP-slicer script (http://www.bruno.postle.net/neatstuff/ip-slicer), but the former, which is a commercial application, is only able to construct a rhombicubioctahedron from a 360x180 degree panorama and the latter only converts an image into orange slices. Both lack additional user options, such as projection correction and incomplete panorama support, which I would like to include in my project.
In my recent discussion with Hugin's group, Bruno Postle - one of it's members - kindly pointed out some solutions to similar problems which I could use as basis for my project. One is his image patterner (http://www.bruno.postle.net/neatstuff/image-patterner), which allows mapping images to arbitrary shapes (but by his words has some problems with performance and is written in perl, so it can't be used directly), and the second is the ability to specify arbitrary 3D meshes as foldable models by seam definition in the well known 3D modelling application Blender (http://www.blender.org/documentation/htmlI/x5336.html). Both present a possibility of parametrization of goal shapes and as such greater flexibility for further expansion of user options. While this would be great, I'm not sure at the moment if it could be developed in the time at hand, but it absolutely is an idea that I would consider, if not otherwise as further development after GSoC.

=== PROJECT TIME PLAN ===

''Pre-summer phase:''

At first I intend to get to know Hugin and it's source code as much as I need to visualize my planned code in it (including planning the GUI modifications). After that I should make some research and study the mathematical basis for my problem, so I could define the required code. At the start of the first coding phase, in June, I will have some exams to pass, so it could be expected that I will be less productive for a period of 2-3 weeks in that month, but I intend to get a head start on the project by beginning coding before the official start of GsoC coding phase one and I plan to already have something to show the mentoring organization at that time.

''Coding phase one:''

After my exams I will most probably be able to commit my full time to the project. Till the 14th of July, when mid-term evaluations begin, I intend to complete the inclusion of at least one of the planned transformations in Hugin with full functionality, so I could show some practical results.

''Coding phase two:''

After mid-term evaluations I would add as many additional functions and user options as possible, before doing the final bug fixes and polishing up the finished code and GUI to complete the project.

=== BIOGRAPHY ===
I was born and raised in Trbovlje, a small town in Slovenia and at the age of 22 I'm in the 4th year of the Computer Software university program at the Faculty of Computer and Information Science in Ljubljana, the capital of our country. My main interest is in computer graphics, although my dream would be to work in a small company designing software and electronic devices of various types, because I like inventing new things.
From my education I have a lot of experience in Java programming and some in Python, C and C++, which would be my programming language of choice, for computer graphics in combination with OpenGL. Recently I have been working with our school's AI lab by doing research in the field of search algorithms and heuristics. I'm not much into photography, which may be strange considering the project I chose for my application, but I think my problem domain is separated enough from Hugin's main functionality that my lack of expertise in the area shouldn't be a problem. Google Summer of Code appears as an opportunity to prove to myself that I can work on real software projects and I hope I will be given the chance to show the same thing to Hugin's project team.

== Michael Ploujnikov, Automatic Test Suite ==
=== Abstract ===
This project is to develop an automated test suite to perform functional (black box) and structural (white box) testing of the Hugin internals as well as the user interface. The goal is to increase developers' confidence in the current code and future changes to the code as well as to improve the user experience of the application.

==== Deliverables ====
* a suitable framework will either be chosen from among existing frameworks or a new one will be developed based on analysis of the project needs and discussion with the core developers
* the test suite will be integrated with the built system
* a number of useful test cases will be created
* procedures for creating unit tests, running tests and analyzing test results will be documented

=== Detailed Description ===
My name is Michael Ploujnikov and I am currently finishing up the last year of a Computer Science program at the York University of Canada.

I have written a lot of C programs for school assignments, just learning the language, personal projects and free/opensource software projects (FOSS)[http://www.ohloh.net/accounts/2615]. Most of my C++ programming experience is in working with FOSS projects.

I shoot hand-held panoramas some of which can be viewed in my public photo gallery[http://plouj.com/g/v/kleinburg/feb-26-2008/]. Making even "regular" panoramas with Hugin is so much fun that I have not had a chance to try anything fancy.

I have not been involved with Hugin in the past. However, I have been involved in programming roles with other free/opensource software projects. My official resume[http://plouj.com/cv/] has some more details about my involvement.

I want to work on this project for a number of reasons. I strongly believe in test-driven development and wish to make it a possible (to some extent) for the current and future developers of a really fun application - Hugin! I am currently taking a university course[http://www.cse.yorku.ca/course/4313/] on software testing and would like to apply the learned concepts while they are fresh in my mind. I want to learn how Hugin works and writing test for the code base is arguably one of the best ways to do so. I want to help the experienced Hugin/panorama developers who volunteer their free time to work on this application by doing a rather tedious and boring task of setting up an automatic test suite for them. As a result, I hope the developers will have more time to spend on less "trivial" tasks. Finally, I want to get a feel for some unit testing frameworks (other than JUnit) that could be useful for me in the future career and FOSS projects.

I'm committed to work at least 40 hours per week for the duration of the GSoC on this project except for the first week in April when I will be busy with final exams, and another one week (to be determined) when I will be gone on vacation.

==== Plan ====
In Chronological order I will
* Finish school exams
* make sure I can run Hugin compiled from source
* Research existing testing frameworks specifically for the C/C++ languages and wx/GTK GUI toolkits
* Research testing frameworks that work well with cmake
* Make a list of test suite features I would like to have
* Modify the test suite feature list by discussing it with core Hugin developers to address their needs
* Decide (with core Hugin developers) whether an existing framework (could be more than one) is suitable for Hugin
* setup the existing framework(s) with stub tests or start creating a new framework (depending on how the previous decision goes)
* write some simple tests
* add concise usage documentation for the test framework on the wiki
* integrate the existing tests (I only found stuff in hugin/trunk/src/hugin1/tests)
* think about the testing techniques learned in school and use an appropriate one to come up with a "good" set of tests cases
* update the documentation because at this point the testing framework will probably have been modified
* write some tests for currently open bugs
* search the google group for proposed test cases and try to integrate them into the suite
* keep adding more tests until time runs out!

== Onur Kucuktunc, Automatic Feature Matching for Panoramic Images ==
=== Abstract ===
Aim of this Summer of Code project is to develop an efficient method for automatically matching local image features. Automatic image stitching, the process of combining images to form panoramic picture, can be performed in two steps: first step is to extract feature points (scale, rotation, illumination invariant), which is handled by "Feature Detection" project. Second step, also the goal of this project, is to find a set of corresponding features between images by using a matching algorithm.

Cover trees (a nearest neighbor matching algorithm) and RANSAC outlier pruning (an iterative method for fitting models in a data set with many outliers) will be implemented in order to find the correspondence between images. EXIF heuristics can also be used for improving the speed of feature matching. The time a picture was taken, and its filename give some information about the order of images in a panorama.

=== Details ===
==== Problem ====
In a panorama photo stitcher software, an automatic feature matching algorithm needs to find the correspondence between feature points and images. The problem is to develop a robust and efficient matching algorithm.

==== Solutions ====
Using nearest neighbor matching and RANSAC outlier with EXIF heuristics seem to increase robustness and efficiency of the feature matching method. Although the complexity of matching n images with a straight-forward approach is O(n^2), we can reduce it to O(nlogn) by using a nearest neighbor algorithm [1]. Cover tree [2] is a remarkably fast nearest neighbor algorithm, which have documentation and implementation in [3]. RANSAC (Random Sample Consensus Algorithm) is an iterative method for outlier pruning. Fischler and Bolles propose and discuss RANSAC model fitting with image analysis in [4]. There is also an overview and pseudo-code for RANSAC in wikipedia [5].

EXIF heuristics can also improve the efficiency and effectiveness of the method. If available, the time a picture was taken, its filename (like DSC_0001.jpg), camera orientation, etc. are useful informations which can both decrease the running time and increase its correctness. Geolocation information in EXIF format also give clues whether or not images belong to the same panorama. "Exchangeable image file format:wikipedia" document [6] gives details about the format.

==== Performance Measures ====
We need to be sure that applied algorithms really improves robustness and speed of the method. Therefore, some comparisons should be made on a dataset, which we manually give matching feature points. Correctly retrieved matching points (hit), retrieved but not correct ones (false positive), and the matching points we could not retrieve (miss) could be plotted to a ROC curve. By this way, we can improve the performance by selecting the right methods and adjusting parameters to the value which give highest hit rate. For example, we can compare k-d trees, which Brown and Lowe used in [7], with cover trees. Running time of the algorithm is also important for our purpose. This can be calculated both by analyzing the complexity of the algorithms, and recording running times in each test.

==== Time Schedule ====
I plan to invest at least 30h or more per week to this project. I will be on vacation for 4-5 days, and another week I will probably be busy with moving my house. But I will balance it by working hard in order not to fall behind schedule. My planned actions will be as follows:
* Before May 26
** Getting to know the developers, mentors, reading documentation,
** Getting used to hugin/panotools, your coding strategy, bug reporting and fixing,
** Finding and/or generating representative datasets for testing and evaluation purposes,
** Reading scientific material, understanding all the concepts of matching and pruning procedures,
** Beginning documentation,
** Deciding inputs and outputs of the project,
** Making contact with the developer of Feature Detection part,
** Making a survey to learn the habits of photographers for effective EXIF heuristics.
* May 26 to July 7-14 (midterm evaluations)
** Combining algorithms by using and adapting existing cover tree and RANSAC algorithms,
** Adding heuristic to implementation, test if it really improves efficiency and effectiveness,
** Preparing the libraries, standalone applications
** Getting ready for mid-term evaluations
* July 15 to August 11-18 or September 1 (final evaluations)
** Testing with a large number of images, offering code for public review,
** Improving heuristics and other parts with respect to the feedbacks and test results,
** Completing documentations, preparing libraries, applications, etc.
** Getting ready for final evaluations

==== Deliverables ====
* A library for automatic feature matching which uses proposed algorithms,
* Documentation created by using the doxygen tool,
* Comparisons of the algorithms, test cases for running time analysis,
* A standalone application (works in cross-platform) that uses the library,
* Fully-commented source codes both for the library and Hugin application that uses the library.

=== Biography ===
I'm Onur Kucuktunc, a MSc student in computer engineering at Bilkent University, Turkey. I'm currently working on our Multimedia Database System (http://cs.bilkent.edu.tr/~bilmdg/), which includes lots of parts related to computer vision. I have taken Basics of Signals and Systems, Image Analysis, Computer Vision, and Pattern Recognition courses. I'm also a photographer.

I have chosen this topic and Hugin/panotools project because last semester for Computer Vision course, I worked on a project titled as "3D Face Reconstruction from 2D Images for Effective Face Recognition". We proposed a 3D face reconstruction method by automatically matching SIFT features, since most of the current implementations still require manual selection of feature points among different images. You can find additional information in my website [8].

I'm mostly experienced in Java and Matlab programming; but I have also written low-level, operating system-related codes in C, and graphic codes (using OpenGL) in C++. As an OSX user, I also believe I will be able to report and fix platform-dependent bugs in hugin. I have i18n and translation experience in some open-source projects, but it will be my first time to actually contribute as a developer.

=== References ===
# J.Beis, D. Lowe. "Shape indexing using approximate nearest-neighbor search in high-dimensional spaces."
# A.Beygelzimer, S.Kakade, J.Langford. "Cover Trees for Nearest Neighbor." ICML 2006.
# http://hunch.net/~jl/projects/cover_tree/cover_tree.html
# M.A.Fischler, R.C.Bolles. "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography." Comm. of the ACM 24
# RANSAC, http://en.wikipedia.org/wiki/RANSAC
# http://en.wikipedia.org/wiki/Exchangeable_image_file_format
# M.Brown, D.G.Lowe. "Recognizing Panoramas."
# Personal website, http://cs.bilkent.edu.tr/~onurk/

== Marko Kuder, Batch Processing ==

=== ABSTRACT ===
Hugin is a graphical user interface using PanoTools software libraries and programs for the creation and editing of panoramic images. Its main function is to join multiple photographs of an area, taken from the same point of view in different directions, into one large panorama. Using several command-line applications like autopano-sift, nona and enblend it offers automatic control point recognition, lighting and color correction and many similar functions.

Some of these functions can be computationally demanding, so Hugin could benefit from the addition of a batch processor, an additional interface which would enable the user to prepare several projects for serial execution. A project to develop such an addition will be the subject of this proposal.

The original description of the idea can be found at:
http://wiki.panotools.org/SoC_2008_ideas#Batch_processing

=== INTRODUCTION ===
Originally I only had time to write one application, but when the deadline was extended, Hugin's team recommended that I should choose an additional project, maybe this one, especially since this is a higher priority project than my previous choice - utility for creating a philosphere. After examining the idea and having a discussion with the group I concluded that it would also be a goal I could achieve in the required time and at the same time present an interesting challenge which would provide some useful experience for my future career as a programmer.

Hugin is a GUI based on top of command-line applications, which means its own code presents only a part of its actual functionality as an application. Most processing is done by separate programs, which are executed by a single command with defined input/output files and multiple switches which specify options the user wishes to use. Most of these options can be selected in Hugin's GUI. When the user instructs the program to execute the processing, a Makefile is created which includes all the instructions needed to produce the final output files.
Because the creation of panoramas usually consists of several stages besides the actual joining (or stitching) of images e.g. color correction, the Makefiles usually include intermediate goals in the form of temporary files, mostly images. These are necessary because of the use of different command-line programs for different operations. However, the user cannot control what is happening with these intermediate files if he/she is not proficient in the use of Makefiles.
Another problem is that Hugin's image processing is not a fast process but neither a very long one. So if the user wishes to create several panoramas quick enough, he has to be near the computer all the time to specify the next project when one finishes.

=== THE PROJECT ===
The main goal of my project would be to develop a batch processor for Hugin which would address these problems. First of all I would include an additional tab in the existing GUI with a manageable list of image processing projects for execution. The basic use of Hugin would stay the same, except that the user would have an option to not execute the processing yet, but add the specified project to the batch instead. The project file and Makefile created by Hugin would be saved and then the user could change all the options and input files to a new project which he could again add to the batch and so on, until the list would include all desired projects which could then be serially executed by the program.
Besides intermediate files, the Makefile produced by Hugin includes some so called 'phony' targets, like 'all' and 'clean'. They are standard parts of Makefiles, which could play an important role in the creation of a batch processor, as other targets are usually actual files created before or during the process. The program could automatically recognize different types of targets and offer to the user additional processing for individual input, output or intermediate images. This processing could be performed by using 'plugin' Makefiles, which could already be included with Hugin. An example of such a Makefile would be to create a QTVR - a Quick Time virtual reality cube - from an input panorama (instructions for making such a Makefile can be found at http://thread.gmane.org/gmane.comp.misc.ptx/8754).
Since one Makefile can have quite a lot of intermediate files, this automatic target recognition approach could show to be a little awkward for the user to use if it should offer the greatest flexibility of extensible intermediate file processing. Maybe the program should only offer the redirection of input/output files between Makefiles. Which design would be the best would show as the project develops, but especially the latter version could be made easier to use by adding a graphic visual editor, where processes could be represented as nodes in a graph and the redirections of files between Makefiles as connections between them. However, in the first version of my program I would keep things simple and the user would just specify input and output filenames for each process in the batch, so they could somehow already be connected, but it would not be very robust, as simple spelling errors would make it work incorrectly.
It would also be a good idea for this batch processor if batch configurations could be portable between different systems, but at the moment I do not know yet how different Hugin's project files are for different systems, so that would be an optional goal.

=== PROJECT TIME PLAN ===

''Pre-summer phase:''

As I mentioned in my previous application, I would probably work less on the project in June, as it is the month of exams, but to compensate I would start working on the project as soon as possible before the official start of coding, so I would not miss any deadlines. By the official start of GSoC coding at the end of May, I should have a deeper understanding of Hugin's Makefile system and a clear plan of how my project should be completed. I even might have some code done by then (maybe the GUI extension).

''Coding phase one:''

By mid-term evaluation I plan to have completed the first part of making a GUI and a Makefile execution queue system, probably already with 'plugin' Makefile support and added options that Make offers like parallel execution of targets.

''Coding phase two:''

After mid-term evaluation if everything would go according to plan, I would try to develop additional functionality for my program, like the mentioned automatic target recognition and intermediate file processing. In August I should present the program to Hugin's team for testing and discovery of bugs, which would hopefully all be fixed until the end of the month.

=== BIOGRAPHY ===
I was born in 1985 in Trbovlje, a small town in Slovenia and I am currently a 4th year undergraduate student at the Faculty of Computer and Information Science, which is part of the University of Ljubljana, Slovenia. I believe I have enough experience in C/C++ programming to be able to work on this project. In my years of education I have also learned Java, Python and some Pascal, but outside of school I did not have much practice yet, which is mostly why I decided to apply for GSoC. My interest is in graphics, computer game programming and electronics, although I have not yet fully decided on my future professional orientation. Recently at school I started working with the AI lab on research of heuristics and search algorithms. Even though Hugin is an application for photographers, which I am not, I believe that developing a batch processor is a goal I could achieve without expert knowledge on camera technology. And from my discussion with Hugin's group it seems they agree with that conclusion.

== Onur Kucuktunc, Extending Hugin's output options for stitching ==

=== Abstract ===
Current version of Hugin has a limited output options. One of the problems is that some small errors after stitching process can be easily removed by using a graphics editor. Giving output support for the file formats these image manipulation/editing applications are using, will improve the usability.

Purpose of this project is to extend Hugin's currently offered output options with other compression techniques and give support for some useful formats, such as multilayered TIFF or PSD. Generation of flash-based (swf) hypertext documents (html) would be another way of extending output options, especially for web publishing.

=== Details ===
As I mentioned in abstract, Hugin's output options are needed to be extended for some reasons. One problem, which is also stated in project idea [1], is that after stitching panoramas may have some ghost objects. For example, a walking person may appear in one scene, but s/he can disappear in the next one, resulting that moving object seems to be semi-transparent after blending. Ghosting can easily be removed if Hugin can output as a layered image format. Popular image manipulation/editing applications like Adobe Photoshop(r) [2] and its open-source alternative GIMP [3] can work on multilayered TIFF and PSD formats. Although specifications for multilayered PSD and PSB formats are not publicized, Panotools already has some code for exporting as PSD, and we may offer some more compression options. On the other hand, GIMP has some codes both for importing and exporting multilayer TIFF files, so it would be easier to support it. OpenRaster is another format which gives multi-layer support, but its specifications are not defined clearly right now. There is a draft specification in [4].

Besides giving support for raster graphics formats, exporting as a vector graphics, such as SVG, will definitely have some other advantages. Generation of swf, js and html seems to be another useful export extension of Hugin for the ones who needs web publishing.

To sum up, extending Hugin's output options has the following parts:
* Multilayered TIFF
* Multilayered PSD and PSB (if possible)
* OpenRaster (with the given draft-specifications)
* A vector graphics format, i.e. SVG
* Swf, html, js (if needed) for web publishing

==== Time Schedule ====
I plan to invest at least 30h or more per week to this project. I will be on vacation for 4-5 days, and another week I will probably be busy with moving my house. But I will balance it by working hard in order not to fall behind schedule. My planned actions will be as follows:
* Before May 26
** Getting to know the developers, mentors, reading documentation,
** Getting used to hugin/panotools, your coding strategy, bug reporting and fixing,
* May 26 to July 7-14 (midterm evaluations)
** Beginning documentation,
** Preparing output libraries
** Getting ready for mid-term evaluations
* July 15 to August 11-18 or September 1 (final evaluations)
** Completing documentations, preparing libraries, etc.
** Getting ready for final evaluations

==== Deliverables ====
* Output library for the proposed file formats,
* Documentation created by using the doxygen tool,
* Fully-commented source codes both for the library and Hugin application that uses the library.

=== Biography ===
I'm Onur Kucuktunc, a MSc student in computer engineering at Bilkent University, Turkey. I'm currently working on our Multimedia Database System (http://cs.bilkent.edu.tr/~bilmdg/), which includes lots of parts related to computer vision. I have taken Basics of Signals and Systems, Image Analysis, Computer Vision, and Pattern Recognition courses. I'm also a photographer. You can find detailed information about me in my website [5].

I'm mostly experienced in Java and Matlab programming; but I have also written low-level, operating system-related codes in C, and graphic codes (using OpenGL) in C++. As an OSX user, I also believe I will be able to report and fix bugs in hugin. I have some i18n and translation experience in open-source project, but it will be my first time to actually contribute as a developer.

=== References ===
# Extend hugin's output options for stitching, http://wiki.panotools.org/SoC_2008_ideas#Extend_hugin.27s_output_options_for_stitching
# Adobe Photoshop(r), http://www.adobe.com/products/photoshop/index.html
# GIMP - The GNU Image Manipulation Program, http://www.gimp.org/
# OpenRaster, http://create.freedesktop.org/specs/OpenRaster-draft.pdf
# Personal website, http://cs.bilkent.edu.tr/~onurk/

[[Category:Community:Project]]

== Tim Nugent, Support Vector Machine-based Sky Identification ==

* I am a 2nd year PhD Student in the Bioinformatics group at University College London, UK.

* I originally trained as a Pharmacologist, attaining a 2.1 BSc (Hons.) from the University of Bristol. I then completed an MRes (with Merit) in Bioinformatics at Birkbeck College, University of London, before working for a bioinformatics company Inpharmatica Ltd., and then as a bioinformatician for a genetics research project at Queen Mary, University of London. I returned to academia and am currently in my 2nd year of a 4 year PhD course in the bioinformatics group; my project concerns the structure prediction of transmembrane proteins using machine learning approaches (more info here: http://www.cs.ucl.ac.uk/staff/T.Nugent/research.html).

* I have programmed in C/C++ for over two years and have recently implemented support vector machine (SVM) and dynamic programming algorithms in C++ into a protein structure prediction program. I have submitted various patches for Hugin, and am currently working on a game that used the SDL libraries.

* I have a strong interest in photography, particularly panoramic and underwater. I am particularly keen to combine these two areas as there are very few examples where they overlap. I'm familiar with a range of stitching tools, but tend to use Hugin as I rarely work on anything other than a Linux box. Here are a few examples: http://www.cs.ucl.ac.uk/staff/T.Nugent/panoramics/

* So far I have not used Hugin/Panotools for anything other than stitching panoramas. However, I'm keen to experiment with HDR photography.

* I'm a developer for the Bioperl project (http://www.bioperl.org) which produces opensource Perl code for the bioinformatics community. You can download the development branch via SVN, which includes some of my modules for drawing graphics using libgd and SVG are available here. I intend to port these to Python for the increasingly popular BioPython project. http://www.cs.ucl.ac.uk/staff/T.Nugent/code.html

* The reason I've chosen to tackle the sky identification problem is that I'd like to apply my knowledge of support vector machine (SVM) classification to a novel problem outside biology. Sky identification ties in perfectly with my interest in panoramic photography, and as someone who has had to manually remove control points from rogue clouds, I feel I'm the ideal person to attack this problem. SVM classifiers have been used in a range of areas from finance to face recognition and there is huge scope for applying these tools to new fields. However, training and cross validating SVMs can be computationally expensive. I'm fortunate to have access to a large Linux cluster at UCL running Sun Grid Engine, which I intend to use for this project (http://www.ucl.ac.uk/media/library/Dell).

* For this implementation I intend to train an SVM classifier to automatically and accurately identify regions in an image that contain cloud, and those that do not, so that control points can be removed or excluded therefore increasing the quality of the alignment. This may also reduce the computation time for control point creation, by reducing the available regions within which to search for control points. This process could also be extended to other regions within an image that do not remain static over a series of photos, e.g. water, waves, other windswept regions etc. I will develop a standalone tool in C++ to score all regions of the image, and return a posterior probability of whether or not that region contains cloud. A scoring function and an adjustable threshold will be used to trim control points from associated Panotools/Hugin file, or create a mask with which to exclude control point searches. I also intend to produce a GUI based version using wxWidgets/GTK etc. The source code will be freely available, and I will provide binaries for all the major platforms.

* As a full time PhD student, I do not have a summer break as undergraduates do, therefore am unable to contribute a 40 hour week to the project. I intend to use any free time I have inside or outside university hours, and envisage working on the project for most evening during the week and for a significant portion of the weekend. While this may not be ideal, I feel the expertise I already have will allow me to 'hit the ground running'. I am also able to start the project immediately so I am highly confident I can finish it by Google's 'pencils down' date. Should I encounter problems, I'm willing to take my annual leave in order to work full time on the project.

* My preliminary plan for the project is as follows, beginning immediately:

1) Familiarise myself with:
# Hugin source code - begun, will continue for duration of the project.
# VIGRA image processing library - begun, will continue for duration of the project.
# libSVM/SVMlight source - already done to a large degree.
# FlickR API - begun.
# Feature/texture classification algorithms (e.g. Gabor filtering).

2) Assemble training and test sets of images containing/not containing cloud, probably from community image sites such as Flickr, using FlickR API/Perl scripts. This will be ongoing in order to obtain maximum diversity, but in the first instance it should take approximately two weeks to collect and then accurately classify enough images (and the regions within them) to begin SVM training.

3) Use VIGRA, and perhaps other approaches, to extract image features from all regions (e.g. RGB data, smoothness, edges etc - motifs common to cloud textures). Add additional global features, for example the (x,y) co-ordinates of the region within the image). Finding the features that accurately discriminate between the two sets may not be straightforward so I envisage this will be the hardest part of the project and may take 2 months.

4) Use these features to train an SVM classifier. Optimise SVM parameters (possibly using genetic algorithm based grid search), cross-validate, re-train, optimise (and repeat..). Use UCL Linux cluster to run many SVM training jobs in parallel. This stage will overlap with (3) to a large degree, finishing two weeks afterwards.

5) Write standalone tool in C++ to score all regions of the image, and return a posterior probability of whether or not that region contains cloud. Create a scoring function which analyses the results globaly - e.g. Score up 'cloud' regions that are in contact with a large number of other 'cloud' regions, and score down those that are not. Use an adjustable threshold to trim control points from associated Panotools/Hugin file. This should take about 3 weeks.

6) Write a GUI based version using wxWidgets/GTK etc. This should take about two weeks. I see this as the least important component of the project as I envisage the tool being run from the command line, or via Hugin, so should other stages of the project fall behind schedule, this part would be sacrificed.

7) Testing, both code (across multiple platforms) and prediction accuracy. This will take 2 weeks but will begin during (5).

[[Category:Community:Project]]

Panorama formats

2008-03-24T04:16:07Z

James Legg: /* Cylindrical */ grammar

{{Glossary|The format of a panorama is defined in broad terms by the [[Projections|projection]]|1}} used to map the full or partial 3D scene onto a 2 dimensional print or screen.
There are several types of projections in use:

=== Full Spherical Formats ===
There are two main spherical formats: [[Equirectangular]] and [[Cubic Projection|Cubic]]. Both are able to display the whole sphere that surrounds us - 360° along the horizon, 90° up and 90° down. Specialized [[Panorama Viewers|viewers]] are needed to view spherical panoramas.

[[Image:Equirectangular.JPG|thumb|250px|equirectangular panorama format]]
==== Equirectangular ====
The equirectangular format is widely used by a couple of [[Panorama Viewers]] as for example [[PTViewer]] and [[SPi-V]]. It consists of a single image with an [[Aspect Ratio|aspect ratio]] of 2:1 (that is, the width must be exactly twice the height).
{{clr}}

[[Image:Cubic.jpg|thumb|250px|cubic panorama format]]
==== Cubic ====
The [[Cubic Projection|cubic]] format uses 6 cube faces to fill the whole sphere around us. The image is remapped to the cubefaces which fit seamlessly.

One very wide spread cubic format is [[Quicktime|QuickTime]] VR. It consists of one file containing the 6 faces as [[JPEG]] compressed images together with a header giving basic information how the panorama should be displayed.

Another cubic format is used by [[SPi-V]]. It consists of the 6 cubefaces in a single row or column. [[SPi-V]] treats any image with an [[Aspect Ratio|aspect ratio]] of exactly 6:1 as a cubic spherical panorama.
{{clr}}

[[Image:little_planet.jpg|thumb|250px|Little planet remapping example © Erik Krause]]
==== "Little Planet" ====
This is an unusal format that remaps a full sphere such that the ground looks like if it was a little planet. See [[Unusual remappings#Little planet|Unusual remappings]] for details.{{clr}}

=== Partial Formats ===
There is a number of possibilities to display partial panoramas - these are panoramas that don't fill the whole sphere in one or the other way. Partial panoramas can be displayed directly if they don't cover more than approximately 120° along the shorter side (that is they can be 360° in one direction but must be 120° or less in the other direction). The main formats are [[Cylindrical Projection|Cylindrical]] and [[Rectilinear Projection|Rectilinear]], but partial spherical panoramas are possible, too.

[[Image:Cylindrical.JPG|thumb|250px|cylindrical panorama format]]
==== Cylindrical ====
[[Cylindrical Projection|Cylindrical]] panoramas can show a full circle along the horizon or a part of it. They are very popular for landscape panoramas. If used for architectural subjects it might be of bother that horizontal lines (except the horizon itself) are bent.
{{clr}}

[[Image:orientation-plate.jpg|thumb|250px|Arc formed panorama example © Erik Krause]]

==== Arc formed ====
A special type of a [[Cylindrical Projection]] where the panorama is arched like on common orientation plates. See details on [[Unusual remappings#Orientation plate (arc)|Unusual remappings]].{{clr}}

[[Image:Rectlinear.JPG|thumb|250px|rectilinear panorama format]]
==== Rectilinear ====
[[Rectilinear Projection|Rectilinear]] panoramas display the subject just like an ordinary (non-fisheye) lens would do. The horizontal and vertical field of view are limited to about 120°. Straight lines stay straight, hence they are good for architectural subjects. But if either field of view is too large they suffer from unnatural looking distortions in the corners.
{{clr}}

[[Image:Equirectangular_cut.jpg|thumb|250px|cutted equirectangular panorama format]]

==== Partial Spherical ====
To partial spherical panoramas applies basically the same as to full sphericals (see above). In most cases they are used to cut off [[Zenith]] or [[Nadir]]. Vertical field of view has to be limited in this case to prevent the viewer from misinterpreting the source images.
{{clr}}

[[Category:Glossary]]

SoC 2008 student proposals

2008-03-24T04:11:25Z

James Legg: /* James Legg, OpenGL Accelerated Hugin Preview */ continuation

SoC 2008 student proposals

2008-03-21T18:45:19Z

James Legg: Added my OpenGL hugin preview project