Development of Open Source tools

From PanoTools.org Wiki
Revision as of 18:48, 8 September 2009 by Seb Przd (Talk | contribs)

Jump to: navigation, search

Panorama Related Open Source tools

Contribution

Why contribute?

The above tools are free for you (and every other user), because volunteers have contributed their skills and time. They provide immense value to the whole community. They provide value to you, don't they? If they do, please consider contributing something back.

How to contribute?

Join the hugin-ptx mailing list to find out what is going on at the moment and how you can help.

If you don't have time, you are most likely a busy professional. You can donate money to Hugin, Enblend/Enfuse on their project pages through Sourceforge.

At the time of updating this text (9-Jun-2009) the building process of hugin is robust and there are more and more people mastering it on the different platform. The 0.8.0 release cycle is coming to an end. Beyond that, a few exciting features such as nona-gpu are in line for future releases and we expect that the Google Summer of Code 2009 will add new features as well.

Don't be afraid of failures in the building process

  • You may encounter errors when following the build processes linked below is high. Don't worry such errors will not compromise your computer.
  • The failure of the building process is actually your success! Every time you report such a failure, with as much detail as possible to how it came about, you are contributing to the progress toward a stable release.

If you are fluent in other languages than English, you can help translate Hugin. There's a Hugin translation guide to help you get started or help when you run into translation problems.


Processes

The goal of the new release process is to decouple the development process from the other processes, avoiding artificial slow downs (trunk freezes).

Hugin uses an asynchronous development process based on Subversion, which means that:

  • trunk never freezes, and committers can continuously add improvements there
  • anybody can branch out a development codeline from trunk to branches/ to implement major changes separately without disrupting the continuous flow.
  • once a months, developers are polled as to whether it is worth issuing a release. if there is enough support and one person volunteers as release manager, that person branches out a release codeline from trunk. that release codeline is equivalent to a trunk freeze in traditional synchronous development. only bug fixes and translations should go into that codeline. Tarballs are to be released from release codelines only.

Development

  • if you have a development branch, nothing changes. you can keep working on it as you please.
  • if you are working on trunk there is an improvement: no trunk freeze.
  • you're encouraged to continue with your usual pace of bug fixing and development.
  • to do things perfectly: commit all your changes to trunk; and those that are bugfixes also to the release codeline.
  • if you forget about it and commit only to one codeline, do not worry. The release manager takes responsibility to port fixes from trunk to the release codeline or the other way around as the need arises.
  • try not to add new features to the release codeline

Translation

  • same as development
  • please do not run extract-messages.sh while a release process is going on

Debugging

Working through Hugin Trackers

Clearing the bugs in the sourceforge bug tracker is an iterative process critical to the release cycle. Feedback from tester is essential for this process. Please take the time to check if the older bugs apply to a current snapshot, if you like to see a release soon.

We'll be releasing frequent snapshots until a release candidate emerges. This is an iterative process:

  1. Volunteers check the release-critical bugs listed in the bug tracker (sort open bugs by priority) against the most recent snapshots (scroll down on that page).
    • Install the latest snapshot.
    • Try to reproduce the bug on your system.
    • If you find that the bug no longer occurs, chances are that it has been fixed. Close it (assuming you have the required access), or simply leave a note that it has been fixed (together with the SVN revision and the system used for testing).
    • If you reproduce the bug, leave a note to confirm that it is still actual. Note the SVN revision and the system used for testing. Post detailed instructions how to reproduce the crash. Provide a test case if you can.
    • Add any comments you have to the ticket in the bug tracker. Let the community know you have tested. The bug-tracker is like a mailing list or forum thread, don't be afraid to post.
    • If you don't want to open an account with Sourceforge, post your observations on the hugin-ptx mailing list. It is possible to post comments to the bug tracker anonymously, but this interrupts the vital flow of feedback between testers and developers. Anonymous bug reports are likely to be discarded.
  2. The developers fix the bugs identified in the tracker.
  3. The builders build new snapshots including the fixes.

Development Codelines

It is recommended to work separately on major changes and integrate them in the main codeline when they are ready. The currently open development branches are listed in the repository. If you want to work on a major new feature, ask for SVN write access and open yourself a branch:

$ svn cp https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/trunk \
  https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/branches/NEW_DEVELOPMENT_BRANCH

The branch owners are strongly encouraged to regularly sync their branch with trunk, i.e. merge the changes that have occurred since the last sync (or branching) into their branch.

When development is complete or has reached a milestone, it may be integrated it trunk. Before integrating in trunk it is recommended to:

  • merge trunk's changes since the last sync into the branch
  • test that the branch builds on the major supported plattforms
  • test that the branch does not break existing functionality (unless the break is intended, e.g. when a new functionality replaces an existing one)

When the development branch has fulfilled its purpose; is supreceded or has been abandoned, move it to the obsolete_branches in the repository

$ svn mv https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/branches/NEW_DEVELOPMENT_BRANCH \
  https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/obsolete_branches/NEW_DEVELOPMENT_BRANCH


Release

Once a month the developers are polled if there is reason to issue a release. If there is enough support for releasing, one developer takes on the role of release manager and undertakes the following steps:

The conditions to declare a release final are:

  • the code builds on the major supported platforms (Ubuntu, Fedora, OSX, Windows)
  • there is no (known) regression, unless intentional. This means: what worked with the previous release should work with the current one.

The next Hugin is likely to be released when these bugs will have been fixed.

Branching Out For Release

1. Identify and add the new strings for translation.

2. Branch out a new release-codeline (replace 2009.2 below with the appropriate version number. 2009 stands for the current year and 2 stands for the next even number in sequence, starting with 0 if the year has changed and this is the first release of the year).

$ svn cp https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/trunk \
  https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/releases/2009.2 \
  -m "Branching 2009.2 for release"

3. In the new release-codeline bump up the version number, e.g.

3.a. get the source

$ svn co https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/releases/2009.2 2009.2
$ cd 2009.2
$ gedit CMakeLists.txt

3.b. edit the following lines in the top level CMakeLists.txt file

# version
set(V_MAJOR 2009)
set(V_MINOR 2)
set(V_PATCH 0)

3.c. edit the Mac version number in mac//Version.xcconfig

3.d. check the changes in

$ svn ci

4. in trunk bump up to the next odd version number, e.g. 2009.3 in this example.

4.a. get the source

$ svn co https://hugin.svn.sourceforge.net/svnroot/hugin/trunk trunk
$ cd trunk
$ gedit CMakeLists.txt

4.b. edit the following lines in the top level CMakeLists.txt file

# version
set(V_MAJOR 2009)
set(V_MINOR 3)
set(V_PATCH 0)

4.c. edit the Mac version number in mac//Version.xcconfig


4.d. check the changes in

$ svn ci

Keeping the Branch in Sync with Trunk

Committers have no obligation to help maintaining the release branch. As a release manager, you want to stay on top of the changes in trunk and synchronize them into your branch if relevant. E.g. if change 4312 to trunk is relevant to the release and has not been applied, do the following:

$ cd trunk
$ svn up
$ svn diff -r 4311:4312 > 4312.patch
$ mv 4312.patc ../release
$ cd ../release
$ patch -p 0 < 4312.patch

then build, test, commit.

Release

1. Start with a clean checkout

$ svn co https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/releases/2009.2 2009.2
$ cd 2009.2

2. Update and commit the Changelog - this can be tricky since after the first branching there may already be significant differences between the trunk and the release codeline. What I found works best for me is to keep synchronized the relevant changes in trunk and in the release codeline; to create the Changelog from trunk; to edit out manually those parts of the trunk changelog that are not relevant to the release, and merge the trunk Changelog from the point of the last release to now into the current release.

$ cd ../trunk
$ svn up
$ svn2cl
$ cp Changelog ../2009.2/Changelog.trunk
$ gedit Changelog.trunk &
$ gedit Changelog
$ svn ci

3. Note down the SVN revision number - the release equivalent in the repository is the releases/2009.2 branch at the revision that you noted down.

4. Extract the relevant Changelog section for Sourceforge, Write a summary for the announcement.

5. Create tarball

$ cd ..
$ mkdir hugin-build
$ cd hugin-build
$ cmake ../hugin -DENABLE_LAPACK=YES -DCPACK_BINARY_DEB:BOOL=OFF -DCPACK_BINARY_NSIS:BOOL=OFF \
  -DCPACK_BINARY_RPM:BOOL=OFF -DCPACK_BINARY_STGZ:BOOL=OFF -DCPACK_BINARY_TBZ2:BOOL=OFF \
  -DCPACK_BINARY_TGZ:BOOL=ON -DCPACK_BINARY_TZ:BOOL=OFF
$ make package_source

6. Build an rpm or deb (depending on your distro) from the tarball as a sanity check. Test what you can test.

$ tar xvfz hugin-2009.2.0.tar.gz
$ mkdir hugin.build
$ cd hugin.build
$ cmake ../hugin-2009.2.0 -DENABLE_LAPACK=YES -DCPACK_BINARY_DEB:BOOL=ON -DCPACK_BINARY_NSIS:BOOL=OFF \
  -DCPACK_BINARY_RPM:BOOL=OFF -DCPACK_BINARY_STGZ:BOOL=OFF -DCPACK_BINARY_TBZ2:BOOL=OFF \
  -DCPACK_BINARY_TGZ:BOOL=OFF -DCPACK_BINARY_TZ:BOOL=OFF
$ make package
$ sudo dpkg -i hugin-2009.2.0-Linux.deb

7. Rename the tarball. For snapsnots, append _snapshotYYMMDD (replace YYMMDD with Year/Month/Day). For betas, append _betaX with X being an incremental number starting with 1. For release candidates, append _rcX with X being an incremental number starting with 1. Eventually the last candidate will be renamed "final" (by removing the appendix) after a few more tests prove that no further candidate is needed.

8. Determine the tarball's checksum for the announcement notice

$ sha1sum hugin-2009.2.0_beta1.tar.gz

9. Upload the tarball to sourceforge, you need to be a 'Release tech' to do this.

10. Tag the release. SVNNR is the revision number noted in 3. TAG_NAME is the same name as given to the tarball, e.g. hugin-2009.2.0_YYMMDD for snapshots and hugin-2009.2.0_rcN for release candidates.

$ svn cp -r SVNNR https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/releases/2009.2 \
  https://hugin.svn.sourceforge.net/svnroot/hugin/hugin/tags/TAG_NAME \
  -m "Branching 2009.2 for snapshot"

11. Paste the announcement into the Sourceforge news system. You need to be a 'News Editor' to do this. Only for the final release, tick the "Submit my news to the Slashdot.org Firehose" box.

12. Mail the announcement to hugin-ptx. Only for the final release, mail it also to other relevant channels such as PanotoolsNG and QuickTimeVR.

13. Wait for feedback and fixes. Volunteers may fix critical bugs. Test that the codeline builds on the major supported platforms. Test that no major functionality is broken. Repeats steps 1-11 as often as necessary.

14. Once the codeline is final, port relevant "polish" commits that have not been ported yet to trunk (similar to merging a development codeline).

15. later on if patches are required, apply them to /releases/2009.2, bump up V_PATCH, tag, and release.

Testing

  • There is only so much testing that can be done with our limited resources.
  • Final releases are expected to build and work on the major supported platforms, however it is possible that a bug slips in due to lack of resources / systematic testing
  • We may issue patch releases in case of security issues or other major issues reported after final release.

Distribution

  • unchanged: the Hugin project releases source code that is tested to build on the main supported platforms: Fedora, Ubuntu, Windows, OSX. It works on the developers machines. YMMV.
  • Building and distributing binaries is left to the users communities. Once there are binaries of appropriate quality level for platforms that do not have a package manager (Windows and OSX) we add them as a courtesy to the SF archive - as usual "WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." See the GNU General Public License for more details.

Download Test Build

If you want to contribute testing but you don't want to go through the hassle of building the code, you can find the latest installer download for Windows and OSX here.

Build your Own Test Builds

If you are ready to go through the building process, here are the instructions.

IMPORTANT: These builds are for your computer. If you decide to share them with others, be aware that you are subject to the GPL, and that the general public may need guidance regarding what you distribute. Read the information in the packaging and distribution section below. If you are unsure, ask on the hugin-ptx mailing list for advice before posting a file for download.

Goal

an infrastructure for on-demand build and distribution of usable test-binaries for the most popular platforms. These builds are meant to enable users to test the newest features and report bugs. Ideally, on Pablo's demand all those who have a build chain will run it against the newest source code to produce the builds.

Process

  1. Experienced users will build the most current hugin and helpers (libpano, enblend, autopano, etc.) for the target platform of their choice, with support from coders.
  2. The build process will be documented for each of the supported platform.
  3. Users willing to spend some time learning how to build will reproduce the documented process.
  4. Power users will script and automate the building process.
  5. Users with packaging skills will package the builds for distribution (installers).
  6. The produced binaries/installers will be made available on the web.

Specific revisions

When building from the repository, some revisions have bugs. This process is meant to build the latest revision so that if the latest revision has bugs these can be identified and corrected. However sometimes these bugs can be more critical than other times. If you need a more or less working version of hugin, try applying the process to one of the following revisions. To stay on top of all changes, consult the revision logs.

hugin

Revision

Branch

Comments

4315

/hugin/releases/2009.2

2009.2_beta2

4008

/hugin/trunk

official 0.8.0 release = RC5

3465

/hugin/trunk

official 0.7.0 release

libpano

Revision

Branch

Comments

1036

trunk

tentative fix for the error introduced in 1017

1017 +

trunk

builds but has a fatal error [1]

1016

trunk

builds and works but messes up locale

enblend-enfuse

Use of CVS has been discontinued in favor of Mercurial. The information below is stale.

Revision

Comments

2009-06-09

builds on ubuntu 9.04 x86. Note: for experimental use there is a temporary fork by Christoph Spiel in Launchpad.

2008-02-07

builds on ubuntu 7.10 AMD64, OSX (used for snapshot package), Windows (used for first snapshot installer)

Supported Platforms

  • If you don't find your preferred platform listed below and you are willing to contribute your time and skills to build hugin on it, feel free to add it to the table. We will accommodate any well supported platform in the regular release process.
  • The persons marked in bold in the credits are known to have currently access to a build chain on the selected platform and can possibly produce a snapshot on request, usually within a few days. If you have contributed to the build system, feel free to add yourself here.
  • Redundancy is good. If you have access to one of the listed platforms, please try to run the documented process below and report success to hugin-ptx. If you think you could do this on a regular basis, enter yourself in the list, in bold.


Platform

Supported Versions

Status

Process

Credits

ubuntu

32bit/64bit
8.04 8.10 9.04

tbd

OK

  • Sébastien Perez-Duarte
  • Yuval Levy
  • Régis B.
  • Kornel Benko

Fedora

32bit/64bit/ppc/ppc64
F-8 F-9

tbd

tbd

  • Bruno Postle

OSX

IntelMac/PowerPC
10.3 10.4 10.5

tbd

  • Ippei Ukai
  • JD Smith
  • Daniel M. German
  • Peter A. Crowley
  • David Haberthür
  • John Riley
  • Roger Howard
  • Harry van der Wolf
  • Charlie Reiman

Windows

32bit/64bit
XP/Vista
(64bit officially supported only after 0.8.0)

tbd

  • Tom Sharpless
  • John Navas
  • Jean-Marc Paratte
  • Yili Zhao
  • Yuval Levy
  • Guido Kohlmeyer
  • Ad Huikeshoven (build automation)
  • Ryan Sleevi (64 bit)

OpenSuse

32bit/64bit
10.2 10.3

tbd

tbd

  • Kornel Benko (10.3)
  • Peter Suetterlin (10.2)
  • Stephan Hegel (10.3 x86_64)

FreeBSD

32bit/64bit
6.3 7.0

n/a

OK

  • Vasil Dimov (6.2/i386)
  • Vasil Dimov (7.0/amd64)

Gentoo Linux

32bit

tbd

OK

  • Thomas Pani

all platforms

a big thank you to Pablo d'Angelo for supporting all of those building efforts.

Stati

Build Chain

  • tbd: looking for responsible
  • OK: mostly automated build process ready on request
  • unavailable: temporarily unavailable (e.g. responsible on holiday)
  • HW-broken: the hardware is temporarily unavailable
  • SW-broken: temporarily dysfunctional, working on a fix
  • broken: nobody is working on a fix
  • unsupported: has been dropped for lack of support

Process

  • tbd: status unknown
  • auto: work as documented and has been automated to a reasonable extent
  • OK: works as documented, could use automation / scripting
  • draft: documented, needs validation / testing / cleaning
  • incomplete: parts are missing (e.g. enblend, libpano)
  • outdated: worked in the past but needs an update
  • obsolete: nobody has the time to update

Packaging and Distribution

Instructions for packaging binaries for distribution will follow. Some important points:

Snapshots

  • comply with the GPL
    • join a text of the GPL in the distribution
    • give access to the source code
    • credit the authors
  • label clearly the snapshot as such, with a reference to the build date and/or the SVN revision number
  • edit the text / readme files that come with the snapshot
    • indicate clearly that it is unstable, experimental software
    • indicate where to find the latest version
    • indicate that the advertised features might or might not work

Release

  • comply with the GPL
    • join a text of the GPL in the distribution
    • give access to the source code
    • credit the authors

Feedback

When running through the building process documented above chances are that something goes wrong. While it is disappointing when the process ends in a flurry of cryptic error messages it is not harmful. This is the nature of software development and you are now part of it. There is still a lot of value in your experience and you can help improve the process and get closer to the hoped for software package. Please don't be ashamed that it did not work. This happens even to the most expert coders. Give the developers feedback on the hugin-ptx mailing list. Only with your feedback they can know that something goes wrong, and a well crafted feedback helps them find out quickly what went wrong and devise a solution. Be a part of the solution, not a part of the problem!

To give good feedback, note down carefully all of this information while you are going through the instructions.

  1. details about your computer. CPU, operating system, other particularities
  2. the revision number of the code checked out with SVN (which appears at the end of the checkout process) or with CVS. Or, if you don't find a revision number, the date and time when you checked out the code.
  3. the last step / command you entered into the command line
  4. a copy of the last few lines displayed, from where you think the error messages started. Don't worry if you copy a couple of lines too many, it is better to give more lines than less lines.

Even the standard feedback is good feedback. For those wishing to dig deeper, you can try

  • to use "make VERBOSE=1" when building hugin.
  • to do a debug build "cmake -DCMAKE_BUILD_TYPE=DEBUG" and use oprofile for profiling.