Development of Open Source tools
| 2010-Nov-22: | Hugin's tracker has moved from SourceForge to Launchpad. If you still find references to the SourceForge tracker in these pages, it may be outdated or it may be related to Panotools or Enblend that have not moved. Please fix if you can. Most pages have been fixed for Hugin's move from a Subversion to a Mercurial repository (2010-May-16) but if you find a reference to SVN, it may be outdated or it may be related to Panotools that has not moved. Please fix if you can. |
|---|
Panorama Related Open Source tools
Contribution
Why contribute?
The above tools are free for you (and every other user), because volunteers have contributed their skills and time. They provide immense value to the whole community. They provide value to you, don't they? If they do, please consider contributing something back.
How to contribute?
Join the hugin-ptx mailing list to find out what is going on at the moment and how you can help.
If you don't have time, you are most likely a busy professional. You can donate money to Hugin, Enblend/Enfuse on their project pages through Sourceforge.
At the time of updating this text (9-Jun-2009) the building process of hugin is robust and there are more and more people mastering it on the different platform. The 0.8.0 release cycle is coming to an end. Beyond that, a few exciting features such as nona-gpu are in line for future releases and we expect that the Google Summer of Code 2009 will add new features as well.
Don't be afraid of failures in the building process
- You may encounter errors when following the build processes linked below is high. Don't worry such errors will not compromise your computer.
- The failure of the building process is actually your success! Every time you report such a failure, with as much detail as possible to how it came about, you are contributing to the progress toward a stable release.
- See also If you encounter errors, Fixing The Hugin Build.
If you are fluent in other languages than English, you can help translate Hugin. There's a Hugin translation guide to help you get started or help when you run into translation problems.
Processes
The goal of the new release process is to decouple the development process from the other processes, avoiding artificial slow downs (trunk freezes).
Hugin uses an asynchronous development process based on Mercurial, which means that:
- the default branch never freezes, and committers can continuously add improvements on the tip.
- anybody can branch out development codelines on their local copy of the repository, and even publish that repository.
- usually developers will branch out to implement major changes; and builders will branch out to clean up releases; without disrupting the continuous flow.
- unlike in the previous Subversion repository, branches are no longer stored in different folders. Give them a meaningful name to distinguish release branches (version number) from development branches (feature being implemented / worked on).
- developers have write access to the official repository on Sourceforge and can push changes and branches to it. The project is very liberal in giving such write access - if you have something to contribute, mention it on hugin-ptx and chances are one of the admins will give you the necessary write rights.
- once a months, developers are polled as to whether it is worth issuing a release. if there is enough support and one person volunteers as release manager, that person branches out a release codeline from the default codeline. That release codeline is equivalent to a trunk freeze in traditional synchronous development. only bug fixes and translations should go into that codeline. Tarballs are to be released from release codelines only.
Development
- if you are working on something big that requires multiple changes to complete, you're encouraged to start your own branch with `hg clone`
- if you are working on incremental improvements to the default branch, feel free to push them any time. There is no trunk freeze.
- you're encouraged to continue with your usual pace of bug fixing and development.
- to do things perfectly: commit all your changes to the default branch; and apply those that are bug fixes also to the release codeline.
- the new thing about Mercurial (coming from Subversion): commit is to your local repository only. Remember to `hg push` your changes from time to time so that others can test them / benefit from them / add value to them.
- if you forget about it and commit only to one codeline, do not worry. The release manager takes responsibility to port fixes from the default branch to the release branch or the other way around as the need arises.
- try not to add new features to the release codeline.
Translation
- same as development.
- please do not run extract-messages.sh while a release process is going on.
Debugging
Working through Hugin Trackers
Clearing the bugs in the sourceforge bug tracker is an iterative process critical to the release cycle. Feedback from tester is essential for this process. Please take the time to check if the older bugs apply to a current snapshot, if you like to see a release soon.
We'll be releasing frequent snapshots until a release candidate emerges. This is an iterative process:
- Volunteers check the release-critical bugs listed in the bug tracker against the most recent snapshot. The most recent snapshot is usually posted on the Hugin-PTX mailing list.
- Install the latest snapshot.
- Try to reproduce the bug on your system.
- If you find that the bug no longer occurs, chances are that it has been fixed. Close it (assuming you have the required access), or simply leave a note that it has been fixed (together with the SVN revision and the system used for testing).
- If you reproduce the bug, leave a note to confirm that it is still actual. Note the SHA1 ID and the system used for testing. Post detailed instructions how to reproduce the crash. Provide a test case if you can.
- Add any comments you have to the ticket in the bug tracker. Let the community know you have tested. The bug-tracker is like a mailing list or forum thread, don't be afraid to post.
- If you don't want to open an account with Launchpad, post your observations on the hugin-ptx mailing list.
- The developers fix the bugs identified in the tracker.
- The builders build new snapshots including the fixes.
Development Codelines
It is recommended to work separately on major changes and integrate them in the main codeline when they are ready. The currently open development branches are listed in the repository along the release branches.
hg pull hg branches
If you want to work on a major new feature, use `hg branch <NEW_BRANCH_NAME>` to open yourself a branch. If you want to publish that branch on Sourceforge, ask for write right and push your changes to Sourceforge with `hg push`
hg branch <NEW_BRANCH_NAME hg push
The branch owners are strongly encouraged to regularly sync their branch with default, i.e. merge the changes that have occurred since the last sync (or branching) into their branch.
When development is complete or has reached a milestone, it may be integrated it default. Before integrating in default it is recommended to:
- merge default's changes since the last sync into the branch
- test that the branch builds on the major supported platforms
- test that the branch does not break existing functionality (unless the break is intended, e.g. when a new functionality replaces an existing one)
When the development branch has fulfilled its purpose; is superseded or has been abandoned, close it
hg up -C <OBSOLETE_BRANCH> hg commit --close-branch -m 'close badbranch, this approach never worked' hg up -C default # switch back to "good" branch
Maturity Criteria
A development codeline - whether under revision control or presented as a patch - is considered mature when:
- a: the functionality it is intended to implement works on the developer's machine ("works for me" condition)
- b: it has been tested to build on the major supported platforms ("does not leave them behind" condition) by at least one contributor for each: Windows, OSX, Linux
- c: it does not unintentionally break existing functionality ("no regression" condition)
When a development codeline reaches maturity, it enters the integration queue.
Integration Queue
The integration queue is the ordered list of new features / development codelines waiting to be integrated in trunk. The prioritization is a collective decision by consensus of the developers. Silence = consent. The discussion, and the latest version of the list, are on the mailing list.
The integration queue is not set in stone: a change in the maturity status of a feature in waiting is good reason to review/change the ordered list. In any case it is reviewed after every release branching.
Release
Once a month the developers are polled if there is reason to issue a release. If there is enough support for releasing, one developer takes on the role of release manager and undertakes the following steps:
The conditions to declare a release final are:
- the code builds on the major supported platforms (Ubuntu, Fedora, OSX, Windows)
- there is no (known) regression, unless intentional. This means: what worked with the previous release should work with the current one.
Release Notes
The Release Notes are an important document accompanying the release. Although it is just a list of new features, it gets distributed widely. Most readers are unlikely to have used Hugin before. When drafting them, think that the target audience is somebody who has an idea of what a panorama stitcher might be, has never heard of Hugin, but is interested in discovering cool new software.
Start drafting them early in the process to allow for translators to translate the Release Notes on the website. Moreover, you will send them as simple text with each released tarball.
Set Up
Set yourself up to edit the release notes on the website. You will need write access to the SourceForge project.
hg clone ssh://${USER}@hugin.hg.sourceforge.net/hgroot/hugin/hugin-web hugin-web
The Editing Cycle
cd hugin-web hg pull hg up mkdir releases/2010.4 nano releases/2010.4/en.shtml hg ci -m "skeleton|draft|updated|final release notes" hg push
The website updates automatically about every hour. Wait enough time and point your web browser to http://hugin.sourceforge.net/releases/2010.4/ to verify the result of your edits.
Release Notes for the First Beta Release in a New Release Cycle
Before posting the beta1 of the release cycle, you want to have at least a working draft of the Release Notes. Edit them directly on the website. List the relevant new features. Use the release notes from past releases as a template. Copy them in text form for the email announcing the beta1 release. Don't link them from the homepage. Translators will want to start working on the text as soon as possible. Try to finish it early in the process. Add a comment at the beginning of the text to detail its current status. Not all translators follow the Changelog of the website and so are not aware if you committed the document as skeleton / draft / update / final.
Release Notes for Follow Up Releases in the Same Release Cycle
It is important to show the changes relative to the previous tarball, however only in the email announcing the new tarball. The HTML page on the website will most likely stay untouoched (unless an important new feature has been added so late in the cycle). Add a section "Changes since previous beta release" (or "Changes since previous release candidate" if we're already at the RC stage) to the text mail you sent announcing the beta1 release. Copy and edit the relevant Changelog entries in there.
Branching Out For Release
1. Identify and add the new strings for translation.
Before running the script, make sure you have wxrc installed. On older Ubuntu, run sudo apt-get install wxrc. On Ubuntu 10.4LTS it is sudo apt-get install wx-common.
hg pull hg up cd src/translations ./extract-messages.sh hg diff hg ci hg push
2. Branch out a new release-codeline (replace 2010.4 below with the appropriate version number. 2010 stands for the current year and 4 stands for the next even number in sequence, starting with 0 if the year has changed and this is the first release of the year).
hg branch 2010.4
3. Bump up the version number in the new release-codeline and check in the changes:
nano ./CMakeLists.txt nano ./mac/Version.xcconfig hg ci -m "bumping version number"
In the top level ./CMakeLists.txt edit the lines
# version set(V_MAJOR 2010) set(V_MINOR 4) set(V_PATCH 0)
In mac/Version.xcconfig edit the lines
HUGIN_VERSION_MAJOR = 2010 HUGIN_VERSION_MINOR = 4 HUGIN_VERSION_PATCH = 0
4. Switch back to the default branch and bump up to the next odd version number, e.g. 2010.5 in this example.
hg up default nano ./CMakeLists.txt nano ./mac/Version.xcconfig hg ci -m "bumping version number"
Edit the same lines as above in the top level ./CMakeLists.txt file and in ./mac/Version.xconfig
5. Verify that the branches are ok and push them to the official repository. Since we intentionally create a new remote branch, we use the -f switch in the push command.
hg view hg push -f
Keeping the Release Branch in Sync with Default
Committers have no obligation to help maintaining the release branch. In fact, it is best if they keep working in default on both code and translations. As a release manager, you want to stay on top of the changes in default and synchronize them into your branch prior to release if relevant.
The Hgk Mercurial extensions will help you with this. Edit your ~/.hgrc file to activate it. In [extensions] section add:
hgext.hgk =
In Hgk the changesets are displayed chronologically per branch with the newest one on top. Branches are indicated visually by a tree on the left. Tags have a yellow background and te top of branches are indicated by a box with green background and the name of the branch, except the default branch. Tags are displayed on yellow background.
Syncing from Default to Release Branch
- Make sure your repo is up to date and get an overview with Hgk:
hg pull && hg up <RELEASE_BRANCH> && hg view &
- Find the first changeset on the default branch after the last time you synchronized and work yourself up to the present.
- Select one changeset at a time. You will see details of the change set in the lower part of the window.
- In the lower left panel, read the changeset log and its revision number.
- Decide if you want to apply it to the Release branch (i.e. if it is a bug fix)
- If you decided that the changeset should apply to the Release branch, check if it has not been applied already by the developer.
- If it has not been applied, apply it, using the <REVISION_NUMBER>:
hg export --git <REVISION_NUMBER> | hg import -
- If you deem this to have been a major change, go through a build/test cycle
- Repeat until all new changes have been applied
- Go through a build/test cycle
- Push your changes to the public repository
hg push
Syncing from the Release Branch to Default
The procedure is the same, just make sure you're working in default:
hg pull && hg up default && hg view & hg export --git <REVISION_NUMBER> | hg import ... hg push
Release
1. Start with a clean checkout
2. Update the Changelog (make sure LC_ALL is set to UTF-8 when doing this). In Ubuntu you may have to do something like `export LANG="en_CA.UTF-8" && export LC_ALL="en_CA.UTF-8"`
3. commit the Changelog
4. tag the release
5. note down the revision/hash for the release
hg up -C 2010.4 hg log --follow --style=changelog > ChangeLog hg ci -m "Updated ChangeLog" hg tag <RELEASE> hg log -l 2
6. Create tarball
cd .. mkdir hugin-tarball cd hugin-tarball cmake ../hugin.hg -DENABLE_LAPACK=YES -DCPACK_BINARY_DEB:BOOL=OFF -DCPACK_BINARY_NSIS:BOOL=OFF \ -DCPACK_BINARY_RPM:BOOL=OFF -DCPACK_BINARY_STGZ:BOOL=OFF -DCPACK_BINARY_TBZ2:BOOL=OFF \ -DCPACK_BINARY_TGZ:BOOL=ON -DCPACK_BINARY_TZ:BOOL=OFF make package_source
7. Build an rpm or deb (depending on your distro) from the tarball as a sanity check. Test what you can test.
tar xvfj hugin-2010.4.0.tar.bz2 mkdir hugin-tarball-build cd hugin-tarball-build cmake ../hugin-2010.4.0 -DENABLE_LAPACK=YES -DCPACK_BINARY_DEB:BOOL=ON -DCPACK_BINARY_NSIS:BOOL=OFF \ -DCPACK_BINARY_RPM:BOOL=OFF -DCPACK_BINARY_STGZ:BOOL=OFF -DCPACK_BINARY_TBZ2:BOOL=OFF \ -DCPACK_BINARY_TGZ:BOOL=OFF -DCPACK_BINARY_TZ:BOOL=OFF make package sudo dpkg -i hugin-2010.4.0-Linux.deb
8. Rename the tarball. For snapsnots, append _snapshotYYMMDD (replace YYMMDD with Year/Month/Day). For betas, append _betaX with X being an incremental number starting with 1. For release candidates, append _rcX with X being an incremental number starting with 1. Eventually the last candidate will be made final (by removing the appendix) after a few more tests prove that no further candidate is needed.
9. Determine the tarball's checksum for the announcement notice
sha1sum hugin-2010.4.0_beta1.tar.gz
10. Upload the tarball to SourceForge, you need to be a 'Release tech' to do this.
- log on to the SF file manager
- left-click to navigate to the appropriate subfolder (e.g. hugin -> hugin-2009.2)
- click on the icon next to the appropriate subfolder and in the context menu select "Uploads here" (or "New folder") if you need a new folder.
- click on "upload file"
- browse to the local file and let the upload begin.
- prepare a release notes text file (e.g. "hugin-2009.2.0.release_notes.txt") and upload it as well, to the same subfolder
- left-click on the release notes text file and click the checkbox "Release Note"
- left-click on the released file. If you want it to be the default download for a specific plattform, click the appropriate checkbox
- in the "Release Notes" dropdown list, select the appropriate ones for this file
- note the SHA1 sum calculated by SF. Compare it with the one determined in step 8. If it is different, delete the uplaoded file and try again.
11. Upload the tarball to Launchpad. You need to be a member of the Hugin Developers Team on Launchpad.
- TODO: describe
12. Draft the release announcement.
- Extract the relevant Changelog section for the release announcement for all betas and release candidates after the first beta in a release cycle.
- Use the Mercurial Tag noted before
- Use the SHA1SUM noted before
13. Mail the announcement to hugin-ptx.
14. (Optional) Paste the announcement into the Sourceforge news system. You need to be a 'News Editor' to do this. Only for the final release, tick the "Submit my news to the Slashdot.org Firehose" box. To do this (not very intuitive): Menu Project Admin -> Feature Settings, in the Project News line, hit "Submit". The project has not been making announcements on Sourceforge for more than a year.
15. Wait for feedback and fixes. Volunteers may fix critical bugs. Test that the codeline builds on the major supported platforms. Test that no major functionality is broken. Repeats steps 1-14 as often as necessary.
16. Once the codeline is final, port relevant "polish" commits that have not been ported yet to default.
17. later on if patches are required, apply them to the branch bump up V_PATCH in ./CMakeLists.txt and HUGIN_VERSION_PATCH in ./mac/Version.xcconfig, tag, and release.
Declare a Release Final
- No tarball is to be released as final without being a release candidate (RC) first. If fixes are committed to the release codeline after the last RC, you have two options: either declare the current RC final and the issue a known issue; or release a new RC and wait again for it to be deemed of final quality. Make the decision by consensus on the mailing list.
- Once you've decided that the current release candidate is final, polish the release notes (this should be a process along with the polishing of the release itself).
- Rename the RC file, e.g. hugin-2010.4.0.tar.bz2.
- Upload to the proper location on Sourceforge (different than betas and release candidates):
- click on the symbol on the right of the "hugin" top level folder and select "new folder". give the new folder a meaningful name
- click on the symbol on the right of the new folder folder and select "upload here".
- upload the tarball
- upload a text file with the release notes.
- left-click on the release notes file and tick the release notes box.
- left-click on the tarball and select the release notes from the dropdown. tick the boxes for all operating systems to which it applies (all but Mac and Windows)
- no need to tag the release as it was already tagged as RC
- Paste the announcement into the Sourceforge news system. You need to be a 'News Editor' to do this. Only for the final release, tick the "Submit my news to the Slashdot.org Firehose" box. To do this (not very intuitive): Menu Project Admin -> Feature Settings, in the Project News line, hit "Submit".
- mail the announcement to Hugin-PTX (subscription required).
- update Hugin homepage with link to the new release notes
Testing
- There is only so much testing that can be done with our limited resources.
- Final releases are expected to build and work on the major supported platforms, however it is possible that a bug slips in due to lack of resources / systematic testing
- We may issue patch releases in case of security issues or other major issues reported after final release.
Distribution
- unchanged: the Hugin project releases source code that is tested to build on the main supported platforms: Fedora, Ubuntu, Windows, OSX. It works on the developers machines. YMMV.
- Building and distributing binaries is left to the users communities. Once there are binaries of appropriate quality level for platforms that do not have a package manager (Windows and OSX) we add them as a courtesy to the SF archive - as usual "WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." See the GNU General Public License for more details.
Download Test Build
TODO
Build your Own Test Builds
If you are ready to go through the building process, here are the instructions.
| IMPORTANT: | These builds are for your computer. If you decide to share them with others, be aware that you are subject to the GPL, and that the general public may need guidance regarding what you distribute. Read the information in the packaging and distribution section below. If you are unsure, ask on the hugin-ptx mailing list for advice before posting a file for download. |
|---|
Goal
an infrastructure for on-demand build and distribution of usable test-binaries for the most popular platforms. These builds are meant to enable users to test the newest features and report bugs. Ideally, on Pablo's demand all those who have a build chain will run it against the newest source code to produce the builds.
Process
- Experienced users will build the most current hugin and helpers (libpano, enblend, autopano, etc.) for the target platform of their choice, with support from coders.
- The build process will be documented for each of the supported platform.
- Users willing to spend some time learning how to build will reproduce the documented process.
- Power users will script and automate the building process.
- Users with packaging skills will package the builds for distribution (installers).
- The produced binaries/installers will be made available on the web.
Specific revisions
When building from the repository, some revisions may be broken. This process is meant to build the latest revision so that if the latest revision has bugs these can be identified, reported and corrected. However sometimes these bugs can be more critical than other times. If you need a more or less working version of Hugin, try applying the process to an earlier revision. To stay on top of all changes, consult the revision logs and the discussions on the hugin-ptx mailing list. This section used to list specific revisions known to work, but the list was hopelessly out of date.
Supported Platforms
- If you don't find your preferred platform listed below and you are willing to contribute your time and skills to build hugin on it, feel free to add it to the table. We will accommodate any well supported platform in the regular release process.
- The persons marked in bold in the credits are known to have currently access to a build chain on the selected platform and can possibly produce a snapshot on request, usually within a few days. If you have contributed to the build system, feel free to add yourself here.
- Redundancy is good. If you have access to one of the listed platforms, please try to run the documented process below and report success to hugin-ptx. If you think you could do this on a regular basis, enter yourself in the list, in bold.
|
Platform |
Supported Versions |
Status |
Process |
Credits |
|---|---|---|---|---|
|
ubuntu |
32bit/64bit |
tbd |
| |
|
Debian |
The Hugin Packagers Team for Debian can be reached at hugin at packages.debian.org. Liaise with them if there are issues with the binaries in Debian (and downstream distributions like Ubuntu) and on backports.org. |
| ||
|
Fedora |
32bit/64bit/ppc/ppc64 |
tbd |
| |
|
OSX |
IntelMac/PowerPC |
tbd |
| |
|
Windows |
32bit/64bit |
tbd |
|
|
|
OpenSuse |
32bit/64bit |
tbd |
| |
|
FreeBSD |
32bit/64bit |
n/a |
| |
|
Gentoo Linux |
32bit |
tbd |
| |
|
all platforms |
a big thank you to Pablo d'Angelo for supporting all of those building efforts. | |||
Stati
Build Chain
- tbd: looking for responsible
- OK: mostly automated build process ready on request
- unavailable: temporarily unavailable (e.g. responsible on holiday)
- HW-broken: the hardware is temporarily unavailable
- SW-broken: temporarily dysfunctional, working on a fix
- broken: nobody is working on a fix
- unsupported: has been dropped for lack of support
Process
- tbd: status unknown
- auto: work as documented and has been automated to a reasonable extent
- OK: works as documented, could use automation / scripting
- draft: documented, needs validation / testing / cleaning
- incomplete: parts are missing (e.g. enblend, libpano)
- outdated: worked in the past but needs an update
- obsolete: nobody has the time to update
Dependencies
Hugin is work in progress and dependencies can be added any time. If your build does not work, see Fixing The Hugin Build.
Hugin depends on the following packages. The list may be incomplete, and there may be some platform-specific dependencies. Details for specific platforms are in the platform documents linked above.
Build-Time dependencies
To build Hugin, you will need:
- wxWidgets >= 2.7.0
- libpano13 >= 2.9.17
- libtiff
- libjpg
- libpng
- libopenexr
- libexiv
- GLEW
- freeglut or glut
- libglu
- libxi
- libxmu
- some boost libraries including -thread -graph -regex -iostreams -filesystem
- gettext
- optionally, lapack
Run-Time Dependencies
At runtime, enblend >= 3.2 is required. Exiftool is optional but recommended. Starting with Hugin 2010.3 and newer, Hugin ships with its own control points generator cpfind. Prior to 2010.3 another optional but recommended runtime dependecy is a control points generator such as autopano-sift-c or panomatic.
Packaging and Distribution
Instructions for packaging binaries for distribution will follow. Some important points:
Snapshots
- comply with the GPL
- join a text of the GPL in the distribution
- give access to the source code
- credit the authors
- label clearly the snapshot as such, with a reference to the build date and/or the SVN revision number
- edit the text / readme files that come with the snapshot
- indicate clearly that it is unstable, experimental software
- indicate where to find the latest version
- indicate that the advertised features might or might not work
Release
- comply with the GPL
- join a text of the GPL in the distribution
- give access to the source code
- credit the authors
Feedback
When running through the building process documented above chances are that something goes wrong. While it is disappointing when the process ends in a flurry of cryptic error messages it is not harmful. This is the nature of software development and you are now part of it. There is still a lot of value in your experience and you can help improve the process and get closer to the hoped for software package. Please don't be ashamed that it did not work. This happens even to the most expert coders. Give the developers feedback on the hugin-ptx mailing list. Only with your feedback they can know that something goes wrong, and a well crafted feedback helps them find out quickly what went wrong and devise a solution. Be a part of the solution, not a part of the problem!
To give good feedback, note down carefully all of this information while you are going through the instructions.
- details about your computer. CPU, operating system, other particularities
- the revision number of the code checked out with SVN (which appears at the end of the checkout process) or with CVS. Or, if you don't find a revision number, the date and time when you checked out the code.
- the last step / command you entered into the command line
- a copy of the last few lines displayed, from where you think the error messages started. Don't worry if you copy a couple of lines too many, it is better to give more lines than less lines.
Even the standard feedback is good feedback. For those wishing to dig deeper, you can try
- to use "make VERBOSE=1" when building hugin.
- to do a debug build "cmake -DCMAKE_BUILD_TYPE=DEBUG" and use oprofile for profiling.