User:Girlliyanli

My name is Yanli Li, I'm a chinese student, doing study on panorama.

Education Background

• 1997.09-2000.07 High school student. No.1 Middle School of KaiFeng County,Henai province, China
• 2000.09-2004.07 Undergraduate. Department of Computer Science,Shandong University,Jinan, Shandong province, China
• 2004.09-2007.07 Postgraduate. Human-Computer Interface and Virtual Reality Lab, Department of Computer Science,Shandong University,Jinan, Shandong, China
• 2007.09- Doctoral student. Virtual Reality lab(one of state key labs),Beijing University of Aeronautics and Astronautics,Beijing,China

Programming Experience

• My main programming language is c/c++. I started to program in it since 2001 and I have never stopped since then. Most of my designs are about image processing. My graduate design which was named System of Text-Image Preprocessing and awarded Shandong University'Excellent Undergraduate Design involved lots of knowledge about image's processing, so I know well about such field.
• When I enrolled in HCI&VR lab in 2004. I began to do study on panorama. At first, I analyzed ptviewer's java code and bulit a panoramic view system running on local machine after port the souce code from jave to c. I also designed a scene-tour system based on panoramic viewer. Later, I began to do study on panoramic stitching. After studying the souce code of Hugin, sift, panotools and enblend, I built a panoramic stitching system based on vc.net to stitch images. It is a totally automatic system: the framework I used is hugin, the control point I used is sift, the matching part is based on kd tree, the blending method I used is Multiresolution Spline, and modification was made on RANSAC so that outliners can been removed more quickly.

Panoramic viewer

• The first panoramic material I consulted wasBig Ben’s panorama tutorial.The first system about panorama I programmed was about panoramic viewer. As we all know,ptviewerwas used to view panorama, but the source code I could find then was wrote in java which was from a java applet. So it seemed if we wanted to view panorama on local machine, we had to port the ptviewer from java to C++. Then it took me about two month to analyze the interface of the source code and wrote some documents about its structure and built a framework and designed several classes and wrote the detail code and remove small bug. Although my tutor advised me use direct3d to speed up. I found it worked well without the direct3D. So all the source code is c++.The principle of ptviewer I think is as following:what should been inputted is a spherical panorama, what we want to get is a frame on the screen. There is a virtual sphere between them. The frame can be created according three parameter of the virtual sphere: pan angle, tilt angle and hfov angle which can be controlled on VC.NET by mouse event, keyboard event and menu or tool. At first, the panorama are projected on the virtual sphere according to two formula, then the frame are projected to the virtual sphere according to two formula controlled by the three parameters, now the relationship between a frame and panorama are built. With the backward projection, we can get the corresponding pixel from panorama to the frame.
• One panorama can only cover view from one site, the information is not enough. If we want to tour around a scene, what should to do? Yes, it needs us to do more. Following my tutor advice, I built a scene-tour system. The scene tour system comprises two parts, one is to build the scene tour, being called scene-tour designer, another is to view the scene tour, being called scene-tour shower. It justly involved interface design without any intricate theory: At first, we should get some material, including the scene's map and several panoramas taken from the scene. Secondly, we input the map and build the relationship between the map's site and corresponding panorama with the scene-tour designer and adjust the directions. Last, we saved and get a file end with ".tour", which is structure we defined. Now we can view this file on scene-tour shower and tour around the scene. To build a demo, I downloaded material from a famous Chinese panoramic websiteJietu

Panoramic stitching

Spherical panoramas which can be viewed through ptviewer are usually stitched from images captured with fisheye cameras. It is inconvenient, expensive and complex. Can we stitch overlapped images taken with hand-hold cameras? Under the guide of this goal, I consulted many articles. M.Brown’s recognizing panoramawas recommended by my tutor. He said if I could build a system according to this article, we would make it. So I began to do it step by step.

• Firstly, we should extract control points(sift) from each image. I read Prof.Lown's siftcarefully and knew clearly how it works. Then I searched on web and found autopano-sift,which is source code of sift in c#. It took me more than one month port it from c# to c++. Later on, the program was modified from time to time, being more efficient and more easily read.
• Secondly, key points should been matched. M.Brown matched key points with Best Bin Fast. I download a source code about kd tree fromANN and found it worked well and fast. As for the outliners' removing(unmatched pairs), we made modification on RANSAC to removed them and our experiment verified its efficiency.
• Thirdly, we should calculate parameters, so that we can build relationship among those images with those parameter. M.Brown used bundle-adjustment to calculate parameter. It is hard for me to realize it and I did not find available source code. Detaining for a month, I changed my mind and started to build a comparatively easier system. It is limited and partial automatic. One of inputted image should be a central image, that is, other images are all overlapped with it. The system can recognize this central image and stitching them together. The projection we used is 8-parameters matrix which is calculate with LM.
• In fact, I have never given up building up a totally automatic system.Thanks to Hugin, I made it at last. After downloading hugin and download vigra and wxwindow. I compiled them successfully with VC.net. I debugged it step by step and gradually realize how it works. Then I began to extract useful code I needed and found the system monolithic. So I decided to build my system on it, using hugin as framework and integrating sift's code and blending'code from Enblend.

I know hugin is a GUI based on panotools, which is a wonderful core code. I have never stop analyzing the principle of it.

Google Summer of Code 2007

Later