User:Girlliyanli

My name is Yanli Li, I'm a chinese student, doing study on panorama.

Education Background

• 1997.09-2000.07 High school student. No.1 Middle School of KaiFeng County,Henai province, China
• 2000.09-2004.07 Undergraduate. Department of Computer Science,Shandong University, Jinan, Shandong province, China
• 2004.09-2007.07 Postgraduate. Human-Computer Interface and Virtual Reality Lab, Department of Computer Science, Shandong University, Jinan, Shandong, China
• 2007.09- Doctoral student. Virtual Reality lab(one of state key labs), Beijing University of Aeronautics and Astronautics, Beijing,China

Programming Experience

My favorite programming language is c/c++. I started to program in it since 2001 and I have never stopped since then. Most of my designs are about image processing. My graduate design which was named System of Text-Image Preprocessing and awarded Shandong University'Excellent Undergraduate Design involved lots of knowledge about image's processing, so I know well about such field. When I enrolled in HCI&VR lab in 2004. I began to do study on panorama. At first, I analyzed ptviewer's java code and bulit a panoramic view system running on local machine after port the souce code from jave to c. I also designed a scene-tour system based on panoramic viewer. Later, I began to do study on panoramic stitching. After studying the souce code of Hugin, sift, panotools and enblend, I built a panoramic stitching system based on vc.net to stitch images. It is a totally automatic system: the framework I used is hugin, the control point I used is sift, the matching part is based on kd tree, the blending method I used is Multiresolution Spline, and modification was made on RANSAC so that outliners can been removed more quickly.

Panoramic viewer

I firstly knew the word "Panorama" was on the lab's discussion group, where my tutor asked me if I had interest to do research on panorama. At that time I knew nothing about it, but had eager to learn a little. So my tutor gave me some material, one is Big Ben’s panorama tutorial[1],the other is panotools. Later, I found panorama was such an amazing image, especially when I viewed it through ptviewer. I thought it will be funny to know more about it. Then my tutor said to me, ptviewer was used to view panorama, it had source code, but was wrote in java, so that the java applet could be easily used on website. He gave me another panoramic viewer PFSview; it could be run on local machine but without source code. It seemed if we wanted to view panorama on local machine, we had to port the ptviewer from java to C++. Then it took me about two month to analyze the interface of the source code and wrote some documents about its structure and built a framework and designed several classes and wrote the detail code and remove small bug. Although my tutor advised me use direct3d to speed up. I found it worked well without the direct3D.After that period, now I know well about its principle: what should been inputted is only a spherical panorama, what we want to get is the image on the screen, which can be called view-image, we can built the two images ‘relationship, in fact, there is a virtual sphere between them. As the change of three parameter of the virtual sphere: pan angle, tilt angle and hfov angle, those change being controlled in VC.NET by mouse event, keyboard event and menu or tool, there are some mathematic formula handling the pixels' correspond between panoramic image and view image. Detail information can been found (here?).

One panorama can only cover view from one site, the information is not enough. If we want to tour around a scene, what should to do? Yes, it needs us to do more. Following my tutor advice, I began to build scene-tour system. It took about two month to finish such a job. The scene tour system comprises two parts, one is to build the scene tour, being called scene-tour designer, another is to view the scene tour, being called scene-tour shower. It justly involved interface design without any intricate theory: At first, we should get some material, including the scene's map and several panoramas taken from the scene. Secondly, we input the map and build the relationship between the map's site and corresponding panorama with the scene-tour designer and adjust the directions. Last, we saved and get a file end with ".tour", which is structure we defined. Now we can view this file on scene-tour shower and tour around the scene. To build a demo, I downloaded material from a famous Chinese panoramic website-Jietu [2].

Panoramic stitching

It’s natural for us to continually study panorama. Of course, the next step is acquiring a panorama. Spherical panoramas which can be viewed through ptviewer are usually stitched from images captured with fisheye cameras. It is inconvenient, expensive and complex. Can we stitch overlapped images taken with hand-hold cameras? Under the guide of this goal, I consulted many articles. M.Brown’s recognizing panoramawas recommended by my tutor. He said if I could build a system according to this article, we would make it. So I began to do it step by step. Firstly, we should extract control points(sift) from each image. I read Prof.Lown's sift carefully and knew clearly how it works. But it was still difficulty for me to program it all by myself. So I searched on web and found autopano-sift. It provided us with source code of sift in c#. I was not afraid of porting it to C++. It took me more than one month made it. Later on, I modified the program from time to time, optimizing it and making it more easily read. Secondly, key points should been matched. M.Brown matched key points with Best Bin Fast. On one discussion group, a doctoral student delivered a speech on his research, of which I learned kd tree works well on multiple dimension searching and there were much material on it. I download a source code fromANNand compiled it, finding it indeed worked well and fast. As for the outliners' removing(unmatched pairs), I made modification on RANSAC to removed them and our experiment verified its efficiency. Thirdly, we should calculate parameters, so that we can build relationship among those images with those parameter. M.Brown used bundle-adjustment to calculate parameter. It is hard for me to realize it and I did not find available source code. Detaining for a month, I changed my mind and started to build a comparatively easier system. It is limited and partial automatic. One of inputted image should be a central image, that is, other images are all overlapped with it. The system can recognize this central image and stitching them together. The projection I used is 8-parameters matrix which is calculate with LM.In fact, I have never given up building up a totally automatic system. Thanks to Hugin, I made it at last. After downloading hugin and download vigra and wxwindow. I compiled them successfully with VC.net. I debugged it step by step and gradually realize how it works. Then I began to extract useful code I needed and found the system agglutinate. So I decided to build my system on it, using hugin as framework and combining sift’ code and blending code from Enblend and removed some unused function such as interface and wxwindow. I did it within two month. I know hugin is based on panotools, which is a wonderful core code. It needs us to read it carefully, utilize it fully and expanding it. I have never stop analyzing the principle of it.

Google Summer of Code 2007

Later