User:Girlliyanli

From PanoTools.org Wiki
Jump to navigation Jump to search

My name is Yanli Li, I'm a chinese student, doing study on panorama.

Education Background

  • 1997.09-2000.07 High school student.
    • No.1 Middle School of KaiFeng County,Henai province, China
  • 2000.09-2004.07 Undergraduate.
  • 2004.09-2007.07 Postgraduate.
    • Human-Computer Interface and Virtual Reality Lab, Department of Computer Science,Shandong University,Jinan, Shandong, China
  • 2007.09- Doctoral student.

Programming Experience

  • My main programming language is c/c++. I started to program in it since 2001 and I have never stopped since then. Most of my designs are about image processing. My graduate design which was named System of Text-Image Preprocessing and awarded Shandong University'Excellent Undergraduate Design involved lots of knowledge about image's processing, so I know well about such field.
  • When I enrolled in HCI&VR lab in 2004. I began to do study on panorama. At first, I analyzed ptviewer's java code and bulit a panorama viewer system running on local machine after port the ptviewer's souce code from jave to c++. I also designed a scene-tour system based on panorama viewer. Later, I began to do study on image stitching. After studying the souce code of Hugin, sift, panotools and enblend, I built a panoramic stitching system based on vc.net. It is a totally automatic system: the framework I used is hugin, the control point I used is sift, the matching part is based on kd tree, the blending method I used is Multiresolution Spline, and modification was made on RANSAC so that outliners can been removed more quickly.

Programming experience about panorama viewer

  • The first panorama material I consulted wasBig Ben’s panorama tutorial.The first system about panorama I programmed was about panoramic viewer. As we all know,ptviewerwas used to view panorama, but the source code I could find then was wrote in java which was from a java applet. So it seemed if we wanted to view panorama on local machine, we had to port the ptviewer from java to C++. Then it took me about two month to analyze the interface of the source code and wrote some documents about its structure and built a framework and designed several classes and wrote the detail code and remove small bug. Although my tutor advised me use direct3d to speed up. I found it worked well without the direct3D. So all the source code is c++.The principle of ptviewer I think is as following:what should been inputted is a spherical panorama, what we want to get is a frame on the screen. There is a virtual sphere between them. The frame can be created according three parameter of the virtual sphere: pan angle, tilt angle and hfov angle which can be controlled on VC.NET by mouse event, keyboard event and menu or tool. At first, the panorama are projected on the virtual sphere according to two formula, then the frame are projected to the virtual sphere according to two formula controlled by the three parameters, now the relationship between a frame and panorama are built. With the backward projection, we can get the corresponding pixel from panorama to the frame.
  • One panorama can only cover view from one site, the information is not enough. If we want to tour around a scene, what should to do? Yes, it needs us to do more. Following my tutor's advise, I built a scene-tour system. The scene tour system comprises two parts, one is to build the scene tour, being called scene-tour designer, another is to view the scene tour, being called scene-tour shower. It justly involved interface design without any intricate theory: At first, we should get some material, including the scene's map and several panoramas taken from the scene. Secondly, we input the map and build the relationship between the map's site and corresponding panorama with the scene-tour designer and adjust the directions. Last, we saved and get a file end with ".tour", whose structure we have defined. Now we can open this file on scene-tour shower and tour around the scene.

Programming experience about panoramic stitching

Spherical panoramas which can be viewed through ptviewer are usually stitched from images captured with fisheye cameras. It is inconvenient, expensive and complex. Can we stitch overlapped images taken with hand-hold cameras? Under the guide of this goal, I consulted many articles. M.Brown’s recognizing panoramawas recommended by my tutor. He said if I could build a system according to this article, we would make it. The other materials he recommended are Hugin, panotools ,autopano-sift, Enblend,So I began to do it step by step.

  • Firstly, we should extract control points(sift) from each image. I read Prof.Lown's sift and autopano-sift's source code carefully and knew clearly how it works. Then I began to port it from c# to c++. It took me more than one month. Later on, the program was modified from time to time, being more efficient and more easily read.
  • Secondly, key points should been matched. M.Brown matched key points with Best Bin Fast. I download a source code about kd tree fromANN and found it worked well and fast. As for the outliners' removing(unmatched pairs), we made modification on RANSAC to removed them and our experiment verified its efficiency.
  • Thirdly, we should calculate parameters, so that we can build relationship among those images with those parameter. M.Brown used bundle-adjustment to calculate parameter. It is hard for me to program it and I did not find available source code. Detaining for a month, I changed my mind and started to build a comparatively easier system. It is limited and partial automatic. One of inputted image should be a central image, that is, other images are all overlapped with it. The system can recognize this central image and stitching them together. The projection we used is 8-parameters matrix which is calculate with LM.
  • In fact, I have never given up building up a totally automatic system.Thanks to Hugin, I made it at last. After downloading hugin,vigra and wxwindow. I compiled them successfully on VC.net. I debugged it step by step and gradually realized how it worked. Then I began to extract useful code I needed and found the system was monolithic. So I decided to build my system on it. I removed the wxwindow and built an interface with MSVC, I also integrated sift's code and Enblend'code into this framwork. The interface is simple and the system is totally automatic, after inputting several images, the system can divided them into parts and stitch each part automatically into cylindrical or planar or spherical panorama.

I know hugin is a GUI using panotools as core engine, which is a wonderful core code. I have never stop analyzing the principle of it and knows its interface well. I also want to rebuilt it, hoping it more easily used in the future for me.

Google Summer of Code 2007

After three years study of panorama, I know that panotools is the most wonderfully open source code and Hugin is the greatest GUI open source code. I appreciate their contribution. But neither of them is perfect. There are much work to do to expend them. Those subprojects showed on websites are all very important. I am sure once they are completed. It will be a milestone in the field of panorama and benefits all researchers. Because I analyzed panotools and hugin for a while and knew them well. I have eager to do something on it, so I apply for this project, hoping have a chance. The items I am interested:

  • Architectural Overhaul of Panotools I have never stopped studying panotool. I read it from time to time and find it amazing. Once I wrote a rough description about its functions'interface. I am passionate with this project.
    • Phase 1(1-2 weeks): Reading source code of panotools and analysing its structure.Writing a document about its structure and interface' description.
    • Phase 2(6-7 weeks): Rebuilting panotools, removing unused source code and writing a clear interface.
    • Phase 3(4-5 weeks): Optimising the souce code and writing commends.Writng document about new panotools.
    • CLASSES:
      • CImg {width,height,**pdata,pitch,yaw,roll,hfov,type}
      • CCtrolPoint {ImgN1,ImgN2,x1,y1,x2,y2}
      • CBlock {CImg* imgarr,CCtrolpoint *cparr}
    • METHODS:
      • GetCP_sift(CImg* imgarr,CBlock* blockarr)
      • GetCP_surf(CImg* imgarr,CBlock* blockarr)
      • Optimise(CBlock block)
      • Stitching(CImg* imgarr, CImg &output)
      • Remap(CImg src,CImg &des)
  • Automatic feature detection for panoramic images The key point I studied most is SIFT(2004). I am now studying SURF(2006)which is said to be faster and more efficient. We all know if the stitching process can be automatic or not mostly depends on if the control point can be extract properly and matched correctly. So it is a very important job. But up to now, there are still some conditions SIFT can work well such as the camera moves largely. Because I have programmed on sift and consulted many materials about other key points, I think it is interesting and I want to continue studying on this field.
  • Anti-ghosting HDR panorama blending and merging algorithm I studied and programmed multiple resolution spine(1983),which was widely used as a better blending method because it can avoid slight ghost and blend seamlessly. The source code I integrated in my system is from Enblend. But when the images are to stitch have a running bus, how can we remove it? I know this problem is tough to handle, but it should be solved, because it's common that the images to be stitched have a moving object. I have some knowledge of image blending and am eager to solve the above problem.

I am very glad when learning Hugin/panotools participates in this year's GSoC. Because I am always studying hugin/panotools and want to learn them more. I am eager to colaborate with people who also interested on this field. I think this opportunity is very meaningful for me, so I will devote myself fully on this project once I am accepted. I am totally free during this holiday. Three month or even more is no problem. That period is long enough for me to finish a task, I am confident. I am sure I wouldn't let you down.

Contact

  • msn:girlliyanli@hotmail.com
  • gtalk：girlliyanli@gmail.com
Retrieved from "https://wiki.panotools.org/index.php?title=User:Girlliyanli&oldid=8768"