SoC 2008 Project OpenGL Preview - Revision history

James Legg: Update since GSOC is complete. Links to soon-to-be new pages added.

2008-08-21T17:46:27Z

Update since GSOC is complete. Links to soon-to-be new pages added.

James Legg: /* Modules */ Updated to reflect current state.

2008-06-30T00:11:27Z

‎Modules: Updated to reflect current state.

James Legg: Added modules suggestion

2008-05-20T23:48:53Z

Added modules suggestion

James Legg: /* Distortions Comparison */ Added a comparison for a rectilinear image at the zenith -> equirectangular

2008-05-13T09:26:42Z

‎Distortions Comparison: Added a comparison for a rectilinear image at the zenith -> equirectangular

James Legg: Added to Community:Project category

2008-05-06T22:10:22Z

Added to Community:Project category

James Legg: Distortion comparison of approximate image transformations, and added somelinks

2008-05-06T20:05:10Z

Distortion comparison of approximate image transformations, and added somelinks

James Legg: blender diagrams for image transformations, more on image transformations

2008-05-06T18:48:25Z

blender diagrams for image transformations, more on image transformations

James Legg: Intro, started explaning image transformations

2008-05-06T17:54:29Z

Intro, started explaning image transformations

New page

The OpenGL Preview project aims to create a fast previewer for Hugin, reducing the time taken to redraw the preview to real time rates. After that, the new previewer can be made more interactive than the current one.

==Image Transformations==
The new previewer will have to approximate the transformations of the images to fit on low resolution meshes. There are two methods for this:

===Remapping Vertices===
Taking a point in the input image, we can find where the point is transformed to on the final panorama. Therefore, we can take a uniform grid over an input image, and create a mesh that shows remaps this grid into the output projection.

This method suffers when the input image crosses certain points in the panorama. For example over the +/-180&deg boundary in a equirectangular image, there will be faces connecting one end to the other, and nothing between the edges of those faces and the actual boundary. Also the poles of an equirectangular image will cause similar problems.

===Remapping Texture Coordinates===
Alternatively, given a point on the final panorama, we can calculate what part of an input image belongs there. Therefore, we can take a uniform grid over the panorama, and map the input image across it.

This does not suffer from problems at poles or borders of the panorama, but instead it breaks down at the borders of the input image.

===Distortions Comparison===
To check that these transformation methods do not distort the image to much, I devised a test.
Using nona to produce a mapping from input image coordinates to output image coordinates, and also creating a projection that goes the other way, I tried this in Blender.

@@ Line 1: / Line 1: @@
-The OpenGL Preview project aims to create a fast previewer for [[Hugin]], reducing the time taken to redraw the preview to real time rates. After that, the new previewer can be made more interactive than the current one. The project is James Legg's [[SoC 2008 overview|Google Summer of Code 2008]] project, mentored by Pablo d'Angelo.
+[[Hugin]]'s OpenGL Preview is a fast but approximate previewer, which is more interactive than the previous one. It was James Legg's [[SoC 2008 overview|Google Summer of Code 2008]] project, mentored by Pablo d'Angelo.
 ==Image Transformations==
-The new previewer will have to approximate the [[remapping]] of the images to fit on low resolution meshes. There are two methods for this:
+The new previewer will have to approximate the [[remapping]] of the images to fit on low resolution meshes. There are two methods it uses for this:
 ===Remapping Vertices===
@@ Line 10: / Line 10: @@
 '''Disadvantages''':
-*This method suffers when the input image crosses over the +/-180&deg; boundary in projections such as [[Equirectangular Projection]], as there will be faces connecting one end to the other, and nothing between the edges of those faces and the actual boundary. We must split the mesh so that it each part is continuous, which means some faces will have to be defined off the edge of the panorama.
+*This method suffers when the input image crosses over the +/-180&deg; boundary in projections such as [[Equirectangular Projection]], as there will be faces with vertices on either side of the seam. In this case the previewer replaces the face with two faces, one for each side.
-*The poles of an equirectangular image will also cause similar problems. At a pole, the image should cover the whole width of the panorama, but this is only one point in the input image. Vertices in the mesh near the pole are define a face that does not cover the whole row.
+*The poles of an equirectangular image will also cause similar problems. At a pole, the image should cover the whole width of the panorama, but this is only one point in the input image. Vertices in the mesh near the pole are define a face that does not cover the whole row. Also the outer rings of disk like projections are poorly done.
 *The detail in the faces might not match up well with the area of the panorama. The faces are all of different sizes and some mappings may put the details in parts that can't be seen as easily as the lower resolution parts.
 '''Advantages''':
-*The mesh resembles the output projection well, and covers up roughly the same area as the correct projection. The edges of the mesh lies along the edges of the image, so we don't need to worry about what happens outside of it.
+*The mesh resembles the output projection well, and covers up roughly the same area as the correct projection. The edges of the mesh lies along the edges of the image, so we don't need to worry about what happens outside of it (except with circular cropping). Most warped images can be displayed reasonably well using this method.
 ===Remapping Texture Coordinates===
@@ Line 22: / Line 25: @@
 '''Disadvantages''':
-*This mapping breaks down at the borders of the input image. The borders of the input image may cross the output mesh at any position, so we need to draw the image  with a transparent border.
+*This mapping breaks down at the borders of the input image. The borders of the input image may cross the output mesh at any position. We clip the faces to the input image to avoid this problem.
-*We must also calculate the extent of the images in the result so the mesh only covers the minimal area.
+*This is inefficient for small images, as they do not cover much of the panorama. Many faces are considered outside of the input image and dropped during clipping.
-*Some faces of the minimal bounding rectangle still don't contain any of the input image at all, processing them would be a waste of time.
+*The quality of the shape of curved images is a little lower than with the vertex coordinate remapping.
 '''Advantages''':
 *There are no problems at the poles and edges of the panorama.
 *The output is of uniform quality throughout, as the faces always have the same density.
 ===Distortions Comparison===
@@ Line 53: / Line 59: @@
 This mesh has 16 rows of faces and 16 columns, spread equally over the output panorama, so there are 256 faces which cover an equal amount of the output.
-Note that at this point I have not attempted to correct the texture coordinates around the boundary of the input image.
+Note that for these images I have not attempted to correct the texture coordinates around the boundary of the input image.
 |}
@@ Line 81: / Line 87: @@
 ==Modules==
-As there are many good ideas for the future of a graphically-accelerated Hugin after this project is complete (See  the [http://groups.google.com/group/hugin-ptx/browse_thread/thread/7c838b0cda383922 discussion on the mailing list]), it is clear that a lot of development can occur using OpenGL and the new preview after I have completed this project. Therefore it is especially important that I provide clean, highly modular and extensible code for future use. Here is how the modules and their interfaces look:
+See main article: [[Hugin's OpenGL preview system overview]]
-−
-; Texture Manager
-: The texture manager is built on top of Hugin's ImageCache, and will perform a similar function but using graphics memory and textures. It is able to set the active texture to one that represents an image. It adjusts the size of the textures it is storing so they fill a sensible amount of memory and provide a even level of detail for the images. This is partially complete, it works fine with real time photometric adjustment (only corrects exposure and white balance) and, photometric correction is in progress, and a logarithmic display for high dynamic range images may be yet to come.
-−
-; Mesh Manager
-: The mesh manager sets up lists of instructions for the graphics system to draw a given image, remapped how it should be in the panorama's output. It uses a mesh remapper for every image to find what faces it needs. This has been implemented.
-−
-; Mesh Remapper
-: This provides the coordinates of points in a mesh. It splits the image into faces that approximate the transformation using the remapping with vertices method above. However, the rectangles of the input image that have their corners transformed to make the face vertices are sized adaptively: in the areas where the remapping is almost linear there are less faces. Also the size of the faces is limited when they become too small to be noticeable. This is partially complete.
-−
-; Renderer
-: This is responsible for drawing the panorama, although it only really compiles a list of render layers and draws them when asked to. Currently it just draws all the active images, as the render layers are not implemented yet.
-−
-; Render Layers
-: Everything the renderer draws would have been put there by a render layer. As the name suggests, they are ordered and can cover each other up. At the end of the project there will obviously be a layer to draw the selected images. Other layers provide indication of image outlines, or perhaps redraw an image in a specific way. Not implemented yet.
-−
-; Image Render Hooks
-: When drawing, we may want to do something different for a selection of the images. A render hook can be set up that changes how an image is rendered. It would provide callbacks for before and after drawing, a condition to drop an image from rendering altogether, or a replacement function for drawing it. An example would be to provide different blending modes. We could subtract an image from the ones behind it by dropping it from the images render layer, and then drawing it again in a different render layer, but subtracting it from the layers beneath. Alternatively, if we wanted something drawn in place but semitransparent, we could turn on blending before it is drawn and turn it off afterwards. These are not implemented yet.
-−
-; Viewer
-: The viewer combines together a renderer and a some basic interface for the tools. It tells the renderer when to render, but just before it does so it lets the mesh and texture managers update. It also stores the view state object. It will be able convert between a mouse position and a position in the panorama. It is implemented as a wxWidget.
-−
-; Input Manager
-: An input manager gets all keyboard and mouse events the viewer sees and passes them to the correct tool. Tools register events they want. When tools are disabled they should give up their events, which can then be taken by a newly activated tool. Not implemented yet
-−
-; Tools
-: Tools can be enabled or disabled. When switching states they must set up or turn off events in the input manager, render layers, and image render hooks. They get access to the panorama data and viewer data. None are implemented yet. A image drag / recentre tool, and a cropping tool is planned. Other tools could be built to work with control points and selection of images for example.
-; GLPreviewPanel
+==Building and Usage==
-: This is a replacement PreviewPanel. It holds the viewer and gives it some screen space. It shows controls for the viewer and tools. It will keep an input manager to decide what to do with user input events. Some tools may be mutually exclusive, for example there might be a few that want to handle mouse events. The preview panel should turn off one tool before starting another in this case.
+The OpenGL preview is not in the main Hugin branch yet. If you wish to experiment, and don't mind compiling it yourself, you can check out the latest version using svn with:
-; View State
+See [[Hugin Fast Preview window]] for usage instructions.
-: As the user changes properties of the panorama, we want something to accept the modifications in real time to control what is drawn. This state should differ from the state of the panorama so the user can cancel their changes and make visible changes during dragging things with the mouse (without creating an undo step for each mouse motion). The ViewState object is what is used to hold the temporary state. It also updates to reflect the other conventional modifications. Finally, this can report what type of changes have been made to the panorama and therefore deduce what expensive calculations need to be done to get the preview up to date, and which things we previously calculated are still valid. It is implemented now.
 [[Category:Community:Project]]

@@ Line 81: / Line 81: @@
 ==Modules==
-As there are many good ideas for the future of a graphically-accelerated Hugin after this project is complete (See  the [http://groups.google.com/group/hugin-ptx/browse_thread/thread/7c838b0cda383922 discussion on the mailing list]), it is clear that a lot of development can occur using OpenGL and the new preview after I have completed this project. Therefore it is especially important that I provide clean, highly modular and extensible code for future use. Here is a proposal for some modules and interfaces:
+As there are many good ideas for the future of a graphically-accelerated Hugin after this project is complete (See  the [http://groups.google.com/group/hugin-ptx/browse_thread/thread/7c838b0cda383922 discussion on the mailing list]), it is clear that a lot of development can occur using OpenGL and the new preview after I have completed this project. Therefore it is especially important that I provide clean, highly modular and extensible code for future use. Here is how the modules and their interfaces look:
 ; Texture Manager
-: The texture manager will be built on top of Hugin's ImageCache, and will perform a similar function but using graphics memory and textures. It will be able to set the active texture to one that represents an image. It should be guaranteed to have a texture for each image when you want to use one, but it won't necessarily give you one that is a good size. It will adjust the size of the textures it is storing so they fill a sensible amount of memory and provide a even level of detail for the images.
+: The texture manager is built on top of Hugin's ImageCache, and will perform a similar function but using graphics memory and textures. It is able to set the active texture to one that represents an image. It adjusts the size of the textures it is storing so they fill a sensible amount of memory and provide a even level of detail for the images. This is partially complete, it works fine with real time photometric adjustment (only corrects exposure and white balance) and, photometric correction is in progress, and a logarithmic display for high dynamic range images may be yet to come.
 ; Mesh Manager
-: The mesh manager will set up lists of instructions for the graphics system to draw a given image, remapped how it should be in the panorama's output.
+: The mesh manager sets up lists of instructions for the graphics system to draw a given image, remapped how it should be in the panorama's output. It uses a mesh remapper for every image to find what faces it needs. This has been implemented.
-; Mesh Transformation Stack
+; Mesh Remapper
-: This will provide the coordinates of points in a mesh. It is a stack so that when only moving the image, we can just recalculate the top of the stack and not have to worry about how distortion parameters are affecting the image. The mesh manager uses the top of this transformation stack. This will also be where coverage tests are performed, so we can identify images under a point.
+: This provides the coordinates of points in a mesh. It splits the image into faces that approximate the transformation using the remapping with vertices method above. However, the rectangles of the input image that have their corners transformed to make the face vertices are sized adaptively: in the areas where the remapping is almost linear there are less faces. Also the size of the faces is limited when they become too small to be noticeable. This is partially complete.
 ; Renderer
-: This is responsible for drawing the panorama, although it only really compiles a list of render layers and draws them when asked to.
+: This is responsible for drawing the panorama, although it only really compiles a list of render layers and draws them when asked to. Currently it just draws all the active images, as the render layers are not implemented yet.
 ; Render Layers
-: Everything the renderer draws would have been put there by a render layer. As the name suggests, they are ordered and can cover each other up. At the end of the project there will obviously be a layer to draw the selected images. Other layers provide indication of image outlines, or perhaps redraw an image in a specific way.
+: Everything the renderer draws would have been put there by a render layer. As the name suggests, they are ordered and can cover each other up. At the end of the project there will obviously be a layer to draw the selected images. Other layers provide indication of image outlines, or perhaps redraw an image in a specific way. Not implemented yet.
 ; Image Render Hooks
-: When drawing, we may want to do something different for a selection of the images. A render hook can be set up that changes how an image is rendered. It can provide callbacks for before and after drawing, a condition to drop an image from rendering altogether, or a replacement function for drawing it. An example would be to provide different blending modes. We could subtract an image from the ones behind it by dropping it from the images render layer, and then drawing it again in a different render layer, but subtracting it from the layers beneath. Alternatively, if we wanted something drawn in place but semitransparent, we could turn on blending before it is drawn and turn it off afterwards.
+: When drawing, we may want to do something different for a selection of the images. A render hook can be set up that changes how an image is rendered. It would provide callbacks for before and after drawing, a condition to drop an image from rendering altogether, or a replacement function for drawing it. An example would be to provide different blending modes. We could subtract an image from the ones behind it by dropping it from the images render layer, and then drawing it again in a different render layer, but subtracting it from the layers beneath. Alternatively, if we wanted something drawn in place but semitransparent, we could turn on blending before it is drawn and turn it off afterwards. These are not implemented yet.
 ; Viewer
-: The viewer combines together a renderer and a some basic interface for the tools. It tells the renderer to redraw when the window needs redrawing. Also it stores where in the panorama the user is looking at and how far they are zoomed in. It can convert between a mouse position and a position in the panorama.
+: The viewer combines together a renderer and a some basic interface for the tools. It tells the renderer when to render, but just before it does so it lets the mesh and texture managers update. It also stores the view state object. It will be able convert between a mouse position and a position in the panorama. It is implemented as a wxWidget.
 ; Input Manager
-: An input manager gets all keyboard and mouse events the viewer sees and passes them to the correct tool. Tools register events they want. When tools are disabled they should give up their events, which can then be taken by a newly activated tool.
+: An input manager gets all keyboard and mouse events the viewer sees and passes them to the correct tool. Tools register events they want. When tools are disabled they should give up their events, which can then be taken by a newly activated tool. Not implemented yet
 ; Tools
-: Tools can be enabled or disabled. When switching states they must set up or turn off events in the input manager, render layers, and image render hooks. They get access to the panorama data and viewer data.
+: Tools can be enabled or disabled. When switching states they must set up or turn off events in the input manager, render layers, and image render hooks. They get access to the panorama data and viewer data. None are implemented yet. A image drag / recentre tool, and a cropping tool is planned. Other tools could be built to work with control points and selection of images for example.
 ; GLPreviewPanel
 : This is a replacement PreviewPanel. It holds the viewer and gives it some screen space. It shows controls for the viewer and tools. It will keep an input manager to decide what to do with user input events. Some tools may be mutually exclusive, for example there might be a few that want to handle mouse events. The preview panel should turn off one tool before starting another in this case.
-The PreviewFrame will then have an option to switch between the existing PreviewPanel and the new GLPreviewPanel.
+; View State
+: As the user changes properties of the panorama, we want something to accept the modifications in real time to control what is drawn. This state should differ from the state of the panorama so the user can cancel their changes and make visible changes during dragging things with the mouse (without creating an undo step for each mouse motion). The ViewState object is what is used to hold the temporary state. It also updates to reflect the other conventional modifications. Finally, this can report what type of changes have been made to the panorama and therefore deduce what expensive calculations need to be done to get the preview up to date, and which things we previously calculated are still valid. It is implemented now.
-===Events===
-Currently there is a PanoramaObserver class, children of this class are notified when the panorama changes. However the viewer is somewhat independent of the panorama, it even makes sense to have multiple viewers of the same panorama. Still, some objects would need to be notified of when the view changes. Therefore we will leave the viewing conditions to the viewer, and provide some more event notifications for when they change. This will be helpful for the texture and mesh managers for instance, since they can change the level of detail as the user zooms on an image. Also any tool that requires this information can also use it.
-−
-Additionally, a texture detail change event would be nice, so that when a higher detail texture is ready for use, we can immediately redraw with the higher resolution. This will allow adaptive detail, since we can delay getting the more detailed texture and keep the user interface interactive for a while, and switch when we are ready.
-Similarly a mesh detail event should be created, this will allow the window to be redrawn after a mesh has been regenerated. We may also need an idle event, so that when we are done processing important things we can consider updating the textures or meshes.
-−
-Perhaps it should be possible to delay these events, so that when the preview window is not visible we don't try to update anything. This would mean you can make many minor alterations to the panorama without waiting for any part of the preview to be recalculated, if it was out of sight.
 [[Category:Community:Project]]

← Older revision		Revision as of 23:48, 20 May 2008
Line 79:		Line 79:

	\|}		\|}
		+
		+	==Modules==
		+	As there are many good ideas for the future of a graphically-accelerated Hugin after this project is complete (See the [http://groups.google.com/group/hugin-ptx/browse_thread/thread/7c838b0cda383922 discussion on the mailing list]), it is clear that a lot of development can occur using OpenGL and the new preview after I have completed this project. Therefore it is especially important that I provide clean, highly modular and extensible code for future use. Here is a proposal for some modules and interfaces:
		+
		+	; Texture Manager
		+	: The texture manager will be built on top of Hugin's ImageCache, and will perform a similar function but using graphics memory and textures. It will be able to set the active texture to one that represents an image. It should be guaranteed to have a texture for each image when you want to use one, but it won't necessarily give you one that is a good size. It will adjust the size of the textures it is storing so they fill a sensible amount of memory and provide a even level of detail for the images.
		+
		+	; Mesh Manager
		+	: The mesh manager will set up lists of instructions for the graphics system to draw a given image, remapped how it should be in the panorama's output.
		+
		+	; Mesh Transformation Stack
		+	: This will provide the coordinates of points in a mesh. It is a stack so that when only moving the image, we can just recalculate the top of the stack and not have to worry about how distortion parameters are affecting the image. The mesh manager uses the top of this transformation stack. This will also be where coverage tests are performed, so we can identify images under a point.
		+
		+	; Renderer
		+	: This is responsible for drawing the panorama, although it only really compiles a list of render layers and draws them when asked to.
		+
		+	; Render Layers
		+	: Everything the renderer draws would have been put there by a render layer. As the name suggests, they are ordered and can cover each other up. At the end of the project there will obviously be a layer to draw the selected images. Other layers provide indication of image outlines, or perhaps redraw an image in a specific way.
		+
		+	; Image Render Hooks
		+	: When drawing, we may want to do something different for a selection of the images. A render hook can be set up that changes how an image is rendered. It can provide callbacks for before and after drawing, a condition to drop an image from rendering altogether, or a replacement function for drawing it. An example would be to provide different blending modes. We could subtract an image from the ones behind it by dropping it from the images render layer, and then drawing it again in a different render layer, but subtracting it from the layers beneath. Alternatively, if we wanted something drawn in place but semitransparent, we could turn on blending before it is drawn and turn it off afterwards.
		+
		+	; Viewer
		+	: The viewer combines together a renderer and a some basic interface for the tools. It tells the renderer to redraw when the window needs redrawing. Also it stores where in the panorama the user is looking at and how far they are zoomed in. It can convert between a mouse position and a position in the panorama.
		+
		+	; Input Manager
		+	: An input manager gets all keyboard and mouse events the viewer sees and passes them to the correct tool. Tools register events they want. When tools are disabled they should give up their events, which can then be taken by a newly activated tool.
		+
		+	; Tools
		+	: Tools can be enabled or disabled. When switching states they must set up or turn off events in the input manager, render layers, and image render hooks. They get access to the panorama data and viewer data.
		+
		+	; GLPreviewPanel
		+	: This is a replacement PreviewPanel. It holds the viewer and gives it some screen space. It shows controls for the viewer and tools. It will keep an input manager to decide what to do with user input events. Some tools may be mutually exclusive, for example there might be a few that want to handle mouse events. The preview panel should turn off one tool before starting another in this case.
		+
		+	The PreviewFrame will then have an option to switch between the existing PreviewPanel and the new GLPreviewPanel.
		+
		+	===Events===
		+	Currently there is a PanoramaObserver class, children of this class are notified when the panorama changes. However the viewer is somewhat independent of the panorama, it even makes sense to have multiple viewers of the same panorama. Still, some objects would need to be notified of when the view changes. Therefore we will leave the viewing conditions to the viewer, and provide some more event notifications for when they change. This will be helpful for the texture and mesh managers for instance, since they can change the level of detail as the user zooms on an image. Also any tool that requires this information can also use it.
		+
		+	Additionally, a texture detail change event would be nice, so that when a higher detail texture is ready for use, we can immediately redraw with the higher resolution. This will allow adaptive detail, since we can delay getting the more detailed texture and keep the user interface interactive for a while, and switch when we are ready.
		+	Similarly a mesh detail event should be created, this will allow the window to be redrawn after a mesh has been regenerated. We may also need an idle event, so that when we are done processing important things we can consider updating the textures or meshes.
		+
		+	Perhaps it should be possible to delay these events, so that when the preview window is not visible we don't try to update anything. This would mean you can make many minor alterations to the panorama without waiting for any part of the preview to be recalculated, if it was out of sight.

	[[Category:Community:Project]]		[[Category:Community:Project]]

@@ Line 51: / Line 51: @@
 |'''Texture Coordinate mapping Approximate Transformation'''
-This mesh as 16 rows of faces and 16 columns, spread equally over the output panorama, so there are 256 faces which cover an equal amount of the output.
+This mesh has 16 rows of faces and 16 columns, spread equally over the output panorama, so there are 256 faces which cover an equal amount of the output.
 Note that at this point I have not attempted to correct the texture coordinates around the boundary of the input image.
 |}
 [[Category:Community:Project]]

@@ Line 48: / Line 48: @@
 |'''Transformation with Nona'''
-This is the remapping the other images are approximating.
+This is the remapping that the other images are approximating.
 |'''Texture Coordinate mapping Approximate Transformation'''
@@ Line 55: / Line 55: @@
 Note that at this point I have not attempted to correct the texture coordinates around the boundary of the input image.
 |}

@@ Line 5: / Line 5: @@
 ===Remapping Vertices===
 Taking a point in the input image, we can find where the point is transformed to on the final panorama. Therefore, we can take a uniform grid over an input image, and create a mesh that shows remaps this grid into the output projection.
-This method suffers when the input image crosses certain points in the panorama. For example over the +/-180&deg boundary in a equirectangular image, there will be faces connecting one end to the other, and nothing between the edges of those faces and the actual boundary. Also the poles of an equirectangular image will cause similar problems.
+'''Disadvantages''':
 ===Remapping Texture Coordinates===
 Alternatively, given a point on the final panorama, we can calculate what part of an input image belongs there. Therefore, we can take a uniform grid over the panorama, and map the input image across it.
-This does not suffer from problems at poles or borders of the panorama, but instead it breaks down at the borders of the input image.
+'''Disadvantages''':
 ===Distortions Comparison===
-To check that these transformation methods do not distort the image to much, I devised a test.
+To check that these transformation methods do not distort the image too much, I devised a test.
-Using nona to produce a mapping from input image coordinates to output image coordinates, and also creating a projection that goes the other way, I tried this in Blender.
+Using nona to produce a coordinate transformation map from input image coordinates to output image coordinates, and similarly creating a projection that goes the other way, I tried these transformations in Blender.