Geometry registration with occlusion resolving:

Step 1. REGISTER the coordinate systems between the source and target videos using the camera parameters.

Step 2. DENOTE the perspective transformation between the two coordinate systems as a 4X4 matrix PA.

Step 3. RESOLVE the the potential occlusion between the sprite and the separated layers in the target video.

Here I picked an image as the target video, and another image as the sprite:


The sprite.


The target image.


The depth map of target image (generated by ACTS)



References: Prof. Guofeng ZhangThe Refilming project