Automatic occlusion removal from facades for 3D urban reconstruction
C. Engels, D. Tingdahl, M. Vercruysse, Tinne Tuytelaars, H. Sahli and Luc Van Gool
In: Advanced concepts for intelligent vision systems, 22-25 August 2011, Ghent, Belgium.

Abstract

Object removal and inpainting approaches typically require a user to manually create a mask around occluding objects. While cre- ating masks for a small number of images is possible, it rapidly becomes untenable for longer image sequences. Instead, we accomplish this step automatically using an object detection framework to explicitly recognize and remove several classes of occlusions. We propose using this tech- nique to improve 3D urban reconstruction from street level imagery, in which building facades are frequently occluded by vegetation or vehicles. By assuming facades in the background are planar, 3D scene estimation provides important context to the inpainting process by restricting input sample patches to regions that are coplanar to the occlusion, leading to more realistic nal textures. Moreover, because non-static and reflective occlusion classes tend to be dfficult to reconstruct, explicitly recognizing and removing them improves the resulting 3D scene.

[Edit]