I agree that it should be possible, and wanted to see how it looks like in reality.
Then I imagined a head popping from a hole in a wall, exported a 3d image and transformed it quickly (alignment is not perfect) into a phantogram. The 3D scene does not look like I wanted, but the effect works.
I attached the test images if you want to see them. [parallel versions]
The image named 'scene' is the export from the modeling software, which also helps to see how it looks like with a regular camera.
The other image named 'result' is the phantogram to see from the side (image at 45° on your left, facing to your right).
It is not as pleasing to view as regular phantograms in my opinion, because turning the head down to view a figurine is quite natural, but in this case with the image on the left, I want to adjust my position to come back in front of it.