Green screen alternative -- Rear projection of 3D image rendered in real time

Davlon
Davlon Posts: 283 Just Starting Out

Too interesting  -- watch the video on the page:

https://www.redsharknews.com/production/item/6575-most-of-this-set-is-cgi-but-it-was-all-filmed-in-camera-in-realtime

"The concept is simple. There are four LED display walls, one behind the actor, two to either side, and one representing the sky. There is also lighting provided by Arri Skypanels. So far so simple.

What makes this all work is that the Unreal Engine 'virtual set' is displayed on the three LED panels, but the system also controls the Skypanel lighting as well. Camera movement is tracked in realtime, so that the viewpoint of the virtual set is moved according to its position. When you're sat on set it just looks like the virtual set is changing perspective. But look through the camera viewfinder and you discover that the set viewpoint is being moved with perfect positioning according to camera position. The result is that the background moves in perfect parallax."

 

Comments

  • Stargazer54
    Stargazer54 Posts: 3,758 Ambassador
    edited August 2019

     Pretty wicked stuff!  A similar rear projection set up was used on First Man.

    https://vfxscience.com/2019/02/25/first-man/

    There are several videos toward the bottom of the link that highlight the process.

  • Davlon
    Davlon Posts: 283 Just Starting Out

    He said it was a 35x60 foot LED screen.   How would that have been built  and color-calibrated so the edges of the panels are perfectly matched for color/contrast?

  • Stargazer54
    Stargazer54 Posts: 3,758 Ambassador

    @Davlon In the October 2018 issue of CineFex they did an article about First Man.  I looked back through it and there was no actual mention of the "secret sauce".   But here are the particulars:

    The LED wall was comprised of 1200 two foot by two foot panels with a 2.8 mm pixel pitch arrayed into a 180 degree half cylinder.  This amounted into 35 million pixels.   The LED panels were synced with Arri 416 cameras and "various" 16mm and 35 mm cameras.  The quote was, "It was tricky to get those to 'sing' in unison".  No kidding.

    The next mention I see sort of goes to your question.  The crew added color correction and camera exposure to make sure blacks were black and whites were white.  Extremely bright light sources tended to digitally "clamp" so they added "more dynamic range" to the plates (but this wasn't explained further).  It was admitted that by shooting on 35mm and16mm they were able to mask LED pixel separations within the film grain. 

    I'm sure the actual process is a carefully guarded secret, for now.  As mentioned in the video above, the crew was able to achieve a lot of in camera reflections "for free" that would have been hard to do or even left out in post.

  • Davlon
    Davlon Posts: 283 Just Starting Out

    Where is this panoramic LED surface now?  Don't tell me they also dismantled 1,200 two foot by two foot panels...

    Found these online:

    https://phantomdynamics.com/elation-evhd2-2-8mm-pixel-pitch-black-face-led-display/

    https://www.christiedigital.com/emea/digital-signage/visual-display-technology/resources/LED-displays

    https://www.christiedigital.com/en-us/digital-signage/products/led-tiles

    I believe they had to shoot film:  had they shot video, I'm guessing there would have been significant, and unpredictable moire due to the dot pitch of the screens interacting with the video scan lines.  Anyone can correct me on this.  

  • Stargazer54
    Stargazer54 Posts: 3,758 Ambassador

    The CineFex article said that the LED wall was built at Tyler Perry Studios in Atlanta.   Either it is still there (which I doubt) or has been dismantled.  

  • Triem23
    Triem23 Posts: 20,604 Ambassador

    I'd assume @Davlon is correct about needing to use film. Otherwise, you're in the same situation as "Gravity," which is the first film I can think of that used projected lightboxes on the actors. What happened on that sow was the lightboxes showing the virtual sets lit the actors perfectly... But then the actors had to be totally roto'd and re-comped into the background.

    Gravity was shot on digital.

    So, here's where we get to what @Stargazer54 wrote: "shooting on 35mm and16mm they were able to mask LED pixel separations within the film grain. "