Working in 3D space

I am trying to produce a music video set in a 3D environment but am having trouble with getting my actors to stay in the proper spots.

I created the environment model in Sketchup, converted it to the proper format (fbx), and brought it into Hitfilm successfully, including textures. The model is a kind of mad scientist's lab where I want my characters to perform. Inside the lab, there are risers where my actors are supposed to stand during the song. I created a composite shot and added a camera layer. I inserted the model, into the composite shot and designated it as a 3D layer. When I brought the model in, I had to rotate it so it was oriented correctly in the 3D space.

I recorded an animated character against a green screen in a VR program called Puppet Play and exported a video. I brought the video into my composite shot and designated it as a 3D layer. I positioned the character onto one of the risers and designated the 3D lab model as the parent layer.

When I zoom in and out with the camera, my character does not stay on the riser. The character seems to zoom at a different rate than the 3D model, almost like it's on a different plane. I'm not sure what I am doing wrong, I used a tutorial (the one with the actor in the helicopter) and followed the directions but am not getting the results I expected. Any help would be appreciated

Answers

  • Stargazer54
    Stargazer54 Posts: 3,963 Ambassador

    @TheParkHopper Once you have initialized the 3D camera, make sure that any layers you want stacked in 3D space have their dimension set to 3D (which it sounds like you have already done). This will allow you to move them back or forward on the Z-axis.

    Once you have your 3D layers set up, it is helpful to change the camera view so that you are looking at the scene from the top or the side.

    You can also set up the Viewport to display both the camera view and other views from the side or top by selecting the drop down marked "View" along the bottom of the Viewport to have multiple views displayed.

    Also, keep in mind that if your models are not all saved relative to the same origin (0,0,0) you may have to fudge their position by making adjustments to the Anchor Point. Hopefully that helps.

  • TheParkH0pper
    TheParkH0pper Posts: 2 Just Starting Out*

    Thank you so much for your help! Sorry for the delayed response. I managed to get everything working and properly place all of my characters into the scene then keyframe camera movements and add lighting and couple of particle effects. One thing I discovered that I wasn't doing before was using the Unrolled 3D setting for my models, I had set them to 3D plane. This video is a big learning experience for me, it's really opening my eyes to some more advanced production processes. But perhaps I'm getting too ambitious for my system...

    Now that I have all my individual composite shots set up and keyframed, I have started the export process. The rendering is going pretty slowly, mainly due to the lighting and particle effects. Rendering without those goes pretty fast. Looking at my GPU and CPU usage, neither ever goes very high. Sometimes they are both under 10%, but never more than around 28-33%. The fans don't kick in because the card is running cool.

    I would have thought the GPU would be used more because of all the 3D elements. I have hardware acceleration turned on in HF settings. To render 3:44 of video, it's looking to take about 24-30 hours. I know my BIOS needs updating, but This is what I am running HitFilm Pro on.

    ------------------------------------------

    1.82TB SSD

    32GB RAM

    BIOS Version/Date   American Megatrends Inc. P4.10, 3/5/2020

    B450 Pro4 Motherboard

    AMD Ryzen 7 2700 Eight-Core Processor, 3200 Mhz, 8 Core(s), 16 Logical Processor(s)

     NVIDIA GeForce RTX 3060 (MSI)

    ------------------------------------------

    Do I have too many lights? I also have shadows turned on for characters and the set. The camera is the only thing that moves in 3D space in the shot, but slowly, and all of the media clips (characters) remain in their spots. This is the guitar composite shot I am currently rendering:

    I have looked through the forums and found some relevant information but still unsure how to get more usage out of the GPU, if at all possible.

  • Triem23
    Triem23 Posts: 20,394 Power User

    A few things to unpack here.

    Lets talk GPU usage first.

    Hitfilm has a bit of a handoff between CPU and GPU. The CPU reads media from source drives, does frame decoding (hardware accelerated video uses a CPU's iGPU, so this is still CPU), renders audio, calculates particle physics and does a few other things. While this happens the (dedicated) GPU is idle.

    The GPU then renders all video and effects filters. While this occurs the CPU is idle.

    The GPU passes frame renders back to the CPU and the CPU writes the output frame to the export file. While this happens the GPU is idle.

    To TL/DR it, Hitfilm's GPU/CPU handoff is what it is, isn't the most efficient use of resources, would basically require a complete ground up recode to change, and there isn't a magic setting to use more GPU. You can get more efficiency from Hitfilm but it's all based around preparing your media assets and project in the smart way.

    Some of the handoff implications here: if you have very high CPU usage - over 60% - it likely indicates either you just have a slow system, and/or you're using poorly optimized video assets (we're not going to fully discuss optimizing video here, but, long story short, video from phones, tablets, screen recorders and downloads from streaming sites are usually encoded in ways designed to save drive space, not be handled efficiently by an NLE. Taking time to transcode footage to an editor-friendly format pays off in edit and render speed. Search this site for the "Adventures in AVC encoding" thread by NormanPCN for the best explanations of this, or see video links below).

    Seeing very high GPU utilization usually means you have either a weak GPU or are actually working in a quite complex scene. In your case your have a mid-level GPU and a pretty complex scene.

    Now, since particle PHYSICS are calculated by the CPU and particle EFFECTS are rendered by the GPU the particle sim is something prone to slow down your system. It also depends on if you're using custom particle textures (especially video clips or 3D models), and, if you're using a custom texture if its built efficiently (quick example-if using a custom particle texture that's a roughly 200x200 [40000 pixels] pixel element but it's part of a 1280x720 layer [921600 pixels] then this isn't efficient... All the transparent space around the 200x200 area still has to be processed. In this example the actual 200x200 texture is taking up less than 5% of the 1280x720 layer. Trimming the entire source texture so it's as small as possible around the actual texture element makes the texture faster to handle - about 20 times faster! - and, if using hundreds or thousands of particles will make a HUGE difference! And this ties into what I said above about preparing projects and media the right way.

    3D models... In your example screen shots it looks like you're using stylised models with small, low res textures. Opening the project settings and turning down Max Texture resolution for 3D models can help here. If you're using simple model textures that are, say, 1080x1080, then the max resolution doesn't need to be 2048x2048. Again, it's just freeing up resources otherwise being assigned to something never used.

    3D lights. Are you using point lights, spot lights or directional lights? Hitfilm uses a rasterized shadow map, so you can get some speed out of Hitfilm by lowering Shadow Map resolution, although this creates blocker shadows.

    Point lights emit in all directions. Obviously, if your only lighting characters on stage it's possible that point light is "lighting" empty space which needs to be calculated. Also, it's not efficient use of the shadow map.

    Directional lights are pretty fast to render, but, because they represent parallel light rays from an infinity plane they render terrible shadows.

    Spotlights are usually the best compromise for most Hitfilm lighting. They are more efficient with shadows than directional lights and you're not lighting empty space like with points. A Hitfilm spot can have a 180° cone, which is quite useful. Imagine for a second we have lights in front of your camera looking at your set. One is a point, the other a spot with a 180-degree cone. The point light is calculating all the space behind your camera where there's nothing to see. The 180-degree cone illuminates everything in front of the camera and nothing behind. It will light the stage in exactly the same way as the point, but with better shadows and faster because Hitfilm isn't wasting time lighting or shadow map pixels on an area invisible to the camera which might not have a thing in it anyways.

    I did say a lot to unpack!

    Ok, a couple video links.

    This is a very long @FilmSensei livestream where he and I go through a lot lot LOT about working with 3D models in. HITFILM.

    In this video skip to 23:40 for a discussion on transcoding video, including tested, optimized settings for faster Hitfilm performance.


  • GrayMotion
    GrayMotion Posts: 1,658 Enthusiast
    edited October 2022

    This is what originally attracted to me to this forums and Hitfilm. Certain users expertise is astounding..

    Not to mention the camera series at Hitfilm U...I still to this day refer to those knowledge sharing vids. In fact I was just there 2 days ago refreshing my memory on a jib and crane setup for an Element scene I'm working on.

  • Triem23
    Triem23 Posts: 20,394 Power User

    @GrayMotion yeah, but it was @FilmSensei who figured out that awesome focus point trick to automate keeping the camera level as you crane up. It's all collaborative. 👍

  • TheParkH0pper
    TheParkH0pper Posts: 2 Just Starting Out*

    @Triem23 Thank you so much for your help! You gave me so much valuable information that I was able to use to speed up my production and finish everything on time. I needed to finish by the end of October because everyone's over the spooky season after the 31st. At least until next year hehe. There are a few things I would have liked to have fixed but...at some point you gotta just put the boat in the water and move on to the next one. Definitely learned a lot from this experience. Very grateful for the help here in the forums. Anyway, here is the result of my first jump into this level of compositing. (Programs used: Puppet Play VR [mocap], FL Studio [composing/mastering], Cakewalk [recording], Sketchup, HitFilm Pro 16 [editing/compositing].

    https://www.youtube.com/watch?v=AEiNrctaLBg

  • Triem23
    Triem23 Posts: 20,394 Power User

    That's great! Yes, I see areas you could nitpick, but that was a really complex video to produce. It works. You got your animations to synch up correctly, the camera moves and editing keep the video dynamic, and that's a good Halloween tune! That was a fun watch and listen as I'm sitting here waiting for the next trick or treaters.