Green screen tracking

Hey guys,

So ive been working on greenscreen footages lately, there is an idea i came up with but i can't render it out on hitfilm after many attempts and research of this forum.

My character has a fake guitar made out of carton and of course painted in green.

I want to replace that fake guitar by a one made out of particles, so ive created a plane with a mask that has the same shape, then ive added atomic particles to it.

My problem is that i can't find a way to track it, so it will follow the original fake guitar.

Thank you for your help !

Comments

  • RobinRobin Website User, HitFilm Beta Tester Posts: 1,671 Enthusiast

    You have a conflict here, as for a greenscreen, you'd ideally want a green surface without any variation on it so you can get a clean key. On the other hand, in order to track it, you'd ideally want a surface or at least a few spots with high contrast for the computer to lock on to.

    Off the top of my head I'd stick some orange markers onto the green guitar, far off from the edge so I can easily mask them out after pulling the key. Then you could use conventional tracking methods on those markers (2-Point Tracker in HitFilm or Mocha) to track the movement of the guitar and apply it to the particles.

  • foleyproductionfoleyproduction Website User Posts: 110

    Thanx, putting orange markers is a great idea, so after keying out, i will see only this 2 markers and i will use it as positions for the tracking.

    But the part i don't understand is, how do you apply it to the particles so it will follow it ?

  • foleyproductionfoleyproduction Website User Posts: 110

    Sorry ive just checked simon tutorial and ive seen i have to create a new point and assign the tracking to this point, then add particles and parent it to the new point

  • RobinRobin Website User, HitFilm Beta Tester Posts: 1,671 Enthusiast

    That depends on how you're tracking it with. If you're using the 2-point tracker in HitFilm (which will probably only work if you don't have much perspective movement going on), just apply the tracking data to a plane and add the atomic particles to that plane.

    If you're using Mocha, it will put out a composite shot with point layers for a surface it has tracked, in that case import the composite shot, create a new plane, set it to 3D, parent it to one of the points, rotate and move it into the right position and apply the atomic particles to that plane. There are a lot of tutorials for tracking with mocha, you could start with this one to get the basics if you're not familiar with mocha yet: 

    The key here is to understand that in this case, you're not tracking the camera movement in relation to a static scene, but in relation to an object. So you set up your tracking planes on the guitar and basically treat those as static, with the rest of the scene and the camera moving in relation to them.

  • JacobBrooksJacobBrooks Website User Posts: 128

    I'm having green screen tracking issues, too.  Is there a way to stop the background on the green screen from moving when you zoom in and out, pan around?  An example would be more of the green background appearing when I zoom in because the media is bigger than the screen.  I don't want it to do that.   I have watched tutorials and such but haven't been able to figure it out.  I know there has to be a simple solution.

  • FilmSenseiFilmSensei Moderator, Website User, Ambassador Posts: 2,295 Ambassador

    I would parent the green screen footage to the outside footage. Anything that you do to the main footage will be mimicked by the green screen footage. Another thing to do would be to create a point and parent them both to the point. Manipulating the point would affect both equally.

  • JacobBrooksJacobBrooks Website User Posts: 128

    The parenting didn't work.  Now what do I do?  I have seen people doing exactly what I am trying to do with this exact program.  But I can't figure it out. 

  • tddavistddavis Moderator, Website User Posts: 4,171 Moderator

    @JacobBrooks I feel your pain.  I have tried and tried and I am missing some critical step in my comprehension of all the tutorials I watched. I absolutely suck at tracking and applying that data to a point, a layer, whatever.

  • MrChrisMrChris Website User Posts: 22

    @JacobBrooks it sounds like you are mixing some 3D and 2D things...   If you are zooming and panning and have a "background" as you mention, then it sounds like you have positioned the background too far back from the foreground in 3D space.  Try making it 1 pixel behind the foreground, so when you move your point of view, the foreground/background looks like 2D and doesn't parallax.

    If you want some movement there then back you POV off a bit so you're not "peeking through" the foreground and seeing the background...

    Another idea might be using spherical warp or otherwise Projecting your background into or onto a sphere which may give the look you want.

    Disclaimer: I'm a permanoob but trying to help. :-)

  • JacobBrooksJacobBrooks Website User Posts: 128
    edited March 2018

    @MrChris thanks for your help, but I don't understand what you are trying to say.  Could someone clarify?  Anyone have any simpiler ideas? 

  • CleverTaglineCleverTagline Moderator Las Vegas, NVModerator, Website User, Ambassador, HitFilm Beta Tester Posts: 3,029 Ambassador

    @JacobBrooks Perhaps you could post a screenshot and/or screen capture video showing the problem you're experiencing.  For screenshots, you'll have to host the image on a site like imgur.com .  For videos, just post to YouTube or Vimeo, then and add the video URL on its own line in a comment, and the forum will auto-embed the video.

  • MrChrisMrChris Website User Posts: 22

    @JacobBrooks sorry!  Not sure I can explain better, but a different way perhaps:

    POV = Point of View (also known as the "New Camera" that is created when you make a 3D layer in HitFilm).

    When you pan or tilt your POV (only -- not zoom), the position of the camera and the objects in your scene are not moving, you're just "aiming" the part of the scene you can see.  So if you have a 3D scene, at least in simple terms, the background will not move against the foreground -- just like IRL -- if the POSITION of your eye doesn't change, but only the ORIENTATION does.

    So you shouldn't have a problem when panning... only zooming.  Why?

    If you ZOOM, or move the position (position and orientation are Transform properties in the Controls of your camera and other layers/objects) then things DO move relative to each other.  That is, after all what 3D means -- the 3rd dimension is "depth"... and zooming is half the magic!

    The other half is having more than one flat image in your scene.  So you have a subject shot against a green screen, and have keyed the green out & replaced it with a background.  Your subject/actor is the "foreground" and the replacement background is, well, the background. :-)  You could have true 3D models/objects in the scene also, or more 2D images, or particle generators in full 3D glory unrolled for the whole galaxy to see...  BUT...

    When you move your camera, or POV -- or if you use your HitFilm powers to move the Entire Universe past your camera instead -- your viewpoint is changing, and therefore you will see "whatever is not blocked by something closer to you", which changes as you move... again like IRL.

    The problem you mention sounds like you're in a 3D space "looking" at 2 images, an actor or foreground, and a keyed-in background, which are not fixed to each other, and so when your viewpoint moves in closer, you "see" more background -- or in this case, some green screen (which makes no sense if you keyed out the green but I'm trying to explain parallax and 3D here) -- that's natural 3D.

    To stop that you need to consider the kind of movement you want in your final product, and ensure the background is big enough and positioned properly so that it looks 'right' at all times.  If you're seeing green, maybe make the background bigger, or in 3D space "closer" behind the layer in front.

    What I mean by that is best shown with an example of a window and a scene visible through it.  If you get close to the window you can see way more of the outside scene.  So if you must get that close, make sure you have a background that is huge enough to cover it off.  Otherwise if the window and background are far off and you DON'T want to get in tight, you might just make the outdoors scene be smooshed up against the window image -- by making it's Position being only 1 or 2 less than the window layer in front of it, so it doesn't move (much).  Of course that would look pretty arse unless it's off in the distance.

    Clear as mud eh?

  • Triem23Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,228 Ambassador

    @MrChris did a pretty good description of basic camera moves, but got some terminology wrong. 

    A camera TILT is looking up and down--like nodding one's head. 

    A camera PAN is rotating the camera left and right--like turning one's head. 

    A camera DOLLY is moving the  closer or farther away--in close, out far. MrChris mistakenly called this a zoom. 

    A camera ZOOM is changing the focal length of the lens to change magnification of the image.

    A camera TRUCK is moving the camera left and right.

    A camera PEDESTAL is moving the camera straight up and down. 

    The names derive from the mounts required back when cameras weighed 70 lbs or more. There's some (technically sloppy) overlap allowed--no one will laugh at you if you say "Pan Up" or "Dolly In," but ZOOM can't be used to describe a camera move. That's a lens function.

    Now @JacobBrooks I'm not certain I fully understand the problem here. You have a greenscreened actor and a background layer. What's moving? Have you keyframed positions of layers in 2D. Do you have 2D and 3D elements in the scene? Is there camera movement in the original footage? I'm not certain what's causing the mismatch, but we'll figure it out. 

    You discussed tracking. What are you tracking, is it the actor or background plate? What do you want to do with the track? 

  • JacobBrooksJacobBrooks Website User Posts: 128

    https://www.youtube.com/watch?v=Ci66Zb2wpHM&feature=youtu.be

     

    Here is my example video everyone wanted to see.  I know the green screen is kind of a weird size so it is hard to get pictures to fit right (I'll have more to ask about that later). But that is not the one I am going to be using for my final product.  As you can see, more of the picture appears as I dolly in.  The same happens when I pan around.  Hope this helps! 

  • MrChrisMrChris Website User Posts: 22

    @Triem23 Thanks Mike, the truth is, I just wanted to help but had never heard the term Pedestal in all my hundreds of hours of ... youtubes. :-)

    I call up and down PITCH, and turning the head is YAW.  Can you guess what I usually do in "3D" space?  Hint - you need a license.

    But I really wanted to help so I melted my phone and sorry @JacobBrooks for the bad terminology.  I think your sample is something like what I thought it was though... you're in good hands now so I'll learn along with you!

  • JacobBrooksJacobBrooks Website User Posts: 128

    @MrChris allright, so what do I do?  Same thing you mentioned above?

  • Triem23Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,228 Ambassador

    @JacobBrooks Now that we can see the video... 

    You're waking towards a greenscreen. I'm guessing your video layer is a 2D layer and so is your background plate.

    I'm assuming you want your background to look like it's like a poster or something, staying the same size as the screen? The current footage you have will make this very difficult. Because you are moving forward to where the greenscreen more than fills the frame you can't point track the corners, because the corners go out of frame, and a point tracker can't track what it can't see. Doing a camera solve with mocha looks almost impossible because you don't really have two planes to track (mocha won't be able to lock onto the greenscreen well once its edges leave frame.

    I'll put more thought into this. Can you describe more fully what you WANT it to look like? 

    @MrChris help is always appreciated. The only real issue I had was saying "Zoom" for a camera move. ONLY because zoom is a lens function, not a camera move. Your basic descriptions were very clearly stated and detailed. All I did was append the glossary. :) 

    Otherwise, don't worry about the terms I tossed out too much. Like I said, no one will really care if someone says "pan up," when it's technically a "tilt up" and no one will care if you say "dolly left" instead of "truck left." These are old studio terms and most of them are fading into obscurity. Never encountering "Pedestal" doesn't surprise me at all--it goes back to when a camera was 70 pounds and you needed a specific (expensive) arriflex camera to do handheld! Older tripods had a hydraulic center column that could be raised and lowered--hence the term.

    Pitch and Yaw would make sense to anyone with any aviation or nautical knowledge. 

  • FilmSenseiFilmSensei Moderator, Website User, Ambassador Posts: 2,295 Ambassador

    @JacobBrooks Are you wanting the background footage to move the same way as the camera moves so as if it appears that you are looking out a window?

  • JacobBrooksJacobBrooks Website User Posts: 128

    @FilmSensei  I don't want it to look like that.  Isn't that what it is doing now?  

    @Triem23  The HitFilm tech support people made it seem like this could be done.  I really have no choice of not showing the corners.  The final product is going to be a news studio with multiple different green screens of different shapes and sizes.  I know you can key different things in the layer with masks or whatever they are called, but that is not my concern now.  And yes, I do want it to look like a poster.  I know what I want to do is possible.  I have seen it done in this program.

  • Triem23Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,228 Ambassador

    @JacobBrooks ok, now we're getting somewhere. 

    Yes, you absolutely DO want your background to follow the camera motion--like (as @FilmSensei said) looking out a window. Except, in this case, you basically want to replace the window. Whether looking out a window or sticking a virtual background on the wall, this is set a set replacement or extension. With a moving camera, that means tracking. 

    What's happening right now is your video footage is moving (because the camera moved), and your background plate is staying absolutely still.

    Now... I didn't say you can't do what you want in Hitfilm, because you can. I'm saying that particular bit of footage is going to be very difficult to match because you somewhat screwed up your shot. Take notes, here's a Sunday workshop. 

    There are two basic types of trackers, both of which can function in 2D or 3D. Point tracking and planar tracking. 

    We'll discuss planar tracking first, both because, if you have Express, you probably haven't bought the Mocha add-on so can't do planar tracking, and also because it's probably not the best solution for this shot anyways. 

    Planar tracking requires two flat surfaces at different angles to each other to track. The tracker looks at.  surface detail and texture problem is, your two planes are the table top and greenscreen itself. The greenscreen has no surface detail or texture, so mocha cannot track it. You cannot track the wall outside of the greenscreen (once it goes out of frame there's nothing to track). Therefore you cannot track this shot via mocha. 

    Point tracking looks for an easy to identify small feature in a frame. A single point track can give you position data. A two point track can give you position, rotation and scale. That's enough information for the shot you want. Problem is, what is there to track? Well, there's nothing on the greenscreen to track. You can't track any of the edges of the greenscreen stand, because as soon as it's out of frame the computer can't track it. In fact, the only option you have is the back edge of the table.

    The two back corners can be used to do a partial two point track of the footage. Being corners, they are easy to identify. But, once the corners leave the frame, nothing more can be tracked. There are no trackable details on the tabletop itself.

    So, you need to two point track as much of the shot as you can. Apply the track data to a point layer, then parent the background layer to the point. 

    This tutorial from @ShinyFilms should cover what you need to know.

    Ok, earlier I said you somewhat screwed up your shot. Also, in both this post and prior post I've said you can't track something outside of the frame. Those go together.

    Ok, just look at the thumbnail for this video:

    See the blue dots on the greenscreens? Those are tracking markers. They exist so that there are points for point trackers to track. Tracking markers can often be eliminated by a second Chroma Key pass, or have to be manually masked out. If there were tracking markers on your greenscreen you'd be able to track the shot past the point where the back corners of the table leave frame.

     

  • JacobBrooksJacobBrooks Website User Posts: 128

    @Triem23

    Where should I set up the tracking markers on HitFilm?  I tried something like this before but it didn't work.  Don't you have to parent the footage to the point or something?

  • Triem23Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,228 Ambassador

    Well, you have to set up the tracking markers on your set, so they exist in the footage. 

    Once the footage is tracked, you'll apply the track to a point, then parent the bg plate to the point. I thought the tutorial I linked covers this? 

  • JacobBrooksJacobBrooks Website User Posts: 128

    @Triem23  I'll take a look at the tutorial, so basically I just put a bunch of trackers everywhere and then I have to manually track the trackers?

  • Triem23Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,228 Ambassador

    @JacobBrooks "a bunch?" depends on the shot. For the specific example shot you gave--camera moving straight forward--you only need two. Again, with the example shot you gave, you can track most of it... Until the back corners of the table leave the frame. 

  • JacobBrooksJacobBrooks Website User Posts: 128
    edited March 2018

    @Triem23 Okay.  I'll just put them up where I think appropriate.  Do I have to manually track them in HitFilm?  I just want to make sure this works.  I don't get how a computer can magically track duct-tape.

  • Triem23Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,228 Ambassador

    @JacobBrooks I feel like you haven't watched the tracking tutorial linked above. 

    You'll have to manually assign a tracker in Hitfilm then let Hitfilm automatically finish the track. 

    It's not magical at all. You have to provide a point tracker something contrasting to work with. In this case there's no contrast on the greenscreen because it's one color. The tape provides a point of contrast. 

    In Hitfilm's tracker you place the red tracking box around the point to track. This is telling Hitfilm "Look for this pattern." The green tracking box defines the search area around the point as it moves. Telling Hitfilm where to look for the thing in the red box on the following frame. No magic. Two user defined parameters allow some pattern recognition to be used. 

  • JacobBrooksJacobBrooks Website User Posts: 128
    edited March 2018

    @Triem23 @MrChris Okay.  I followed the tutorial and it worked!  What do I do for a shot when I pan across?  I tried doing that and it didn't work.  Is it just me, or is there something different I have to do?

Sign In or Register to comment.