Key / mask using chroma colour on part of a moving item

aarondcaarondc Website User Posts: 264
edited January 2015 in Post-production techniques

Say I record a video of me drumming. If I colour the end of the sticks, chroma green or something, could I use that

1. for tracking

2. to key out the colour and just replace it with a generic drumstick white

3. then link a particle emitter (thingy) to the drum stick ends and have sparks or what have you exploding as they hit the cymbals

If I then add similarly coloured dots to the cymbals could I use that to dynamically track a plane (?) applied to the cymbal for particle bounce or whatever effect I want to do there?

This is probably a question better answered by reading a web page somewhere discussing green screen and motion tracking with blue dots or something, so if there is  such a page, or book, or video, please let me know and I'll go read or watch it.

Thanks!

Comments

  • AxelWilkinsonAxelWilkinson Staff Administrator, Imerge Beta Tester, HitFilm Beta Tester Posts: 5,242 Staff
    edited January 2015

    Coincidentally, Simon actually addresses a lot of those specific questions in this tutorial, where do does a very similar thing to what you describe starting around 8:15:

    In general, tracking drumsticks is going to be tricky at best, due to the motion blur caused by their rapid movement. Similarly with keying, the motion blur makes them a bit transparent so that the background colors blend with the sticks, making it difficult to get an accurate selection. But, it will probably depend somewhat on what song it is, and how vigorously you are drumming.

    As far as tracking cymbals, if you decided to try it you probably won't need tracking markers, unless your cymbals have a brilliant finish. Mocha's planar tracker should be able to grab onto the texture of the cymbals without them. but keep in mind that  you will need to first track stationary planes to get a camera solve, before tracking the moving cymbal and exporting the data.

  • aarondcaarondc Website User Posts: 264

    Thanks for the response, Axel. I also appreciate the tutorials you and Simon are doing.

    If I could modify my question above slightly:

    is there any value in having deliberately coloured items on objects or people, to help with tracking motion, assuming the items in question will not be moving too fast, and are distinct enough to chroma key out?

    Or am I making more work than is necessary?

  • Triem23Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,300 Ambassador

    To weigh in here, I think the answer has a lot of "it depends." Since in another thread, you described yourself as a new user I'll try to give some detail here to help you further refine your question.

    If you're tracking using Hitfilm's 2D optical flow tracking, then then tracker is trying to lock onto a "point of contrast," which needs to have a certain amount of difference between nearby pixels to see. Let's say, for example, I am tracking an actor's eye to add some glow--The inner corner of the eye is going to be a small point with some color and brightness difference--the different color of eye to skin and some shadow. Because this tracker is looking for a point, fast motion can blur the detail you're after and make it impossible for the point tracker to lock on--Axel brought this up about drumsticks, and it also applies to everyone who thinks they can paint a stick green and track it as a lightsaber--doesn't work with fast motion.

    Now, if you're tracking in Mocha Hitfilm, mocha is looking for areas of textured pattern and tracking it's relative motion, but you're tracking an entire area of texture. However, this does mean mocha is far better at tracking things with motion blur than a point tracker. However, since mocha does need some sort of texture contrast, something that's (for example) a pure chromakey green isn't going to have any texture for mocha to look at, so it gets confused.

    Mocha MIGHT be able to track drumsticks, but those are thin and don't have too much texture to them, so it's looking at a small area to track it's motion. A point tracker won't be able to lock onto it.

    (Here's a tutorial where Axel discusses using mocha to track an object--the key thing here, as Axel noted above, is that you have to do a camera solve in mocha before you can track an object)

    Now, sticking with drumsticks for a moment, bright colored drumsticks might make it easier to do certain effects--like you might be able to grab a chroma green stick and color correct it back to a white, even with blur. It might help if you wanted to do glows or similar effects. For motion tracking, it probably wont help too much for the reasons discussed above. There's a good chance you'll just have to manually keyframe a point if you want a drumstick emitting particles. (Same goes for lightsabers. Even ILM is still doing manual roto for those)

    Another concern with a colored object or actor for keying purposes is lighting and shadow--if you want to, say, track an actor and you put your talent in a bright green shirt with the intent of keying that out, it needs to be the tightest shirt you can find, because a loose shirt is going to have folds and shadow that will ruin the key.

    So, chances are it's making more work than needed. One the other hand, you're being smart and trying to solve these issues before you shoot and can plan ahead.

    Hope this is helpful.

  • aarondcaarondc Website User Posts: 264

    "it also applies to everyone who thinks they can paint a stick green and track it as a lightsaber--doesn't work with fast motion"

    "sticking with drumsticks for a moment"

    @Triem I am not sure if you read my adjusted question? I specifically mention "assuming the items in question will not be moving too fast".

    I appreciate you are trying to help, but my questions is very specific. I have not mentioned the item(s) but they are not drumsticks, which we have already determined are moving too quickly for motion tracking. They are not lightsabers, or anything that is moving too quickly.

  • Triem23Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,300 Ambassador

    Anything moving fast enough for motion blur won't track with a point teacker, period. Anything without enough texture can't be tracked in mocja, period. My general information answers your specific question, period. 

  • aarondcaarondc Website User Posts: 264
    edited January 2015

    I think we are having a communication problem. I wrote "assuming the items in question will not be moving too fast".

  • aarondcaarondc Website User Posts: 264

    You also sound like you are getting upset. Not helpful.

  • aarondcaarondc Website User Posts: 264

    eg: If I put a point on a snail, and then film it, will that point help in motion tracking?

  • Triem23Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,300 Ambassador

    OK, sorry, you're correct, that last response was a little too snippy. I apologize.

    Going back a couple of posts, the generalized discussion of point vs mocha tracker applies in all cases.

    Regarding the current, specific example, Hitfilm isn't set up for full motion-tracking like using a mocap suit with all the dots. A bit off topic, but tossing it out there. For tracking a snail, yes, the dot would give you something specific to track. A single point can only track x/y position. With TWO points, you can track x/y position, rotation and scale, which, in many cases is enough to fake it being a 3D track. That said, then you have to remove the dot on the snail in post, so you're adding a bunch of steps. In most cases--at least with the snails common to my area of southern California--there is enough contrast an detail on the snail to track it based on it's own shell markings. For a single point, I'd track near the center of the shell. For two-point tracking I would try tracking the edge of the shell near the rear of the foot and the other edge of the shell where it joins the neck.

    I still have to fall back on "it depends," because it really does. Other things to think about include, say, camera movement. If I have a nice, static tripod shot and I'm tracking a moving object, then Hitfilm's 2D tracker is the tool I'll look at first. If I'm handheld, but basically standing still, I can also look at the 2D tracker. If I'm moving around on a slider or dolly and I'm moving sideways or vertically relative to my object, I'll still look at the 2D tracker. If I'm moving on a slider or dolly and moving towards/away from the object, again, the 2D tracker. If I start tiltinging my camera a lot, or if I'm moving around/circling an object, then I'm thinking mocha. For things like colored objects and tracking dots, I need to think about lighting, because if my tracking dot/object goes from bright light to shadow, then I'm not going to be able to grab it to correct with a key or color selection tool, since it's color changes.

    Sorry, it's a lot of info, and a lot to cover, and when uncertain of the specifics of a shot I have to fall back on generalites because I'm trying to covers as many options as come to mind. Hence the above paragraph talking about several camera options, since I don't know how you're planning your shot.

  • StephansBilderweltStephansBilderwelt Website User Posts: 523

    Triem23, very good explained. 

  • aarondcaarondc Website User Posts: 264

    Cheers Triem, much appreciated.

  • SimonKJonesSimonKJones Moderator Website User, HitFilm Beta Tester Posts: 4,450 Enthusiast

    Something else to consider is what you're going to be compositing. This is something that came up on a recent project of mine, where some tracking markers were put on walls in order to apply bullet impacts in post.

    The problem with that example is that the bullet impacts only appear halfway through the shot, and the markers were much larger than the bullet impacts. This meant that I actually spent more time removing the markers than I did compositing the impacts. And, as it turns out, the wall surface was easy to track in mocha anyway - so the markers weren't needed in the first place.

    If you're going to be compositing something which will naturally obscure the marker, then it's not a consideration (eg, if you comp a large gun on top of the snail...you know you want to). But if the marker is going to be visible all the time, do factor in the time required to remove it in post.

  • aarondcaarondc Website User Posts: 264
Sign In or Register to comment.