Jittering Track Points

torburwelltorburwell Website User Posts: 2 Just Starting Out

I just got CamTrackAR and have been disappointed in the jitter in my scene once I bring it to blender. I have checked and it is not the footage that is shaky, but only the tracking points that I put in the scene while using the app. If someone could help it would be greatly appreciated.

Comments

  • JimmyGunawanJimmyGunawan Website User Posts: 2 Just Starting Out

    I've experienced the same issue. I have a feeling it's a bug with Blender import. I don't know if it's the same wobble / jitter when the tracking data is loaded into After Effects.

  • keithhamkeithham Website User Posts: 3 Just Starting Out

    I've had the same jittery issues in After Effects. It's hard to say if it's an FXHome issue or an ARKit issue. I've played around with Apple's AR before and the models you place do drift or jitter some as you move your device around. It's not like if you have a good track from After Effect's native camera tracking in post. I think it's a limitation of ARKit and the hardware on your device. I would think a device with LiDAR (which I don't have) would work the best.

  • GrayMotionGrayMotion Website User Posts: 1,368 Enthusiast

    I have an iPhone 12 Pro Max...jitter is pretty prevalent across the entire video actually. The model is locked on the point but the entire scene is shaking. . It looks like the hand held action is just too sensitive? I also noticed in FXHome's demo video that the entire scene also jitters.

    What you think? Maybe locking down the iPhone on a tripod and then film pans and tilts will eliminate that jitter?

  • Triem23Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,330 Ambassador

    Phone sensors are tiny little rolling sensor CMOS sensors and are subject to serious wobble issues (my guess is, if you look carefully your footage is actually distorting and microstretching), additionally, while I'm not an iPhone owner I believe you're getting VFR footage. Both rolling shutter wobble and VFR will cause drift. Additionally, the core tech - the Apple ARKit - has its own limitations. It's ARKit that limits the footage to 1440p not 4k.

    None of this is the fault of FXHOME, and none of this is anything FXHOME can fix. CMOS wobble is inherent in the hardware and can only be fixed by using a global shutter chip (to my knowledge the iPhoneXs was the only iPhone with global shutter (update - global shutter for the IR sensor, not the main camera). Timing issues caused by VFR can only be fixed by actually recording CFR. Again, this is an issue for Apple.

    Using a gimbal will eliminate a lot of jitter. Using a tripod will eliminate more. Even then the very act of panning, tilting or moving the camera causes rolling shutter distortion. Small sensors (phones) have much more distortion than large sensors (Cinema camera).

    The tracker assumes everything is still. Rolling shutter distortion reads to a tracker as MOVEMENT and it messes up the track. VFR changes the cadence of movement and throws off the track.

    In other words CamtrackAR is probably performing as well as the limitations in the underlying tech allow, and there ain't nothing else FXHOME can do.

    For those who don't know what rolling shutter is, I leave this article.


  • GrayMotionGrayMotion Website User Posts: 1,368 Enthusiast
    edited November 19

    Thanks for that Mike.

    I just wanna add that I have seen pretty solid tracking using the iPhone AND a camera (using an Elgato HD60) inside Unreal...defiantly no " jitter" in the footage. Solid tracking.

    I'm going to do some testing with Camtrack AR though. Everything I've done so far is fairly close to the camera. Nothing too far away (+10 feet). I'll get me a hand-held device as well as another smaller Manfrotto tripod w/ball head (Compact Action Smart) and do some shooting and see what I can see... (um, See what I can see - sounds like a mechanic working on a car.)

    Anyhoo...I noticed the first AR video that FXHome showcased in Jul 31, 2020 they are using a hand-held device and there is very little "jitter". The second video released Aug 20, 2020 - the intro - you can defiantly see the jitter in this entire shot. Now the video released Oct 29, 2020 the jitter is hardly (if at all ) noticeable. So - rig difference to be sure - I'd imagine :-)

    Thanks again bud 👍

Sign In or Register to comment.