Drifting issue

Hey all, loving Camtrack so far but have one issue. I'm currently compositing green screen footage that was on a slider. Once I take the tracking data to my virtual set and composite the two, my green screen object is drifting on the floor and not staying in one spot. is there any way to make this not happen?