Why the speed effect doesn't behave well (maybe?)

CervierCervier Website User Posts: 156 Just Starting Out

I believe I might have have discovered a flaw in the" speed effect" algorithm. Let me explain. First, I did a test. I cut a 10-second long clip, made it a composite shot and dropped a Speed effect on the layer. I set a "1x" keyframe at the very beginning and at the very end, and a "2x" keyframe at 5" and RAM-rendered the clip. It accelerated to 2x its speed, as expected, but stopped there, having reached the last frames of the original clip. The playhead went on to the end of the compt timeline without displaying anything.

(This looked odd to me, as I would have expected the playhead to move faster, then slower, because the timeline is graduated in seconds and not in frames. I didn't know what to make of this, being far from an expert.)

Ok, now let's move on to the experimentation I did out of curiosity. Just for the fun of it (yes, that's one of the many things I have fun with ;) ), I tried to figure out what should happen to frames when the speed of a clip is changed.

First look.
Let's say there are 20 frames, numbered from 0 to 19 (it's easier to start at 0 than at one for the formula that's to come), and occupying positions 0 to 19.
Let's say we want to double the speed, which means skipping one out of two frames. Or adding 2 to the frame number.
The new frame in position 0 is still frame 0,
The new frame in position 1 is frame 2 (0 + 2)
The new frame in position 2 is frame 4 (2 + 2)
The new frame in position 3 is frame 6 (4 + 2)
etc.
New frame at position N is frame 2xN.

The last occupied position is #9 (the tenth) with frame #18. The length of the clip is halved, as expected.

Second look.
Now, let's look at something a bit trickier: non integer coefficient. Let's say we want to increase the speed by 50%, which means we multiply it by 1.5. And, let's call K that coefficient. So K = 1.5
The new frame in position 0 is still frame 0,
The new frame in position 1 is frame 0 + 1.5 = 1.5, but there's no frame #1.5, so let's round that to the closest integer, 2.
For the new frame in position 2, if I add 1.5 to the number of the frame in the previous position, I'll do 2+1.5=3.5, rounded to the nearest integer will yield 4; again, I'd really be adding 2 instead of 1.5. What I really should do is keep track of the "floating point" number of the frame. The frame in position 1 is frame "1.5", so for position 2, I'll do 1.5 + 1.5 = 3.
New frame in position 3 will be frame (3 + 1.5 = 4.5), that I round to 5 (but remember 4.5), etc.
The new sequence of frames will be 0 (skip #1), 2, 3, (skip #4), 5, 6, (skip #7) etc.

To avoid the rounding issues, I thought at that point the easiest method would not be to add K to the previous frame number, but to use the formula:
NewFrameNumber at Position P = rounded(K*P)

Third look.
This works if K is constant on the set of frames. But what if it varies?
This is what I tried (spoiler: before realizing it was wrong).
I did a test. I took a clip that was 24 fps, 10 seconds long and exported it as a sequence of 240 png files (numbered from 0 to 239). My goal was to manually remove the to-be-skipped frames and import back the folder.
Using a spreadsheet, I prepared a column of 240 lines. I filled that column with numbers from 0 to 239, representing both the "position" and the number of the "original frame". In the next column, I put the changing K value: a 1 on line 0, a 2 on line 120 and back to 1 on line 239. In between, I increased the K value by 1/120 so it went in a linear fashion fro 1 to 2 then back to 1.
In the third column, I kept the "floating number" of the new frame. For instance, in position 5, the new frame would be (number of frame in position 4) + (K of position 4).
The next column was just the third but rounded to the closest integer. That gave me a list of frames to keep, that I turned into a list of frames to remove and I deleted the relevant files in the PNG folder.

I imported the new image sequence and played it.
It behaved in the same way as the comp I described at the beginning of this text!  It accelerated to 2x and pretty much stopped there!

Fourth and final look
Then I realized what was wrong. I wasn't using the correct K values! I shouldn't use the K relevant to the "position", but to the previous "new frame"! In other words, the K value is attached, not to a position, but to a frame that can move along the timeline because of the change of speed!.

So, the formula for the "new frame number" should be:
NewFrameNumber = PreviousNewFrameNumber + K(PreviousNewFrameNumber).
In my spreadsheet, I looked for the K value that was, not on the same line, but on the line with the same number as the PreviousNewFrame (rounded).
I tried that and it worked perfectly!

I should have spotted it at first glance: in my first try, the numbers of the frames I had to skip were 12, 21 (jump 9), 27 (jump 6), 33 (jump 6), etc. The size of the jump quickly got to 2 and never rose agains, which isn't consistent with a speed returning back from 2x to 1x.
For my second try on the other hand, the jumps went from 9 to a couple of 6s to a mix of 4 and 3 to 2, stayed at 2 for a while then went back up to 7 -that's consistent.


Conclusion
Since my first try based on an erroneous algorithm behaved exactly like the Speed effect, I wouldn't be surprized if the algorithm used by HitFilm had the same flaw. A small one that was easily corrected by changing one formula (I am aware of the fact it's more complicated to correct a software than a draft spreadsheet).

Comments

  • NormanPCNNormanPCN Website User Posts: 3,945 Enthusiast
    edited October 2019

    "It accelerated to 2x its speed, as expected, but stopped there, having reached the last frames of the original clip. The playhead went on to the end of the compt timeline without displaying anything."

    "(This looked odd to me, as I would have expected the playhead to move faster, then slower, because the timeline is graduated in seconds and not in frames. I didn't know what to make of this, being far from an expert.)"

    The playhead speed is always constant. It is real time. It is the timeline speed. The speed effect only changes what frames are displayed. If speed is above 1.0, then some frames are skipped and there will be empty/black frames beyond a certain point in the clip/layer. If speed is 2.0, then that time half way through the clip.

    So if you have a 10 second clip at 30fps, so 300 frames, and you have a speed 2.0 then the speed effect tries to crams the 300 frames into a 150 time period. Hitfilm does this by skipping frames. This speed above, Hitfilm will run out of frames because the clip boundary is a hard boundary for the speed effect. This is different that how other editors operate. So when Hitfilm runs out of frames you get the empty/blank frames being played. You cannot trim those off because that changes the last/end/boundary frame the speed effect can feed on. This is one reason why the speed effect a total PITA compared to other editors with regards to rate control.

    If your speed is 0.5, then Hitfilm would take 150 frames and make them take 300 frames of timeline time. Hitfilm does this by duplicating existing frames.

    Changing how many frames are played relative to the original frame count in the source, changes the apparent *speed* of movement in your playback.

  • CervierCervier Website User Posts: 156 Just Starting Out

    Yes, I know all that As I wrote, the problem comes from the fact HitFilmchecks the position of the frame to assess what the speed is at that point, instead of looking at what frame is in that position.

    Let's say there are 100 frames, set in 100 positions. We put a "x1" keyframe in position 1, a "x2" keyframe in position 50, and a "x1" in position 100. I don't know exactly what HitFilm does, but it behaves as if it wrote a different "K factor" (what the speed is multiplied by) in every position: for instance, the speed when the playhead reaches position 50 should be 2x. And when it reaches position 75, it should be around K=1.5, halfway between 2 and 1. I'll write this as "K(75) = 1.5" ("K of 75 is 1.5").

    The problem is, there are chances position 75 will never be reached, because of the dropped frames! And here, attaching keyframes (or K values) to positions doesn't work.

    The solution is to attach keyframes (K values) to frames. For instance, if "K=2" is attached to frame#50 (and not position#50) then wherever this #50 frame lands is where the speed should be 2x. Let's say it's moved to position 35. K of 36 is 1 + 35/50 =apx 1.68. With the wrong way of calculating what the next frame should be (i.e looking at K of position 36, the speed should be 1.72, still increasing towards 2! But with the correct one, i.e. looking at K of "original" frame 50 (same as K of position 50), the speed should now be 2, starting to decrease towards 1.

    Mind you, this means that speed will not vary linearly relatively to time, but relatively to the frame number. Relatively to time, it will increase abruptly (e.g. at 25 fps in the above example, speed will go from 100% to 150% in .84s and from 150% to 200% in .56s).

  • roundcubethreeroundcubethree Website User Posts: 1 Just Starting Out
    edited August 30

    Temporal speed could use a lot of improvements.

    The clip with temporal speed added should reflect the "time" occupied on the timeline. As of now any clip with temporal speed needs to be on another clip layer because it skews the sequence of the timeline if the clip is in between any other clips. Its also guesswork to know where the end frame of a clip is when temporal speed is added. [spam link removed]

  • FlyingBanana78FlyingBanana78 Website User Posts: 419 Enthusiast

    I too find the speed effect to be quite confusing. Maybe it is the wording of the settings for it I dunno I just don't mess with it anymore cause whenever I do I end up messing all sorts of the video up and just have to delete it and bring it back into the layer stack from media.

  • triforcefxtriforcefx United StatesModerator, Website User Posts: 1,059 Moderator

    @roundcubethree While this is an interesting topic, please don’t bump threads that haven’t been touched in almost a year. The link you posted is also suggestive of bot behavior, so consider this a warning. If you do it again, you will likely be banned. I’m gonna close this thread now.

This discussion has been closed.