Audio Lag/Slippage: Can you help get it fixed?

Palacono
Palacono Posts: 3,425 Enthusiast
edited December 2018 in HitFilm

I'm going to tag a few experts here because then we'll get a good range of opinions and hardware, but the more the merrier to help get this confirmed > fixed so feel free to chip in, whoever you are. :)

@Triem23 , @CleverTagline, @NormanPCN , @Aladdin4d

Here is the problem when a clip is repeatedly exported - although even the first export is wrong:
https://i854.photobucket.com/albums/ab106/pickaname2/Poke_Stuff.jpg

Finally having a PC that can playback in realtime, I noticed a 1 frame lag/slippage in the audio of this clip from Tobias at Surfaced Studio (not to be confused with the new Text Effect of a similar name, although he regularly does tutorials for Hitfilm... ) Basically, his voice doesn't line up with his mouth in Hitfilm, but the original does in Media Player.

It was present in a Composite shot and the NLE. Could have just been my eyes/ears, so I exported it and loaded it back in. Yep, it's delayed. Repeat with that clip - more delay added. Repeat: more delay. You can do this indefinitely and it never stops slipping. Tried exporting from inside a Comp and the NLE. Same results.

This was on a Win 7 machine with a single CPU and a GTX 580. Also same results on a Dual CPU Win 10 machine with a Quadro 4000.

Reported to Support and it can't be replicated. Only thing I have that Support apparently doesn't is: Xeon CPUs in both PCs (single and double). Be surprised if that's a factor, but you never know...

Also both mine are NVidia cards. No idea what Support used.

Tested with several earlier versions of Hitfilm and they all do it too; just never noticed before.

Can you help get a wider range PCs/Macs involved and do a similar test - only one export is required to confirm it, then zoom waaaay in on the timeline slider  - and then Support will have more to go on and be better positioned to fix it?

If you report your findings here they should see them. Or if you're one of da Beta crew, you could let them know directly, I guess?

I exported with either the Vimeo or YouTube HD settings, although I don't think export settings are important; it's evident inside Hitfilm itself immediately you use the clip.

The Tutorial clip I used is very short (7 seconds) so if you all use the same clip (does it with others too, it's not the clip itself) it's a known quantity and very quick to do.

Here's the page you can download Tobias' tutorial clip from.
https://www.surfacedstudio.com/downloads/tutorial-files/motion-tracking-in-hitfilm-express

Cheers! :)

Comments

  • CleverTagline
    CleverTagline Posts: 3,337 Ambassador
    edited December 2018

    Interesting issue you've found here. I took it for a spin, and initially had the same result on my Mac using the latest version of Express. Just to make sure we're both doing the same thing, here's my process:

    1. Import the original clip, drop it onto the timeline. Export Contents using the default YouTube 1080 preset.
    2. Import that exported clip, and drop it onto the timeline. Mute the video and audio of the first version. Export Contents, YouTube 1080.
    3. Repeat step 2 with the most recent export.

    Each export that is re-imported has the audio shifted later than the one from the pass before it.

    However, I found a way to avoid the problem. It's fairly simple, and is something that we constantly tell people to do before editing: transcode.

    I took the original and transcoded it to ProRes 422 (File 1). Imported, added to the timeline, then exported with the YouTube 1080 preset (File 2). Imported that back in, and there's no delay between the two.  If I export File 2 with the YouTube 1080 preset again and import it, the delay begins. However, if I export File2 (no delay) as ProRes 422 and import that, it's still in sync with the original ProRes version (File 1).

    Here's what I find a little odd, though. We often tell people that transcoding fixes variable frame rate issues, which can be common with files compressed for viewing and not editing, and which can often lead to audio sync problems. However, the original MP4 from Tobias checks out in MediaInfo as having a constant frame rate, so I ran one more test.

    Imported the original MP4 file from Tobias, and this time exported as ProRes 422. Imported that, and the delay is there. So something with the compression in his file is causing the delay from the very first import, but it's not a VFR issue. It looks to be something with the audio compression specifically. 

  • Palacono
    Palacono Posts: 3,425 Enthusiast
    edited December 2018

     @CleverTagline Thanks, that's very helpful to know I'm not alone. I'd actually already Transcoded the file when I first discovered it, but thought it easier to point you at the original, as that did it too.

    Yes, yours is a similar process, although to try and tie it down: I've closed Hitfilm and reloaded it and done 'New' etc. etc. between each Export and done it from a Comp (first place I spotted it) and the NLE. None of which makes the slightest difference...which would be another bug if it did.

    I transcoded it with Shotcut, which is a small , free Video Editor and it's basically got a shell wrapped around ffmpeg.exe. I found it useful to be able to set custom M and N values for smoother playback in Hitfilm and it also works well as a VFR > CFR converter....unlike Handbrake.

    Shotcut doesn't fall over with 23.976 and 59.94 fps clips, which Handbrake unfortunately does. That messes up CFR and turns it into VFR(!?), in those instances, which is not helpful at all.

    What did you initially Transcode it with to ProRes 422 before you used Hitfilm to Export as ProRes? Unless I'm missing something: you may actually have found another bug if Exporting as ProRes 422 'shifted it back', because I can tell it's wrong when played back inside Hitfilm before it's ever been exported, which is how I first spotted it.

    So, neither of my machines are unique and you were able to reproduce it, which is interesting when Support couldn't with either an original clip, or a ShotCut version.

    Oh, and I asked Tobias, and that clip is the default .MP4 format that is exported from Premiere.

    Here's something I thought I'd never have to ask you: What are the specs of your PC, GPU, CPU, OS? :D

  • NormanPCN
    NormanPCN Posts: 4,194 Enthusiast

    Looking at this file I notice that there are multiple durations listed. By MediaInfo and by looking at the MOOV/MP4 data blocks. In MediaInfo you can see the audio has two durations listed. Duration and Source duration (which is longer). So the actual audio track is longer that the duration of the clip. 

    Duration : 7 s 174 ms
    Source duration : 7 s 211 ms

    Depending on too many things we cannot know about what is going on inside Hitfilm this difference could very well be the source of the slip in  Hitfilm. The transcode that CleverTagline did probably just cleaned up (fixed) the differing track durations in the Prores transcode result.

    File handling is not Hitfilm's forte so something may be mishandled/misinterpreted here. If my memory serves me, I have seen multiple durations listed in the past but it is rare. Not that is saying much, given a low volume, but I do tend to stick my nose into the uglies just out of curiosity.

    The uglies.

    The file (mvhd) has a duration of 648960 with a timebase of 90000.

    The video track (trk 1) has a track header (trkhd) duration of 645645 with the file timebase. It has a media header (mdhd) duration of 172172 with a video timebase of 24000 (24000 / 1001).

    The audio track has a track header (trkhd) duration of 645645 with the file timebase. In the media header (mdhd) is has a duration of 346112 with a timebase of 48000.

  • CleverTagline
    CleverTagline Posts: 3,337 Ambassador

    "What did you initially Transcode it with to ProRes 422 before you used Hitfilm to Export as ProRes?"

    MPEG Streamclip. It's all I've ever used for transcoding.

    "you may actually have found another bug if Exporting as ProRes 422 'shifted it back', "

    Nothing got shifted back (i.e. un-delayed) that I can see. I think you may be mis-reading my comments. I'm going to edit my note above to clarify which files I'm referring to when talking about my first ProRes test sequence.

    As for my system specs, I'm running macOS High Sierra on a late-2014 Mac mini. 2.6 GHz Intel Core i5, 8GB of RAM, Intel Iris 1536 MB GPU.

  • Palacono
    Palacono Posts: 3,425 Enthusiast
    edited December 2018

    @NormanPCN yes, I saw that in MediaInfo for the original. Weird what Premiere produces.

    When I transcode it with Shotcut there is only one duration, which should have cleaned things up; but the audio length dropped from 7s 253ms to 7s 252ms on first export (new numbers again). Thereafter it remains the same 7s 252ms on each subsequent export, but keeps getting 'sonically' (?) shifted by Hitfilm, which doesn't seem to be able to handle it's own exported files...

    Can you reproduce and the usual OS, CPU, GPU etc. question? ;)

  • NormanPCN
    NormanPCN Posts: 4,194 Enthusiast
    edited December 2018

    Windows 10, 16GB ram, i7 4770k @4ghz, GTX 1080, Hitfilm Pro 11

    The Hitfilm export takes the longest duration of the audio track. What MediaInfo calls Source Duration. That becomes the file header duration from Hitfilm. The video track duration remains the same as the source. The audio track keeps the long duration of the source and that becomes it's only duration.

    I think the shift probably has something to do with the differing durations. I think one can get slight differences in audio and video duration when the audio is recorded separately from the camera. In camera basically forcibly makes the audio duration the same and video. This shift would seem to be Hitfilm mishandling the difference in some way. That different duration amount may be the shift amount.

  • Palacono
    Palacono Posts: 3,425 Enthusiast

    @CleverTagline OK, edited my post, thanks for the clarification. @NormanPCN thanks and good to know Xeons aren't a factor and it's basically just there.

    Any more for any more?  

     

  • NormanPCN
    NormanPCN Posts: 4,194 Enthusiast

    I don't think the shift is happening on import or timeline place. I see the audio shift in Vegas. From the Original and 2 Hitfilm exports.

    I tried this with some of my GoPro media. I see the shift on export.

    This media had a slight difference in the audio and video duration. The audio track header and media header durations were not different as in the PokeBall media.

    General
    Complete name : D:\User Data\Norman\Video Projects\test Projects\play media\GoPro.MP4
    Duration : 55 s 659 ms

    Video

    ID : 1

    Format : AVC

    Duration : 55 s 656 ms
    Frame rate mode : Constant
    Frame rate : 29.970 (30000/1001) FPS

    Audio
    ID : 2
    Format : AAC LC
    Codec ID : mp4a-40-2
    Duration : 55 s 659 ms
    Bit rate mode : Constant
    Bit rate : 128 kb/s
    Channel(s) : 2 channels
    Channel layout : L R
    Sampling rate : 48.0 kHz

     

  • Palacono
    Palacono Posts: 3,425 Enthusiast
    edited December 2018

    @NormanPCN I first spotted it on timeline placement. It was the delay in Hitfilm playback that made me investigate. But useful to know GoPro clips also do it. 

    I also first confirmed in Vegas, but again thought it easier to show how to see the results in Hitfilm. 

  • NormanPCN
    NormanPCN Posts: 4,194 Enthusiast

    Export type does not appear to matter. Happens with Cineform AVI export as well.

  • Triem23
    Triem23 Posts: 20,679 Ambassador

    I'm just gonna tag in... Oh, @DannyDev today. Although if the same shift happens in Vegas I'm more likely to blame the video files with the wonky mismatched audio, this is good info. 

  • NormanPCN
    NormanPCN Posts: 4,194 Enthusiast
    edited December 2018

    Vegas handles and exports the file(s) just fine. Vegas shows the shift of the Hitfilm exports. I only used Vegas as a test to see if Hitfilm was doing something to/with the files on import or if the exports were shifted.

    Actually HF could be shifting on import and thus the exports are shifted. This would mean even the original is shifted. Only way to test that is to measure the offset of the first sound blip from file start against something known to be good.

  • Palacono
    Palacono Posts: 3,425 Enthusiast

    @NormanPCN Yes, that's what I meant too. Used Vegas to see the waveforms  initially, did not export from Vegas, but useful to know you also exported from Vegas and it handles them correctly.

    I think the problem is likely on Import, rather than in the act of placement, as when it's placed in a comp or the NLE, that's where the first shift of the original was evident, hence my exporting, reloading etc. etc. So I was comparing the original during playback in Hitfilm (lagging) to the same original played back in Media Player Classic (correct). I agree that something with sound blips would be the ultimate test.

    As it seems to be MP4 that's the problem input format - but we have a small sample set so far - I'll try putting some GoPro footage through GoPro Studio and put some of that Cineform.AVI output into Hitfilm, then export that as Cineform again, rather than using Hitfilm to go MP4>AVI, as you did. Maybe, like ProRes, that will work correctly?

  • Triem23
    Triem23 Posts: 20,679 Ambassador

    @NormanPCN yeah, I read what you posted incorrectly. I'm still jet-lagged from returning from New Zealand to California. 

  • Palacono
    Palacono Posts: 3,425 Enthusiast
    edited December 2018

    @NormanPCN I put a 1080p @23.976fps GoPro clip through GoPro Studio and imported the .AVI file, then loaded them both into Hitfilm. Not even exported from Hitfilm yet to see what gets better/worse.

    Clapping my hands for the sync and the original .MP4 has the audio spike lagging about 1 or 2 frames behind the clap (top audio, bottom video).

    The .AVI file actually has the audio shortened, with hatched marks at the end, with the clap audio spike (lower audio, upper video) looking closer to where it should be. But that's just a coincidence, as if you look at the other spikes - other large one was the rebound click of the 'Record button' - they're almost the same, so it was the same audio as the MP4 clip, just shrunk/sped up a bit.

    https://i854.photobucket.com/albums/ab106/pickaname2/clap-handGoPro.jpg

    Are GoPro incapable of exporting an MP4 file as an AVI file without messing up their own audio? Quite possibly, but this comparison shows that the original audio from the MP4 file is lagging...somewhere, caused by...something and the AVI audio is even worse and pretty useless. This clip was only 2 seconds long... 

    I then exported both the .MP4 and the .AVI as Cineform AVI and loaded them back in.
    https://i854.photobucket.com/albums/ab106/pickaname2/GoPro-Clap2.jpg

    What you're seeing here is: the GoPro.AVI had the audio stretched back out to be the same as the original, slightly laggy, GoPro MP4 audio. That's GOPR6021.AVI>Cineform AVI.AVI

    The other pair are: GOPR0621.MP4 (original) > GoPro MP4.AVI and it's now lagging further than before, which is expected, as exporting anything tends to make it laggier each time.

    Finally, in Vegas, the audio spike lines up with the clap image. In this first image the original .MP4 is the top track and the GoPro Studio .AVI is the lower one. The MP4 seems to line up with the spike. As this is a frame ahead of the image in Hitfilm, the second image below is the same frame as shown before, where you can see the audio has already passed, but on the same frame in Hitfilm it hasn't.

    As the GoPro Studio Cineform Exported .AVI file has different audio to the MP4 version: it looks like GoPro mess things up at their end too, which isn't helping.

    But, comparing the positions of the spikes and frames: what might have appeared to be a single frame lag in Hitfilm on import, might be a 1.5 or 2 frame lag.

    https://i854.photobucket.com/albums/ab106/pickaname2/Vegas Clap1.jpg
    https://i854.photobucket.com/albums/ab106/pickaname2/Vegas Clap2.jpg

  • CleverTagline
    CleverTagline Posts: 3,337 Ambassador

    I did a similar test to @Palacono, only with the original MP4 file from Tobias. Transcoded it to ProRes, then dropped both the ProRes and the MP4 onto the timeline. The MP4 is delayed later than the ProRes version.

  • Palacono
    Palacono Posts: 3,425 Enthusiast
    edited December 2018

     @CleverTagline Thanks for the update. Depending on when you started reading the post above: I've added to it twice. :)

    This is all a little confusing, and I guess for bullets and explosions, no one really notices; but if your actor's lines aren't sync'd properly, that could be more of an issue. Especially as it all depends on what you used to Transcode with in the first place.

    GoPro Studio? Bad doggy! Go to your bed! Actually, Handbrake, with your inability to handle 23.976fps an 59.94 fps files: you can join him...

  • St3v3C
    St3v3C Posts: 21

    I'm late to this party but just to add one more data point I see the same issue here on Tobias's footage. I exported Cineform in .avi and when I drop it onto the Vegas timeline the audio is delayed by approximately one frame at 23.976fps. This is on a Dell laptop, Win 7 with i5 and  GeForce 830m. Driver 389.08.

    Since I manage my audio in Vegas it is unlikely that I would stumble on this but it makes a nice reverb effect. ;-) And should be fixed!

  • Palacono
    Palacono Posts: 3,425 Enthusiast

    @St3v3C Thanks for the extra data point. I also use Vegas for editing, which is why I'd not spotted it before.

    Plus before my PC was upgraded I couldn't actually play back anything at realtime speeds in Hitfilm to be able to see it. Now I can't not see it.

This discussion has been closed.