Express 2017 Crashes while Importing long videos

RichardMcGillRichardMcGill Website User Posts: 4

Just like the title says, this issue is really starting to wear thin and doesn't do it in Hitfilm 4 Express.

Regardless of file size, bitrate, resolution or framerate, or even run through handbrake, you try to Import anything that's around 2 hours long and the software just rolls over and dies every single time, I have to go into task manager to find it "not responding" to quit out of it, yet I do the same in Hitfilm 4 and the video is there instantly.

Also before you ask what my specs are.

CPU: Intel Core i7 6700K 4.6ghz OC
CPU Cooler: Corsair H100i v2 Liquid Cooling System
GPU: Nvidia GTX 1080 Founder's Edition
RAM: Corsair Vengeance LPX 16gb DDR4 3000mhz
Mobo: Asus Maximus VIII Ranger
HDD: Samsung 950 Pro 256gb M.2
HDD2: Seagate 2TB Hybrid SSD
PSU: Corsair RMx 650w

I seriously would have upgraded to Pro if it wasn't for this issue as I use it for my gaming YouTube channel, thankfully I didn't uninstall Hitfilm 4 Express as the 2017 trial had the same problem and I had my doubts.

Is anyone looking into this?

Comments

  • NormanPCNNormanPCN Website User Posts: 3,948 Enthusiast
    edited May 2017

    If Windows says "not responding", then Hitfilm has (probably) not crashed or hung.

    It is scanning the media to determine if it is constant frame rate (CFR) or variable frame rate (VFR). It is doing this by doing a full frame decode which is slow. With a 2Hour long media video this can take a while. It will eventually finish. This is all part of the VFR support feature in Pro 2017 and Express 2017.

    It is doing the scan from the user interface thread. The one processing mouse and keyboard inputs. While doing the scan those messages will not be processed and Windows marks HF as not responding. This is not good.

    1/10 second rule (Petzold).

    You mentioned nothing about your media specs. I bet that it is AVC/H.264 video in a MOV/MP4 file. AVC typically has a high decode overhead compared to other formats.

    Is anyone doing anything. For my part, I have reported to FxHome about the implementation of this feature. The fact that it appears to be doing a full frame decode, verses a frame timings scan. edit: The CPU usage is too low for a decode. I traced the call stack on one of the reads during the scan. It seems to be doing a full demux via MC.

    Hitfilm also appears to be doing some amount of this new file scanning on every project open. It does not appear to be saving scanned VFR/CFR information for future use. At least that is the case with a test project I created.  An initial slow scan can be tolerated but on every open, not so much.

  • PalaconoPalacono Website User, HitFilm Beta Tester Posts: 3,442 Enthusiast

    Why is it scanning the files? Isn't the info it needs in the header? Header says CFR? Load like HF4E does it. 

    It would be nice if we knew what it was doing with the cache files now. Are these a little like Vegas does to cache the audio? 

  • NormanPCNNormanPCN Website User Posts: 3,948 Enthusiast

    @Palacono I cannot say exactly why Hitfilm is scanning., I just know it is. It is a recent thing so I assume it is related to the VFR support.

    I know one thing Hitfilm does with the cache. The Hitfilm cache stores resampled audio. If your project sample rate is different than the media sample rate Hitfilm resamples and stores that in the cache. It stores some stuff related to video in HFPK files in the cache. I think that is new and related to the VFR support. It was over a year ago I wondered what the HF cache was so I emptied it and loaded various things to see when something(a file) shows up in the HF cache. It is kinda like the sidecar files that Vegas stores. Just in a central location. Like the Vegas SFK files are data about how to display audio waveforms efficiently and fast.

    As for VFR/CFR in the header. I don't know all formats but I know MOV/MP4 quite well. Yes, MOV/MP4 have this information in the header. It is the stts box.

    Here is a VFR file.

    Here is a CFR file.

    You can see the stts header data is quite small. The CFR file has a single entry in the array. The VFR has many but is still a small table relatively speaking. At most one entry per file frame.

    AFAIK, AVI technically does not support VFR but it can be shoehorned into the spec legally with null chunks for empty/unused frames to fill gaps appropriately. So in AVI one must scan the file chunks. Looking for blanks. But then since the file is CFR the runtime/playback just needs to understand what to do with empty/blank video chunks. Meaning use the previous frame again.

    Don't know about MXF (XDCAM) or AVCHD (mpeg transport stream).

    Realistically the Mainconcept libs HF is using can encapsulate (hide) all these container format variations. Maybe doing a full demux does this. This simplifies the programming but it does look to induce a significant overhead. The easy way is often slow(er). Big files, or a lot of files, will have a noticeable slowdown in load times. 4K is a major OUCH since they are big no matter the length in time.

    One slow scan can be tolerated but scanning every file on every open is wrong IMO. I've tested this with AVC media and support has my test case. My test case was 4 55 second UHD AVC fast decode media files. The project load in 2017 is about 10x slower than in HF 4. 3-4 seconds goes to 35-40 seconds.

    Even in the MediaInfo utility you can see that some container formats do not have as extensive header metadata like MOV/MP4 has. MediaInfo can display various specs very fast for some and others not so fast. The file has to be scanned for some.

    We users of CFR media are being burdened by overhead to support VFR. I don't think that is right. I can understand that FxHome wants the VFR thing transparent and automatic given its user base. Especially typical users of VFR media.

    IMO you cannot make CFR use slower because of VFR.

    I would prefer a global preference to disable VFR detecting scanning. That option can be defaulted on. That is quick and dirty. Then maybe have a right click media panel option to do a VFR scan. An option that would be remembered if HF detects the file changed.

    99.9% of VFR is going to be in MOV/MP4 containers. I am not sure about AVI and VFR otherwise I might be inclined to go 100%. Realistically MXF and AVCHD will never be VFR. If the MXF/AVCHD containers support VFR the cameras and apps that output that stuff always do CFR.

    In MOV/MP4 it is trivial and ultra fast to detect (stts) and analyze VFR frame timings. So if HF only supported VFR in MOV/MP4 containers and assumed CFR in all other containers as it has in the past, then you can be automatic and transparent with VFR detect and so friggin fast nobody would know you are doing the VFR thing. What you give up with this approach is VFR in other container formats, which may not even support VFR. IMO.

    Maybe the stts box existence in MOV/MP4 is not absolutely mandatory. I've never seen a file without it. Shadowplay output has it and so does OBS.

    Supporting every theoretical possibility is great and awesome so long as it does not harm. IMO, there is "harm" (media load performance loss) in the implementation as of this writing.

  • RichardMcGillRichardMcGill Website User Posts: 4

    I'm fairly new to the whole video editing and learning as I go along, so some of your acronyms have gone over my head lol

    As for my media, my Elgato software exports them as h.264. MP4 and I always give them a run through handbrake to make sure they have a constant frame rate.

This discussion has been closed.