Went from Pro 2021.1 to 2021.3, and with RAM preview gone, "timeline cache" is S...L...O...W

JBaymore
JBaymore Posts: 354 Enthusiast
edited January 11 in HitFilm

Hi all. Likely there is something I am doing wrong. I'm known for my "operator error" stuff. 😉

Prior to doing this last Point Three upgrade, I used the RAM preview a LOT when working. I do a lot of detail type stuff... and keep the resolution up to see what I am doing. So dropping the quality of the "working images" is not really a useful thing for me much of the time. I need to see the more subtle details of the edge of the mask, or the way the effects is appearing. Stuff like that.

When I compare the "timeline cache" rendering and the RAM preview situation... it SEEMS to me that for the same project that the cache approach is WAY slower to give me what I need, at least for me. Seems to have slowed down my workflow. I'm working on a scene shot that I started out working on in the older version... and now am working on it in the new version....so the "speed" comparison is somewhat apples to apples.

When the timeline has not cached, .... the playback goes in HUGE big "jumps" like a series of still images selected in series at random spacings from the progression of frames. Didn't do that before.

Machine specs are Asus Prime Z390 Mobo w i9-9900K Coffee Lake @ 3.6 G, 64 GB DDR4 RAM, 1 TB Samsung SSD as C (OS and programs):, 1 TB SSD as D: (video project stuff), LG Bluray burner as E:, 4 TB SATA HD as F: (archive storage), RTX-2080Ti 11 GB video card, in a Thermaltake full tower with 1000 W power and both fans and liquid cooling. Windows 10 Pro with all updates in place.

Any thoughts on this? Appreciate the help in advance.

best,.....................john

«1

Answers

  • Stargazer54
    Stargazer54 Posts: 3,555 Ambassador

    @JBaymore Your specs are certainly not the issue. Personally I turn off timeline cache and only use it as needed by clicking on the icon with the plus sign on it on the right hand side of the timeline (usually under the bottom right of the viewer window).

    Reaching at straws: Do you have an on board GPU that may be conflicting with the RTX-2080-Ti? Might have to go into the NVidia control panel and dedicate the 2080 only to HF? Is is possible your virus software is getting in the way? What do you see about resource usage when you run task manager? Any other programs stealing or competing for resources?

    Other than that, I would recommend that you contact FXHome support - https://fxhome.com/ask. They will be better equipped to work through the issue. Again, I can't see that your hardware would be the culprit here.

  • JBaymore
    JBaymore Posts: 354 Enthusiast
    edited January 12

    @Stargazer54

    Thanks for the reply.

    Yeah... I designed the machine specifically for video and audio editing... and it is not too old........ so it is decent. Although time IS slipping by and it seems to be getting "less decent" by the day. Sigh.............................. (it's only money.)

    The important point here for me is that FXHome said the recent updates focused on performance. I went from using the Point One version to the Point 3 version. But in my case, I feel that I got decreased performance. So that is odd, given that my machine is easily still decent for this version.

    I just went into the Nvidea Control Panel and made a couple changes to see if that makes a difference. Will try more compositing work shortly. I've always run the defaults for everything on that video card. So this is a "new thing" for me. But with it sitting at the defaults for the older HitFilm Pro version and for this new version.... there is a sudden noticeable "slow down" feeling. Without changing anything on the machine but the version of Hitfilm.

    Kind of "ditto" on the anti-virus program. Why that suddenly would affect things is sort of crazy. Nothing changed there with the anti-virus program. Unless maybe Hitfilm's code being revised now causes a conflict with something it didn't before. (BTW... anti-virus is Bitdefender.)

    You said "Do you have an on board GPU that may be conflicting with the RTX-2080-Ti? Might have to go into the NVidia control panel and dedicate the 2080 only to HF? " In "Device Manager" the only video card that shows is the RTX one. As best I can tell I just changed it in the Nvidea Control Panel from set at the default "use all GPUs" to use just the RTX card one. Not having used that Nvidea Control Panel before, I am not sure WHAT aspects to change and HOW to optimize the Hitfilm Pro program. Anyone have suggestions for optimal settings there for me?

    I'll look at Resource Manager again soon here....but in the past Hitfilm has had my computer pretty much in 'snooze mode' overall. Never seeing it taxing the machine much. If anything, historically it doesn't seem to really USE the 16 cores.

    I'll go talk to FXHome support also soon.

    best,

    .................................john

  • Stargazer54
    Stargazer54 Posts: 3,555 Ambassador

    @JBaymore Definitely should get with support because they need your information to run a potential issue to ground.

    As to the Nvidia Control Panel - you should be able to right click on the desktop and get a pop up window that has the Nividia Control Panel listed as an option. Below is a screen shot of how I have mine set up for HF. Note that under "OpenGL rendering GPU" I have mine set explicitly to my RTX 2070 and not Auto-Select. Not sure if this really makes any difference since I only have one GPU.

    You can check if you have more than one GPU by typing in "Computer Management" in the Win 10 search window on the lower left of your desktop and opening Device Manager. Then look for Display Adapters. If your 2080 is the only one listed then your system has only detected one GPU. AFAIK, HF should default to a dedicated GPU card rather than an onboard chipset. But it is always good to check and make sure HF sees the right GPU.


  • JBaymore
    JBaymore Posts: 354 Enthusiast
    edited January 12

    @Stargazer54 ,

    Thanks again for the help.

    I changed the setting in the Nvidea Control Panel to my RTX card (like what you have set). I also changed the "Power Management" setting below that to "performance". As I said above.... in Win Device manager ... there is no other video showing other than the RTX card. So no conflict with an on-board graphics adapter.

    I just did some compositing work.....and unfortunately I couldn't notice a change for either the better or for the worse. It is still feeling "slow" like it was. So basically.... no change doing that stuff.

    And when the timeline is NOT pre-rendered (either automatically or when I press the button)......... the advancing of the video in the video window is WAY horrible when compared to what I got before I installed the "Point Three" version. The playhead line will move to the right for a second and there will be absolutely NO change at all on the video showing in the window... and then suddenly a new frame (many frames later than the previous one) will show up and just freeze there.... and the playhead line will continue to advance to the right and then a new frame will pop up after maybe a second or two...and so on until the end of the clip. So sort of what I am seeing is the first frame..... the 33rd frame....the 61st frame..... the 90th frame and so on. Totally un-useable that way.

    So to work, I have to pre-render everything to see what I've just done. I did not have this issue before the new version. I'd get some "jumpy" playback when I hadn't RAM previewed something....but nothing even CLOSE to this. And when I used the RAM preview.... it gave me a workable solution faster on the same kind of composite.

    I just checked performance in Performance Monitor and Task Manager while HF was pre-rendering a 25 sec. clip with a 3-d model in it a couple of times. CPU use was hitting only about 10% consistently overall. However, CPU core 2 was sitting at 100% use consistently the whole time while most of the other cores showed little use or none at all. There was TONS of RAM still left unused. (How do I check the graphics card's GPU and memory usage????)

    So this isn't the CPU or the RAM getting overworked. I think.

    best,

    ..................................john

  • JBaymore
    JBaymore Posts: 354 Enthusiast

    Just created a ticket with FX Home.

  • Triem23
    Triem23 Posts: 20,244 Ambassador

    Random thought regarding Anti-virus.

    Some, but not all anti-virus programs scan all files upon creation and/or opening. In the case of the Timeline Cache, Hitfilm is creating individual files for each frame (so when altering something at 0:01:00:00 it doesn't alter/rewrite from 0:00:00:00 to 0:00:59:2x)

    Now, because the cache creates files for each frame we can see that the minimum specs for Hitfilm should include SSD drives - the cache will be particularly slow on an HDD.

    But, back to anti-virus. It's worth checking the settings to see if Bitdefender is constantly scanning everything. If so then that's really impacting cache speed. It might be worth a test where you shut bitdefender off, and maybe disable your Wi-Fi and see if that speeds things up a bit. If so, then we've learned adjusting the anti-virus settings can improve Hitfilm performance.

  • JBaymore
    JBaymore Posts: 354 Enthusiast

    @Triem23,

    Thanks for the thoughts. Luckily... I am running SSD's for the OS, the HF program, and also the video data I'm actively working on. Program is on C: and data is on D:. But yeah.... if it was a SATA type HDD........ that would really add time in.

    I just went and told Bitdefender to lay off of Hitfilm in a general sense (a check box in the setup). No noticeable change so far. Now to dig deeper into it and see if that keeps it from scanning files HF creates. And if there is a specific way to stop/override that function if it does.

    I hate having to shut off AV software to do work in individual programs......... but I WILL test that idea soon. I just feel "vulnerable" with the fiber optic connected while the AV is off. So that means turning off the fiber connection too. No wi-fi running on the computer....all is Cat5 wired here.

    Phil in support is actively working on this. So far it appears that nothing is "jumping out" yet. He's still asking for more info, step by step.

    best,

    ................................john

  • Triem23
    Triem23 Posts: 20,244 Ambassador

    Phil is good hands. 👍

  • JBaymore
    JBaymore Posts: 354 Enthusiast
    edited January 17

    Well.... got an "answer" from Hitfilm support. Not the one I expected or wanted.

    Summary of that answer............

    It's your machine for this kind of project in the Point Three version. Turn stuff down now during compositing/editing to lower settings (to like DRAFT!!!!!!) to get decent performance (like you had before).

    Apparently my machine specs have slipped to being barely OK.

    Sigh.........................

    Machine specs are Asus Prime Z390 Mobo w i9-9900K Coffee Lake @ 3.6 G, 64 GB DDR4 RAM, 1 TB Samsung SSD as C (OS and programs):, 1 TB SSD as D: (video project stuff), LG Bluray burner as E:, 4 TB SATA HD as F: (archive storage), RTX-2080Ti 11 GB video card, in a Thermaltake full tower with 1000 W power and both fans and liquid cooling. Windows 10 Pro with all updates in place.

  • Stargazer54
    Stargazer54 Posts: 3,555 Ambassador

    @JBaymore I always leave my settings to Draft while editing & compositing. You can't really tell the difference and if it speeds up playback while editing that's not a bad idea. The real difference (as I understand it) is Draft is still a high quality image, but Final will apply anti-aliasing to the Viewer. This can of course, slow things down and you really only need anti-aliasing when you render/export. Anti-aliasing is On by default during export, anyway. So the only reason to set your Viewer to Final is to check edges on titles or other graphics. But again I leave it off, because it gets turned on during the render.

    My system is very similar to yours - Asus Z97 Mobo, i7-4th gen, 32 GB, RTX-2070 Super, SSD, Win 10. Although, I keep my footage on a video grade HDD and write to a second HDD . With my system I can drag on 4K footage shot from my GoPro and only experience is just little when I'm working in 4K but it isn't enough to bother me.

    So I am wondering if it is your footage that's causing a slow down? Where did it originate from? Have you tried transcoding to a more editor friendly codec such as Cineform?

    Another option would be to let HF do a Proxy render on your footage. That would speed up the edit and yet still use the original footage during the render/export.

  • Triem23
    Triem23 Posts: 20,244 Ambassador

    @Stargazer54 Lynn, minor corrections here. I point you both to sections 4.2.5 and 4.8 in the manual.

    There are two sets of controls in the Viewer which affect playback. "Quality" and "Resolution."

    Stargazer wrote that anti-aliasing is toggled via setting the Viewer Quality to "Full." This is incorrect. Quality settings do not affect anti-aliasing. Anti-aliasing is actually a Resolution setting. 1/4, 1/2 and Full Resolution have no anti-aliasing. There is a Resolution setting for "Antialiased" which is Full Resolution with anti-aliasing.

    Quality settings toggle off or on the following options (copied from section 4.2.5)

    • 2D Effects: Toggles rendering of 2D effects on and off. By default, this option is enabled in Final, Draft, and Quick modes.
    • Lights: Toggles rendering of 3D Lights on and off. By default, this option is enabled in Final and Draft modes.
    • Shadows: Toggles rendering of 3D Shadows on and off. By default, this option is enabled in Final and Draft modes.
    • Reflections: Toggles rendering of Reflections on and off. By default, this option is only enabled in Final mode.
    • Motion Blur: Toggles rendering of Motion blur on and off. By default, this option is only enabled in Final mode.
    • Depth of Field: Toggles rendering of Depth of Field on and off. By default, this option is only enabled in Final mode.

    I additionally note that "Motion Blur" refers to motion blur on a moving layer. The Motion Blur EFFECT is covered by the "2D effects" toggle.

    The "Depth of Field" refers to Hitfilm's built in Camera DoF. The Lens Blur EFFECT is covered by the "2D effects" toggle.

    Of course the toggles for the Quality Settings can be changed in the Options Menu, but I think the defaults are spot on.

    As a side note I either hadn't realized, had forgotten, or it's a new enhancement, but the Viewer can also toggle color bit depth between 8, 16 and 32-bits (depending on Express with proper add-on or Pro). I'll need to check my own viewer settings, but I've been in the habit of changing Project Settings to 8-bit for most editing, then going to 32-bit for lighting and color grading. Having this in the Viewer Menu is much more convenient, and eliminates the possibility of my forgetting to set the Project back to the correct depth before a render! So, I got to learn a new thing while checking the manual to write this post! 👍

    Let's talk Timeline Cache for a second.

    Timeline Cache replaces RAM preview in Comp Shots and can be used on the Editor Timeline. The advantage of RAM preview is faster rendering, since frames are placed directly into memory. The disadvantages are one will always have a lot more drive space than RAM, and making any changes, at all to the Timeline invalidates the RAM preview.

    Timeline Cache is writing individual frames to separate files in the relevant folder. Of course writing the file is slower than paging to memory - and, in my opinion, effective use of the Timeline Cache absolutely requires an SSD... The 100-200ms seek time on a HDD adds 1/10-1/5 a second of delay to every frame written. At (true) 24 fps for a 1-minute Timeline that's 1440 frames, so an extra 144 to 288 seconds - 2.5 to 6 minutes - just to write the cache files (and that's not taking into account an HDD also having to grab original media files, if any, so for every media file read add another 144 to 288 seconds to the cache build. SSDs have negligible seek time.

    Now, the advantage to the cache is it's persistent. Unless you directly change a cached frame, it just stays on the drive and becomes a single "final" frame read to the Timeline. Changing a cached frame requires a recache, but only for the changed frames. The rest is left alone.

    Timeline Cache is helpful when creating a Comp, but, once it's "locked," you're better off pre-rendering.

    On the Editor Timeline, once the cache is built it speeds things up a lot.

    I'm general I turn off automatic caching. Caches can be manually built with this button under the Viewer.

    You can also set In/Out points to cache. I'll usually set up In/Out points, start a cache, then go use the restroom or get a drink or pet a cat for a few minutes to let the cache build so I'm not staring at it. This was how I dealt with RAM previews, too.

    Ok, let's talk a bit about system resources.

    As I understand it, Hitfilm does use multiple cores, but only a single core per media stream. John, in your case, I'll assume you're using a single video stream. In this case you're seeing relatively low use on core 1 - only being used when reading frames from the source media - while Core 2 is pegged at 100% because it's building the cache, so is constantly working. Cores 3+ are idle. In theory this means adding more media streams don't add any slowdown or bottleneck to the CPU because it just dedicates more Cores to the streams. This doesn't take into account bandwidth limits on drives or additional GPU time needed to render multiple layers.

    Additionally Hitfilm uses CPU/GPU in SERIES not PARALLEL. So, while the CPU is reading/writing media frames to/from drives, doing audio mixing, calculating particle physics and performing other CPU tasks, the GPU is idle. When the GPU calculates and renders frames the CPU is idle. In general this means the more CPU used the less GPU used and vice versa. You'll never see Hitfilm using all CPU threads and the GPU at, say, 90% at the same time.

    Knowing THAT helps read Task Manager. Let's say we've got one video layer and we have some effects on it, and maybe a text overlay or procedural graphic as a lower third. Here I'd expect to see maybe one CPU thread at a high usage rate, or about 12% total CPU (1 Core of an 8-Core processor would be 12.5% total CPU time). Maybe I'll see 10% GPU. This would actually indicate efficient and fast use of resources in the pipeline... The 10% GPU would mean that the GPU is finishing quickly (and has plenty of time to add more effects before GPU bottleneck) and a 12%ish use of CPU (95% one Core) meaning the GPU is finishing quickly enough to keep the CPU busy.

    100% GPU use would mean you have a complex frame taking significant render time, and you'd see very low CPU use because the CPU is waiting for the GPU a lot.

    John, again, the Task Manger example you gave indicates fairly efficient Hitfilm use. Low use of resources on Core 1 and GPU means Hitfilm is building frames quickly while that 100% use of Core 2 is the Timeline Cache being built in the background.

    Yeah, the Timeline Cache IS slower than RAM preview to build... But, when the Timeline Cache was in Beta another Tester and I went back and forth with the devs on this - we both preferred RAM preview in Comps, but the two different systems "fought" each other and it wasn't possible to keep RAM preview AND the Timeline Cache. It was decided the benefits of caching outweighed the disadvantages, but I suspect the Timeline Cache is a big part of the huge min specs jump in HF 2021.3.

    Lynn, in your case, you mentioned you're using HDDs for source and render drives. Hopefully you have an SSD you can use for caching? Otherwise, as discussed above, caching to HDD is S-L-O-O-O-O-W!

    AE users went through this same transition a few years ago when AE removed RAM preview for Timeline Caching.

  • assensy
    assensy Posts: 17 Just Starting Out*

    Very well written and explained but does not provide a solution to the problem. It's almost like the joke about the flying balloon and the Microsoft building.

    A lot of people are complaining about the new HitFilm features. Instead of explaining why this is happening, they should bring back the RAM player and give a choice: RAM preview or cache.

    I've already lost hope that a new HitFilm update will appear without a bunch of problems. I'd rather have an update with bugs fixed than a few new features that don't work properly.


    Translated with www.DeepL.com/Translator (free version)

  • Triem23
    Triem23 Posts: 20,244 Ambassador

    @assensy HF 2021.3 lists 36 bug fixes in the change log. Under "Enhancements," there are things listed like "better font support" and "better loading of GLB models," 5 "Enhancements" like that (such as "changing proxies now triggers 'unsaved' state for the project") could be called "bug fixes" as they fix problems. So, 41 problem fixes.

    HF 2021.2 lists 65 bug fixes. 3 Enhancements that fix problems for 68 total fixes.

    HF 2021.1 (16.1) lists 23 bug fixes.

    HF 16.0 lists 55 bug fixes and 4 Enhancements that fix problems for a total of 59 problem fixes.

    Or 191 bug and problem fixes in a year of development. Congratulations, you're getting your wish for Hitfilm releases that fix bugs.

    As far as bringing back RAM preview goes, as I noted in my prior post, the systems are incompatible, and RAM preview ain't coming back. As a Beta Tester/Ambassador (but NOT an FXhome employee) I'm bound by NDAs, so I can't really say more than that, other than to repeat that, in Beta, at least two of us had this argument/discussion with the devs and QA team. I do know work continues to improve the speed of the Timeline Cache. The changelog for 2021.3 included four enhancements and bug fixes specifically related to media decoding and the caching process itself. I also know that the Timeline Cache was in development for well over a year before it went live.

    Now, as far as the cache goes, there isn't really a "problem." The system works as designed and built. Yes, it is true that it takes longer to write a file to a drive than store to RAM, but that's called "physics," and unless you can change the speed of electrical signals, speed up data busses, and improve the bandwidth of drives, there's little more to do.

    Also, as stated in my prior post, the AE community dealt with this when their RAM Preview was replaced with Timeline Caching in 2018. Avid replaced RAM Preview with Timeline Caching in 2013. Sometime after 2011 FCPX removed Quickview (RAM preview) with Background Render (Timeline Cache). Davinci Resolve never had RAM Preview, just a Timeline Cache. Vegas Pro has a RAM Preview but no Timeline Cache. That's a grand total of zero NLEs with both a RAM preview and Timeline Cache - and every NLE that added a Timeline Cache lost the RAM Preview. Since no other company has both, I REALLY have to believe the Hitfilm devs when they say the two systems are incompatible... Cuz if I'd had an example of an NLE with both, I'd have challenged the dev team.

    There are a few ways to optimize Hitfilm performance, but the majority of them come down to the individual artist understanding the smart and the less smart ways of building projects. I'm reminded of a user who just didn't want to build nested/embedded Composite Shots. I argued with him for nearly a month about why his 150+ layer Comp Shot was dumb and how he could improve performance with a little nesting and pre-rendering. He finally listened, and things got faster. I'm thinking of a whole bunch of tutorials I've seen where the tutorial creator does something silly like create a 1920x1080 1080 plane layer then immediately create a circle mask in the middle... Starting with a 1080x1080 plane removes 46% of the pixel data - wasted data because it's all outside the mask.

    Here's an interesting blog focused on improving preview speed in After Effects. Everything in it applies to Hitfilm. In fagt6, in Ae they ran tests relating to the thing I brought up about using a 1080x1080 plane with a circle mask instead of 1920x1080 plane. To no one's surprise the conclusions are smart creation of assets, and proper use of systems like nesting and pre renders speed everything up.

    Ok, let's discuss this little gem of a statement: "I've already lost hope that a new HitFilm update will appear without a bunch of problems."

    Pick ANY software or operating system you can't say the same thing about... The May through December updates of Ae have 71 bug fixes. If you choose to go to the Microsoft site, you can look up all the known bugs in Windows 10 and 11. Feel free to look at all the console games with "Day 1" patches, or all the Steam games that release monthly patches.

    A reality of software development is that there are hundreds of millions of lines of code involved. Depending on the software there might be third party libraries involved - in the case of Hitfilm there's OpenGL, OpenCL and (in Express) OS libraries. Earlier in 2021 Hitfilm Express started using the OS native video encoders/decoders (Pro uses third party encoders). Right when that happened, a subset of users started reporting problems with video exports. Well, the problem turned out to be with the Microsoft encoders - that's not a Hitfilm problem but FXhome got the blame. Above I mentioned there were recent bug fixes dealing with video encode and decode? That's FXhome devs writing a patch into Hitfilm to fix something Microsoft screwed up.

    There are bugs which arose after the release of Hitfilm 2021.2 that didn't come up in testing - because they related to certain hardware combinations that none of the testers have... And before you think, "well, test on everything," I've already done this math. Just in the last two generations of hardware, between Intel and AMD there were almost 80 CPUs that could run Hitfilm. Between Nvidia and AMD there were over 25 GPUs that could run Hitfilm. That's 2000 different hardware combinations that are capable of running Hitfilm, and we haven't even looked at INTEGRATED GPUs, motherboards, drives, RAM and other system variables that could cause problems. Intel is on the 12th-gen CPUs but Hitfilm is compatible with 6th-gen and up. CPUs. Add in the AMD lineup from 2016 on and that's several hundred CPUs. Take into account all the GPUs and that's over a hundred different GPUs ranging from MX series to Quadros and everything in between. Now we're getting into tens of thousands of hardware combinations.

    Now we have the variables of other software running on everyone's systems. Some anti-virus software scans each and every file when it's both opened and saved... And if you're running a Timeline Cache on that 24fps, 1-minute Timeline with 1440 frames and you're running one of the anti-virus programs that scans every file when saved or opened, then your Timeline Cache performance is going to be really slow because of something FXhome had nothing to do with. There are some known issues with Dell Backup and Recovery (software known to cause problems with other programs as well). Every Dell/Alienware computer has some issues with Hitfilm and other software because of something Dell wrote. There's an issue some users have had with dual-monitor setups (not me with any of my monitors), which, in some cases turned out to be an interaction with Asus Sonic Studio. Asus Sonic Studio is an audio settings manager, so it's certainly not the first place anyone would look to solve a monitor problem. I have no idea if any of FXhome's testing machines or if any of the machines owned by outside testers are ASUS, but, if not, well... You can see why it took a year to figure out the conflict. So, bugs are gonna happen.

    Yup, it's another long post describing the issue without a solution - cuz there isn't one.

  • JBaymore
    JBaymore Posts: 354 Enthusiast
    edited January 18

    @Stargazer54 ,

    Thanks for the feedback/thoughts.

    "I always leave my settings to Draft while editing & compositing."

    I have always had my settings set to "final" in the past since building this machine (just over 2 years ago), and Hitfilm used to run at an acceptable level for me with most anything I threw at it. It is appearing that it is possible that this is just 'the way it is' with Hitfilm Pro moving forward with a machine with specs like mine. It was pretty cutting edge 2 years ago......... guess that 'current' window is closing faster than it used to. I'm hoping that is not the case... that it is 'something else'. This "update", at least form me, resulted in a HUGE performance change all at once. And although marketed as being done for a lot of improving performance, mine moved backwards FAST. I think that is what is the "insult to injury" aspect here.

    Summary of workflow changes so far for me with the new version: 1.) Proxy/cache is way slower than RAM preview was to render a clip or a section of a clip. 2.) Playback in the viewer without RAM preview or caching done is totally un-useable at the same settings I use to use (a 'slideshow').

    "With my system I can drag on 4K footage shot from my GoPro and only experience is just little when I'm working in 4K but it isn't enough to bother me."

    I rarely use any 4K stuff. However I AM in this particular piece of work I am comparing from 2021.1 to 2021.3 versions.... for one file used in the project. Mainly for what I do I am running projects in HD -1920 x 1080 ....and all are set to output/export in that format. I know my hardware is not "up" to running everything in 4K. Plus I don't have any 4K camera gear to generate the main footage anyway.

    "So I am wondering if it is your footage that's causing a slow down? Where did it originate from? Have you tried transcoding to a more editor friendly codec such as Cineform?"

    The only video footage in the project is 'commercial stuff' from Production Crate (I have a paid account). It is all "effects". I do use a lot of stuff from Action VFX (own many paid assets from there also) and Production Crate for VFX work. I've never trans-coded anything in the past for Hitfilm...and had decent performance before up til and including 2021.1.

    The testing clip involved here is composed of a 3-d (lidar scanned) .fbx model of a cave interior. The whole composite shot is in 3-d, @ 30 fps. Bit depth at 8 bit, 4x MSAA. There is no motion blur set to on for anything. It is in the same state (for testing) when it was in 2021.1 and now in 2021.3 versions, for consistency. I've stopped actively working on it to change things. It is sort of the benchmark at the moment.

    A Hitfilm virtual camera is set in back inside the cave a bit, and is slowly dollying forward toward the cave entrance opening (which opens to the "world". Going from dark-ish to light. Eventually our green-screened actor will be overlaid in the foreground (from behind) walking toward the cave entrance (but this part is not implemented yet). Dolly speed eventually will get varied a bit based upon the actor's actual walking pace.

    There are two lights involved. One is "the sun" and is set at an angle to impact the area immediately inside the cave entrance. The second is "bounce" from the outside world that is illuminating back into the cave interior a bit. Shadows and ambient occlusion are set to on for the 3-d cave model. The lights are not set to illuminate any of the stuff outside the cave's interior.

    That outside "world" is composed of a couple still frame planar elements set in the 3-d distance. One is a Creative Commons still image of a landscape with some mountains, and it is set at some distance from the cave model to give depth and parallax. The sky from that image was masked out, and behind that mountain image in 3-d space is a still sky image from Production Crate. During the dolly forward the sky image is slowly being moved to the right to create simulated movement in the clouds in the image.

    Near the entrance to the cave there was a small hole in the cave's mesh. I have a 200 x 200 3-d plane rotated and sitting behind that small hole and color matched to the cave wall there in order to "hide" the hole.

    There is a 4K Production Crate clip of some blowing ground mist sitting just inside the cave entrance in 3-d space, and it is set with limited opacity. There is a 2K Production Crate clip of a pine tree blowing in the breeze set just outside the cave entrance in 3-d space that can be seen as the camera moves forward. Behind that in 3-d space is a short 2K Production Crate clip of a couple birds flying by.

    That's the shot in question here used for an example between the two program versions.

    I know that I can now start changing settings and also do stuff like trans-code. And likely I will be. But I am still hoping that there is "something else" that is a glitch rather than a total negative performance change from Hitfilm running on my machine specs. This machine was an expensive build 2 years ago, done with an eye toward the future. Not sure the budget is ready for another new cutting-edge machine just yet.

    I've "supported" FXHome / Hitfilm for a while with my wallet... and am currently "pre-paid" on my renewals for even another year ahead now. I had intended to keep up with that "paid in advance" approach to help support the product development. But now I'm wondering if moving back to 2021.1 is the solution for the foreseeable future for me... and if my Hitfilm upgrades are at an end for a good while until it is time for a new machine (couple years). Replacing expensive computers every 2 years or so is NOT in my budget. I also wonder how this "now you need a really robust machine" change in the program is going to impact Hitfilm sales/renewals. Seems to me the userbase for Hitfilm (even Pro) is not the "serious" pro market but is more indies and hobbiests. Will they keep up with rapidly escalating hardware requirements?

    best,...........................john

  • NormanPCN
    NormanPCN Posts: 4,088 Enthusiast

    @Triem23 "The 100-200ms seek time on a HDD adds 1/10-1/5 a second of delay to every frame written.'

    What is the source of the 100-200 ms. Just curious.

    --

    "Vegas Pro has a RAM Preview but no Timeline Cache."

    Nothing automatic. When I was using Vegas. Vegas did have a manual timeline pre-render. You could choose various regions as desired to pre-render. Manual pre-render of the bits that are done'ish but slow/complex to render on the fly and then ram preview for the bit you are current working on. You are more likely to have enough ram for a single bit than all the bits. Thus everyone having a background cache to disk. Memory issues are especially true with 4k and all that.

    Hitfilm just needs to be efficient.

    --

    ...but then that is the rub. Hitfilm is never efficient. Pretty much across the board (even 2D pixel graphics, e.g. glow). This is being picky to some extent but JBaymore reporting here should be heeded as one hell of a warning. Even the use of HW decoders for AVC is being done badly. A feature that brought much needed performance to the Hitfilm timeline. Performance is being left on the table. Luckily, most will not see this but a thread some time back brought this to light.

  • JBaymore
    JBaymore Posts: 354 Enthusiast
    edited January 18

    @Triem23

    I just got help from Bitdefender on how to tell the anti-virus program to "ignore" all to do with Hitfilm without having to totally shut off the program. They did not specifically SAY that it did scan the files upon creation and or opening....................so I asked for that info again.

    Will go and look at their instructions and implement that change.

    Will post here if that makes any difference for me. And if I get an answer to the scan question.

    best,......................john

  • Triem23
    Triem23 Posts: 20,244 Ambassador

    @NormanPCN non-scientific, mom-comprehensive estimates based on looking up specs for USB powered HDDs I own and based on looking up specs of other users having similar complaints/issues on Facebook FB groups, Reddit and Discord. Which comes out to be about a dozen drives.

    Saw your question a couple hours ago and did a little more overall research and find that current ENTERPRISE drives and RAIDS can have seek times as low as 5 to 25ms while many internal drives are closer to 50-100ms with only the external USB drives (especially the 5600rpm ones) having seek times over 100ms. A specific user on Reddit had a slow as sin drive with a seek time of about 225ms.

    So, strictly speaking, my estimated range was a little off, and skews lower than what I've typed in this thread. In future discussions of the topic I'll use the wider potential range of times with 50-79ms as my "median" range. Under 50-ish would be particularly fast, over 120-ish would be particularly slow.

    But, even if we take my 200ms seek time example above - adding about 2.5 minutes to a cache of a 24 fps, 1-minute Timeline - and correct down to a 50ms seek time... Well, let's first discuss an embarrassing math error. 288 seconds is closer to 4 min, 40 seconds, not 2 min 30 seconds... But correcting 200ms seek time delay to 50ms seek time still added about 1 minute 12 seconds to that cache of a 1-minute Timeline. 25ms seek time would still add 36 seconds, which is still quite significant.

    So, thanks for asking the question, because you got me to spend a couple more hours refining my numbers so the discussion will be more accurate, but, even adjusting seek times down, we still see an HDD is really going to slow down caching significantly! And, of course, I used a (true) 24 fps example. Higher frame rates merely compound the issue.

    We can still bottom line the conclusion that a user really needs an SSD to have any hope of a useful cache.

    @JBaymore yeah, I don't know if changing the anti-virus settings will help... It depends on if Bitdefender is one of those aggressive "scan everything" programs, but maybe you'll get lucky and that will have been a bottleneck. 👍 Here's hoping! That would be an easy "fix!" If it does work, let me know and I'll make a note for other Bitdefender users.

  • Stargazer54
    Stargazer54 Posts: 3,555 Ambassador

    @Triem23 ...you mentioned you're using HDDs for source and render drives. Hopefully you have an SSD you can use for caching? Otherwise, as discussed above, caching to HDD is S-L-O-O-O-O-W!

    Believe me I understand this. That is why all my cache is on SSD.

    I've mentioned this before. I read footage from one HDD and write to another (like I'm rolling from one tape deck to another). This all works just fine and I have no performance issues.

    When I built my machine, large sized SSD's (1TB and above) hadn't proliferated enough for it to be cost effective. Of course if I built a new machine today, I would make different choices. But like @JBaymore , I can't afford to build a new machine every couple of years. (However, my rule of thumb for the lifetime for a digital video editing system is about 4 to 5 years.) Mine was built around 2014 - and I upgraded the GPU a couple of years ago. So the clock is ticking.

    The irony is JBaymore's system is beefier than mine so not sure why he is having issues. But Mike you may have hit on a potential solution, though. @JBaymore - as suggested, if you have multiple comps, you would be best served to build an overall comp with the other comps nested inside and then pre-render the nested comps.

    From the description of your project, pre-rendering comps might be the way to go.

  • JBaymore
    JBaymore Posts: 354 Enthusiast

    @Triem23

    "We can still bottom line the conclusion that a user really needs an SSD to have any hope of a useful cache."

    Yeah.... it appears that way. Glad I went that route 2 years ago with two 1 TB ones as the C: and D:. While likely SSDs are 'the future' for almost everything (until "Quantum Xtals" arrive 😉 ) ................. and the cost has come WAY down now on SSDs........... I'm guessing that a LOT of people that already use Hitfilm (Express and Pro) have machines that do not have them installed. And those considering purchasing it too.

    So this program change decision by FXHome is potentially quite impactful. New minimum requirement really needs to state a SSD for program and data files. If my experience with the viewer playback when it is not pre-rendered/cached is an indication of performance.

    best,.................................john

  • JBaymore
    JBaymore Posts: 354 Enthusiast

    @Stargazer54 ,

    "However, my rule of thumb for the lifetime for a digital video editing system is about 4 to 5 years."

    Yeah... mine too. Going to "outdated hardware tech" in two years is a little much. Programmers always push hardware capabilities to the max.... but that is really pushing it.

    "The irony is JBaymore's system is beefier than mine so not sure why he is having issues."

    THIS is the thing! At the start of this thread... it seemed that both you and Triem23 quickly looked at my machine specs and basically said "not your machine". Others are not absolutely flooding the forum here saying the same thing for problems (a few are).

    So maybe the real source of the issue here is that with my machine I had the "luxury" of working with my settings very high and still getting Hitfilm performance that was OK for me. And also I got away with doing limited use of having to be layering comps and stuff like that. Maybe other folks have had their settings pulled back all along, and have been layering comps, and the like and have not noticed the issue as "quickly" as I have.

    It is hard to compare "apples to apples" across different people's workflows, procedures, and machines.

    best,.............................john

  • Stargazer54
    Stargazer54 Posts: 3,555 Ambassador

    @JBaymore I think we are on the same page here on the hardware side. Like I said, I think pre-rendered comps are the way to go. I don't utilize that very often when I am just doing a straight video edit (i.e. documentary or info-mercial). But if you have complex layered effects it just makes sense to break that up into different comps that can be pre-rendered.

  • NormanPCN
    NormanPCN Posts: 4,088 Enthusiast
    edited January 19

    @Triem23 Okay thanks, but not buying your seek numbers. Waaaay back in the day, late 80s, early 90s, drives commonly had a 28ms seek time and they were 3600rpm drives back then. HD have not gotten slower. Probably have not gotten (much) faster in seek as well. Now seek and access can vary depending on how one wants to benchmark. Seek may not include rotational latency depending what you are reading and/or measuring. I don't care what anyone says. I just know what I have measured for myself or what a reputable review site might have benchmarked. It ain't never been anywhere near 100ms. Usually < 30ms "average" access (seek + latency)(read) for stuff is good working order.

    I've no idea what overhead USB can add. Also, some low power drives might be on the slower side and not because of spindle speed. That can vary by many factors and anything else on the same bus as the HD. It's just not a sensible thing to do intensive I/O over USB. USB was never designed for that. Lots of reading or even some bulk writing is okay. If you have a decent/suitable/appropriate amount of ram, reading can be partially or largely negated by the system disk cache. Something I showed in a post years ago on my old machine which had a spindle disk.

  • JBaymore
    JBaymore Posts: 354 Enthusiast
    edited January 19

    I kind of find this marketing on the current Hitfilm Pro page a tad ironic at the moment:

    "Work at lightning speed using HitFilm’s multi-core and GPU accelerated system. With our unique sequence and compositor prefetcher, we have improved your playback performance with no compromise on your workflow.

    Edit your videos without interruptions. Our high-performance system and threaded rendering means your interface will always be snappy and fast, no matter what’s rendering underneath. Creative software that gets things done."


    Also this on the minimum/recommended specs page:

    "HitFilm Pro is multi-core and GPU accelerated. It has seen some vast improvements under the hood providing superior performance and speed like you’ve never seen before."

    MIN: 6th Generation Intel® Core™ i5  REC: nothing is listed

    MIN: 8 GB REC: 16 GB RAM

    MIN: NVIDIA GeForce GTX 10 series REC: NVIDIA GeForce GTX 20 series 

    Nothing listed for drives.

  • JBaymore
    JBaymore Posts: 354 Enthusiast
    edited January 19

    Here's a phone recorded clip of the playback quality in the viewer in version 2021.3 of the clip in question here under discussion, showing the level of performance without having the clip already rendered/cached. It is pretty "un-useable" as is. When cached/rendered already and I play it back it is just FINE (but to do that process takes longer than it did to RAM preview it). In version 2021.1, this exact same clip has a little playback roughness without the RAM preview having been done, but it was WAY better than this. :

    https://www.dropbox.com/s/wamy6a4je6gtwep/PlaybackJumpy.mp4?dl=0

  • JBaymore
    JBaymore Posts: 354 Enthusiast

    Well.... I just followed the specific Bitdefender instructions for telling the program to "ignore" Hitfilm that they gave me. But I'm not completely sure that those selections in Bitdefender are yet doing exactly what I want. In fact, I'm still not sure IF Bitdefender even accesses each file as it is created and run. But I'm working this direction "just in case it does".

    I can tell Bitdefender to ignore certain .exe files. So I've told it to ignore Hitfilm.exe and also HitfilmRenderClient.exe. Are there any other of the .exe files that I should exclude?

    I can also tell Bitdefender to ignore certain folders. So I've added the folders that contain the exports/cache stuff too. Anything else you can think that I should add to that list?

    One other option that I have NOT yet implemented is that I can tell Bitdefender to ignore completely looking at a certain file extension type. This is HAVE NOT yet done... and don't really want to. Telling it to ignore all .mp4's or the like seems to me to be inviting a disaster. I apparently can't "target" it to not open .mp4's that are only from Hitfilm. Thoughts on that one?

    So with the first two Bitdefender actions already in place, I just tested that reference Hitfilm clip. Unfortunately, there was no noticeable change in playback performance without the previous cache/render in place.

    So........ chasing this aspect is looking less likely to be the issue.

    best,...................................john

  • Triem23
    Triem23 Posts: 20,244 Ambassador

    @JBaymore 2021. 3 did come with a huge min specs change, and I have already made a recommendation to Staff that the min specs also include "SSD for Timeline Cache." I suspect the worry there at the FXhome end is confusion from users thinking they'd need to have SSDs to just run the software, when that isn't correct...

    Come to think of it, I want to ask @TheBenNorris for a clarification.

    Ben, current min specs list Nvidia 10xx GPUs. Does that mean any and all integrated GPUs are now considered under spec for 2021.3 and up?

  • triforcefx
    triforcefx United StatesPosts: 1,340 Moderator

    @Triem23 just a bit of speculation, but I believe it has more to do with the age, driver, and feature support of the hardware, rather than the actual raw power


    I do think some optimizations could be made under the hood to the timeline caching…

    I haven’t looked at it super closely, but it might be a good idea to limit timeline caching only to “extra” processor cores and let the rest of HitFilm run on the first 2 cores.

    I think HitFilm has always had a hard time “task switching”, but this was largely alleviated when the multi-threaded interface was introduced around v12.

  • CedricBonnier
    CedricBonnier Posts: 1,202 Staff

    @Triem23 The minimum specs listed on the website are correct. We've found issues with the performance of hardware decoding (or it might have been encoding, I can't remember on top of my head) on NVidia 9XX series cards, hence why we bumped the min specs. The integrated Intel cards can still perform well in those situations so the 620 and above are supported.

    @JBaymore Your project does seem really slow to work with and I would be curious to play with it. Would it be possible to send it to support? If you still have the version that was saved on 2021.1 that would be most useful for us to compare performance on the two.

    Adding timeline cache has proved much more difficult than initially thought. We found some bugs and performance issues in drivers, which made it hard to make the feature work for everyone. If you are still encountering worse performance in 2021.3 than 2021.1 then please do contact us via support.

  • Triem23
    Triem23 Posts: 20,244 Ambassador

    @CedricBonnier thanks, muchly. Good to know the integrated GPU is still viable. 😁

    I'm not surprised there was an issue with the Nvidia 9xx series. Many years ago I reported an issue with my Nvidia 980m 8GB card. I think Hitfilm was only seeing 4GB of the VRAM, but I don't think you guys ever figured out the problem. Annoying, cuz I spent an extra $200 for that extra 4GB of VRAM and Hitfilm wasn't the only program that treated it as 4GB. Either way my 2060m outperforms my 980m by WAY more than it should based on the benchmarks for the cards.

    Once every few generations chip makers seem to swing and miss. The 6xx series was also problematic and often performed slower than the corresponding 5xx card.

    To clarify my prior post, I didn't mean to imply the specs listed on the website are incorrect, but, that if FXhome were to recommend SSDs specifically for timeline caching users would misread that and think an SSD was required to run the software. 👍

  • JBaymore
    JBaymore Posts: 354 Enthusiast

    @Triem23

    The "truth in advertising" thing with that approach is that in the marketing work, FXHome touts the timeline cache business as a main feature/benefit of the 2021.3 (and likely future) product. So timeline cache is an integral part of Hitfilm. This is a bit of a technicality... but I think an important one.

    This situation almost brings up a concept where there is a "chart" that lists what program functions you can use in the core program based upon what hardware you have to run it on. Luckily, they let someone test drive the program before buying..... so that should suffice.

    best,

    ..................................john