When to remove noise, before, during or after editing?

TMFilmsHD
TMFilmsHD Website User Posts: 80
edited February 2015 in Practical Filmmaking

After partially filming my first scene of the new short film I'm making, I a little disappointed by the amount of noise I had in some shots. I was thinking of using noise reduction software since HitFilm 2 Express doesn't have the noise reduction feature, but I don't really know what to do best, in combination with the MOV DNxHD format.

(By editing I also mean applying a grade which worsens the noise a bit) 

-Should I convert the video to MOV DNxHD, reduce noise, and start editing? 

-Should I reduce noise, convert to MOV DNxHD and start editing?

Or should I convert to MOV DNxHD, start editing, export the scene, reduce noise, import it into the project with the rest of the scenes later on? 

P.S.

My camera (Nikon D5200) has got an inbuilt noise reduction feature which I set to low/normal. There's some options for higher noise reduction too, but won't that affect the quality of the image in the end? I have no experience with this function. 

What do I do to get the best looking image in the end?

Example shot, no DNxHD used, MAX ISO 800 + Grade (Not the worst noise) 

 

«1

Comments

  • SimonKJones
    SimonKJones Moderator Website User, HitFilm Beta Tester Posts: 4,450 Enthusiast

    Generally speaking, you want to keep the amount of conversion to a minimum. Every time you convert, you open up the possibility of losing data. You can convert to lossless codecs, of course.

    I'd probably denoise as the first step, then export to a stable codec of some sort (DNxHD, ProRes if you're on a Mac, etc), then use those files as my main working files. Denoising as the first step makes sense, though, because you're working with the original data, without any further processing having happened to muck things about.

    HitFilm 3 Pro has a new denoiser which is pretty powerful, which can be tested in the demo.

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80

    Excellent explanation, thank you Simon.

    I tried exporting to Youtube from HitFilm 3 Pro demo, but it wouldn't let me. It doesn't recognize the youtube account or password. I know I entered the password right, so could this have to do with the fact that I have multiple channels under 1 e-mail account? 

  • Robin
    Robin Website User Posts: 1,671 Enthusiast
    edited February 2015

    This is an issue that has come up multiple times, and has to do with the google security policies. Google blocks authentication attempts through third-party software it deems "less secure". You either have to use two-factor authentication and generate an app-specific password for HitFilm, or you have to tell google explicitly to accept those less secure apps. Have a look at this page: https://www.google.com/settings/security/lesssecureapps

    @ The HitFilm team: Maybe include a notice about this behaviour in the export screen when exporting to YouTube? Seems to be a common problem.

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80
    edited February 2015

    @Robin, thanks for the pointer. I tried turning it off and exporting to Youtube, but it still wouldn't let me. Not sure what's the problem now, maybe I'll need to restart my PC. I'll let you know if anything happens.

    While trying to come up with a good editing flow (noise reducing and converting wise) I can't find any standalone Video-noise-removal programs. All that's out there is plugins, plugins and more plugins. 

    Would anyone happen to know any standalone video noise remover?

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast

    I don't know of a stand alone denoiser but if you're looking to avoid costs there are several free noise reduction plugins for VirtualDub like the MSU denoiser filter that might do the trick. There are different approaches to denoising so you might have to experiment to find a plugin that does the best job for your video. As a bonus VirtualDub can also be set up to encode to DNxHD or ProRes so you can denoise and encode at the same time.

    See this thread in another forum for setting up VirtualDub to encode to DNxHD or ProRes. These profiles in a post about halfway down the page

    https://encodingtalk.com/threads/virtualdub-save-direct-to-mp4-fraps-to-youtube-with-external-encoders.2446/page-2

  • Triem23
    Triem23 Moderator Moderator, Website User, Ambassador, Imerge Beta Tester, HitFilm Beta Tester Posts: 18,948 Ambassador
    edited February 2015

    NEATVideo has a stand-alone version, I think. Although the VirtualDub option Aladdin mentioned is quite useful, and free. NEATVideo is about a hundred bucks. NEATVideo's approach is similar to HFP3's, so if you figured out Hitfilm's denoiser, NEATVideo's will be cake. 

    EDIT: never mind, NEATVideo no-longer offers the standalone, it seems. 

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80

    Thanks for the infos. NEATVideo indeed no long offers the standalone, the problem was that there were leads to it on the internet, but the program itself is no more.

    So I´m experimenting with Virtual Dub at the moment, but  it doesn´t seem to be able to import .mov (quicktime) files. 

    I'm wondering, (looking at Aladdin4d's post) since this program seems to be open to adjustments, if I can make it import .mov files? 

    Because... 

    .mov files -> converting to .avi -> denoise in virtualdub -> export -> convert to DNxHD -> import into hitfilm

    ..seems like a ton of extra work and diskspace.

    ---

    I'm also experimenting with DaVinci Resolve Lite, it seems like there's ways to remove video noise manually and it's able to export to DNxHD straight away, for editing in HitFilm later on. Not sure if anyone's got experience with this, but it I think I'll have to learn how this program works.

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast
    edited February 2015

    Sorry I didn't realize you were starting with an MOV or I would have added info on how to open it in VirtualDub. VirtualDub supports import plugins too so try this:

    http://gral.y0.pl/~fcchandler/Plugins/QuickTime/

    If you can open the file in Quicktime this should allow you to open it in VirtualDub. If for some reason it doesn't work there is another similar plugin here: (Last entry in the list)

    http://www.tateu.net/software/

    As a last resort there is also an Avisynth plugin at the top of that list called QTSource. Usage instructions for both are included in the zip files. 

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80
    edited February 2015

    Alright so I tried the QuickTime plugin. I made a ''plugins64'' folder in the folder where VirtualDub is installed, and placed 'quicktime64.vdplugin' there. I also had the option to make a plugins32 bit folder but I think I need 64.

    After this I tried importing .mov files into virtual dub, but I would get the same message as before. (Can't recognize .mov file type or some sorts...)

    So I tried your second option, the last entry on the list, placed the 'Quicktime' config file in the VirtualDub folder (as instructed), tried importing and it wouldn't work. 

    Now down to the last resort, the first entry on the list, all the Usage instructions from the zip.file tells me is  that ''OMFToolkit.dll must be somewhere in your system path, this is the dll made available by... blablabla'' 
    So that doesn't help me much either, or am I misinterpreting this line?

    The ''plugins'' folder already existed, but the instructions told me to make new ones (32/64.) Am I doing something wrong here? (At this point I only succeeded importing a photo).
    I appreciate anything at this point :) I appreciate the help.

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast

    First make sure you have the latest stable 32 bit version. 

    http://sourceforge.net/projects/virtualdub/files/virtualdub-win/1.10.4.35491/VirtualDub-1.10.4.zip/download

    For this you want the 32 bit version because there aren't that many 64 bit plugins yet. Once you extract the Zip a plugins32 folder should already be there. Place the Quicktime.vdplugin in that folder, start VirtualDub, go to File-->Open Video file.... In the file requester that open there is a "Files of type:" dropdown. Select Quicktime, browse to your clips and try to open one.

    Another possible plugin to try if that doesn't work:

    http://www.virtualdub.org/beta/DShowInputDriver-0.93.zip

    This uses whatever DirectShow filters you may have to decode a video. Follow the steps above to use only this time instead of selecting Files of type: Quicktime, select DirectShow Input Driver

     

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80

    Thanks @Aladdin4d, I think I had an older version installed. Anyway, I followed your first steps, and now I could select files to import Quicktime .mov files. I selected one of my videos I shot for my film, and I could import it. However it gave me an error message saying, ''Missing Codec (AVC1)'' The video became black in VirtualDub but I could play the file and I heard the audio that came with it.

     

    I tried the second thing as well, placed ''DShowInputDriver.vdplugin'' into the same plugins32 folder as Quicktime.vdplugin, I could select ''Directshow Input driver'' but couldn't import the mov. files, (it didn't see them) but I could import a JPG file. (just a test so that it indeed is able to import something.. ;)  )

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast

    Hmmm ok you're probably going to hate me but try one last plugin

    http://sourceforge.net/projects/virtualdubffmpeginputplugin/files/V0823/VirtualdubFFMpegPlugin_0823.zip/download

    This one has a plugin and a folder that both need to be in the plugins32 folder. We know you can open an MOV container so the Quicktime plugin is working so hopefully this will finally take care of any missing codec.

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80

    Don't worry, I'm doing anything you want me to do... I just want this to work so I can leave you alone and I can get on with my film... hehe

    I put the ''ffdlls'' folder and the ''FFInputDriver.vdplugin'' file into the plugins 32 folder, nothing changed.  ''Missing Codec (AVC1)'' Error came once again, black screen, I can hear the audio.

    Not sure what to do. I can indeed  import .mov files but nothing's changed since last plugin. I have no clue what's going on, you seem to know much more about this stuff than I do. 

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast

    In the "Files of type:" drop down you should have a new entry for FFMpeg Supported files so be sure to try that. 

    If you have a codec pack like CCCP or K-lite installed I would recommend un-installing it unless you have to have it for something else and then install just the FFDShow DirectShow filters.

    If you still can't open it can you upload a clip to dropbox or something similar?

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80

    Oh shit it worked. I tried the FFMpeg Supported Files thing and I could open the videos  and see them play and have the audio and all that... thanks for your help dude, you've been generous. 

    I've never heard of those codec packs so I'm not gotta bother looking for  them either.. ;) If I still had problems I would upload a vid to dropbox for sure but I guess I don't have to. Now I gotta see how this program works...

    Thanks @Alladin4d you're awesome. I'll definitely put you in the credits if you like lol

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast

    Woot!

    VirtualDub is a fairly simple program but it tends to do the simple things extremely well. There's tons of info on how to use it out there too. Here's one tutorial on noise reduction: (part of a larger tutorial on VirtualDub)

    http://archive.arstechnica.com/guide/audio-visual/video-cleanup/m-cleaning-5.html

    An older one from the old FXHome forums:

    http://oldfxhomeforums.s3-website-us-east-1.amazonaws.com/21394

    And here's where you can grab to MSU Denoiser I mentioned earlier

    http://compression.ru/video/denoising/index_en.html

     

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80

    Thanks alot @Alladin4d I'll take a look. 

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80
    edited February 2015

    I still have one question though. I'm pretty satisfied with the result of the noise removal, (I used the Neat Video plugin)  but now I have to export (obviously). I could save the previously .mov file as an avi, and convert the avi to DNxHD to edit with in HitFilm. 

    I exported as AVI and I noticed it isn't even exporting in HD, but 640x 424.............Have I been stupid and not seen this coming or is HD export possible?

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80

    My bad, forget about my last post. I can already export HD. 

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast
    edited February 2015

    Ok so you're next step would be to set up VirtualDub to use an external encoder like I linked to earlier. To expand on that a little bit there are two versions of FFMpeg you can use to get there. The first is FFMBC which is a fork of the FFMpeg project dedicated to broadcast and professional usage. It originally provided ProRes and DNxHD at a time when the generic FFMpeg didn't. Since then the generic FFMpeg has grown to support those codecs but FFMBC still has better support for some of the other "professional" codecs. A list of those is here:

    https://code.google.com/p/ffmbc/

    With a Windows build of the most recent version here:

    http://www.videohelp.com/tools/ffmbc

    However unless you need to encode to something on that list other than ProRes or DNxHD you should probably go with a recent generic build of FFMpeg because it tends to be faster. Zeranoe is the place to get good FFMpeg Windows builds

    http://ffmpeg.zeranoe.com/builds/

    I would take Zeranoe's advice and just download the static 32 bit build to start with.

    Since this is going to be for VirtualDub I would extract whatever build you chose to a folder inside your VirtualDub folder and name it something like VDub_FFMpeg. (Or FFMBC if you went that route). This way you'll never get this FFMpeg confused with another version that might be on your system. FFMpeg is the basis for a lot of encoders/transcoders and video viewers so even an average user can end up with several different versions floating around.

    Now we can copy riffraff's initial profile and save it. You can use Notepad for this just be sure to select "Save as" and set the "Save as type:" to All Files (*.*) and save the profile. I saved mine as ProRes_DNxHD.vdprof in the VirtualDub folder.

    {
    "description": "VirtualDub external encoder profile collection",
    "externalEncoders": {
    "sets": {
    "mov ffmbc prores+pcm": {
    "videoEncoder": "V ffmbc prores",
    "audioEncoder": "A ff pcm 48k",
    "multiplexer": "M ff mov",
    "description": "mov",
    "extension": "mov",
    "processPartial": true,
    "useOutputAsTemp": false
    },
    "mov ffmbc dnxhd+pcm": {
    "videoEncoder": "V ffmbc dnxhd",
    "audioEncoder": "A ff pcm 48k",
    "multiplexer": "M ff mov",
    "description": "mov",
    "extension": "mov",
    "processPartial": true,
    "useOutputAsTemp": false
    }
    },
    "profiles": {
    "M ff mov": {
    "name": "M ff mov",
    "program": "C:\\Program Files\\ffmpeg\\bin\\ffmpeg.exe",
    "commandArguments": "-i \"%(tempvideofile)\" -i \"%(tempaudiofile)\" -vcodec copy -acodec copy -f mov \"%(outputname)\"",
    "outputFilename": "%(outputname).audio",
    "type": 2,
    "inputFormat": 0,
    "checkReturnCode": true,
    "logStdout": true,
    "logStderr": true,
    "bypassCompression": false,
    "predeleteOutputFile": true
    },
    "A ff pcm 48k": {
    "name": "A ff pcm 48k",
    "program": "C:\\Program Files\\ffmpeg\\bin\\ffmpeg.exe",
    "commandArguments": "-i - -vn -c:a pcm_s16le -ar 48000 -f wav \"%(tempaudiofile)\"",
    "outputFilename": "%(outputname).wav",
    "type": 1,
    "inputFormat": 1,
    "checkReturnCode": true,
    "logStdout": true,
    "logStderr": true,
    "bypassCompression": true,
    "predeleteOutputFile": true
    },
    "V ffmbc prores": {
    "name": "V ffmbc prores",
    "program": "C:\\Program Files\\ffmpeg\\bin\\ffmbc.exe",
    "commandArguments": "-f rawvideo -s %(width)x%(height) -r %(fps) -i - -pix_fmt yuv422p -vcodec prores -qscale 1 \"%(tempvideofile)\"",
    "outputFilename": "%(outputname).mov",
    "type": 0,
    "inputFormat": 0,
    "checkReturnCode": true,
    "logStdout": true,
    "logStderr": true,
    "bypassCompression": false,
    "predeleteOutputFile": true
    },
    "V ffmbc dnxhd": {
    "name": "V ffmbc dnxhd",
    "program": "C:\\Program Files\\ffmpeg\\bin\\ffmbc.exe",
    "commandArguments": "-f rawvideo -s %(width)x%(height) -r %(fps) -i - -pix_fmt yuv422p -vcodec dnxhd -b 185M \"%(tempvideofile)\"",
    "outputFilename": "%(outputname).mov",
    "type": 0,
    "inputFormat": 0,
    "checkReturnCode": true,
    "logStdout": true,
    "logStderr": true,
    "bypassCompression": false,
    "predeleteOutputFile": true
    }
    }
    }
    }




  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast
    edited February 2015

    Start VirtualDub, select Options then External Encoders... In the pop up window click the Import button and import the profile you just saved. Now click the Encoders tab. For each entry, double click it and in the window that opens there is an entry labeled "Program" with a browse button on the right. Click the browse button and browse to your copy of FFMpeg.exe and then click Ok. After you've done that for every entry you'll be able to encode straight to either ProRes or DNxHD by going to File-->Export-->Using External Encoder... and selecting the profile

    EDIT: I forgot to add a link to the DNxHD codecs for playback and importing to Hitfilm in case you didn't already have them.

    http://avid.force.com/pkb/articles/en_US/download/Avid-Codecs-LE-v2-5

     

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80

    Thanks again @Aladdin4d. I took the FFMpeg and placed the files and folders in a new folder in the virtualdub folder as you instructed. I copied the notepad text as well and saved it as ''All Files''and placed it in the Virtual Dub folder.

    I just don't understand one thing, (after clicking the encoders tab in Virtual Dub) which is ''For each entry, double click it and in the window that opens there is an entry labeled "Program" with a browse button on the right.'' 

     

    What exactly do you mean by 'for each entry ' ? If I open the 'encoders' tab, there's nothing there and I have the options 'Add',  ' Import' , 'Export' and 'Ok.'

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast

    When you first go to Options-->External encoders and the window opens did you import the profile you just saved? This would be on the "Encoder Sets" tab. After you import the profile you should end up with 2 entries in the "Encoder Sets" tab and 4 entries under the "Encoders" tab

    The 2 entries on the "Encoder Sets" tab should be like this

     

    The 4 entries on the "Encoders" tab should be like this

    These are the entries you need to double click and then browse to your FFMpeg location. Just in case something is wrong with the code I posted try using this profile:

    https://www.dropbox.com/s/q5uuvkrax378n6w/ProRes_DNxHD.vdprof?dl=0

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80

    Thank you, at first I didn't understand but I got it.

    I followed the steps and I imported a video, selected 'Export' (Using External Encoder) and selected ''mov ffmbc dnxhd+pcm'' 

    I couldnt play the file in windows media player, but VLC media player could. 

    HitFilm could also import the file, I could preview it but it didn't show anything but it would run. When I placed the file in the timeline the video was plain white. The same thing happened in HitFilm Pro (demo) but the video became black instead of white... so what's going wrong now? lol

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast

    Ummm I don't know.......I'll check some things and let you know.

     

    And now back to the studio

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80

    hehe thanks I appreciate it...

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast

    So I just did some new test clips to see if I could duplicate the problem here and couldn't so can you post a clip of the raw footage and I'll try with that?

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80
    edited February 2015

    https://www.dropbox.com/s/sn3t5d3fbmkheer/DSC_1044.MOV?dl=0

     

    Either one, it's just a test shot I used so go ahead :)

     

    By the way, about the white file, it's now black and in the media properties it says Codec: Uncompressed, ''The video stream is not supported'' filesize 2.85 gb.

    In the timeline it says ''offline material.''

  • Aladdin4d
    Aladdin4d Moderator Website User, Imerge Beta Tester Posts: 2,509 Enthusiast

    I just ran this through VirtialDub making a DNx QuickTime file and it worked perfectly in Quicktime, Vegas and HitFilm Pro. I'm uploading the DNx clip now for you to try on your end. I'll send you a message when it's done.

  • TMFilmsHD
    TMFilmsHD Website User Posts: 80
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!