@Chris2014 Sorry I forgot to answer it before but yes I have used Video Container Changer with mts files several times before. It keeps a log so you could try checking that to see of there's any clues as to why it failed.
With Vegas when you go to render it will always present full RGB to the encoder so the encoder will encode it that way unless you tell it otherwise.
Transcode time - I'm not sure what's going there because I get faster results than that when it's all I'm doing. VirtualDub won't be any faster because it's the codec doing all the work which would be the same for both.
Batch Processing - I almost always batch process either while I'm sleeping or at a low priority while I work on something else. Usually with VirtualDub but sometimes with Vegas. To do this in Vegas first make a folder in Documents called Vegas Script Menu. Now download JHM_AddRegionsToEvents.js and put it in the folder you just made. The next time you start Vegas you'll have a new entry in the script menu (Tools-->Scripting). If you haven't already made a rendering profile with your settings for the codec you're using you'll need to do that too.
Now place the clips you want on the timeline, go to Tools--.Scripting and select JHM_AddRegionsToEvents. This will create a region for every clip you just placed on the timeline. Next go to Tools-->Scripting and select Batch Render.Use Browse to set the location and base file name for your rendered clips. Select the render profile you created for your codec, tick Render Regions at the bottom, click ok and go to bed
If you set the number of threads that MagicYUV uses to 1 it will take longer but you can start another instance of Vegas and work on something else or do whatever else you want.
With your encodes the difference here is the mp4 has I, P and B frames, which means you can get a better-looking compression at the cost of slowing down previews. The HRxHD is all I frames which means high compression can look worse.
An I frame stores the whole image. P and B frames are storing differences between frames. Mp4 may only store an I frame every 150 to 300 frames, so, to display frame 70, the computer has to look at frames 1 and 150, AND 2-69 AND 71-149 just to show 70. This uses a lot of computer power to decode but keeps file size down.
HRxHD is all I-frames, so to look at frame 70 it looks at frame 70. Less processor load, more disc space.
In general, you would use the intermediate (transcoded HRxHD) to edit, then replace it with the original footage for final render. In both Vegas and Hitfilm you can right click on media in the media pool and select "Replace" or "Relink" and select the replacement file.
If you got Vegas and Hitfilm as part of Vegas Suite then you have Production Assistant, which has a better batch renderer than the JHM script Aladdin linked.
I'm in a bit of a rush and so am going to do a brain dump and run.
@Chris2014, yes in AviDemux MP4v2 Muxer is correct presuming you've select 'copy' for the audio and video streams.
DNxHD36 is usually what people use as a low quality proxy, it's way too low if you're going to use those files for your final render. I'd go for 185, I went for 220 as I was too lazy at the time to determine what was the lowest bit rate that would still have acceptable quality but I think 185 is a common value.
mts -> m2ts. Microsoft have a patent for FAT32 and long file names, I think most SD cards are formatted with FAT32 and so to get around the patent devices use the old school 8.3 file names. The software provided by the camera manufacturer usually fixes up the extension when importing the files.
@Aladdin4d, "Mpeg 3 (MP3)" I presume you mean the audio files in which case it's MPEG-2 Audio Layer III.
"Since the lowest one is more than 3 times what your source footage is you have no reason to ever go beyond that." Where does the magic number 3 come from, seems a little low?
Those two comments may come across as unfriendly but please don't take them the wrong way, I appreciate the work you've done investigating the intermediate codec options, I'm just a bit dyslexic so it takes me ages to write stuff down and as stated above I'm in a rush.
@__simon__ No offense taken at all
The recording specs of a camera will list the bit rates for each mode in addition to everything else. The bit rates are a relative indicator of how much data is contained in the decompressed stream. So all else being equal 35 Mbps is better than 28 Mbps and 50 Mbps is better than 35 irrespective of what codec is being used to supply that stream.
As with going to 4:2:2 from 4:2:0 at this point you can't create something from nothing (Actually for color you can kind of if you scale the resolution down but that's a whole other discussion) So if you take 50 Mbps footage and go to 185 or 220 all you've really done is create a large file with a lot of blank data in it. Are there situations where it would be useful to go a higher bit rate? Sure if you know you're going to end up rendering 5 or 6 times before you end up with your final product but increase it by four times? No way not ever. As @Triem23 alluded to a common broadcast workflow is edit in an intermediate then replace it with the original footage just before the final render. making the high bit rates totally pointless under any circumstances. Keep in mind that Avid is not used for just broadcast video but for film as well and their codecs were developed to handle not just the standard video bit rates but also the higher bit rates wanted for film scans. As a reference point for broadcast work 24 Mbps is the extreme low end, 35 Mbps is commonly used for news, 50 Mbps is the go to standard for primetime and top end broadcast cameras top out at 100 Mbps. @Chris2014 is recording to a 17 Mbps stream so DNxHD 36 is over twice the original bit rate and even though it's a lossy codec with the right settings he'll end up with a 1:1 copy of his original footage.
MP3? What you point out actually goes to what I was trying to explain. Sony adds the 2 to try and ease confusion.
What a thrilling thread ! @Aladdin4D weird that mts conversion works for you.some selected error messages:[mp4 @ 00a8f9c0] track 2: could not find tag, codec not currently supported in containerCould not write header for output file #0 (incorrect codec parameters ?)About conversion time, I can say that my i7 's giving all it's got.Batch in Vegas, thanks a lot, I'm familiar with batch scripts, although I didn't have the add regions to event cheers!@Triem23, thanks i'm now familiar with all these details explained in various links about compression BUT you're saying :"In general, you would use the intermediate (transcoded HRxHD) to edit, then replace it with the original footage for final render"and @Aladdin4D you're saying : "As @Triem23 alluded to a common broadcast workflow is edit in an intermediate then replace it with the original footage just before the final render" but earlier in the thread you said"Finally yes it is better to read large files from hard drive(s) than it is to force your editor to decompress highly compressed video especially when it comes time to render."meaning you advise to keep the intermediate at all time, never relinking to the original.That's why I said that I didn't care about the time it takes to render my 4 min final video, compared to the hours needed to transcode the original footage into intermediate codecs. So I guess it was a mistake, and the intermediate, as the name suggests, is just for editing.@Aladdin4D you're saying "@Chris2014 is recording to a 17 Mbps stream so DNxHD 36 is over twice the original bit rate and even though it's a lossy codec with the right settings he'll end up with a 1:1 copy of his original footage."That's my point, it's not. There's visible degradation of the image despite being at 36Mbps when my original file is at 17Mbps@_simon_ ok for mp4v2 !cheers everyone!
@Chris2014 For that error if I had to guess I would say for whatever reason it doesn't like the audio of your clips. I'm basing that on ffmpeg interpreting Track 2 to be the audio.
For the workflow @Triem23 was talking about yes that is common mainly in newsrooms. Ideally they want camera footage to be 50 Mbps 4:2:2 or better. 50 Mbps for the quality and 4:2:2 for cleaner keying (Studio! Field can be 35 Mbps 4:2:2 or even lower). If they know they won't be keying the footage they will accept 4:2:0 at a lower bit rate in many cases. They do a lot of keying, picture in picture and lower thirds. What they are not doing is heavy grading, heavy compositing or other vfx so other than keying they really aren't doing much to the actual footage at all because ideally they don't have to. That's a benefit of a relatively large equipment budget that supports really good cameras operated by very experienced and talented people. (Not saying you aren't!) When they ingest footage they will go to a lower bit rate intermediate to use as a "proxy" for easy editing. When it's time for the final render they'll replace the proxy with the higher bit rate camera footage and render to bit rate that's appropriate for broadcast and adequately supports their generated media.
Now because you mentioned music videos I assumed you weren't working in a news room and would be wanting to do a lot of heavy grading, heavy compositing, vfx and in general doing quite a bit to your actual footage. In this case I would stay with an intermediate all the way through because it makes it easier to do all of those things, can withstand those things without degrading better and your source footage bit rate is low enough that you could encounter dropped frames in a final render just like in the other thread I linked to earlier.
As far why you aren't getting a 1:1 I don't know and I should have said "visually 1:1" because there will almost always be some mathematical change but I was in a hurry at the time and didn't. Anyway I do mts to the lower bit rate intermediates all the time without seeing a degradation. There are many times when I've dropped both the intermediate and the original on separate tracks, set the composite mode of the top track to Subtract and see nothing but black when I scrub through the timeline. Not all the time by any means but often. I will say you do have to be dead on with all your settings in Vegas for project properties, render properties and the encoder settings. Depending on what render template you start with I've seen things default back to interlaced in the final render even though it was definitely set for progressive and other strange results because something was off elsewhere.
One last thing to end today's novella and I'm not calling anybody's work or gear garbage but the old computer maxim of Garbage In - Garbage Out still applies here. Can you get some benefit of increasing the bit rate somewhat? Sure no question but increasing it 3 or 4 or even more times over the original? Massive overkill and it doesn't gain you anything. At some point in your work depending on what all your doing or adding to the project you may very well want or need to render to a considerably higher bit rate to preserve what you've created but that doesn't mean it's a good idea to take 17 Mbps footage and run it all the way up to 185 or 220 Mbps when you first ingest it.
@Aladdin4D man, excellent, thanks so much, all clear now!You're right about your guesses about heavy processing and compositing for my videos.About Vegas export to DNxHD36, I'll check the settings and do tests.Anyway, thank you so much for taking the time to explain with so many precise details!
@Chris2014 Good luck and do take the time to experiment until you find a sweet spot with a codec and bit rate that works best for you! That's the most important thing.
@Aladdin4d, You seem to be under estimating the bit rate required of an intra-frame only codec, such as DNxHD, to obtain the same decompressed picture quality as an inter-frame codec such as H.264.
Here's a quote from this page: http://www.avsforum.com/forum/161-camcorders/1467038-mp4-versus-avchd-bitrates.html
"I have, as have many, compared even 145Mbps well-known compression schemes (e.g., Pro Res) to 28Mbps AVCHD and found no difference (both sampling at 8-bit 4:2:0) in quality from the same camera."
@__Simon__ Sorry to disagree but no I'm not. AVCHD is a very lossy codec and simply increasing the bit rate in another codec regardless of type will not miraculously recover what is already lost. Avid themselves recommend DNxHD 36 for 24 and 25 fps AVCHD workflows and only going up a step (for @Chris2014 that would be DNxHD 85) if absolutely needed because going any higher will gain you nothing except a much larger file. I'm going to go out on a limb and assume that Avid knows a heck of lot more on the subject than either one of us (or even both of us together) and that they're telling the truth.
If cranking up the bit rate to say 9 or 10 times the original really gave you all the benefits of having recorded at a higher bit rate then AVCHD would be the only codec ever needed but it doesn't and we do have many other higher bit rate camera codecs out there because of it some of which do get some benefit by going to 185 or even 220. Is there a benefit to going higher? Yes but how much higher has a direct correlation to the bit rate of the original and diminishing returns drops you to zero benefit at much lower bit rates than you seem to think.
Again keep in mind this is for ingesting footage. The holy grail is a workflow that doesn't require another intermediate rendering step in the process but sometimes it's required. In that case you do probably want to go to higher bit rates to preserve what you have done and added.
On a side note Avid also recommends DNxHD 36 for ALL offline (proxy) 24 and 25 fps workflows even for 100+ million dollar budget movies like Iron Man which is why you'll see it referred to as a proxy but that doesn't mean it was designed to be a proxy setting like ProRes has. It's quite the opposite as DNxHD doesn't have any proxy setting like ProRes.
I'm stuck again I've opted to transcode to DNxHD36 (size is reasonable) but when choosing AVI as a container (because mov is limited to 32b on PC) the DNxHD codec doesn't appear.That means it's whether .AVI in magicYUV with its HUGE files and ultra slow transcoding or .MOV in DNxHD badly handled by PCs because of the 32bit limitations.
@Chris2014 DNxHD only supports MOV or MXF as a container. Don't forget there's also the Grass Valley codecs that do support AVI as well as MOV. The big difference with Grass Valley HQ is there are no set bit rates there's a user selectable slider for size/quality so you'll have to experiment to find the sweet spot.
Comparing the MTS file and the DNxHD36, there's very obvious degradation, although when I use "substract" mode in Vegas to compare the 2 it's all blackI've made screen caps x2 bigger so that you can see the compression zones No need to specify which is which
@Aladdin4D OK thx, I'll try the HQX. A note about the pictures above, IMHO they tend to illustrate what @_simon_ says about mts
@Chris2014 Actually you are getting some weird anti aliasing that I can't explain. Excepting that there's only .02% difference between the pics. Including the anti aliasing there's a .82% difference. Vegas is probably ignoring the anti aliasing and that's why you're getting black when you check it. Again I can't explain why you're getting the extra anti aliasing as I haven't seen anything quite like it.
@Aladdin4D . it's better with DNxHD85, but the video track is only visible in an editor, not a player (VLC) EDIT: works in QTime player. But who uses that ? GrassValley HQ/HQX : registered,downloaded, installed, rebooted, tried all codecs and all options, not working on AVI
"An error occured while creating the media filethe selected codec doesn't support the current render setting"settings are the ones from the mts, 1920x1080, square pixels, audio track PCM uncomp, then I disabled audio.Then I tried in .MOV, and a different option menu appeared, and it works.But I'm back to square 1 as QT movies are in 32bit
@Chris2014 I'm sorry man you're having all kinds of issues that I just don't have answers for!
If it's better for you and it works in your editors run with it!
@Aladdin4D Yeah no problem, I'm just reporting. As you've done a list of codecs, I'm warning that GrassValley HQ/HQX is not working on AVIAnd that DNxHD at 36Mbps is not good enough as an intermediate codec for a 17Mbps MTS . The "square" zones you can see on the picture, are moving around when the movie's playing and are visible. Not good as I'm doing green screen keying,
Can you upload a short clip of your original to dropbox or something?
Is there a way to crop just 5sec of the MTS without re encoding?
Ummm try this one?
While editing, I'll use DNxHD 36 for most video, but for anything I'm going to mocha track, green screen, etc..., I'll use a higher bitrate, such as 220.
While going from a 17Mbps AVCHD to a 180Mbps DNxHD file isn't going to magically make your footage better (can't be better than the original) higher bitrates may still be advantageous for shots you feel might be tracked or have heavy compositing and effects on (not color correction, mind you)--the reason for this is you're starting with compressed footage, which has artifacts, then you're re-compressing the footage, which adds MORE artifacts. The higher bitrate intermediate will help eliminate additional artifacts, but it's going to happen (if you want to see this in action, take a JPEG. Open it. Save as a new JPEG. Open saved file. Resave as a new JPEG, open this third file, resave as a fourth file. Now compare file four to file one. Four should look a lot worse).
This is another reason why these files are referred to as "intermediates" or "proxies" and why you replace them with the original before render. You're making a file that's going to respond better during editing, but getting rid of it before final render to maximize quality.
To respectfully disagree a bit with Aladdin, it's not just news agencies that edit in proxies, then replace with original footage on output--this has been standard practice on dramatic shows and big budget films since the 90's (In the 90's when digital video was starting out you'd render a 320x240 proxy. When we jumped to HD, proxies were often in standard def, and now for 4K and 5K work most proxies are still at 1080. For all the shots where you're just cutting and trimming for timing you don't NEED pristine quality footage or even "100% scale"--you only need pristine footage for shots with compositing.)
That said, if you've started with Long-GOP, 4:2:0 footage, it may be worth it to render your final master to an I-frame, 4:2:2 codec or 4:4:4 codec. The 4:2:2 won't magically improve the original 4:2:0 footage, bu if you've added layers, overlays, glows, etc then the 4:4:4 codec will keep the quality of your additions. Then you can transcode that 4:4:4, I-frame footage into anything else (mp4 for youtube, mp4 for Blu-ray, etc) quickly while keeping a pristine master. In my own workflow, I am not needing 10 or 16-bit video output, so I render out my (VFX) footage as PNG sequences (This is lossless, but huge folders, but it's the best-quality 8-bit output possible at half the size of uncompressed video), then usually render my final output masterto a 4:4:4 I-frame codec to re-transcode for delivery.
Chris, I think you have Vegas, correct? In that case, the best way to chop a short segment from the original is to check the file properties and make certain you know what codec and settings were used to record the original. Also try and find out the I-frame count. Try to select a region starting and ending with an I-frame, then render out of Vegas with the same codec/settings. Vegas has a "Smart Render" function where, if it sees that your output format is identical to the import format it will say "NO RECOMPRESSION NEEDED" and basically just write that segment to a new file without transcoding. You want to make sure you start and end with I-frames to keep I-frame placement the same so there isn't any recompression needed, since frame 1 (frame 0 if we count like the computer) is always an I-frame with P-B frames built from there. If you want to take the time on a project, taking advantage of Vegas's Smart Render is a great way to prep files for transcode, since you can do an initial snip/trim of anything you know is garbage in the footage. As long as you match the I-frames Vegas will do a bit-perfect copy without recompression. I've even been know to toss original footage for "first-trim" footage if my re-encode was "NO RECOMPRESSION NEEDED."
I use proxies all the time. In my previous post where I mentioned using DNxHD 36, that's only as a proxy while editing. When I'm ready for a final render, all are replaced with DNxHD 220. I normally edit on a laptop with a 670m GPU, and using the proxies seems to help.
@rgbii I assume by rendering a 220Mbps intermediate you've gone to a high enough bitrate to avoid adding many new artifacts to your source footage, while keeping the speed advantage of the DNxHD vs your (I assume) mp4 source footage?
@Triem23, yes. I'm sure I don't need to go that high, but I do that to try and minimize adding any new artifacts. I do this rather than go back to the original footage since I don't have to worry about the colorspace issue we've discussed before. I've read over the links you've shown before about adjust colorspace in vegas, but I find this easier and keeps the color/contrast consistent with all the programs I use, not just HF and Vegas. I also don't have to remember to fiddle with that in Vegas
My original footage is normally .MOV from Canon DSLR's, which is H.264, and a mix of Interframe (IPB,IPP) and Intraframe (ALL-I).
@Aladdin4D thanks for that link, I've opted to instead film a short sequence, as I've been installing so many programs and codecs these 3 past days, I'm getting lost. I'm trying to send you a private message with the link through this board but clicking on "message" on your page does nothing. New activity maybe ?about the new footage, just re-did a test comparing mts and DNxHD36, same "zoning" as old footage.@Triem23 indeed I'm on vegas and familiar with "no recompression needed", which I used when I was editing DV, but I've never been able to find a setting that would enable me to trim & export a .mts (m2ts) without recompression. My Sony A65 captures at 17Mbps, I've tried (in Vegas) with Sony AVC, MainConcept AVC, different rates and whatnot, never once got the "no recompression needed". I remember vaguely (3y ago) reading on creativecow that this cannot be done with m2ts files.@Triem 23 I see you export as PNG, I've been trying to export movie+alpha channel from Vegas, so I chose .mov and PNG but the resulting movie is not read smoothly at all by Hitfilm@Aladdin4D some notes I could add on your intermediate codec page for windows users: Before that I'd tried several codecs supposed to include alpha channel DNxHD even though you have options in the menu to include, compress or uncompress alpha, there's no alpha channel in the resulting movie (checking in HitFilm media properties: 32bit, alpha from file ticked, but there's no alpha). Googled and found out this was a known problem for windows usersGrassValley HQ/HQX option: with alpha, but .mov export (from Vegas) crashes at 10% and the saved export preset always reverts the "animation" option to 12 fps (instead of the saved 25fps for me)..avi export crashes at 0%UtVideo export is not working at all: error in the codec message.I just managed with MagicYUV, and compressed alpha ! So if you're on Windows that seems to be the intermediate codec to use for several reasons
@Chris2014 Thanks for the notes and reminding me of some things I should add!
DNxHD with alpha - I totally forgot about this issue! Here's the thing, the alpha channel is there and works perfectly BUT only in Avid products or products that license DNxHD from Avid rather than just using the Avid LE codecs. In the real world that means Avid Media Composer, Adobe After Effects/Premiere and I'm pretty sure Resolve will recognize and use the alpha channel but Edius, Vegas, Final Cut and HitFilm won't. (I think anyway, I can't check all of them) For whatever reason Avid encodes the alpha channel as the exact opposite of what everybody actually uses. I don't know if they still have to but at one time Adobe users had to make sure a "Use Avid Alpha" box was ticked to get things to work.
MagicYUV with Alpha - Actually I'm surprised this worked so good score! Traditionally an alpha channel in an AVI has to be stored uncompressed which meant the entire AVI had to be uncompressed. Slightly more recently an alpha channel could be compressed as long as it was mathematically lossless meaning when decompressed it was bit for bit identical to the original. PNG is a mathematically lossless compression scheme so it became quite common to use Quicktime PNG for alpha channel work but as far as I know there's never been an AVI equivalent. The closest I can think of would be CineForm which uses wavelet compression like jpeg 2000 and it can store an alpha channel. All together it just led to MOV and PNG being the container of choice for storing an alpha channel. MagicYUV also uses a mathematically lossless scheme so I imagine that's why it can pull off an alpha channel in an AVI.
Grass Valley and UT Video - I have not gotten around to testing every variant with or without alpha yet but so far I haven't had either one crash yet.
@Aladdin4D, I should have added that MagicYUV + Alpha only works with .mov ( so 32bit limitation). I couldn't get any AVI to add alpha, the option in Vegas is available only for uncompressed formats.GrassValley: Export in .mov from Vegas12 on windows 7 64bit, if the option "compressed depth": 24bpp color : works OKif you choose alpha in the menu, there's no alpha in the resulting .mov (due to 24bit I guess)
if 32bpp color : codec error, with or without alpha.In both cases the saved template doesn't store the changes made to the codec option "animation", maybe unimportant ?AVI export error : the selected codec doesn't support the current rending setting (I've tried so many settings, size, audio, no audio...)Have you ever tried exporting in AVI ?
UTvideo in AVI :UT Video pro YUV 422 10bit VCMthe selected codec doesn't support the current rending setting UT Video RGB VCM, UT Video RGBA VCM both make Vegas crashUT Video YUV420 BT.601 VCMUT Video YUV 420 BT.709 VCMUT Video YUV422 BT.601 VCMUT Video YUV 422 BT.709 VCM export works but the movie has lots of black frames flashing like a strob when played in Hitfilm. UTvideo in .mov : All of them generate an error "the selected codec doesn't support the current rending setting "
Grass Valley - I have exported to AVI several times but don't recall ever trying with an alpha channel.
UT Video - Have not tried 422 10 bit but I have tried RGB. Have never tried RGBA or MOV. Playback for the rest has been fine for me in HitFilm even going up to 4k.
MagicYUV with Alpha as an AVI. You're right with trying an AVI with alpha. Vegas will only let you tick that option for uncompressed AVI. There is a work around though. Go ahead and render to MOV and change the container to AVI afterwards I just tested this and it does work. I've uploaded an MOV and the "converted" AVI
I used ffmpeg with the following command to do it:
ffmpeg -i MagicYUV_Alpha_QT.mov -vcodec copy -r 24 MagicYUV_Alpha2_AVI.avi
The -r option sets the frame rate. Setting it here like this won't change the frame rate but if you don't set it or set it to something other than the source frame rate you can get some strange results.
EDIT: UT Video RGBA, Grass Valley HQ and HQX samples with Alpha uploaded. Still testing a couple of things but will let you know how I did it in a bit
It looks like you're new here. If you want to get involved, click one of these buttons!