Hitfilm Pro 2017- No Superwhites?

I've been having a problem grading footage from my Panasonic SD900 and my friends X920 camcorders. Both these cameras record Superwhites (over exposed values above 100 on the Waveform scope), which allow highlights to be recovered during grading. Unfortunately, Hitfilm cuts these values off at the 100 mark, losing the extra highlight information. I've tried grading it, but simply can't recover information which is no longer there. I have tested the same footage on Premiere CS6 and the Superwhites are there (see example) and I was able to grade it with no problems.

Is there a way to enable Superwhites in Hitfilm and if not, will this issue be fixed in later versions?

http://filmdigital.net/Hitfilm-NoSuperwhites.png

Phil

«1

Comments

  • edited February 9

     Just corrected the link above.

     

  • Triem23Triem23 Moderator
    edited February 9

    Hitfilm's scopes seem to render differently, but all the information should actually be there.

    "Superwhites" only refers to luma values between 235 and 255, because luma 235 is the maximum value allowed in broadcast TV.

    Come to think of it, I think you may have your HF scopes configured wrong. Look at the scopes panel and check which color space your scopes are showing. I'm guessing your scopes are in Rec 2020 and should be in Rec 709.

    If you aren't familiar with those controls for the Scopes, then I recommend this tutorial.

    https://www.youtube.com/watch?v=zZgf1Pm193M

  • The information isn't there, look at my example. Even if you ignore the values, you can see on the image that there is visibly less information in the highlights. It cuts off lower down.

    http://filmdigital.net/Hitfilm-NoSuperwhites.png

    I fully understand what Superwhites are how they work. I've spend about 3 hours doing tests and have successfully recovered highlights and highlight detail on all my Premiere test, which have all failed on Hitfilm.

     

  • Common misunderstanding which is based on false interpretation of Rec. 709. Luma of 235 is defined to be reference white in 8 bit video. Peak level is a different pair of shoes and nowadays almost all broadcasting stations in EBU allow for peak levels above 235. Even in CCUs of broadcasting houses it is common to use a certain BBC preset which sets peaks to 103/105 %.
    Also for the digital broadcast itself there is no need for legalizing/limiting peak levels. Nice reading -> here.

  • Triem23Triem23 Moderator

    Yeah, looking at your screen shot, there are two "Z-shaped" areas on the first hump to the left. In your Premiere scopes they are both above the 90% line, in your Hitfilm scopes both below the 90% line. This is why I think your Hitfilm scopes are in the wrong color space. 

  • Sorry, meant to say. I had previously changed the Waveform from Rec 2020 to Rec 709. The example shows Rec 709, but my original point stands, the superwhites have been clipped and can't be recovered in grading because of it.

    Ignore the numbers and values. Look at the arrows I've added. You can see the cut off point for the highlights is lower.

    http://filmdigital.net/Cutoffpoint.png

    It's not just the scopes, but the results of the graded videos. I can see recovered highlights and highlight detail in the Premiere Graded footage, which I have not been able to recover on Hitfilm.

    I've been shooting and editing video for 20 years and am technically very competent.

    If you don't believe me, or my example, then borrow a Panasonic SD900 or X920 camcorder. Record footage in AVCHD (1080p50) with some slightly over exposed areas and put the footage in Hitfilm. You'll see for yourself that the superwhites are clipped. Compare to Premiere Pro. Try grading in both packages and you'll see what I mean

  • Similar problems appears when doing a HitFilm roundtrip from Vegas Pro. Using full level video in Vegas Pro, it will be back clipped when having processed via HitFilm.

  • @Triem23 sorry to butt in, but why do the scopes only go from 0-100 in 8 bit mode, but -20 to 120 in 16/32 bit modes (same colour space)?

    Must have missed that in your videos, sorry.

  • Triem23Triem23 Moderator
    edited February 9

    @Palacono no clue. In fact since my default Hitfilm setup is 16-bit I didn't know the scopes did that until you (I think) mentioned it. A question for a dev. 

    @avalon Vegas has its own color space oddities. If you output from Vegas using Main concept mp4 it's always outputting 16-235. Similarly Hitfilm is full level internally but always exports mp4 to 16-235. Cineform may be different. 

    http://www.glennchan.info/articles/vegas/v8color/vegas-9-levels.htm#codecTable

    As of Vegas 13 the above linked chart is accurate. I doubt Vegas 14 changed anything. 

    Phillip you may have found a legit issue. I'll tag the overworked @CedricBonnier

  • Personally, I think the measurement system used on Waveforms isn't that important. As long as you know where your Solid Black and Full white lines are. 0 and 100 on Hitfilm and 0.3 and 1.0 on Premiere PAL video. My point isn't really about how it's measured. Look closely at the shapes of the top of the waveforms. You can clearly see it's clipped lower down on Hitfilm and that the extra (Superwhite) highlights are there in Premiere.

    http://filmdigital.net/Cutoffpoint.png

    It's not just the Waveforms that have proved this to me. I can see the recovered highlights and highlight detail on the Premiere graded footage and have not been able to achieve this on Hitfilm. Shame, as I feel it has a lot of promise and could knock Adobe off it's pedestal in time.

    I bought Hitfilm as a replacement for my Premiere CS6 and wouldn't have spent the money on it, if I'd have known about this problem. It's not something that will effect most DSLR users as most DSLRs don't record Superwhites and have the much better system of allowing flattened colour profiles. I however have literally thousands of hours of footage shot on my AVCHD Panasonic SD900, which I can't properly grade on Hitfilm. Looks like I'll have to stick the Premiere CS6 for a little longer. 

  • Palacono, I think you've hit the nail on the head. I started a new Hitfilm project and set the Rendering to 32 bit. I had been working in 8 bit before, which is probably what caused the clipping. Oddly enough, I'd trying changing the last project from 8 to 32 but it didn't seem to work. Starting a new project on 32 bit seems to have sorted it :-)

    http://filmdigital.net/32bitproject.png

    I will do further tests.

  • Triem23Triem23 Moderator

    Phillip see what happens at 16 bit just for comparison. I believe CS6 defaults to 16-bit, and 32 is usually overkill. 

    Oddly enough I'm in an argument with Adobe right now. It goes something like this. 

    Adobe: recent testing has indicated you are not running Genuine Adobe Software. 

    Me: Here's the support chat transcripts and email chain where you at Adobe assured me this was a licensed reseller and I am buying genuine Adobe product. 

    Adobe: recent testing has indicated that you are not running Genuine Adobe Software. 

    Me: Yes, I am, and here's the emails from you proving it. 

    Adobe: you should upgrade to CC. 

    Me: Shove your ransomware, and validate my Genuine Adobe Product that Adobe assured me was Genuine Adobe Product before I purchased it. Here's another copy of Adobe Support assuring me I was buying Genuine Adobe Software. 

    How I suspect this will play out. 

    Adobe: Tough. We either lied or made a mistake. Would you like to upgrade to CC? 

    Me: No. Thanks to GIMP, PD Howler, Vegas and Hitfilm I have no pressing need for your ransomware--expect me to share the story of how you guys screwed me everywhere. 

  •  Still not so sure? Scopes look better, but still struggling to recover highlight detail in Hitfilm, where the detail is easily recovered in Premiere.

  • Yea, I've been going off Adobe for a while now. They have become that super greedy corporation who can't see anything beyond dollar and pound signs. I'm one of a number of people I know who simply refuse to go down the Creative Cloud route. At least FXHome are passionate about their products and listen to their customers. That's quite rare now-a-days.

  • Aladdin4dAladdin4d Moderator

    @Palacono The different scope ranges have to do with color spaces, bit depth, and integer vs. floating point. YUV data, even 8 bit, can contain values that can't be accurately presented in an 8 bit integer RGB space. You'll either clip or introduce rounding errors converting YUV->RGB when you have those kinds of values. Increasing the bit depth and going to floating point give you the extra headroom to get around that and prevent rounding errors. 

    Premiere tries to keep everything YUV throughout the pipeline unless you do something that requires working in RGB like an effect at which point it switches to a 32 bit floating point RGB space. 

     

  • @PhilipWainman if you could send us one of those files exposing the issue where it is possible to retrieve data from highlights in Premiere but not in HF, that would help us a lot. We won't fix it today but given time we will definitely see what we can do about it.

    Regarding why in 8 bit the range is 0-100, if I remember correctly it's because the data could not go beyond so we reduced the range. As it is possible in 16 and 32 bit, the range is larger.

  • @Aladdin4d well what confuses me is loading in some 8 bit GoPro footage and the sky is bleached out White 100% and the shadows go down to 0%.

    In 8 bit mode the line clips at 100. in 16/32 bit mode it also clips at 100, but when I increase Brightness or Contrast it will go up to 120 (and beyond) and down to -20 (and beyond).

    But when I flip between 8 and 16/32 bit mode, the image looks identical. If I use the White Balance Pipette (Effect Off, but Pipette still active) I can't find higher than 1.0 in the blown out sky, or lower than 0.0 in the shadows, so...what am I seeing?

    Are the scopes in 16/32 bit modes showing that while there are potential SuperWhites (and SuperShadows?) I'll never be able to see them because when rendered they'll be clipped anyway?

    It's also looks a little weird that values can go right off the range of the scopes in 16/32 bit mode if you whack up the Contrast enough, but they 'clump' around (not at) 100% and 0%. Again, it looks identical in 8 or 16/32 bit modes in the Viewer when you do that.

  • Triem23Triem23 Moderator

    @CedricBonnier Scopes in 8-bit mode should still go (at least) 0-110...

    Remember, 100 IRE is a luma value of 235. 110 IRE is a Luma value of 255. Don't even think of arguing that point. a large part of my work for the last decade has been video for broadcast. ;-) I'll concede I'm giving you NTSC numbers, and PAL might be different.

    There are cameras that shoot 8-bit that still shoot superwhites up to IRE120.

  • edited February 9

    "As of Vegas 13 the above linked chart is accurate."

    I know this chart, but I would not take it as base for any in-depth considerations about proper level handling.

    In Vegas Pro if you output/render signals of 16 - 235 to MainConcept MP4, it will remain 16 - 235. 
    If you output/render signals of 0 - 255 to MainConcept MP4 it will remain 0 - 255.
    Vegas Pro does not touch the level range.

  • Hitfilm expands most footage on import from 16..235 to 0..255, so it will clip camera output not marked as full range. In this case 235..255. I know AVC footage can be so marked but I don't think anything else has that. DSLRs, digicams action cams and such record full range 0..255 and mark the file as such. Most camcorders do not mark as such and many do capture a 16..255 range. I am no expert on that but I do remember a thread years ago on the Vegas forum where people were posting the levels ranges captured by various cameras. 16..255 was common.

    I don't know if this is your exact problem but it is something Hitfilm will do. I've create test media and project to test this. I have stated I don't think Hitfilm should auto expand levels on import and this situation is one reason. We can always expand levels in our software but we can never get clipped levels back.

  • edited February 9

    CedricBonnier

    Ok, I'm still having the same problems with this clip, even in 32 bit. Luckily it's only 12 seconds long (38 Megs), so I've uploaded it to my webspace.

    You can download it here.

    http://filmdigital.net/scarecrows.html

    and here's a reminder of the Waveform differences

    http://filmdigital.net/Cutoffpoint.png

    Thanks

    Phil

  • I'd like to show a different approach. Maybe you could download this small test file which is H.264 720p25 (MainConcept AVC, zipped for download purpose).

    Unzip and import this file into HitFilm Pro and tell me what text you could read in the frame?

  •  It's behaved pretty much how you'd expect. Bang on the 0 and 100 lines. Results were the same when I switched from 32 bit to 8 bit.

    http://filmdigital.net/levels_waveform.png

  • edited February 9

     Yes, same here. There is text information in the frame but because of the clipping which happens in HitFilm this information is gone and no way to get it back. At least I didn't find a way to recover. Using a 32 bit float point project and trying to recover the lost information using level corrections does not help.

    Import same file in Vegas Pro and text information is still there, similar in Catalyst Edit, where it could easily be restored.

    It is true though there are file formats where such clipping in HitFilm does not happen, e.g. CineForm AVI.

  • Triem23Triem23 Moderator

    As Norman noted Hitfilm seems to assume 16-35 input and expands to 0-255 unless therebis a full-range flag. 

  • edited February 9

    Yes, I think that's it. And I second what Norman said: "We can always expand levels in our software but we can never get clipped levels back."

    The way it works now causes doubled trouble for Vegas Pro usesers if they would like to do their color corrections within HitFilm Pro. It's a back and forth of expanding/compressing levels at the expense of quality (especially for 8 bit color depths).
    And Norman is right about the way video cameras commonly record levels. As long as reference white is set to 235 and video data does not exceed 254, this is pretty fine according to Rec. 709.
    There is little need to clip data between 235 and 254 within import, editing or compositing processes and in various cases not even for export.

    So I find it's a smarter way to have options how levels are handled for imported media and while editing/compositing processes.

     

  • Triem23Triem23 Moderator

    Of course Hitfilm Pro owners can grade in Vegas using all those lovely Ignite plug ins. :-) 

  • Yes, true of course. :)

  • I suggest using Davinci Resolve to bring the super whites down (you will have to mark the clips as full range or Resolve will also clip them). Your blacks will be lifted a bit, but I prefer having lifted blacks and not so clipped peaks to completely burnt peaks and proper black levels.

     

    The whole 16-255 brightness (present in the two sony cameras I have owned) is super annoying. I used to use Vegas and I would never know what my videos would look like after rendering.

  • One thing to remember. Levels below 16 and above 235 are not illegal. Just out of normal/standard range. What happens to those levels at the time of display normal depends on the adjustment of the video display device. They might be clipped and they might not be depending on device display settings.

    Remember that computer displays are not "video" display devices. They are rigid 0..255 and here is where the 16..235 video clipping (range adjust) often comes in. A video signal looks flat on a computer monitor. Realistically the 16/235 => 0/255 range adjust is fine for final playback. However, cameras may capture extended range to allow extra latitude/recovery in editing.

    Also, note that Hitfilm does a 0/255 => 16/235 adjust on render to AVC/MP4. This is fine for normal/standard playback. Since HF only ever previews on a computer display the internal data wants to be 0/255 to look proper. That is likely the reason HF is doing the 16/235 => 0/255 adjustment on import.

     

Sign in to comment