A Story about Proxies

Shortly: Proxy files are HUGE.

So, yesterday I was editing a gaming video with HitFilm Express 2017. Editing was really slow and laggy. I decided to make a Proxy of all the video material (30min facecam 1080@25 and 30min gameplay 1080@60). I left it proxying for the whole night.

I have installed HitFilm into my 250GB SSD (with Windows and programs). 

In the morning, the proxying of my facecam material was complete but the gameplay had stopped due to an Error 36. I started proxying the gameplay again for my school day. When I came home, it said again "Error 36". I gave up with proxying the gameplay and continued with editing. Later, I decided to see how much there's space left on my SSD.

34GB.

I was really confused because last I checked, it said 178GB. I went through all my folders and finally found the problem from C:\Users\Name\AppData\Local\FXHOME\HitFilm Express 2017\Proxies. I deleted all the Proxies and searched a way to change the Proxy folder destination. It is found at File > Options > Proxy > Media Directory.

If you have separate SSD and HDD, I recommend you to move the Proxy directory to your HDD and/or delete your Proxies when you don't need them anymore. Also, you can limit the size of your Proxy directory.

I just wanted to share this experience with you guys to prevent people from killing their SSDs.

Comments

  • Yes, Hitfilm proxies are massive. They are not intended to be a proxy to make hard to decode media files easier to edit. They are lossless renders intended to for use with effects heavy composites to help with playback of those. Being lossless they do have a heavy I/O burden. It is a bit unfortunate the word proxy was used for them given the more common use of the term in the video industry. It is not a misuse of the term. They are a proxy for the source. They are really a full render quality.

    To help with normal editing playback performance. Consider these sources.

    https://www.youtube.com/watch?v=Yqhl0mRuTng

    https://hitfilm.com/forum/discussion/42349/transcoding-for-better-performance-and-easier-editing

     

  •  @NormanPCN

    Thank you for your answer! This will be helpful! :)

  • I think Triem said that Hitfilm's use of the word "proxy" is not in accord with other NLE software. My understanding is that a proxy in other software is exactly as you suggested, a way to make editing quicker/easier (often using lower-resolution-versions of footage?) and a more appropriate term would be 'pre-renders'. I am just a dumb hobbyist, however.

  • Triem23Triem23 Moderator

    @THX1139 correct. Most software uses "proxy" to indicate a low-res, low-quality file replaced at render. Hitfilm's "proxies" are full-res, maximum quality files suitable for final rendering. They're "pre-renders."

    It's a silly thing, but Hitfilm's misuse of "proxy" is my biggest issue with the software because it leads to situations like Nomachio described.

    Nomachio, you're not alone. Every month or so a similar post appears--a new user discovers proxies are flipping huge.

    Putting them on an HDD is a mixed solution. Proxies are so large an HDD can bottleneck streaming them. If you're going to proxy EVERYTHING you're better transcoding before Import. NormanPCN linked a decent video on the topic. 

  • Hitfilm really needs a quick and easy way to take a range of codec media and convert it to the best format (with size options) for the user. Having to go outside causes a lot of frustration and bother that unfortunately a lot of people just get turned off by. Yes you could dump a load of comp shots and convert them all, but I am thinking more a list of media here, click output options then dump out here.

  • Triem23Triem23 Moderator

    @Andy001z like maybe "Add to Export Queue" as a right-click context menu option in the media bin? Highlight clips, send to Export, then, in Export, Highlight all clips and drag on your export preset?

  • Err @Triem23 are you suggesting it already does this or that this is a good idea? It is a good feature either way.

  • Triem23Triem23 Moderator

    Hold that thought--let me check.

    *Opens Hitfilm*

    Suggesting that as a possible implementation. I'm off to the Wishlist.

     

  • @Triem23 get that Hitfilm signal out and let's get the guys and girls at Hitfilm to move this one up the queue.

  • edited October 16

    One slight problem...when it gets Variable Frame Rate footage it can occasionally decide that the frame rate is not what it actually is - I had it thinking some 23.976fps footage was 24fps -  so...that would mean it would then export it at the wrong rate and cause problems when it reimported the 'editor friendly' version

    tl;dr needs to work correctly/better with VFR and/or allow you to override it and work like it mostly did with HF4.

  • Triem23Triem23 Moderator

    @Palacono my general advice remains to transcode VFR footage outside of Hitfilm. There's a reason no other NLE even tries to parse that crap correctly, and I commend FXHOME for trying, but... Hell, I shot some July 4th footage, all at "29.97" fps, but my phone dropped as low as 20fps on some shots. Hitfilm thought some of the footage was 30p, some 29.97, some 25 and some 24.

  • edited October 16

     @Triem23 follow up to the problem: the footage I was using was already 23.976fps CFR and I was using Handbrake to adjust the quality/ i/bframes etc. and it spat out the VFR that confused Hitfilm 2017.

    Every. Single. Time. Even when overrides were selected to force CFR. 

    Dialed it back and found that make no adjustments at all, just straight in>out, still produced VFR.

    Gave up on that and used Shotcut instead. Perfect CFR and buttery smooth  playback (even backwards) in Hitfilm.

    Weird as they're both (I think) using ffmpeg under the hood and it can't be version specific because: same version for both.

  • Triem23Triem23 Moderator

    @Palacono that is odd. I've never had Handbrake do that! 

  • edited October 16

    @Triem23 Get Tobias' 'follow along' files (23.976 fps) for Tracking in Hitfilm from his Tutorial here (See comments for link) Hitfilm Tracking Tutorial

    Chuck it through Handbrake and it will turn into this:

    Width                                    : 1 920 pixels
    Height                                   : 1 080 pixels
    Display aspect ratio                     : 16:9
    Frame rate mode                          : Variable
    Frame rate                               : 23.976 (24000/1001) FPS
    Minimum frame rate                       : 23.974 FPS
    Maximum frame rate                       : 23.981 FPS

    Hitfilm thinks it's 24fps.  Shotcut spits it out as constant 23.976 as it should be and it loads as that into Hitfilm as 23.976. Overlay both tracks with a Difference blend mode and they get out of sync pretty quickly.

    Norman checked it out for me a few months ago and it's a bug in Handbrake that only seems to affect 23.976 fps ...that I know about. But then again I stopped looking when I found Shotcut worked, so could be others.

    I tried to find the original references to it we discussed about 4 months ago, but failed because of the forum software's high search suckitudinosity index. :(

  • Aladdin4dAladdin4d Moderator

    @Palacono Handbrake uses Libav, a fork of FFMpeg, for input decoding and encoding of Mpeg 2 and Mpeg 4 ASP (Divx/Xvid). H.264 encoding is handled by x.264, part of the VideoLAN (VLC) project. FFMpeg uses x.264 for encoding H.264 as does Libav but for whatever reason the Handbrake developers choose to use x.264 directly instead. Shotcut uses FFMpeg for everything. 

    The bug appears to be unique to Handbrake. FFMpeg, VirtualDub Filtermod w/x.264 and x.264 by itself all handle 23.976 correctly.

  • @Aladdin4d Ah, well that does explain the differences. :) Did you just test it with all those programs, or is this a widely known issue?

  • Aladdin4dAladdin4d Moderator

    @Palacono

    • Shotcut - Taking your word for it ;)
    • FFMpeg - Tested
    • x264 CLI - Tested
    • Libav CLI - Not tested but as a fork of FFMpeg I expect it works the same as FFMpeg
    • VirtualDub Filtermod - Tested. The x264 configuration is pretty much the same as Handbrake with the exception of NormanPCN's extra options. "keyint=8:bframes=0" is --keyint=8 --bframes=0
  • @Aladdin4d You're the end of a leg. :)

  • The Handbrake 23.976 output oddity is interesting and should never cause an issue in Hitfilm or any NLE.

    The variable in the variable frame rate output is actually an alternating frame rate on each frame. One frame a touch low in frame rate and the next frame a touch high and repeat that two frame sequence. The average of the two being 23.976. Audio should never really go out of sync with that oscillation.

  • Anyone tried use Vidcoder ? They are based same code Handbrake but better and updates.

    Both Handbrake and Vidcoder look familiars but more features, updates on Vidcoder.

    http://vidcoder.net/

    Will try out that shortcut.

    Thanks.

     

     

     

     

     

  • edited November 14

    (Staff note: some spam here has been purged with cleansing fire)

  • Spam! Kill it with fire. 

  • I would suggest also get more storage. If you are into videoediting, and gaming videoediting maybe you would like to consider a larger HDD or SSD. Keep the SSD for your programs and OS, and have the HDD for the huge files (and video resources, audio resources, projects, proxies, etc)

Sign in to comment

Leave a Comment