Frame serving

Many of you are probably wondering what the heck is frame serving. Well it's a method of serving video from one application to another without having to render to an intermediate first. It's useful when you're pressed for time, need to render to a format your software doesn't support, want to render using a different encoder (Handbrake instead of Main Concept) or you need to import a clip that's in a format your software doesn't accept. 

HitFilm does not support frame serving but it can still be useful to users especially those that use another NLE like Vegas or Premiere Pro or have footage in a format HitFilm doesn't like. Frame serving can also help with things like taking an OpenEXR sequence rendered with HitFilm and converting it to a deep color format for encoding as a 10 bit ProRes, DNxHD or Cineform video. 

Hopefully this thread will grow with more useful examples but for now I'm going to start with frame serving from Sony Vegas to HitFilm because it could come in handy when your version of HitFilm isn't the right one for integrating with Vegas. The requirements for this example are:

Not needed but recommended

NOTE: Premiere users can follow along too. If you have an older version of Premiere Pro the DebugMode FrameServer might work but for newer versions there is the Advanced FrameServer which is an updated fork of DebugMode just for Premiere.

Before you get started on downloading things I have a warning - Do not get distracted by the 64 bit and/or multi-threaded versions of Avisynth! The 32 bit single threaded version is stable and generally works with everything if slow on complex operations. The other versions - not so much. Start out slow with the version listed here and once everything is working and your comfortable with Avisynth then and only then move on to other versions if you really feel the need.

Let's start by downloading and Installing the DebugMode FrameServer. The installation is very straight forward and when it's done Vegas will have a new rendering option called DebugMode FrameServer. Not very imaginative but it works.

Next move on to Avisynth. During installation just accept the defaults.

Now the Pismo File Mount Audit Package. Again you can just accept the defaults.

Finally AVFS. This does not have an installer, just a zip file that contains the code source and a small exe file. This exe file needs to be placed in a location covered by your system PATH. The documentation says the Windows folder works so if you're comfortable with that copy it there. Note you'll get a warning form Windows requiring administrator approval to do that and if you aren't comfortable with that remember any location covered by PATH is fine.

Now that all the important bits are installed there are a few things to consider before digging in. One is that DebugMode is now a Render as option with only one default template that can't be customized because the frame server gets all of its properties from your Vegas project properties. That means if you want progressive output your project properties should be set to progressive (not necessarily - more on this later). If you want alpha channel output your project needs to be set to 32 bit float, video levels. IMPORTANT: If you do set your project to 32 bit float be sure to check your video levels!!! See Glenn Chan's excellent article on Color Spaces and Levels in Sony Vegas for more info on this. Last thought on this part - If your doing a lot of complex things to whatever you want to frame serve your serving speed will only be as fast as Vegas can render it. This is fine if your goal is to use a foreign encoder like Handbrake but it will make editing in another app painful so consider and plan accordingly.

It's also a REALLY good idea to get organized and plan ahead on where you're going to keep your working files. I made a folder on my C: drive called frameserver with two more folders in it called scripts and debugmodeout. You should also make notes on your project in case you ever have to go back to it. Frame serving means there are no intermediate files left so you have to keep track of what clips you served. If you took my advice and installed Timeline Tools too it has a "Notes" tab just for this kind of thing.  

Can we finally get to actually doing something?? Umm yeah sure

Anyway for this first example we will not be dealing with audio for reasons I'll cover in a bit so for now fire up Vegas, set your project properties for the output you want and drop a clip on the timeline. Like rendering with any other project you can render the entire project or set a loop region and render just that. When you're ready select File-->Render as, set your output path and file name, select Project Default under DebugMode FrameServer (*.avi), check whether to render the whole project or just the loop region under Render Options and then click Render. If everything is working you should see the following window:

DebugMode Render Options

Select RGB24 when you aren't dealing with alpha channel data or RGB32 when you are dealing with alpha channel data. Make sure Write audio as PCM samples in signpost AVI is not checked and click Next. After a few seconds you should be greeted with this:

DebugMode Serving Window

Congratulations! You are now frame serving from Vegas! What's happened is DebugMode has created a signpost or stub AVI file that represents the served output. Applications that support frame serving like VirtualDub can open and work with this file just like it was a real AVI file. Others that don't like HitFilm need more help and that's where Avisynth comes in.

As briefly mentioned in the requirements above Avisynth takes scripts for input and has it's own language with a whole bunch of internal functionsinternal filters and can use hundreds of external filters but we're not doing anything that complex so all we need to do is create a script that defines a source video and a source filter, something like a codec, to use to read the source video. The scripts are just simple text files with AVS as the extension instead of TXT and if you accepted the defaults when you installed Avisynth you should now have a new right click option to create a new Avisynth script. Browse to where you're saving your scripts, right click and select New then Avisynth script and name the script however you like. If you double click it, it should open in Notepad and be blank. Now you can define your source filter and the path to your source video like so: (Make sure it's all on one line)

OpenDMLSource ("C:\Path\To \served.avi").KillAudio

OpenDMLSource is the internal source filter we want to use. Avisynth deals mainly with AVI files and OpenDMLSource is an AVI source filter that works well with stub files like we have here. The next part is just the path to the Debugmode stub file enclosed by (" "). The .KillAudio is there for insurance. don't forget the . and I promise I will talk about audio soon. Once you have everything entered save the file. Not Save As, just save. If you do a Save As, Notepad will default to saving it as a regular text (.txt) file.

Now you have to make a choice. Either open a command prompt and execute avfs.exe followed by the name and path to your AVS file or right click the AVS file and select Open with and then browse to where you stashed avfs.exe. If you do the Open with option you can set it up so that every time you double click an AVS file it gets mounted as a virtual folder but if you do that you will no longer be able to edit a file just by double clicking it. On the other hand opening a command prompt and typing everything in is a pain too. For what it's worth I set it so that double clicking an AVS file automatically mounts it and I use the Open with option to open one in an editor. Either way you go you should end up with a command prompt window that tells you to press CTRL+C to quit (don't until you're done with the clip) and a new folder on your C: drive called Volumes that has your frame served video mounted as a virtual folder in it. If everything went well the virtual folder will have three files, the video, a copy of the AVS file and an error.log file. The error log is created even when everything is fine. If you don't see an AVI file then check the error log for clues as to what isn't working. 

If everything is working you now have a frame served video that can be opened in HitFilm along with most any other program that handles video or even another instance of Vegas. When you're done with the clip just bring up that command prompt window and CTRL+C to stop Avisynth. Debugmode can be stopped by clicking the Stop button. All that leaves is talking about audio.

Apparently Debugmode has a bug that'll glitch served audio. The fix is to add an empty event to the end of whatever is being served that's exactly 1 second in length. To me that's just annoying and you're better of rendering a separate wave file first then importing both it and your served video into HitFilm but if you run into a situation where you absolutely have to have the audio in the served content Avisynth will help you there too. To start go ahead and render out your audio as a wave file then frame serve the video like normal only this time change your Avisynth script to something like this:

Video = OpenDMLSource("C:\Path\To \served.avi").KillAudio

Audio = WAVSource("C:\Path\To\audio.wav")

AudioDub(Video, Audio)

And mount it. This time you'll have a clip with audio and separate audio files mounted in the virtual folder.

Since I've already written way too much I'm going to stop here for now. This is just barely scratching the surface of frame serving so if you have questions or are just bored be sure to check out the Avisynth Wiki

Comments

  • Nice writeup.

    I have frameserved from Vegas directly to ffmpeg for encoding. Those willing to go that route do not need the File mount or virtual file system drivers since ffmpeg supports Video for Windows directly via AviSynth. However, the ffmpeg command line may not be a lot of people's cup of tea.

    I have read about the 1 second audio issue. I've personally not run into it. I have been using Avisynth+. Not Avisynth proper. Maybe that has something to do with it. Avisynth+ is a fork of Avisynth since the later stagnated in development.

  • Aladdin4dAladdin4d Moderator
    edited December 2015

    @NormanPCN Supposedly the audio bug is with Debugmode itself instead of Avisynth but I haven't personally encountered it either. I think it turns up more often with people frame serving specifically to Handbrake. In any case rendering the audio separately avoids it completely.

    I know about and have used Avisynth+ but in a strange twist it's now the stagnant project while Avisynth has leaped ahead this year. I decided to stick with the "official" version for the basics like this because when I get around to covering the more advanced stuff, especially deep color it'll be with Vapoursynth which can also import the plain version scripts and filters without too much trouble.

    I also plan on covering serving to FFMpeg at some point but that's a post for another day unless of course you plan on tackling a tutorial for it :)

  • Yep, the Handbrake folks are the ones I mostly read about with the 1 sec audio thing.

    Interesting turnabout with Avisynth+. Initially I tried Avisynth but it could not serve the audio. It needed the PCM samples in the AVI. I tried synth+ and it could serve the audio and I have not bothered since. Except when ffmpeg changed and I needed to update synth+. It was problematic finding an updated install, which the development stalling you mention most likely explains.

    I'm kinda Hitfilm boy these days, so frameserving is not on the map. Then again, Debugmode is now open source so if Hitfilm had an output hook, then frameserving could be brought to Hitfilm with less effort.

  • Nice, I will give this a try tomorrow. Thank you for sharing your knowledge. 

    @NormanPCN yeah the ffmpeg command line was not my cup of tea. I plan on trying handbrake. I already downloaded it and debug.

  • The only issue with Handbrake (as far as I know) is Handbrake specifically wants to output h.264 (I could be wrong here). Works great, and I use it often. But, I think MPEG Streamclip is more flexible. 

    Good information here. BBookmarked this for reference. 

  • Most people wanted Handbrake to use the x264 AVC/H.264 encoder. The best out there. Higher visual quality and tons faster as well. Mainconcept really is slow.

    MPEG streamclip has not been updated since 2012 by their site. That's not really a problem, but you are stuck with what you got.

    Using ffmpeg gives you the kitchen sink, at a complexity cost. x264 for AVC output. That was my 99.x% use. But you can do ProRes, DNxHD, XDCAM EX and most things. No DNxHR or Cineform.

  • Here's the link to the Debugmode open source files in hope that some awesome person(s) will pick up the ball & run with it ;)

    https://github.com/satishsampath/frame-server

  • @goodrichm I'm definitely not any kind of programmer so I haven't even looked at the source but I suspect it's tied to Video for Windows which gives it some hard limitations. The most obvious would be color depth. For most things you would be limited to RGB24 or RGBA32 and possibly 422 10 bit v210 so not really a good fit for HitFilm's 16 bit float per channel capabilities. Something based on Vapoursynth would be a better choice. There's no audio support but it handles up to 32 bits float per channel. 

    Using Vapoursynth and FFMpeg I have encoded an OpenEXR sequence from HitFilm to 422 10 bit ProRes and DNxHD.  Switching to VirtualDub instead of FFMpeg I've also encoded the same sequence to 422 10 bit Cineform.

    @NormanPCN Basic support for DNxHR was added to FFMpeg at the first of October but as far as I know only two of the HR profiles were tested because the guy adding the support only had two test sequences to work with. I don't know if that support is in any of the current Zeranoe builds so I guess I should add checking on that to my to do list. Now that the advanced features of Cineform are unlocked for third party tools I'm hoping we'll see an encoder that follows the Cineform SDK and accepts piped input like FFMpeg sometime soon.

Sign in to comment