Tracking a shot that circles a subject

Hi Guys,

I'm planning a shot where the camera circles a person revealing a CG element in the background that goes all the way around. Very much like the early scene in Maze runner after the lead character has run and face planted having exited the lift. He then spots the Maze wall all around him as he looks around the camera circles him, 

Obviously shooting this isn't too much of an issue, i'm more concerned about tracking it in mocha. Would it by very similar to the tutorial on traking long pans by @AxelWilkinson


  • Triem23Triem23 Moderator

    In short, yes.

    Also, that linked webinar covers techniques for tracking long shots where splines move out of frame. 

    The key is knowing when to add a second spline to an existing track (say a long wall that's in-frame for close to 180 degrees of pan) and when to add a new tracked object. 

    If you stage your shot where ground/floor is in view the entire time, you should be able to track that by detaching the spline from the track and letting ground move through the spline for the whole shot. 

  • Thanks @Triem23, I seem to find myself saying that a lot :-)

    Will this still work of the camera is proscribing a circle around the subject rather than just panning 360 degs on a tripod? 

  • Triem23Triem23 Moderator

    It should, except you'll need to track multiple planes for parallax. 

  • Excellent @Triem23, multiple planes it is then

  • edited August 2015

    When tracking something how important is it that the grid is accurately applied to the plane of the surface it's supposed to be attached to, and what's the best way to do this?

    In that video his grids weren't particularly accurately applied, especially as regards the verticals. He had walls that were upright, but his grid/plane wasn't, so does it work out the orientation of the wall being tracked when it does the solve, or do you get the 5 points only where you placed them, so some would be 'floating' off the wall?

    I've tried tracking some textures on ground surfaces where it was hard to work out where the original ground plane was, and over time during the track in mocha the grid kept drifting. Repeatedly adjusting it to try and get it to remain in place was like trying to nail jelly to a wall, so I gave up and did it by hand with individual points and the Quad Warp  (which worked fine for a blemish coverup), but the process seems a lot less automated than I'd hoped.

  • Triem23Triem23 Moderator

    @Palacono, with mocha there's often an "It Depends" factor. The thing about mocha is it's just solving a bunch of rectangles moving relative to each other and guessing. The fact that it works as well as it does amazes me. 

    So, planes, I try and match em up, but, if I remember correctly, in the webinar, he was only accurate with the point he was attaching something. Everything else could slop a bit. Also note that was a live Webinar of a shot he had previously done. The points he didn't align would be useless anchors. Yes, they would float off the wall. 

    For mocha, in general, I suggest watching tutorials on the imagineer channel. Particularly roto, beauty and tracking/solving tuts. 

    Now, mocha has a couple of ways to tweak a track. First is the align track module, which is basically a four-point tracker. Second is adjusting splines. The main thing is to apply your correction at the point of furthest drift. You may need correction, but not too much. 

  • edited August 2015

    @Triem23 OK, so I'm not missing anything. Darn. You just hope the central point is enough and then inevitably do some more 'by eye' of the thing you're applying to the plane when you get it back in Hitfilm. Yikes. That's not worked for me to an acceptable level so far. The sliders controlling the angles are far too coarse and I inevitably go past the angle I want in both directions and start typing in smaller and smaller decimal digits to try and get it closer. My request for a SHIFT, or ALT or CTRL+ALT method of slowing those sliders down is (possibly) at the bottom of the stack.

    I did wonder if @SimonKJones was skipping a step when getting rid of the car in the Projector Tutorial video a little while back as he blithely said "just rotate the clean plate to angle "such and such"" (if I could do that, I wouldn't need the talcum powder), but it seems you really do have to just mess about until it looks kinda right. And incidentally, in that tutorial: his plane does drift, so....hmmm.

    I don't understand why 3D trackers aren't generating a 3D point cloud after they've tracked some points in 2D, then attaching a plane to them as a best fit that goes through them all. In fact they could do this without you having to mess about with any kind of a grid at all, other than perhaps setting the approximate area you wanted.  You could have multiple planes generated if you were tracking say, a set of stairs, and be able to generate a plane for each step.

    It  all seems so needlessly 'hand made' at the moment, and is why I did that little spreadsheet to calculate how to get a plane to go through multiple points a while back rather than having to just completely guess, which sucks.

    If Pix4D can generate a 3D Point cloud from multiple photos taken from different angles (and it doesn't need very many, either)  and then allow you to manipulate that and fly through the cloud at any camera position, then solving for a plane moving about a bit in a scene and giving you 4 or 5 points that are actually on it should be mathematically trivial at this point.


  • Triem23Triem23 Moderator

    @Palacono SOME 3D trackers absolutely do track a point cloud. After Effects' camera tracker is a point cloud, then you move your mouse through the cloud trying to grab the points that make a plane.

    That said, the "hand made" aspect of all camera tracker, whether planar or point cloud comes down to this--the computer doesn't know what it's tracking. Point clouds are looking at how a lot of individual points are moving in parallax relative to each other, and a planar tracker is looking at rectangles moving relative to each other (in the case of the planes, it's guessing perspective based on corner points), but the computer only sees pixel data--it doesn't know that position X is a stair or a floor or a wall or the tip of a lamp post, it just knows things are moving in a certain way and extrapolates a camera and world position based on parallax--incidentally, this is why sometimes a mocha track will place objects 200 units away from each other and other times 20,000. Both values would be mathematically valid solves of the same parallax data, but 20,000 would be two orders of magnitude higher than 200.

    I'm not familiar with Pix4D, but quickly looking over their website and software it's a fundamentally different process. Rather than taking a single viewpoint and generating data by measuring temporal/spatial shifting, it's taking multiple viewpoints and merely doing spatial extrapolation. However this leads me to my last point on camera trackers and video.

    Another issue with point and planar trackers is the resolution of the source image--Pix4D is potentially working with higher resolution images than a video tracker, and pixel resolution is going to make a difference in the accuracy of a track. 1080p video is a fairly low frame size: roughly 2 Megapixels. even 4K video is a "mere" 8 Megapixels. Obviously, in mocha, I can't place a surface corner more accurately than a pixel.

    Incidentally, mocha seems to work best if you give it a very large tracking spline but shrink the surface down as much as possible--I think that, since the track is based on pixel movement within the spline area that a smaller surface gives more "error correction" since you're placing a small object in a large search area.

    Bottom line is successful, consistent mocha tracking is an art form, and one I've not gotten down myself yet (Sometimes I get perfect tracks in five minutes, sometime I spend a day messing around then go to another method.). Sometimes a mocha track can be fixed by simply moving the spline a bit. It's touchy. ;-)

    Couple of short tips that may or may not help: On the left sdie of the mocha interface, below the layer stack, is a panel called "Layer Properties." In this panel you have several controls that don't have a specific tutorial, and their functions come up as "oh, by the ways" in other tutorials. You'll note In/Out points for a layer--this is good if you've tracked something mostly well, but the last few frames drift (like if the track is going out of frame). You can change the in/out points to snip those bad frames. The "Insert Clip" dropdown gives options to add a fill to your surface--the mocha logo and some grids (In mocha pro you can load your own image). These can help see what a tracked surface is doing more clearly than the default grid. The "Iink to track" dropdown is powerful. Above I briefly mentioned unlinking a track from it's spline? This is where you do it. When unlinking a track from it's spline, mocha still tracks motion data, but leaves the spline where it is on screen. This can be useful if you have a large object in frame for an entire shot with a lot of camera movement--say a very long wall in frame for a near 180 degree pan. You can ALSO attach a spline to a previously tracked layer. This is more useful for roto work than camera solves, and is covered in the recent "Color correction with mocha" tutorial released by Imagineer. Still, it's good to know.

    Finally, remember Hitfilm can import tracker data from outside software if it's in .ma format. I've been known to track a shot in AE CS6, and bounce the AE track through an "AE to MA" script to import into Hitfilm. I've just had one or two shots I could NOT track in mocha to save my project!

  • Pix4D can be amazing but there's really a lot that goes into building a dataset for it to work with and many times that includes manual points and manual scale measurements. It also really prefers you use a known camera/lens otherwise you have to calibrate for whatever you are using. It also reads the EXIF data from the images and geotag information if that's available. Ideally you also tell it the altitude a given photo was taken at. All of this extra info gives it a huge leg up compared to something like Mocha because with the info Pix4D can (and does) make a lot of assumptions that most other trackers can't.

  • Been away, so...
    I've played with Pix4D a bit and found that you can ignore all the GPS data, EXIF stuff and still get great results out of only a handful of photos. I used manual point selection in a few cases where detail was poor, but that's pretty much what you do in Mocha anyway, but in that you're also having to try and choose points that are co-planar.  I also use an 'unknown camera' as it's a GoPro with a non-standard lens, so it knows nothing about it. It manages OK.

    I'd simply like to be able to say to a tracker: " Trust me: these 4 points are on the same plane, use a bit of basic maths to follow it some other points around them in a 3D cloud and generate new corner points that are near to, and lie on, the plane I approximated when I gave you the original corners.  I.e. we need something like in music programs: Quantising, where they drag a note to the nearest beat if you tapping along to your virtual instrument isn't quite close enough. It should see that your 4+1 points are floating when compared to the 3D cloud versions and drag them to the plane's surface.

    Thanks for the other Mocha info @Triem23 and I'll play with it some more (not got Pro though and can't justify it) but if Mocha is relying on me to place the points correctly, and I get it wrong, then none of the generated points are going to be accurate. If the points are floating off the surface because the plane I approximated wasn't correctly aligned with the actual plane in the scene, then it might only touch at one point, if at all, and all the points will drift if the camera movement is anything other than fairly simple.

    Not even the centre one can be relied on, as the larger the rectangle: if the top is correct, but bottom is floating by X amount, then the centre is floating by X/2.

    I find the variation in the size of the units when Mocha does solve can make it very difficult to apply a texture to something. E.g. if it thinks the camera is 20000 units from the surface instead of 200, then I need a gigantic texture and have to scale it down. Then you quickly hit maths accuracy issues in Hitfilm, as it doesn't like scaling large textures down too far and you get lots of aliasing unless you apply lots of blur.

    We need a 3D cloud system where you select 4+1 (centre) points for tracking, nominate one of them to actually be on the surface (you can't predict with certainty that you've got more than one right) and the others are tracked then quantised to be on the plane that passes through the one you originally nominated as the anchor point. Done and done. With all the other whizzy maths involved in programs these days, it seems weird when some things are still limping along like they were a decade ago when we had so much less processing power.

    With Mocha being so flaky: for simply blemish removal, I've tried to approximate this manually and got fairly good results with tracking 4 individual points I know are on a plane and in approximately a projected rectangle when I select them. I don't have to be accurate about this though. Then, doing a sort of reverse Projector: grab a screenshot, cut out the shape, remove the blemish, then stretch it into a rectangle and save as a .PNG again and use Quad Warp to apply it back to the 4 corners. Quad Warp will then reapply it to whatever approximation of a rectangle I selected, with the benefit that all 4 points are definitely on that plane.  Works better than Mocha sometimes.

    Only issue then is the accuracy of Quad Warp in actually drawing the texture, which can tend to wander about a bit and produce slightly weird results if the texture is particularly detailed. See where I yelled into the wind about it here , but it works OK for textures that aren't too noisy. Be nice if that was improved though.

Sign in to comment