Autonomous Camera Motion Control and Object Tracking

N

nmathew

Hey guys,

I'm an amateur film-maker and robotics graduate student in Waterloo, Canada and I've recently been playing around with new and interesting tools for camera motion control. I've used the new DJI Ronin handheld stabilizer gimbal, built a camera dolly rig using an RC car as well as used a drone to film a couple of sweeping shots of an object in a field.

I quickly realized that it's actually pretty difficult to control the camera while simultaneously running around with a handheld unit, driving the motorized dolly or flying the drone in the air. For example following a soccer ball with my camera or keeping some object centered in the frame while taking panning or sweeping shots was hard without having at least 2 other guys helping out.

Anyway, so I'm a roboticist and hacked together this working prototype that uses the video stream to track anything I want, using computer vision algorithms.

Here's a quick youtube video to show what I mean:
https://www.youtube.com/watch?v=QJW9ReKU-0I

So far, I programmed in smooth-motion and the ability to track and frame anything I select in a box. In the future I am going to make an android/ipad app so that you can just multi-touch define a subject with 2 fingers.
After looking into the control interfaces for most of these gimbals and motorized tripod heads I'm pretty sure I can make them follow my commands, in addition to commanding the motion paths of the drone or rc car itself to follow something (ie. a bike, animal, car etc.)

I was just wondering if anyone out there would also interested in this tech? It's easy enough for me to make and share, and any feedback on cool ideas to implement would be great

Cheers,
Neil
 
Sort by date Sort by votes
fun stuff. The tech is definitely there for an off the shelf build.

What you have now is an excellent tool for sports \ action style film, but cinema is different and would be more challenging.


What would be cools would be to teach the camera how to follow the actor in a cinematic way, it would need to use facial recognition, enough to pick a particular face out of one or two other faces. There are some simple rules in film making that could help you achieve a better "cinematic" shot.

These are conventions that we have come to consider "cinematic" of course they are broken all the time, but a good place to start.

Rule Of Thirds:
Have the camera move the subject to a "third intersect" for example a face where the eyes are on a line 1/3 from the bottom and 1/3 from the right side of the frame. Composing a face its better to "crop" the top of the head then to trim the chin...

Panning Speed: There is a maximum panning speed that should be followed. We want to avoid both rolling shutter and strobe effect. I don't know exactly what it is but it is much slower than your movements.

Combine your technique with some frame boundaries.

On a set, we set up the frame very carefully. The guy holding the boom mic knows that it can come down only so far, the camera operator knows not to pan to far to the right or else he'll pick up the light stand, etc.
So we need to tell your cambot to only tilt "this" far up and pan "this" far left.. etc..

Ramping into the movement.
On a set we will start a pan BEFORE the subject enters the frame and then follow the subject as it moves through the frame. Telling the cambot to "start panning and then pan with the subject as he approaches the center of the frame"

Focus. Depth Of Field.
Being able to keep a subject in focus as it moves closer and further away from the lens is critical. Its very hard to do by hand. There are high end super sonic sensors used in Hollywood that tell the focus puller the exact distance to the subject, I used a similar cheap versions on my own autonomous bots when I was into that. There are already electronic devices for turning focus on the lens easy enough to integrate.
 
Upvote 0 Downvote
You must log in or register to reply here.
Back
Top