motion-detection using an optical flow-field

Over the past couple of weeks I’ve been working on a fun new project with a few friends. Not going to give away too much yet, but part of the project involved setting up a webcam pointing out the window of our apartment to take photos of people as they walk by on the street… see?

Since we want to only take photos of people from the front (walking right to left), one of the problems that I ran into while working on the motion-detection script in Processing is how to get a good idea of the direction of motion in the frame.

Originally I created a little class that allows you to draw boxes on the screen in order to detect a change in the average brightness of the pixels in the target areas. This is what it looked like:

By checking to see the order in which the two areas were “hit” I could tell what direction the person was moving in. This worked pretty well for a while, but I quickly ran into a problem when there were a group of people walking in the wrong direction:

When there were multiple people in the frame, I would still be hitting both sensors and generating a lot of “false positives” (this street gets a lot of traffic). I knew that I had to find a better way to sense the direction of motion. A little digging around led me to this rad Optical Flow Processing sketch created by Hidetoshi Shimodaira (video). Here he has derived the motion in the video and visualizes it with a grid of colorful vectors. The color and direction of the lines signifies the motion in the frame. Blue = left, red = right, purple = up, and yellow = down.

So, I ported this code over into a class. By finding the sum of all the vectors’ horizontal forces, this class lets me check the overall direction of motion in the frame at any given point in time. If the sum is positive, I know that the motion is going left to right. Negative means it’s right to left. And the resulting captures are way more accurate:

Besides the practical value, I also just really love the aesthetic of these colorful flow field vectors. This one in particular is a favorite. You see only a foot in the bottom-left of the frame, but the flow field trail gives you a sense for the energy of that person who just jumped through the frame, like motion lines in a comic book or cartoon.

And another fun effect results from having multiple bodies moving in opposite directions in the frame. I love how here you can see the opposing forces crashing into each other:

The OpticalFlow class I created from Hidetoshi Shimodaira’s sketch can be found on GitHub HERE.

12 thoughts on “motion-detection using an optical flow-field

    • The law is clear about “reasonable expectations of privacy”, and when you’re walking on public property there is almost no expectation of privacy, except for – in limited cases – spoken conversation.

      That’s what legally allows the dozens of cameras on every block of the five boroughs to track your every move.

      These creative people are simply webifying people watching. I think it’s excellent.

    • Sarah, my question in response to you is, “How much privacy do you expect while walking down the street?” As disturbing as it may seem, you are photographed/videoed/record at least 20 times a day while walking on urban public sidewalks and that too is not a violation of your privacy.

      Now, if the concern is that they are posting the images to the web, then I would simply point out that there are hundreds of /street/traffic cams pointed at numerous corners/intersections all over most American cities. We are being recorded, plain and simple.

      Now, if the actual concern is that this is poking fun at hipsters…well, that’s just crazy hilarious…you know, ’cause they’re hipsters.

  1. Pingback: New Fashion Blog Lets You Spy In Real-Time on Williamsburg Hipsters - Gawker | Latest NewsLatest News

  2. Pingback: New Fashion Blog Lets You Spy In Real-Time on Williamsburg Hipsters | Fashion

  3. Pingback: New Fashion Blog Lets You Spy In Real-Time on Williamsburg Hipsters | Fashion News and Reviews

  4. Pingback: New Fashion Blog Lets You Spy In Real-Time on Williamsburg Hipsters | Fashion|News|Gossip|DecadesFashion|

  5. Pingback: Styleblaster, A Real-Time Fashion Blog Critiques People Walking in Brooklyn

  6. Pingback: Styleblaster, A Real-Time Fashion Blog Critiques People Walking Down The Street in Brooklyn

  7. Pingback: Styleblaster, A Real-Time Fashion Blog That Critiques People Walking Down The Street in Brooklyn | Dank Logic News

  8. Pingback: Guy Behind New Williamsburg Street Style Webcam SPEAKS | Fashion

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>