The other day we deployed a new site to the production servers only to find our video player was bugging out. Specifically in Firefox, it seemed that the file size and video length was not being interpreted correctly. Load complete events and video play complete events were being fired prematurely, if not instantaneously. It other browsers the bytes loaded of the net stream never updated, but the video played fine. It worked perfectly in our dev environment and had passed QA, but for some reason broke when deployed to production. This lead us to believe that it was environmental and  differences between the server configurations were at fault.

CAUSE

I’ve seen this type of thing before and wrote about it here. Using an HTTP proxy (I use Charles), I looked at the differences in responses to the F4V requests from the two servers. Both were returning a “plain-text” mime-type. Not a good thing, but still did not explain the differences in behavior. The only difference was the production server was using gzip compression for F4V files. I did a quick search on the web and found only two documented cases of malfunctioning F4Vs and gzip compression, so I figured it was worth writing about.

SOLUTION

The solution is to exclude F4Vs from gzip compression in your apache configuration. Unfortunately, we don’t have administration rights to out client’s servers, so we defaulted to the lower quality and slightly larger file size of FLV, which fixed the issue.

REFERENCES

1. http://www.longtailvideo.com/support/forum/Setup-Problems/21791/MP4-and-Firefox-3-5-7-#msg143431
2. http://stackoverflow.com/questions/1100888/f4v-player-works-on-every-site-but-one

It works like this. Once a color is selected from the video, using a seriess of thresholds, the ColorIsolator drops all pixels that don’t fall within the defined range of the target color. The remaining pixels are set to the target color and getColorBoundsRect is used to pinpoint the remaining blob of color. It’s really simple. I had the general idea, but I didn’t understand how to define a range of  color until I found Klingermann’s code.

I would consider this just a starting point. It actually works very well, but there are a number of things that can be done to improve it. Here are my recommendations to try out:

1. Try using HSV color values rather than RGB.
2. Introduce a palleteMap in place of or in addition to the saturation/contrast ColorMatrixFilter.
3. Add blob detection to ignore anomalies/artifacts of the target color.

Download source here.

All I ask is for constructive criticism and to let me know if and where you use this. A credit to myself and Mario in your source code comments would be nice.

Thanks to Justin Windle (soulwire) for a great exchange of ideas for this.

For plotting points, I used this blog post as a reference. The musical notes I exported from Garage Band. Also of note, the USGS does not have a cross domain policy set up for Flash, so I had to proxy the RSS feed with a PHP script. I will probably be adding more features, refining the musicality and adding a UI to this, so check back again sometime.

Click screenshot below to launch it.

UPDATE: Looks like the USGS not only moved their feed, but changed it a bit. For now, this won’t work.

Click on the image below to launch the swf. You can use the sliders to adjust the image for your lighting conditions. Click on one of the boxes to assign the colors to the channel. Check out my blog post prior to this one to see where I did most of my research. I’ll release abstracted code after the project is complete.

UPDATE: Source code now available here!

That said, I’ve been doing a ton of research and experimentation on motion and color tracking. There is a lot of good stuff out there. A lot of it useless, but along the way I came across a few key blog posts that really got me rolling. if you are trying to get your head wrapped around all of this stuff, check out these helpful blog posts:

Color Isolation
http://www.quasimondo.com/archives/000614.php
Mario Klingemann’s blog is a gold mine of info relating to bitmap data. This blog post reveals how to pick out a color range and drop the rest of the colors. I had to translate this to AS3 for my uses. I’ll release that translation soon.

Blob Detection
http://play.blog2t.net/fast-blob-detection/
Blob detection is key to dropping little pieces of bitmap data that you don’t want or to track multiple blobs of the same color. Read the post and you’ll see how it’s useful.

Motion Tracking
http://blog.soulwire.co.uk/2008/06/06/webcam-motion-detection-tracking/
This is motion tracking it it’s simplest form. This is the post I started on.  Ultimately, you may want to combine some of the above techniques with this one. Justin Windle is a cool guy and he has lots of good info on his blog that can help you with your generative art.

Edge Detection
http://blog.inspirit.ru/?p=297
I just found this one most recently. Really interesting. I don’t have a use for this now, but I will be keeping an eye on where this goes. This blog also has a lot of good info and I will be adding this one to my regular reading list.

Automated/Adaptive Threshold
http://www.quasimondo.com/archives/000690.php
http://blog.inspirit.ru/?p=310
One of the main issues when dealing with bitmap data from a webcam is the lighting conditions the user is in. Adaptive threshold analyzes the bitmap data and applies the appropriate threshold to get the best contrast. I haven’t yet dived into the code, but this look extremely useful.

And there you have it!
The best motion and color tracking blog posts to start with. I hope this help others on the same path as I am. Hopefully I will be taking what I’ve learned from all of this and releasing it in a nice tidy code package for everyone to use.

Around the same time, Aaron Smith and I worked on a project with our NY office. It was a good experience. Aaron wrote a good post on it, here. We also added a few new features to Guttershark, we made really good use of Xviews and Aaron added some new stuff to the Service Manager.

Also through our NY office, I will be working on new project involving motion tracking through the webcam. I’ll be posting some of my findings and experiments here. Expect a new ColorIsolation and possibly a new MotionTracking class to be added to my SVN repo after i finish the project.

I used FlarToolKit in conjunction with Papervision3D. On top of that, I created the foundation of a FlarToolKit wrapper that introduces stabilization to reduce jitter and access points to retrieve the the orientation of the FlarBaseNode. I also created a new DisplayObject3D class that gave me the ability to render the stream of gold. All of these will get their own blog post in the near future. For now, check out the site and you can check out the new code base in progress in my google code repo.