REDBLIND

SYMPTOMS

The other day we deployed a new site to the production servers only to find our video player was bugging out. Specifically in Firefox, it seemed that the file size and video length was not being interpreted correctly. Load complete events and video play complete events were being fired prematurely, if not instantaneously. It other browsers the bytes loaded of the net stream never updated, but the video played fine. It worked perfectly in our dev environment and had passed QA, but for some reason broke when deployed to production. This lead us to believe that it was environmental and  differences between the server configurations were at fault.

CAUSE

I’ve seen this type of thing before and wrote about it here. Using an HTTP proxy (I use Charles), I looked at the differences in responses to the F4V requests from the two servers. Both were returning a “plain-text” mime-type. Not a good thing, but still did not explain the differences in behavior. The only difference was the production server was using gzip compression for F4V files. I did a quick search on the web and found only two documented cases of malfunctioning F4Vs and gzip compression, so I figured it was worth writing about.

SOLUTION

The solution is to exclude F4Vs from gzip compression in your apache configuration. Unfortunately, we don’t have administration rights to out client’s servers, so we defaulted to the lower quality and slightly larger file size of FLV, which fixed the issue.

REFERENCES

1. http://www.longtailvideo.com/support/forum/Setup-Problems/21791/MP4-and-Firefox-3-5-7-#msg143431
2. http://stackoverflow.com/questions/1100888/f4v-player-works-on-every-site-but-one

When I was researching color tracking with Flash, there were surprisingly few examples. The examples I did find did not have any source to download. So, I am offering up my source code from  my color tracking experiment. The ColorIsolator class is based on an old AS2 algorithm by Mario Klingemann (Quasimondo) for selective color. I’ve translated it here to AS3 and applied it to video from the webcam.

It works like this. Once a color is selected from the video, using a seriess of thresholds, the ColorIsolator drops all pixels that don’t fall within the defined range of the target color. The remaining pixels are set to the target color and getColorBoundsRect is used to pinpoint the remaining blob of color. It’s really simple. I had the general idea, but I didn’t understand how to define a range of  color until I found Klingermann’s code.

I would consider this just a starting point. It actually works very well, but there are a number of things that can be done to improve it. Here are my recommendations to try out:

1. Try using HSV color values rather than RGB.
2. Introduce a palleteMap in place of or in addition to the saturation/contrast ColorMatrixFilter.
3. Add blob detection to ignore anomalies/artifacts of the target color.

Download source here.

All I ask is for constructive criticism and to let me know if and where you use this. A credit to myself and Mario in your source code comments would be nice.

Thanks to Justin Windle (soulwire) for a great exchange of ideas for this.

Here are the results so far for my color/motion tracking project. Turns out that you don’t have to use a difference filter when tracking colors. All you basically have to do is isolate a color range using multiple thresholds. Then run getColorBounds to get the coordinates of the color blob. The one thing I do have to add is blob detection, but with vibrant colors I don’t seem to need it.

Click on the image below to launch the swf. You can use the sliders to adjust the image for your lighting conditions. Click on one of the boxes to assign the colors to the channel. Check out my blog post prior to this one to see where I did most of my research. I’ll release abstracted code after the project is complete.

UPDATE: Source code now available here!

I don’t usually like to write blog posts about other people’s blog posts. I try to create original content that people will find useful, but that is very hard to do on the web. I like to sleep, watch TV and play video games, so it’s hard keep up with all the 24/7 coders out there.

That said, I’ve been doing a ton of research and experimentation on motion and color tracking. There is a lot of good stuff out there. A lot of it useless, but along the way I came across a few key blog posts that really got me rolling. if you are trying to get your head wrapped around all of this stuff, check out these helpful blog posts:

Color Isolation
http://www.quasimondo.com/archives/000614.php
Mario Klingemann’s blog is a gold mine of info relating to bitmap data. This blog post reveals how to pick out a color range and drop the rest of the colors. I had to translate this to AS3 for my uses. I’ll release that translation soon.

Blob Detection
http://play.blog2t.net/fast-blob-detection/
Blob detection is key to dropping little pieces of bitmap data that you don’t want or to track multiple blobs of the same color. Read the post and you’ll see how it’s useful.

Motion Tracking
http://blog.soulwire.co.uk/2008/06/06/webcam-motion-detection-tracking/
This is motion tracking it it’s simplest form. This is the post I started on.  Ultimately, you may want to combine some of the above techniques with this one. Justin Windle is a cool guy and he has lots of good info on his blog that can help you with your generative art.

Edge Detection
http://blog.inspirit.ru/?p=297
I just found this one most recently. Really interesting. I don’t have a use for this now, but I will be keeping an eye on where this goes. This blog also has a lot of good info and I will be adding this one to my regular reading list.

Automated/Adaptive Threshold
http://www.quasimondo.com/archives/000690.php
http://blog.inspirit.ru/?p=310
One of the main issues when dealing with bitmap data from a webcam is the lighting conditions the user is in. Adaptive threshold analyzes the bitmap data and applies the appropriate threshold to get the best contrast. I haven’t yet dived into the code, but this look extremely useful.

And there you have it!
The best motion and color tracking blog posts to start with. I hope this help others on the same path as I am. Hopefully I will be taking what I’ve learned from all of this and releasing it in a nice tidy code package for everyone to use.

Personally, I think the whole Augmented Reality thing is going to get played out real soon (if it hasn’t already). Which is why I’m glad I had the opportunity to build this before the huge wave of AR widgets, apps, and sites hits the masses. This landing page, startwithapencil.com, I built for McCann NY in support of the One Show.

I used FlarToolKit in conjunction with Papervision3D. On top of that, I created the foundation of a FlarToolKit wrapper that introduces stabilization to reduce jitter and access points to retrieve the the orientation of the FlarBaseNode. I also created a new DisplayObject3D class that gave me the ability to render the stream of gold. All of these will get their own blog post in the near future. For now, check out the site and you can check out the new code base in progress in my google code repo.

Looks like my fellow McCannite devs in Belgium beat us too it. If you want to try it out, download the QR code, then hit this page to launch the application. Good job guys.

Today I spent some time playing with FLARToolKit, an AS3 port of the C based ARToolKit. If you have a webcam, launch the application, download and print this QR code PDF and place in front of your camera.

Flash AR Test from redblind on Vimeo.

I will post source once I clean it up.

Just messing with Pixel3D in Papervision3D. 2000 particles randomly positioned in a virtual sphere that tween to another random position on the virtual sphere. No sphere primitive used. Just Pixels and Pixel3D.

 Testing out mapping a panoramic image to the inside of a cylinder.

Source available here.

Panoramic image by the amazing artist Mario Doucet at dusso.com.

I’ve added a simple Trig class to RedLib. Provides easy access to common trig calculations for use in AS3.

• Calculate Distance between two objects.
• Calculate angle of an object reletive to another. Useful for pointing a MovieClip at another MovieClip.
• Convert radians to degrees and degrees to radians.
• Calculate X and Y coordinates given a distance and radius from an Object.
• More to be added.

Source available in Google code here.

Update: Looks like some dude, Sekati, has a Trig class almost identical to this one. Check it out on google code. The rest of his library is pretty snazzy, too. On a side note, even though you may end up using an existing library, it’s always to good to try it yourself first. You learn more that way.