More Augmented Reality with FLARM

I finally got a chance to work on my idea of automating the adaptive threshold bias by looking at the bitmap data coming in. Basically, you want to set the bias -ve for a dark scene and positive for a lighter scene. What I came up with was using the histogram() method of the bitmapData class and summing the lowest 20 values for the dark number and the highest 20 for the light. Comparing these then tells you if the scene is light or dark. I decided to make the assumption that each colour channel would be roughly even so I only worked on the green channel but it would be trivial to expand it for all three.

I tried timing it getting Date() at the start and end of the function but it always comes back 0 so we can say it takes < 1millisecond.

I have some trace statements in too to check the results it’s coming back with but those were commented out for the timing. I added the method to the FLARManager class and I’ll pass the code on to Eric so he can see if he thinks it’s worth including.

Here’s the method in case you want to play with it:

public function calibrateThreshold():Boolean{
var histo:Vector.<Vector.<Number>> ;
var channel:Vector.<Number>;
var darkSum:Number = 0;
var lightSum:Number = 0;
//var startTime:Date;
//var endTime:Date;


try {
//startTime = new Date();
histo = this.flarRaster.bitmapData.histogram();


//assume even colour spread so use Green
channel = histo[2];


//sum the darkest and lightest values
for (var c:int = 0; c < 20; c++) {
darkSum = darkSum + channel[c];
lightSum = lightSum + channel[c + 235];
}


if (darkSum > lightSum) {
//scene is dark, set threshold bias -ve
this._adaptiveThresholdingBias = -0.5;
//trace("Scene is DARK");
}else if (lightSum > darkSum) {
//scene is light, set threshold bias +ve
this._adaptiveThresholdingBias = 0.5;
//trace("Scene is LIGHT");
}else {
//even - set threshold bias 0
this._adaptiveThresholdingBias = 0;
//trace("Scene is even");
}
} catch (e:Error) {
// this.flarRaster not yet fully initialized
return false;
}
//endTime = new Date();
//trace("calibrate took: " + (endTime.getTime()-startTime.getTime()));
return true;
}

I left the trace statements and date code in but commented out.

Advertisements

Learning Flex – Lesson 14, Using Formatters and Validators

Flex has formatter classes that can be used to format raw data into customized strings. You can combine these with data binding to be able to format multiple fields simultaneously. All formatters are subclasses of the Formatter class and built in versions include:

CurrencyFormatter, DateFormatter, NumberFormatter, PhoneFormatter, ZipCodeFormatter.

Each formatter has properties related to it’s type so a CurrencyFormatter has currencySymbol and precision properties and a PhoneFormatter has an areaCode property (area code to add to a 7 digit number, -1 means do not add and is the default). They all have pretty sensible defaults and more data can be found in the Adobe reference docs. To use a defined formatter, just call it’s format() method providing the value to be formatted.

All validators are a subclass of the Validator class. The following are provided validators but you may define your own:

CreditCardValidator, DateValidator, EmailValidator, NumberValidator, PhoneNumberValidator, SocialSecurityValidator, StringValidator, ZipCodeValidator.

All validators have a source attribute which is the id of the control that is being validated and defines where any error message will appear on failure. The property attribute defines where the actual information being validated is stored (ie if we’re validating a TextBox called myText, the source would be myText and the property would be text). If the validation fails, the control is highlighted in red and hovering over it produces an error message.

Like formatters, validators can have properties related to it’s type so a ZipCodeValidator has a domain property (which can be “US Only“, “US or Canada” or “Canada Only“) and a CreditCardValidator requires source and property attributes for cardType and cardNumber.

By default, the validator listens for the ValueCommit event on the source component which corresponds to when the user leaves that component. You can change this behavior or call the validate() method to force validation.

Often, just detecting the validation error is not enough, you want to prevent the user from proceeding until they fix the error. In this case, you need to listen for the ValidationResultEvent (calling validate() directly will return an object of this class). The event type property will either be VALID or INVALID.

Regular Expressions

Regular expressions can be used to help design custom validators. These are special strings that define a pattern a client string should match. You can find out more about defining regular expressions here.

The RegExp class allows you to define a regular expression string using it’s constructor via new which has a second optional parameter for flags (i=case insensitive, x=white space ignored etc). Alternatively you can define a literal expression directly for a RegExp variable. In this case you don’t need to escape the ‘/’ character (because you’re not providing it as a string) but you do need to put one at the start and end of the pattern – traditional regex style (any flags would go after the second /).

To execute the regex, use the search() method of the String object you are checking, passing the RegExp object you’ve created. This will return -1 if there is no match for the pattern (or the position in the string where the pattern begins).

You can build a custom validator by extending mx.Validators.Validator. Call the super() method from the constructor and override the protected method doValidation(value:Object):Array. The doValidation() method returns an Array of ValidationResult objects. Start by clearing your results Array then call the super class version of the method (super.doValidation()). If the input value object is not null, call it’s search() method providing your RegExp. If it’s not found, call the push() method on the results Array object to add a new ValidationResult object. This should specify the properties isError:Booleantrue if there’s an error, subField:String – the name of the subfield (if any) the error is associated with, errorCode:String – the error code for the problem (eg “notNumber”) and the errorMessage:String – a longer, more descriptive error message (eg “account ids cannot contain letters, they are 7 digits long”).

You can set the required property on a validator to true to ensure that if the field is left blank, it fails validation. You’ll need to call the validate() method directly for this at a point you know the user “should” have entered the data (eg on a submit button click).

Learning Flex – Lesson 13, Implementing Navigation

You could implement navigation of an app purely using states but Flex also has a ViewStack class container which is a collection of other containers, only one of which is visible at a time.

Note that all children of a ViewStack must be containers – a runtime error will be thrown otherwise.

The ViewStack class does not have any inherent capabilities to switch which child is visible. This must be done using other built in tools or by developing your own.

An example of a built in control is the LinkBar. If you set the dataProvider of a LinkBar to the id of a ViewStack, a link will appear for each child displaying the label of the child container.

If you want to do things yourself, the selectedIndex (zero based) property of the ViewStack determines which child is displayed. Alternatively you may use the selectedChild property and provide the id of the child component you wish to display.

The TabNavigator is an extension of ViewStack which builds tabs for each child container and uses the label property of the child to determine what to display in each tab. Clicking on a tab will display the relevant child.

By default, the first child of a ViewStack is displayed. Deferred instantiation (or lazy loading) means that the other children are not created until they are needed. This is important to note if other components reference values in a ViewStack child. You could avoid this by setting creationPolicy="all" on the ViewStack but this would be a bad idea as it eliminates the efficiency gains of the deferred instantiation process. It’s better to avoid links to child view values or disable the component requiring the data until the child view is visible.

Date Components

Flex has a DateChooser component which presents a calendar to the user who can then choose a day, month and year. The selectedDate property stores the chosen value in a Date object.

An alternative is the DateField component which looks like a text box with a calendar icon beside it. The user can click the icon to get a calendar or enter the date directly in the field. The date chosen is also stored in a selectedDate property.

Note that when accessing Date objects, the month value is zero indexed (ie 0=Jan, 11=Dec).

Augmented reality via FLART, FLARM

There’s been plenty of Augmented Reality examples about lately (see this and this for instance). A lot have been done using FLART which is a port of of an AR library by a Japanese guy with the moniker Saqoosha fortunately, others like Mikko Haapoja have done a lot of work to figure out how to use it. After starting to read up on this, I discovered the work of Eric Socolofsky working as Transmote who has developed a really nice wrapper for FLART, the FLARManager.Not only that, but he’s got some really nice examples you can download and explanations of what’s going on.

There are various parameters you can tweak when using FLARM so one of the first things I’ve done is to play with one of his example files to create an app to allow you to change them all on the fly and see what effect they have. Here’s a picture of it in action:

tuning FLARM

tuning FLARM

The circle shows the marker has been recognized, the two values in the top left corner are the current frame rate and the confidence the app has in the marker.

So far, I haven’t been able to make anything dramatically better than the defaults he provides but I have found that pushing the Adaptive Threshold bias (especially in the wrong direction) or setting blurring more than about 2 seems to be a bad idea. I have an older PC (circa 2003) that I think will really struggle with this based on the sort of frame rates I saw with the GE app running on it so I plan to try this out on that and with different lighting conditions to see if I can learn anything useful from that.

Something else I’m thinking about is using run time inputs to help improve parameters (especially in the slow/low light type conditions). For instance, you could use the frame rate as a basis to decide your smoothing and marker update (if the frame rate is low, you want to smooth over less frames as the image could have changed a lot and you can afford to have a higher marker update threshold for the same reason – bigger changes between frames). Another potential change is figuring out your Adaptive Threshold bias based on the amount of light you’re reading from the incoming frames, light make it +ve, dark -ve.

Here’s the source for the app in case you want to try it (you’ll need to have the latest FLARManager and don’t forget to compile for Flash 10).

<?xml version="1.0" encoding="utf-8"?>

<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="vertical" width="900" creationComplete="startup()">

<mx:Script>

<![CDATA[

import examples.*;
import mx.containers.*;

private var flarWindow:FLARManagerTutorial_2D;

private function startup():void {
var panel:Panel = new Panel();
flarWindow = new FLARManagerTutorial_2D();
panel.percentWidth = 75;
this.addChild(panel);
panel.rawChildren.addChild(flarWindow);
}

private function updateSmoothing():void{
flarWindow.flarManager.smoothing = smoother.value;
trace(“smoothing set to: ” + flarWindow.flarManager.smoothing);
}

private function updateAT():void {
flarWindow.flarManager.adaptiveThresholdingSpeed = atSpeed.value;
trace(“AT speed set to: “+flarWindow.flarManager.adaptiveThresholdingSpeed);
}

private function updateATBias():void{
flarWindow.flarManager.adaptiveThresholdingBias = atBias.value;
trace(“AT Bias set to:” + flarWindow.flarManager.adaptiveThresholdingBias);
}

private function updateBlurring():void {
flarWindow.flarManager.sampleBlurring = blurring.value;
trace(“Blurring set to: ” + flarWindow.flarManager.sampleBlurring);
}

private function updateMarkerThreshold():void {
flarWindow.flarManager.markerUpdateThreshold = markerUpdate.value;
trace(“updateMarkerThreshold set to: ” + flarWindow.flarManager.markerUpdateThreshold);
}

]]>
</mx:Script>

<mx:Tile direction=”horizontal” width=”100%”>

<mx:Label text=”Smoothing”/>
<mx:Label text=”AT Speed”/>
<mx:Label text=”AT bias”/>
<mx:Label text=”Blurring”/>
<mx:Label text=”Marker Update”/>

<mx:HSlider id=”smoother” minimum=”0″ maximum=”6″ snapInterval=”1″ tickInterval=”1″
labels=”[0,3,6]” thumbRelease=”updateSmoothing()” />

<mx:HSlider id=”atSpeed” minimum=”0″ maximum=”2″ snapInterval=”0.1″ tickInterval=”0.1″
labels=”[0,1,2]” thumbRelease=”updateAT()” />

<mx:HSlider id=”atBias” minimum=”-0.5″ maximum=”0.5″ snapInterval=”0.1″ tickInterval=”0.1″
labels=”[-0.5,0,0.5]” thumbRelease=”updateATBias()”/>

<mx:HSlider id=”blurring” minimum=”0″ maximum=”6″ snapInterval=”1″ tickInterval=”1″
labels=”[0,3,6]” thumbRelease=”updateBlurring()” />

<mx:HSlider id=”markerUpdate” minimum=”20″ maximum=”200″ snapInterval=”20″ tickInterval=”20″
labels=”[20,100,200]” thumbRelease=”updateMarkerThreshold()” />

</mx:Tile>

</mx:Application>

I also made a few changes to the FLARManagerTutorial_2D file. The first was to make the FLARManager instance public so I could mess with it in my app and the other was to get the frame count and confidence displayed. Here’s the code I added to the constructor:

var framerateDisplay:FramerateDisplay = new FramerateDisplay();
confText = new TextField();
confText.defaultTextFormat = new TextFormat("_sans", 10, 0x33FF33, false, null, null, null, null, "left", 30);
confText.background = true;
confText.backgroundColor = 0x000000;
confText.text = "Waiting...";
this.addChild(confText);
this.addChild(framerateDisplay);

Then just update the confText in the event handlers for marker added and marker updated like this:

this.confText.text = evt.marker.confidence.toFixed(5);

I’m sure there are far cleaner ways of doing this but it was enough to get up and running. Let me know if you try this out and get any interesting results.

New Google Maps 3D

Google have updated their Flash API for maps to add 3D perspective maps. I stumbled upon their Geo developers blog which has a post on it here. Obviously I’m a little behind the times on this API as when I downloaded the SWCs they were now up to version 1_16 and my old ones were 1_8. I tried updating a simple test app I had to use the new map format and it was pretty quick and easy so if you’ve been playing with maps, give it a go.

I’ll be keeping an eye on that blog every so often to make sure I don’t miss out on anything else cool they might come up with!

Flex Security

If you’re using BlazeDS/Livecycle or some other backend integration services, you might want to take a look at deblaze. It’s a security tool that can be used to figure out exposed remote methods and other info. It gives you some examples about how you could use it and pointers to make things more secure.

Learning Flex – Lesson 12, Using Drag and Drop

All Flex components support drag and drop and a subset have additional functionality which greatly simplifies implementation. List based components have this capability which introduces two main properties: dragEnabled, a boolean value (defaulted to false) which specifies if the control can act as a drag initiator and dropEnabled, a boolean value (defaulted to false) which specifies if the control can act as a drop target.

Flex will decide what to drop based on things such as liked named properties between the source and destination. If it can’t figure anything from this, the whole object will be dumped and you’ll see something like [object Object]. To avoid this and work with components that do not have the enhanced capability, there are various events we can use.

Drag Initiator Events

mouseDown, mouseMove (MouseEvents class) – although not specific to drag/drop operations, these mouse events are used for components that do not have a dragEnabled property. The mouseDown event is fired when the user selects a control and holds down the mouse button. The mouseMove event is fired when the mouse moves.

dragComplete (DragEvent class) – this event fires when a drag operation is finished as the user releases the mouse button, either when the data drops on a target or the drag is abandoned.

Drag Target Events

All these events are part of the DragEvent class.

dragEnter – fires when a drag proxy (the image displayed during the drag) moves into the target component.

dragOver – fires when the user moves the pointer over the target.

dragDrop – fires when the mouse is released over the target.

dragExit – fires when the user drags outside the target but does not drop the object.

There are several operations that need to take place for drag/drop to be implemented;

At the source:

  • recognize the user may be about to drag (mouseDown event)
  • assign relevant data to a DragSource object

At the destination:

  • recognize something is being dragged into the component (dragEnter event)
  • make sure this is an object that can be accepted by this component
  • recognize if the object is dropped (dragDrop event)
  • process the dropped object

For the source component, there are two classes that must be included: mx.core.DragSource, mx.managers.DragManager. Next, specify an event handler for the mouseDown event on the component. It will require the following parameters – the component, the data to be passed, the event and a format String. The format is used by potential destinations to determine if the object being dragged in is valid for dropping on them.

In the mouseDown event handler, define a new DragSource object and use it’s addData() method to add the data object to be passed and the format string for that data. Note that you could store multiple data types with different formats in this DragObject so that different destinations could represent the object in different ways (maybe text in one and an image in another for example). Finally, call the static method DragManager.doDrag() specifying the object you’re dragging from, the DragSource object you’ve just created and the mouseDown event.

For the destination, start by defining an event handler for the dragEnter event, passing the event and the acceptable format string for this component. In the handler, the first thing to do is check that the dragged object contains a valid format. This is achieved by calling event.dragSource.hasFormat(formatName). The method returns true if the format is found. If the format’s ok, call the static method DragManager.acceptDragDrop(IUIComponent(event.target)) using the event.target rather than the destination container keeps the method more generic and re-usable.  This will remove the red ‘X’ on the dragged visual indicator when it enters the component.

The next stage is to define an event handler for the dragDrop event. Pass in the event and the acceptable format. In the event handler, start by defining a temporary Object variable to hold the dropped data. The data can be retrieved by calling event.dragSource.dataForFormat(formatName) . We need to provide the format name as there could be multiple supported formats. Finally use the addItem() method on the container’s dataProvider to show the data at the destination.

If you are overriding the default behavior of  a drag/drop enhanced container (to prevent [object Object] showing up for example) you only need to define the dragDrop event handler. Note that the format name associated with data in a DataGrid is “items“. One additional task is to add event.preventDefault() to the end of your handler so the default drop behavior is not executed.

One of the other drag/drop behaviors you can change is the drag proxy. A common choice is to make this an image representing the data. By default, the drag/drop operation removes the drag proxy from the source so if you have the image in your source container and provide it’s id as the proxy, it will be removed. To prevent this, declare a new version of the image. Withing the mouseDown event handler, define a new Image object and use it’s load() method to load it from the file system. By default, the width and height of a new image are set to zero so you should provide values for these properties to be able to see it (80×80 is a good starting point). Then in the doDrag method call, add the image object as the fourth parameter.