Ginos Tech Blog Feed

Moving Around in VR

Fri, 31 Dec 2021 00:00:00 GMT

Oculus sample framework provides some prefabs to enable player movement in a Unity VR Game.

Lets add one to our VR scene.

The provided Player Controller Prefab uses custom hand meshes as controllers and enables player stick movement, so first remove the static OVRCameraRig added in the first article in this series.

Next drag the Player Controller prefab from the Assets > Oculus > SampleFramework > Core > Locamotion folder into your scene.

When you build and run movement is enabled with the left thumbstick, while rotation is enabled with the right stick and via HMD rotation.

However, pushing the left stick all the way forward will show a red laser pointer and player movement will stop. This is part of the teleportation functionality provided by the prefab, which we are not using for this demo, so lets disable it.

Go to the LocomotionController child of the Player Controller and uncheck the teleportation components.

That should leave only stick based movement in the scene.

Build and run and you can now walk around your beautiful terrain.

To prevent the player falling off the edge of the world, simply create some box colliders at the edges of the terrain that the player can't pass through.

Adding Terrain to a VR Game

Sun, 26 Dec 2021 00:00:00 GMT

Add a terrain object to the scene (GameObject > 3D Object > Terrain). This will add a square terrain of 1000 meters to the scene.

Click on the "Create Neighbouring Terrains" and add a 2x2 set of terrains.

Now paint the terrain using the terrain painting tool and brushes.

Now to color the terrain using textures.

Import terrain assets and speed tree from the Standard Assets package on the unity asset store.

Click on a terrain tiles, then Create a terrain layer and select the texture for the terrain.

If your terrain is shiny, click on the cog and select the diffuse material instead of the standard one.

Add the layer to the other tiles by selecting them and choosing Edit Terrain Layers > AddLayer menu. Select the same layer you set up for the first tile. Adjust the material on the tile also if they are shiny.

Create a few more layers, selecting the textures you want.

Now, click on a layer in the properties window, and paint the terrain using the layer texture.

Now add some trees. Import the free Speed Tree asset from the asset store.

In the Free_SpeedTrees folder, click on each tree and regenerate the materials.

Add a tree prefab to paint with.

Add a couple of tree models then you can paint trees, mass place trees, and randomly rotate trees.

You can add details such as grass and other vegetation in almost the same way as trees. Select the detail tab, select a grass texture and paint away.

Build and run the app, and enjoy your new terrain in VR.

Adding a Sky Box to Unity VR Game

Tue, 21 Dec 2021 00:00:00 GMT

VR game was set up in Getting Started article.

We will be adding the following skybox to the project to make it look a little more interesting.

https://assetstore.unity.com/packages/2d/textures-materials/sky/fantasy-skybox-free-18353

Add the skybox to your assets.

Open Package Manager in Unity and download/import the skybox assets into the project.

Import the skybox that you want to use, here I'm using FS000_Day_05.

To use the skybox, go to the lighting configuration screen.

Create new lighting settings.

Click on the Environment tab. Select the skybox material you imported earlier and select Skybox as the source for environment lighting.

Note unity documentation seems wrong here, they tell you to do this on the scene tab, but the settings actually live on the environment tab.

Now return to the game and the scene should be rendered in the skybox.

Getting started with VR Development with Unity and Oculus Quest 2

Mon, 20 Dec 2021 00:00:00 GMT

This article is all about how to get started building VR games on the Oculus Quest 2 using Unity.

I followed this guide on the Oculus website: https://developer.oculus.com/documentation/unity/unity-gs-overview/

Go here and set up Standalone headset and unity as defaults.

Install Oculus app and pair the quest 2 https://play.google.com/store/apps/details?id=com.oculus.twilight
Enabled developer mode on the Oculus quest app

Install Unity, I'm using 2021.2.7f1. Ensure to include android build support.

Create a new 3D project in Unity.
Enable device for debugging and development

Follow this guide: https://developer.oculus.com/documentation/unity/unity-enable-device/

Install Oculus integration package from Unity Asset Store. Search for Oculus Integration - install the free package

https://assetstore.unity.com/packages/tools/integration/oculus-integration-82022

Import Oculus Integration using package manager

I got this warning when importing. Clicked Use OpenXR for now, but its says its easy to switcvh to legacy backends if I need to later.

Configure build settings for the Unity project

Configure player settings App name and version

App identification and location

Install Oculus XR Plugin

This will install the plugin from package manager.

Select target device

Configure Rendering Settings for Oculus Quest

Configure quality settings

Create an android manifest

Delete the main camera in the scene
Drag OVRCameraRig Prefab from Assets > Oculus > VR > Prefabs Configure as per https://developer.oculus.com/documentation/unity/unity-add-camera-rig/
Drag controller prefabs onto the OVRCameraRig

Ensure to set the Controller property in the inspector to match the anchor

Create a plane so you don't fall into oblivion.

Build and Run the Game

Make sure Oculus device is switched on.

Once the app is deployed to Oculus you should ne able to see the controllers and animations.

JavaFX Scrollpane Draw At Centre

Tue, 31 Aug 2021 00:00:00 GMT

I'm creating a JavaFX application which has a ScrollPane containing a Canvas. I wanted to render an image at the centre of the viewport, regardless of the content scroll/panned position.

I created a helper method that takes the scrollpane and the content and returns a Point2D containing the centre point of the currently visible area of the Canvas.

package com.example.infrastructure.utilities;

import javafx.geometry.Bounds;
import javafx.geometry.Point2D;
import javafx.scene.Node;
import javafx.scene.control.ScrollPane;

public class ScrollHelper {
    public static Point2D getViewportCentre(ScrollPane scrollPane, Node content) {
        Bounds viewport = scrollPane.getViewportBounds();
        Bounds contentBounds = content.getLayoutBounds();

        // horizontal parameters
        double horizontalMin = scrollPane.getHmin();
        double horizontalMax = scrollPane.getHmax();
        double horizontalValue = scrollPane.getHvalue();
        double contentWidth = contentBounds.getWidth();
        double viewportWidth = viewport.getWidth();

        // vertical parameters
        double verticalMin = scrollPane.getVmin();
        double verticalMax = scrollPane.getVmax();
        double verticalValue = scrollPane.getVvalue();
        double contentHeight = contentBounds.getHeight();
        double viewportHeight = viewport.getHeight();

        double horizontalOffset =
                Math.max(0, contentWidth - viewportWidth) * (horizontalValue - horizontalMin) / (horizontalMax - horizontalMin);

        double verticalOffset =
                Math.max(0, contentHeight - viewportHeight) * (verticalValue - verticalMin) / (verticalMax - verticalMin);

        return new Point2D(horizontalOffset + viewportWidth / 2, verticalOffset + viewportHeight / 2);
    }
}

The hvalue and vvalue are percentage values representing the scroll position. The method calculates the vertical and horizontal offset of the viewport, then adds half the viewport width and height to find the centre point.

When testing the method I got an error java.lang.IllegalStateException: Toolkit not initialized, indicating that the JavaFx runtime was not initialized correctly, so I needed to make use of a framework for testing JavaFX applications in JUnit5.

I added the dependency to by gradle.build....

dependencies {
    ....
    testImplementation('org.testfx:testfx-junit5:4.0.16-alpha')
}

Then for the test, I used the JUnit5 ExtendWidth attribute to initialize the JavaFX runtime. For testing I picked some easy values to work with, a scrollpane viewport size of 100x100 with a canvas size of 200x200. The scene contains a Group as the root so that I can call applyCss and layout methods to calculate the layout before I test the result.

package com.example.infrastructure.utilities;

import javafx.geometry.Point2D;
import javafx.scene.Group;
import javafx.scene.Scene;
import javafx.scene.canvas.Canvas;
import javafx.scene.control.ScrollPane;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.testfx.framework.junit5.ApplicationExtension;

import static org.assertj.core.api.Assertions.assertThat;

@ExtendWith(ApplicationExtension.class)
class ScrollHelperTest {

    @Test
    void getViewportCentre_No_Scroll_ReturnsExpectedPoint() {
        Group root = new Group();
        Scene scene = new Scene(root, 320, 240);

        ScrollPane scrollPane = new ScrollPane();
        scrollPane.setPrefViewportWidth(100);
        scrollPane.setPrefViewportHeight(100);
        scrollPane.setHvalue(0);
        scrollPane.setVvalue(0);
        root.getChildren().add(scrollPane);

        Canvas content = new Canvas();
        content.setWidth(200);
        content.setHeight(200);

        scrollPane.setContent(content);

        root.applyCss();
        root.layout();

        Point2D point = ScrollHelper.getViewportCentre(scrollPane, content);

        assertThat(point.getX()).isEqualTo(50);
        assertThat(point.getY()).isEqualTo(50);
    }

    @Test
    void getViewportCentre_WhenScrolled_ReturnsExpectedPoint() {
        Group root = new Group();
        Scene scene = new Scene(root, 100, 100);

        ScrollPane scrollPane = new ScrollPane();
        scrollPane.setPrefViewportWidth(100);
        scrollPane.setPrefViewportHeight(100);

        // scroll to a certain position
        scrollPane.setHvalue(0.5);
        scrollPane.setVvalue(0.5);

        root.getChildren().add(scrollPane);

        Canvas content = new Canvas();
        content.setWidth(200);
        content.setHeight(200);

        scrollPane.setContent(content);

        root.applyCss();
        root.layout();

        Point2D point = ScrollHelper.getViewportCentre(scrollPane, content);

        assertThat(point.getX()).isEqualTo(100);
        assertThat(point.getY()).isEqualTo(100);
    }
}

To Lean more about JavaFX, read the docs here https://openjfx.io/

Memoized Function with TTL

Sat, 19 Sep 2020 00:00:00 GMT

Lodash.memoize creates a function that memoizes the result of the function passed as a parameter. This is great if you want cache the result of an expensive function.

However, lodash.memoize doesn't support a TTL or expiry, but it does let you customise the underlying cache it uses.

So, if for example you want to cache the responses you are retrieving from an API heres a simple way to add TTL functionality to memoize:

import memoize from 'lodash.memoize';

const DEFAULT\_CACHE\_EXPIRY = 60000;

// create a custom cache based on Map
export default class CustomCache extends Map {
  constructor() {
   super();
  this.expiry = DEFAULT\_CACHE\_EXPIRY;
  }

  has(key) {
    // when checking the cache also check the expiry
    return super.has(key) && super.get(key).expiry > Date.now();
  }

  set(key, value) {
    // when setting the value, add an expiry property
    value.expiry = Date.now + this.expiry;
    return super.set(key, value);
  }
}

// provide the custom cache for memoize to use
memoize.Cache = CustomCache;

// example function fetching data from the
const fetchData = async () => {
  return fetch(`https://someapi-example-api`)
  .then((response) => response.json())
  .then((response) => response.data);
}

// create a memoized function
const cachedFetchData = memoize(fetchData);

// set the cache expiry
cachedFetchData.cache.expiry = 30000

...

// later, call the memoized function as follows
const response = await cacheFetchData();

A custom cache can be created that extends the Map type.

An expiry property is added to the cached value in the set method before saving it in the Map. The has method then checks the expiry of the item if it exists in the cache.

We can then set memoize.Cache to our custom cache implementation.

When we wrap the function we want to cache, memoize exposes the .cache property that can be used to set the desired TTL for the cached items.

You can then call the cached function as needed. If the value is in the cache and not expired, it will be returned, otherwise the inner function will be called.

Caveats:

This approach will work if the value returned from your function is an object. You would have to extend the approach to support scalar values.
By adding the expiry in this way property you are mutating the object, which is a bit yucky and could have side effects. This could be extended to be more functional, but hey, I said "simple approach".

Amazon EC2 Container Service Deployment using Terraform

Thu, 31 Aug 2017 00:00:00 GMT

Overview

We have a number of individual micro-services and want to have a continuous delivery system with an on demand deployment to various internal and external environments.

After using codeship for a while we decided to use Jenkins CI to implement our pipeline to give us a little more control. Furthermore, to reduce costs we have containerized all of our services using Docker and deployed to Amazon EC2 container service, moving away from an Elastic Beanstalk deployment model with 40 instances, to three ECS clusters with just six instances.

In this article I’ll go through the entire pipeline and point out some of the key patterns that have enabled us to do this.

Sample source code for the article can be found here: https://github.com/ginocoates/pipeline-samples

Caveats/Points to Note

For now we have opted for continuous delivery (on-demand) over continuous deployment. This is because we need to control what features go to each environment so that we don’t impact integrators adversely.
Database deployments are handled separately. We have a policy of making DB deployments backwards compatible and we rollout DB changes and smoke test them before rolling out corresponding application code.

Tools Used

Jenkins CI - Open source automation server.
Jenkinsfile - All pipelines have been implemented using the Pipeline plugin with the Jenkins file pulled from source control.
Terraform - Allows you to define your infrastructure as code using a json like syntax. We used terraform to define our entire ECS cluster, including cluster, cluster instances, application load balancer, target groups, security groups, ecs services and task definitions for each of our services.
Custom Orchestration Tools - Some custom scripts written in bash to drive the other tools.

Environments

We have a number of environments in use as Doshii for different purposes. The goal of our pipeline is to promote a set of container images through these different environments on demand.

Test - This is an on-demand ECS cluster that is created as needed and destroyed by a Jenkins scheduled job each evening at 8PM to save costs.
Beta - Functionality that has passed system testing in the test environment will be promoted to the Beta environment on demand. This is also the environment that most of our integration clients use for their integration development.
Staging - Where we test containers with live credentials before deploying to the live cluster. This environment is hitting the live database.
Live and Sandbox - Live and Live like environments. Some customers are sandboxed in a live like environment while they are onboarded.

Jenkins Setup

Jenkins is running as a docker container on a small instance. This is fronted by a load balancer to simplify SSL termination. The Jenkins servers are tucked away inside a VPC private subnet with individual security groups and network ACL’s in place.

The figure below shows the basic architecture of the jenkins servers.

To fire up more beefy slaves on demand we use the Amazon EC2 plugin. The slaves will created as needed and will be terminated after the idle timeout.

In the AWS EC2 plugin you can configure a lot of details about your slave, including the AMI ID to use, the instance size and security group details.

]

We use a custom built AMI, built using Hashicorp’s Packer, which includes all the tools and utilities we need to perform our build. Pre packing the ami in this way shaves time of your build process, otherwise you’d have to install dependencies every time a slave is created.

You can also set the number of jobs you would like to allow on the instance. Best approach is to limit this to the number of processors on the machine (e.g. 2 for an M4.Large instance).

]

You can also configure things like the instance profile to use for your instance. Ensure that the instance profile has adequate permissions to execute the build tasks that will be run on the instance, especially if your build interacts with AWS resources in any way.

Importing Upstream Builds

In Jenkins we are using the Extensible Choice Plugin to create list of upstream builds to deploy. This plugin allows you to specify a groovy script to build the choice list. For example, to import builds from the deploy-test job, we used the following script in an Extensible Choice parameter.

import jenkins.model.Jenkins
import hudson.model.AbstractProject
import hudson.model.Result
import hudson.util.RunList

def builds = \[\];
RunList<?> runs = Jenkins.instance.getItem("deploy-test").getBuilds().overThresholdOnly(Result.SUCCESS).limit(100)
runs.each { it ->
 builds.add("${it.number}-${it.displayName}")
}

return builds

As a convenience we set the the display name of a build to be the git commit message if the build is successful. This is then visible in the Extensible Choice parameter and helps with traceability of changes deployed throughout the Jenkins pipeline. This can be achieved with a simple groovy script to get the commit message from git. e.g.

def setDisplayName(){
 def comment = sh (script: "git log -1 --no-merges --format=%B", returnStdout: true,)
 comment = comment.trim();
 currentBuild.displayName = comment;
}

Pipeline Overview

In general our deployments go through a number of stages. Individual services are built and tested on check-in. We deploy the latest docker images to Test on demand, perform system testing and then promote those images through the different environments.

]

Each of our microservices have a separate job in Jenkins, which uses ‘docker build’ to create the container image. If tests pass we tag the image with the build number and push this to EC2 container registry.

If the build is successful we will also create an artifact groovy file that records the tag of the container image we published to ECR. For example, for service1 the output file for build 90 would contain:

env.SERVICE1_BUILD=90

The idea behind this artifact is that it’s imported by the downstream deployment project. The groovy file is loaded and the environment variable thats created is used to set the image tag to deploy for that service (more on that later).

If the deployment is successful, we re-archive the env.groovy files in the deployment build so that they can be referenced in downstream deployments.

The image below gives an overview of how the deployment pipeline works:

]

We use the Copy Artifact plugin to copy the artifacts across to the deployment build. The code to copy the last successful build within a pipeline is quite simple. The snippet below copies the env file for the target service to the import directory of the deployment build.

 $class: 'CopyArtifact',
 projectName: 'service1-build',
 filter: 'output/service-1.env.groovy',
 fingerprintArtifacts: true,
 target: 'import',
 flatten: true,
 selector: \[$class: 'StatusBuildSelector', stable: false\]\]);

Sample pipeline scripts can be found here:https://github.com/ginocoates/pipeline-samples/tree/master/jenkins/pipeline

Orchestration

We’ve created a simple tool for orchestration using terraform. This tool wraps terraform in a docker container and sets up some conventions allow terraform to be used more consistently on local and in Jenkins (ripped from an idea by an awesome DevOps I know, Vinny Carneiro).

The tool uses the following folder conventions to manage terraform source code:

Orchestration Root Folder

config - variable and terraform state config for deployed stacks
- region - A regional stack deployment - e.g. ap-southeast-2
  - stack-name
    - tf-config.tf - Configure backend state for terraform
    - variables.tf-vars - A set of variables for this stack instance
src
- stack-name - A set of terraform files for a stack
- modules - Reusable terraform modules

For example our source folders for ecs clusters are structured as follows:

Orchestration Root Folder

config
- ap-southeast-2
  - appcluster-test
    - tf-config.tf
    - variables.tf-vars
  - appcluster-beta
    - tf-config.tf
    - variables.tf-vars
src
- appcluster-test
  - main.tf
- appcluster-beta
  - main.tf
- modules
  - app-cluster
    - main.tf

These conventions allows us to independently control features of our cluster for each environment, via the variables.tfvars file in the config folder.

To run the orchestration tool we pass in the source folder name and the region as follows.

./orchestrate [tf cmd] [stack-name] [region]

e.g.

./orchestrate apply appcluster-test ap-southeast-2

The tool will use a docker volume to set the source code context to the appropriate stack folder and load the configuration from the corresponding config folder.

Note: To use this tool in Jenkins the slave instance profile has been given adequate permissions to manage our ECS resources using terraform.

Sample source code for our ECS cluster can be found here: https://github.com/ginocoates/pipeline-samples/tree/master/terraform

Deploying to ECS using Terraform

To deploy to ECS using terraform we use a aws_ecs_taskdefinition resource which is rendered using the service.build variable, as follows:

data "template\file" "service1-task-definition" {
 template = "${file("${path.module}/task-definition.tpl")}"
 vars {
 BUILD = "${var.service1-build}"
 container-image="service1"
 container="service1"
 }
}

resource "aws\ecs\task\definition" "service1" {
 family                = "ecs-service-service1-${var.environment-name}"
 container_definitions = "${data.template\file.service1-task-definition.rendered}"
 task_role_arn = "${var.task-role-arn}"
}

To deploy a specific build of a service we can override these variables at the command line when calling the orchestrate tool.

In Jenkins, we first load the upstream groovy files to create the environment variable, then pass the environment variables to the orchestrate tool, as follows:

// load upstream build artifacts to set environment variables appropriately
load "import/service1.env.groovy"
load "import/service2.env.groovy"

// initialize the state
sh './orchestrate init appcluster-test ap-southeast-2'

// run the deployment, specifying the tags to use in the task definition.
sh "./orchestrate apply appcluster-test ap-southeast-2 \\
-var service1-build=${env.SERVICE1\_BUILD} \\
-var service2-build=${env.SERVICE2\_BUILD}

Under the hood, this will generate a new task definition in ECS and issue an Update command to the service. ECS will then schedule any new tasks as required.

Staging and Blue Green deployments

For blue green deployments we considered creating a whole new staging cluster on demand, smoke testing this, then perform a DNS switch when we want to roll this new cluster out to live. However, we didn’t like this approach because it means that there may be some lost requests as the DNS switch occurs, and some possible downtime for our clients.

Instead we decided to break live deployments into two steps. The first step is to stage our containers against the live environment and ensure they are operational. This creates a staging ECS cluster on demand, configured against a different DNS where we can smoke test our service containers in a live environment before releasing them to the wild.

The second step is the live deployment. Since our terraform based deployment updates the service with a new task definition, we just tell ECS to deploy the images we have previously staged. ECS takes care of scheduling the new tasks,routing new requests to these tasks, while bleeding connections from the old tasks before killing them. This results in zero downtime for our clients.

A Multi-Threaded Background Processing Pipeline for .NET

Sun, 23 Apr 2017 00:00:00 GMT

In this article I'll be illustrating an architecture for a background processing pipeline in .NET.

I use a similar architecture as part of my PhD project to process large amounts of data coming from Kinect using Open CV on a background thread. In my application each frame contains approx 9MB of data and Kinect runs at a rate of 30 fps. That's about 270 MB per second flowing through the pipeline!. This pipeline pattern allows that volume of data to be processed without impacting the UI thread or the rate that Kinect frames are received.

The pipeline code and a sample project can be found on GitHub at https://github.com/ginocoates/BackgroundPipeline.

Pipeline Classes

The diagram below shows the main classes in the pipeline. .NET Generics are used to allow the pipeline to process any kind of frame object. The pipeline uses BlockingCollection which wraps a ConcurrentQueue as the background queue. The BlockingCollection blocks the executing thread and waits for items to be added to the collection before allowing the thread to continue. The ConcurrentQueue allows thread safe access to the background queue from both the UI and background threads.

The pipeline holds a collection of modules implementing the IPipelineModule interface. At run-time, frames added to the background queue are passed to the Process method of each of the registered modules.

The pipeline also uses a PipelineTimer class to calculate some run-time metrics such as frames processed. Event notifications are sent to consumers periodically. E.g. this can be used to display the processing frame rate on screen.

Finally the pipeline raises a number of other useful run-time events:

FrameStart - The pipeline is just about to process a frame.
FrameComplete - The pipeline has completed processing a frame.
QueueComplete - The pipeline queue has been fully processed.

How it Works

The sequence diagram below shows the basic operation of the pipeline.

The BackgroundPipeline.Start method is called, which creates the BlockingCollection, starts the timer and creates the background thread.
Once created the thread loops, trying to take items from the queue until BlockingCollection.IsComplete is true. Inside the loop the BlockingCollection will block the thread until there is something to process. Once TryTake returns a frame the frame is passed to the Process method of each module registered with the pipeline.
The consuming application then calls BackgroundPipeline.Enqueue to add work to the pipeline. Any work added will unblock the background thread.
To end process the consuming application calls BackgroundPipeline.Stop which stops the timer and calls BlockingCollection.CompleteAdding. After this no more work can be added to the pipeline. The background thread will process any remaining frames and exit the loop.

Note that the background thread is created using TaskFactory.StartNew passing TaskCreationOptions.LongRunning to ensure that the scheduler creates a separate thread. The code to create the thread and process the items in the queue is shown below. Inside the loop we use the BlockingCollection.TryTake method with a timeout of infinite. This method blocks if the queue is currently empty:


processingTask = Task.Factory.StartNew(() =>
{
    while (FrameQueue != null && !FrameQueue.IsCompleted && !cancellationToken.IsCancellationRequested)
    {
        try
        {
            // blocks until frame is available
            T frame;

            if (FrameQueue.TryTake(out frame, -1, cancellationToken.Token))
            {
                ProcessFrame(frame);
            }
        }
        catch (OperationCanceledException)
        {
            Debug.WriteLine("Queue processing cancelled.");
        }

    }

    OnQueueComplete();

}, cancellationToken.Token, TaskCreationOptions.LongRunning, TaskScheduler.Default);

Sample Application

In the git repo I've provided a sample WPF application that gets data from a Kinect sensor and processes it using the background queue. The goal of the sample is to ensure that the background processing does not impact the performance of the UI or throughput of Kinect frames. The sample shows the FPS of the UI thread, the background pipeline, along with the number of frames still to be processed on screen.

The sample provides a dummy module that simulates a background load by looping for a pre-defined period of time. i.e. the Process method simply waits for 40 ms before continuing:


var end = DateTime.Now + TimeSpan.FromMilliseconds(40);
while (DateTime.Now < end) ;

This module also computes the interval between KinectFrames as they arrive.

When Stop is clicked the sample will create a report.csv file containing the UI and background calculated FPS. The diagram below shows the main classes in use in the sample.

Note that the KinectFrame class stores the Kinect data using unmanaged code, and is therefore IDisposable. By doing this we can stave off expensive Gen2 garbage collection events when dealing with large amounts of data. Note that the sample targets the x64 platform so that we can allocate more than 2GB of memory at run-time.

Pipeline Performance

The chart below illustrates the performance of the sample app over 60 seconds. Metrics for the fps for Kinect (RenderFPS) and pipeline are shown, along with the number of frames yet to be processed in the queue. The Interval metric is the amount of time between Kinect frames being processed by the dummy module. This metric allows us to monitor if there are large gaps in the data which might indicate a performance problem when getting frames from Kinect. Total process memory used and number of Gen1 and Gen2 collections are also shown.

As you can see the UI thread runs at a constant 30 fps, while the background pipeline is processing through the frames at 25 fps without impacting performanceof the UI. The interval between frames processed is also constant throughout the capture. This is important as it means we are not losing any Kinect frames due to performance bottlenecks in the background. Gen2 collections are also kept to a minimum and don't impact performance of the application.

Conclusion

The pipeline presented allows background processing of intensive operations while not impacting UI thread performance. The pipeline uses .NET concurrent collections to achieve thread safe access to the queue and enable processing in the background. Intervals between Kinect frames in the queue were constant, indicating that the pipeline processing does not impact the performance of the thread interacting with Kinect.

For more information on the performance of .NET Concurrent Collections, see this MSDN article:

https://blogs.msdn.microsoft.com/pfxteam/2010/04/26/performance-of-concurrent-collections-in-net-4/

The code for the pipeline and sample can be found on Github here: https://github.com/ginocoates/BackgroundPipeline.

If you want to know more about programming with the Kinect SDK, Read the docs or pick up this book on Amazon.com.

Kinect for Xbox One Sensor

Kinect for Xbox One Adapter

Kinect for Xbox One Colour Depth Map using Helix Toolkit and WPF

Sat, 21 Jan 2017 00:00:00 GMT

The Helix Toolkit is one of the easiest ways I've found to quickly add 3D features to a WPF Application.

KinectV2 (Kinect for XBox One) is a motion capture device designed for gaming. It comes with a free SDK making it super easy to build KinectV2 apps for windows. In just a few lines of code you get access to the various streams captured by the KinectV2 sensor.

In this Article I'll be demonstrating a simple way of showing a 3D color depth map inside a WPF application using the HelixToolkit along with the Microsoft SDK for KinectV2.

The depth map will be rendered using the X, Y, Z coordinates from the KinectV2 depth camera stream and the coloured using a texture generated from the RGB camera stream.

The image below shows the output from the program capturing a fairy that lives with me and shows up from time to time.

The code for this article is available for download on github here.

Dependencies

We'll be using three main dependencies from Nuget for the sample:

Galasoft.MVVMLight - Used to create ViewModels for our views. All of our interactions with Kinect 2 will be within this class.
HelixToolkit.WPF - Provides 3D model and rendering features. In this sample we'll be using the HelixViewport3D and PointsVisual3D classes.
Microsoft.Kinect - The Kinect for Windows 2 SDK

In order to run the sample on windows you will need to purchase a Kinect for XBox One sensor and adapter to allow it to be used on Windows. Click the images below to pick them up from Amazon.com

Kinect for Xbox One Sensor

Kinect for Xbox One Adapter

Application Architecture

The following diagram shows the structure of the application.

Class structure for the sample

The MainWindow hosts a HelixViewport3D which will display our PointCloudModel.
The MainWindow's ViewModel has a reference to the Kinect 2 Sensor and handles events to update our PointCloud model.
The PointCloud model is a subclass of the Helix PointsVisual3D class.
To avoid expensive operations on the UI thread, the PointCloud is updated on a separate thread via a PointCloudWorker, a custom subclass of the BackgroundWorker class.
Depth and Colour frame data is communicated between the PointCloudWorker and MainViewModel via the KinectData class.

The Viewport

The MainWindows hosts a DockPanel. The HelixViewport3D is the lastchild control so will fill the window. I've added a View menu bound to some of the viewport controls for convenience. The snippet below shows how easy it is to add a 3D viewport to the window.


<helix:HelixViewport3D HorizontalAlignment="Stretch"
   VerticalAlignment="Stretch"
        Background="Silver"
        x:Name="mainViewport"
        CameraRotationMode="Turnball">

        <helix:HelixViewport3D.DefaultCamera>
            <PerspectiveCamera
                Position="0, 0, -2"
                LookDirection="0, 0, 1"
                UpDirection="0, 1, 0" />
            </helix:HelixViewport3D.DefaultCamera>

        <helix:DefaultLights/>

</helix:HelixViewport3D>

We are using a Perspective Camera positioned 2 meters behind Kinect, 0,0,0 is the Kinect sensor origin.

We are also looking in the same direction that Kinect is looking, which is +Z in CameraSpace. We've also set the up direction to be +Y in Kinect CameraSpace.

Finally, we will use a default set of lights for the scene.

To display our model in the viewport, MainWindow attaches to the KinectScene class exposed by its ViewModel. Note I have exposed the ViewModel and Viewport as properties of MainWindow for convenience.


public MainWindow()
{
    InitializeComponent();
    this.Loaded += MainWindow_Loaded;
}

private void MainWindow_Loaded(object sender, RoutedEventArgs e)
{
   // attach the kinect scene to the viewport to display it
   this.ViewModel.KinectScene.Attach(this.Viewport);
}

Interacting With Kinect

Inside the MainViewModel we are interacting with KinectV2. We initialize Kinect to send us events for Depth and Colour frames. We then handle MultiSourceFrameArrived events and update our KinectScene with data from Kinect.

An important point in the code is that we are using the CopyFrameDataToIntPtr functions to acquire the Kinect frame data for processing. These versions of the procedures outperform the versions that don't use an IntPtr since they are manipulating memory buffers directly.

Also note that we are using a utility class to map the frame data to the coordinate spaces we need for rendering the depth map.

First the Depth frame is mapped to Camera space so that we can position the points in 3D with respect to the Kinect sensors coordinate space.

Second the Depth frame is mapped to colour space which will give us the location of RGB pixels that correspond to the Depth frame positions, allowing us to colour the depth map appropriately.

At the end of the event handler, we pass the Kinect frame data to our scene to update the point cloud.


public MainViewModel()
{
   // initialize the scene with the kinect coordinate mapper
   this.KinectScene = new Kinect2Scene();

   // create the Kinect sensor
   this.sensor = KinectSensor.GetDefault();

   // We'll be creating a colour 3D depth map, so listen for color and depth frames
   this.frameReader = this.sensor.OpenMultiSourceFrameReader(FrameSourceTypes.Depth | FrameSourceTypes.Color);

   // handle events from Kinect
   this.frameReader.MultiSourceFrameArrived += FrameReader_MultiSourceFrameArrived;

   this.sensor.Open();
}

private void FrameReader_MultiSourceFrameArrived(object sender, MultiSourceFrameArrivedEventArgs e)
{
   var frame = e.FrameReference.AcquireFrame();

   if (frame == null) return;

   var colorFrame = frame.ColorFrameReference.AcquireFrame();
   var depthFrame = frame.DepthFrameReference.AcquireFrame();

   ...

   var pinnedDepthArray = System.Runtime.InteropServices.GCHandle.Alloc(kinectData.DepthPixels, System.Runtime.InteropServices.GCHandleType.Pinned);

   try
   {
       IntPtr depthPointer = pinnedDepthArray.AddrOfPinnedObject();
       depthFrame.CopyFrameDataToIntPtr(depthPointer, (uint)Kinect2Metrics.DepthBufferLength);
    }
    finally
    {
       pinnedDepthArray.Free();
    }

    // map the depth pixels to CameraSpace for 3D rendering
    MapperUtils.MapDepthFrameToCameraSpace(this.sensor.CoordinateMapper, kinectData);

    // map color pixels to depth space so we can color the depth map
    MapperUtils.MapDepthFrameToColorSpace(this.sensor.CoordinateMapper, kinectData);

   // update the scene using the kinect data
   this.KinectScene.Update(kinectData);

   ...
}

The PointCloud Model

The PointCloudModel is a simple class that exposes a ModelVisual3D. It also controls the SampleSize (how many pixels from the depth map we will use) and the point size for the 3D depth map.

The PointCloud class is created and added as a child Visual3D of the model.

Finally the Update() function passed the KinectData to the PointCloud to that it can update itself.


public class PointCloudModel
{
        // for performance reasons we will not use every depth map point
        public static readonly int SampleSize = 10;

        // controls the size of the point cloud points
        public static readonly int PointSize = 4;

        PointCloud pointCloud;

        // expose the point cloud externally
        public ModelVisual3D Model { get; set; }


        public PointCloudModel()
        {
            // initialize the point cloud
            this.pointCloud = new PointCloud(SampleSize);
            this.pointCloud.Size = PointSize;

            // add the point cloud to the model
            this.Model = new ModelVisual3D();
            this.Model.Children.Add(this.pointCloud);
        }

        public void Update(KinectData data)
        {
            // update the point cloud using kinect data
            this.pointCloud.Update(data);
        }
}

The PointCloud class has two main components, the texture, which is a WriteableBitmap, and the PointCloudWorker.


public class PointCloud : PointsVisual3D
{
        // The RGB texture taken from the colour stream
        public WriteableBitmap texture;

        // a background worker to update the point cloud as data arrives
        PointCloudWorker worker;

        ...
}

In the constructor for the class we initialize the texture, setting the model material to be an ImageBrush that uses the texture.

We also add a ScaleTransform3D to the model. This renders the model as the KinectV2 sees it. If we don't do this the model would be reversed on screen.

Finally we initialize the PointCloudWorker class that does the grunt work.


 public PointCloud(int sampleSize)
 {
            this.sampleSize = sampleSize;

            // initialize the texture to the size of the RGB frame
            this.texture = new WriteableBitmap(Kinect2Metrics.RGBFrameWidth, Kinect2Metrics.RGBFrameHeight, Kinect2Metrics.DPI, Kinect2Metrics.DPI, PixelFormats.Bgr32, null);

            // setup the texture brush to colour the point cloud using the RGB image
            var materialBrush = new ImageBrush(this.texture)
            {
                ViewportUnits = BrushMappingMode.Absolute,
                Viewport = new System.Windows.Rect(0, 0, this.texture.Width, this.texture.Height),
                AlignmentX = AlignmentX.Left,
                AlignmentY = AlignmentY.Top,
                Stretch = Stretch.None,
                TileMode = TileMode.Tile
            };

            this.Model.Material = new DiffuseMaterial(materialBrush);

            // reverse the model on the x axis so we see what Kinect sees
            this.Model.Transform = new ScaleTransform3D(-1, 1, 1);

            worker = new PointCloudWorker(this.sampleSize);
            worker.RunWorkerCompleted += Worker_RunWorkerCompleted;
}

In the Update method, we ask the PointCloudWorker to update the PointCloud if its not busy. Depending on performance this may mean that we don't update the cloud for every Kinect frame we recieve.


public void Update(KinectData data)
{
            // if the worker is still updating, just bail
            // note: this may add some update delay depending on system performance
            if (this.worker.IsBusy) return;

            // send the data to the worker
            var workerArgs = new CloudWorkerArgs
            {
                Data = data
            };

            this.worker.RunWorkerAsync(workerArgs);
}

When the PointCloudWorker completes its job, we replace the points in the PointsVisual3D with the new points generated by the Worker, and update the texture with the latest RGB pixels. We also set the TextureCoordinates in the Mesh to those created by the worker.


private void Worker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
        {
            // get the result from the worker and update our model
            // We will just replace the texturecoordinates and points with the results from the worker
            var workerResult = e.Result as CloudWorkerResult;

            this.Mesh.TextureCoordinates = new PointCollection(workerResult.TexturePoints);
            this.Points = new Point3DCollection(workerResult.PointCloud);
            this.texture.WritePixels(
                                  new Int32Rect(0, 0, Kinect2Metrics.RGBFrameWidth, Kinect2Metrics.RGBFrameHeight),
                                  workerResult.Data.ColorPixels,
                                  Kinect2Metrics.ColorStride,
                                  0);
}

The PointCloudWorker

This is where we do the grunt work.

In the constructor we create a set of indexes that are used to sample the depth map and update the pointcloud and texture vertices arrays.

The KinectData is passed as part of the event args to OnDoWork. We use a Parallel.For loop to update the point cloud asynchronously and efficiently.

Note that for the texture vertices we need four points (two triangles). We want each cloud point to take on the colour of the single pixel in the RGB texture, so each vertex points to the same point in the texture.

Finally, we update the point cloud X,Y and Z coordinates, taking care that of -Infinities (in fact if we include -Infinities in the point positions, the model will not render at all).

When we are done we set the resulting point cloud in the DoWorkEventArgs.Result property.


public class PointCloudWorker : BackgroundWorker
{
        Point[] texturePoints;
        Point3D[] pointCloud;
        private int[] pointIndexes;

        public PointCloudWorker(int sampleSize)
        {
            // select a set of indexes based on the sample size
            this.pointIndexes = Enumerable.Range(0, Kinect2Metrics.DepthFrameWidth * Kinect2Metrics.DepthFrameHeight)
                .Where(i => i % sampleSize == 0).ToArray();

            // initialize the arrays of points with empty data
            this.texturePoints = this.pointIndexes.SelectMany(i => Enumerable.Repeat(new Point(), 4)).ToArray();
            this.pointCloud = this.pointIndexes.Select(i => new Point3D()).ToArray();
        }

        protected override void OnDoWork(DoWorkEventArgs e)
        {
            // if we have no depth data, just bail here
            var args = e.Argument as CloudWorkerArgs;
            var data = args.Data;

            // update the texture coordinates and points in parallel
            Parallel.For(0, this.pointIndexes.Count(), (i) =>
            {
                var j = this.pointIndexes[i];

                // update texture coordinates, we need four vertices per point in the cloud
                // however, all texture coordiates can point to the same RGB pixel so the
                // point assumes that colour

                var colorPoint = data.MappedDepthToColorPixels[j];

                var k = i * 4; //there are 4 texture vertices per cloud point

                this.texturePoints[k].X = float.IsNegativeInfinity(colorPoint.X) ? 0 : colorPoint.X;
                this.texturePoints[k].Y = float.IsNegativeInfinity(colorPoint.Y) ? 0 : colorPoint.Y;
                this.texturePoints[k + 1].X = float.IsNegativeInfinity(colorPoint.X) ? 0 : colorPoint.X;
                this.texturePoints[k + 1].Y = float.IsNegativeInfinity(colorPoint.Y) ? 0 : colorPoint.Y;
                this.texturePoints[k + 2].X = float.IsNegativeInfinity(colorPoint.X) ? 0 : colorPoint.X;
                this.texturePoints[k + 2].Y = float.IsNegativeInfinity(colorPoint.Y) ? 0 : colorPoint.Y;
                this.texturePoints[k + 3].X = float.IsNegativeInfinity(colorPoint.X) ? 0 : colorPoint.X;
                this.texturePoints[k + 3].Y = float.IsNegativeInfinity(colorPoint.Y) ? 0 : colorPoint.Y;

                // update the 3D positions in the point cloud
                var cloudPoint = data.MappedDepthToCameraSpacePixels[j];

                if (!float.IsNegativeInfinity(cloudPoint.X))
                {
                    this.pointCloud[i].X = cloudPoint.X;
                    this.pointCloud[i].Y = cloudPoint.Y;
                    this.pointCloud[i].Z = cloudPoint.Z;
                }
            });

            // return the result
            var workerResult = new CloudWorkerResult
            {
                Data = data,
                TexturePoints = this.texturePoints,
                PointCloud = this.pointCloud
            };

            e.Result = workerResult;

            base.OnDoWork(e);
        }
}

Program Output

Here is a set of images of the back of my office. The first picture is front on, the second is an isometric view. The HelixViewport3D allows you to rotate the view by right-clicking and moving the mouse.

For performance reasons in the code I have sampled every 10 pixels from the depth map. The sampling amount can be adjusted in the PointCloudModel class.

And that's it!, we have a nice and simple 3D rendering of what Kinect can see.

If you want to know more about programming with the Kinect SDK, Read the docs or pick up this book on Amazon.com.

Kinect for Xbox One Sensor

Kinect for Xbox One Adapter

Backing up your Multi-Site Wordpress Blog to AWS S3

Fri, 06 Jan 2017 00:00:00 GMT

Today I'll be demonstrating how to back up a wordpress blog to Amazon S3 storage using the free version of the BackWPup plugin. If you are already hosting your wordpress site on AWS, then backing up to S3 makes sense. S3 is a robust, secure and easy to use cloud storage solution. S3 is not free but pricing is fairly reasonable.

BackWPup is a nice little plugin that supports backup of wordpress to a number of destinations, including Dropbox, FTP and Amazon S3. The S3 integration works very well once the configuration is complete. They have both a free version with limited features and a supported version with many more. BackWPup also supports Amazon Glacier which may be more cost effective, however this feature is only available on the PRO version.

In order for the plugin to work properly and for the S3 setup to be secure you need to use a combination of S3 bucket policies, IAM policies, groups and users.

In this article we'll be configuring a complete backup of a WordPress multi-site installation including database, files and plugins. I will assume that you already have signed up for an AWS account. Lets go….

Step by Step

Click on MySites > Network Admin > Plugins > Add New. Search for BackWPup and click "Install Now".

Once the plugin is installed, click on "Network Activate"

Click on My Sites > Network Admin > Dashboard. When activated you should see a BackWPup menu in the sidebar.

AWS IAM Setup

First we'll create an IAM policy that lets the BackWPup create, list and upload to s3 buckets in our account.

Navigate to the AWS IAM system and click on Policies > Create Policy

Click on Create your own policy

Create a policy that allows s3 bucket management called s3-manage. Fill in the fields as below and click Validate Policy then Create Policy.

Now we'll create a user group that has this policy attached. Go to AWS IAM and click on Groups > Create New Group. Call the group wordpress and click next

Search for the s3-manage policy that we created earlier and select it.

Click next, review the changes and create the group.

Next we'll create a new user that belongs to the wordpress group and can access s3 in our account. In IAM click on Users > Add User. Call the user wordpress and give the user programmatic access.

On the next step, search for the group we created earlier and add the user to group...

On the last step AWS will display the new users Access Key and Secret Key. Take note of these for configuring the BackWPup plugin later. Also take note of the users arn (e.g. arn:aws:iam::[youraccountid]:user/wordpress). We will use this for creating the S3 bucket policy later.

S3 Bucket Setup

Next we'll create the S3 bucket and add a bucket policy to control access to it. In the AWS Console navigate to the S3 and create a bucket called com.yoursite.wordpressbackup

Click on Properties > Permissions > Add Bucket Policy

Enter the following bucket policy into the dialog. This policy will allow the wordpress user to list the contents of the bucket and put, get and delete objects. Note: The resource property should contain the arn of the bucket you created, and the Principal should contain the ARN:

{
  "Version": "2012-10-17",
  "Id": "Policy1483571552699",
  "Statement": [
    {
      "Sid": "Stmt1483571510816",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::youraccountid:user/wordpress"
      },
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::com-ginocoates-wordpressbackup"
    },
    {
      "Sid": "Stmt1483571549812",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::youraccountid:user/wordpress"
      },
      "Action": ["s3:DeleteObject", "s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::com-ginocoates-wordpressbackup/*"
    }
  ]
}

Click save and S3 has now been secured for access by the wordpress user.

Configure BackWPup Plugin

Now we are ready to configure the backup plugin to talk to S3. Let's create a job and configure it. Back in your Wordpress installation, click on BackWPup > Add Job.

Enter the desired job name

Select the destination as "Backup to an S3 Service"

For now set the schedule to manually so that we can test.

On the S3 tab select the region where you created the S3 bucket and enter your S3 bucket url

Now enter the Access Key and Secret for the wordpress user created earlier

If your settings are correct you should see a list of available buckets in the Bucket selection area. Select the one we created earlier.

Save settings. Now switch to the jobs screen and run the job.

The job progress will be displayed. If everything is setup correctly the progress bar should reach 100%

Now switch back over the AWS console and navigate to your S3 bucket. Your newly created backup should be available.

Now you can go back to the settings of the job and set the backup to run on a schedule (e.g. using WPCron). Finally, you should take the time to test restoration of your backup to, say, another test site to ensure that if works as expected ;)

Learn more about Wordpress...

Creating an RStudio Server using Terraform on AWS

Wed, 04 Jan 2017 00:00:00 GMT

I've created a set of Terraform files to quickly provision an RStudio server on AWS using one of the great ami's from Louis Aslett http://www.louisaslett.com/RStudio_AMI/

The files contain the following variables:

local_ip - the local IP address to be given SSH access
region - the aws region for the deployment
instance_type - the type of instance to create. see: https://aws.amazon.com/ec2/instance-types/ zone - the availability zone the server should be created in
disk_size - the size of the EBS volume to provision
keep_data - whether the EBS volume should be kept on server termination
keypair_name - the pre-created key pair name to use
private_key - the private key to use to connect via ssh, i.e. the key from the key pair
rstudio_password - the password to set for RStudio server

There is also a privision.sh and libraries.R scripts that can be modified to setup the server and install addtional R libraries once its created.

The code is available on github here: https://github.com/ginocoates/rstudio-server-terraform

To provision a server, create a variables.tfvars file, and use terraform to do the grunt work. e.g. create a tfvars file as follows:

"local_ip" = "[your local ip address]"

"region" = "ap-southeast-2"

"instance_type" = "t2.micro"

"zone" = "ap-southeast-2a"

"keypair_name" = "datascience"

"private_key" = "~/.ssh/datascience.pem"

"rstudio_password" = "verysecurepassword"

"disk_size" = "100"

"keep_data" = "false"

Then provision the server as follows

terraform plan -out=terraform.plan -var-file=variables.tfvars

...

terraform apply terraform.plan

Terraform should report as output the public IP and DNS of the newly created server. Navigate to the IP and the RStudio prompt should be displayed.

Learn Terraform and RStudio Server...

Creating and Destroying AWS S3 Buckets in two Regions with Terraform

Tue, 03 Jan 2017 00:00:00 GMT

I ran across an issue with Terraform where I couldn't destroy an Amazon S3 bucket created in a region other than the one provided at the prompt.

To get around this I had to configure an AWS provider for each region I wanted to add a bucket to, and use the alias property to reference the provider from the s3 resource.

Without the region based providers security errors were showing up when trying to manipulate the bucket in different regions.

For example, Create two buckets in different regions:

resource "aws_s3_bucket" "s3_bucket_1" {
 bucket = "com-ginocoates-testbucket1"
 region = "us-east-1"
 }

resource "aws_s3_bucket" "s3_bucket_2" {
 bucket = "com-ginocoates-testbucket2"
 region = "us-east-2"
 }

Plan and apply them...

gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform plan
 provider.aws.region
 The region where AWS operations will take place. Examples
 are us-east-1, us-west-2, etc.

Default: us-east-1
 Enter a value:

+ aws_s3_bucket.s3_bucket_1
 acceleration_status: "<computed>"
 acl: "private"
 arn: "<computed>"
 bucket: "com-ginocoates-testbucket1"
 force_destroy: "false"
 hosted_zone_id: "<computed>"
 region: "us-east-1"
 request_payer: "<computed>"
 versioning.#: "<computed>"
 website_domain: "<computed>"
 website_endpoint: "<computed>"

+ aws_s3_bucket.s3_bucket_2
 acceleration_status: "<computed>"
 acl: "private"
 arn: "<computed>"
 bucket: "com-ginocoates-testbucket2"
 force_destroy: "false"
 hosted_zone_id: "<computed>"
 region: "us-east-2"
 request_payer: "<computed>"
 versioning.#: "<computed>"
 website_domain: "<computed>"
 website_endpoint: "<computed>"

Plan: 2 to add, 0 to change, 0 to destroy.

gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform apply
 provider.aws.region

The region where AWS operations will take place. Examples
 are us-east-1, us-west-2, etc.

Default: us-east-1
 Enter a value:

aws_s3_bucket.s3_bucket_2: Creating...
 acceleration_status: "" => "<computed>"
 acl: "" => "private"
 arn: "" => "<computed>"
 bucket: "" => "com-ginocoates-testbucket2"
 force_destroy: "" => "false"
 hosted_zone_id: "" => "<computed>"
 region: "" => "us-east-2"
 request_payer: "" => "<computed>"
 versioning.#: "" => "<computed>"
 website_domain: "" => "<computed>"
 website_endpoint: "" => "<computed>"
 aws_s3_bucket.s3_bucket_1: Creating...
 acceleration_status: "" => "<computed>"
 acl: "" => "private"
 arn: "" => "<computed>"
 bucket: "" => "com-ginocoates-testbucket1"
 force_destroy: "" => "false"
 hosted_zone_id: "" => "<computed>"
 region: "" => "us-east-1"
 request_payer: "" => "<computed>"
 versioning.#: "" => "<computed>"
 website_domain: "" => "<computed>"
 website_endpoint: "" => "<computed>"
 aws_s3_bucket.s3_bucket_1: Still creating... (10s elapsed)
 aws_s3_bucket.s3_bucket_1: Creation complete
 Error applying plan:

1 error(s) occurred:

* aws_s3_bucket.s3_bucket_2: Error putting S3 ACL: TemporaryRedirect: Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.
 status code: 307, request id: 9D4A2839B33A45F4

We get an error, however, the two buckets were created successfully. We can query s3 to verify...

gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ aws s3 ls | grep bucket
 2017-01-03 14:16:01 com-ginocoates-testbucket1
 2017-01-03 14:15:59 com-ginocoates-testbucket2

Lets try to destroy the buckets...

gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform destroy
 Do you really want to destroy?
 Terraform will delete all your managed infrastructure.
 There is no undo. Only 'yes' will be accepted to confirm.

Enter a value: yes

provider.aws.region
 The region where AWS operations will take place. Examples
 are us-east-1, us-west-2, etc.

Default: us-east-1
 Enter a value:

aws_s3_bucket.s3_bucket_2: Refreshing state... (ID: com-ginocoates-testbucket2)
 aws_s3_bucket.s3_bucket_1: Refreshing state... (ID: com-ginocoates-testbucket1)
 Error refreshing state: 1 error(s) occurred:

* aws_s3_bucket.s3_bucket_2: error reading S3 bucket "com-ginocoates-testbucket2": Forbidden: Forbidden
 status code: 403, request id: E081D8D9A009E943

Forbidden error, since the AWS provider is initialized with us-east-1, we can't delete the s3 bucket that was created in us-east-2. What I think this means is that the default provider is locked in to the region it was initialized with and can't call the s3 API in another region to refresh the state of bucket 2.

Lets try to fix it. Configure the AWS provider so we can create buckets in different regions. We can do this with alias properties.

provider "aws" {
 alias = "us-east-1"
 region = "us-east-1"
 }

provider "aws" {
 alias = "us-east-2"
 region = "us-east-2"
 }

resource "aws_s3_bucket" "s3_bucket_1" {
 bucket = "com-ginocoates-testbucket1"
 region = "us-east-1"
 provider = "aws.us-east-1"
 }

resource "aws_s3_bucket" "s3_bucket_2" {
 bucket = "com-ginocoates-testbucket2"
 region = "us-east-2"
 provider = "aws.us-east-2"
 }

Lets plan these changes...

gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform plan

aws_s3_bucket.s3_bucket_1: Refreshing state... (ID: com-ginocoates-testbucket1)
 aws_s3_bucket.s3_bucket_2: Refreshing state... (ID: com-ginocoates-testbucket2)

No changes. Infrastructure is up-to-date. This means that Terraform
 could not detect any differences between your configuration and
 the real physical resources that exist. As a result, Terraform
 doesn't need to do anything.'

So TF now sees the state as correct. Can we now destroy the s3 buckets successfully in each region?

gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform destroy

gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform destroy
 Do you really want to destroy?
 Terraform will delete all your managed infrastructure.
 There is no undo. Only 'yes' will be accepted to confirm.

Enter a value: yes

aws_s3_bucket.s3_bucket_2: Refreshing state... (ID: com-ginocoates-testbucket2)
 aws_s3_bucket.s3_bucket_1: Refreshing state... (ID: com-ginocoates-testbucket1)
 aws_s3_bucket.s3_bucket_1: Destroying...
 aws_s3_bucket.s3_bucket_2: Destroying...
 aws_s3_bucket.s3_bucket_2: Destruction complete
 aws_s3_bucket.s3_bucket_1: Destruction complete

Destroy complete! Resources: 2 destroyed.

Yes! Now is it possible to move bucket2 to us-east-1? Lets first recreate them.

gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform apply

aws_s3_bucket.s3_bucket_1: Creating...
 acceleration_status: "" => "<computed>"
 acl: "" => "private"
 arn: "" => "<computed>"
 bucket: "" => "com-ginocoates-testbucket1"
 force_destroy: "" => "false"
 hosted_zone_id: "" => "<computed>"
 region: "" => "us-east-1"
 request_payer: "" => "<computed>"
 versioning.#: "" => "<computed>"
 website_domain: "" => "<computed>"
 website_endpoint: "" => "<computed>"
 aws_s3_bucket.s3_bucket_2: Creating...
 acceleration_status: "" => "<computed>"
 acl: "" => "private"
 arn: "" => "<computed>"
 bucket: "" => "com-ginocoates-testbucket2"
 force_destroy: "" => "false"
 hosted_zone_id: "" => "<computed>"
 region: "" => "us-east-2"
 request_payer: "" => "<computed>"
 versioning.#: "" => "<computed>"
 website_domain: "" => "<computed>"
 website_endpoint: "" => "<computed>"
 aws_s3_bucket.s3_bucket_1: Still creating... (10s elapsed)
 aws_s3_bucket.s3_bucket_2: Still creating... (10s elapsed)
 aws_s3_bucket.s3_bucket_1: Creation complete
 aws_s3_bucket.s3_bucket_2: Creation complete

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Lets use change testbucket to to use the provider us-east-1. In theory this should destroy the bucket and create a new one in the desired region. Fingers crossed....

provider "aws" {
 alias = "us-east-1"
 region = "us-east-1"
 }

provider "aws" {
 alias = "us-east-2"
 region = "us-east-2"
 }

resource "aws_s3_bucket" "s3_bucket_1" {
 bucket = "com-ginocoates-testbucket1"
 region = "us-east-1"
 provider = "aws.us-east-1"
 }

resource "aws_s3_bucket" "s3_bucket_2" {
 bucket = "com-ginocoates-testbucket2"
 region = "us-east-1"
 provider = "aws.us-east-1"
 }

Plan the changes...

gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform plan

aws_s3_bucket.s3_bucket_1: Refreshing state... (ID: com-ginocoates-testbucket1)
 aws_s3_bucket.s3_bucket_2: Refreshing state... (ID: com-ginocoates-testbucket2)
 Error refreshing state: 1 error(s) occurred:

* aws_s3_bucket.s3_bucket_2: error reading S3 bucket "com-ginocoates-testbucket2": Forbidden: Forbidden
 status code: 403, request id: 83855B6D6780B04D

Oops, error. The state has the bucket in us-east-2, but the tf files reference us-east-1. So the provider is initialized with the us-east-1 endpoint when we run terraform plan. But it can't refresh the state of the bucket in us-east-2 as its configured to use a different AWS endpoint. Kinda makes sense.

So, with TF we can't destroy a bucket in one region and recreate it in another with this setup. Something to watch out for!

To achieve the move we'd have to remove bucket2 from our tf files, apply to allow TF to destroy it, then add it back with the desired region, and apply these changes.

Learn Terraform Here....

Kanban - Feel the flow

Mon, 02 Jan 2017 00:00:00 GMT

In this article I will be talking about my favorite way to manage flow of work through a software team, Kanban.

Kanban is a great way to gain team agility without requiring changes to existing processes. Kanban allows these processes to be improved over time with minimal impact to how a team works, while giving full visibility on the process and bottlenecks within the delivery system. 25% of respondents on the version one state of agile survey said they were using Kanban . In a nutshell, Kanban…

Helps visualize the development process and bottlenecks within the delivery system
Enables agility within existing development processes
Improves the speed of delivery of software to the business users requesting it
Enables a continuous improvement culture within a team
Uses an SLA based scheduling approach
Uses frequent delivery of usable software on an agreed schedule with business users
Improves team morale

What is Kanban?

Kanban is a technique for managing software development in an efficient way. It is not a development process in itself and can be used to improve the performance existing development processes (e.g. Scrum). As it doesn't mandate drastic changes in the development process and it can be used as a change agent that gradually improves delivery performance over time with minimal impact to staff and processes.

At its core it is a visual system designed to help:

Visualize the development process as a pipeline of stages
Visualize the current state of items being produced within the process.
Visualize current resource assignments for items within the process.
Visualize blockages to items being produced within the process.
Visualize Work in Progress Limits within the system (see below).
Visualize lead times for classes of work items within the process.
Maximize the throughput of bottlenecks within the system in order to increase the flow of items through the system.

In manufacturing a Kanban card is a visual signal that more inventory needs to be produced and is used to control waste in the system. Kanban was developed by Taiichi Ohno, at Toyota, to find a system to improve and maintain a high level of production.

In Kanban for software development we use a physical board and system cards to display work items within the development process. Typically the board is divided into the various columns representing the phases of the development process. As items are developed the cards are pulled through the different columns in the board by development resources that become free. At a glance, the Kanban board gives a clear picture of where things are within a current release. Each card on the board gives a clear status for items being developed.

Example Kanban Board

Below is a sample similar to board formats that I've used in the past:

Each column is a step in the value chain. The ready for dev column is a set of groomed items in priority order, most important at the top.
The lead time for delivery is written clearly on the board beside the ready for dev column.
Each column has a work in progress limit which represents the capacity of the resources working in that column, or the limits of the bottleneck of these resources (see below for details)
The definition of ready/done for each column is written in a section at the bottom of the board. This makes it easy for team resources to check whether they have done all important tasks before the item can move.
Different coloured cards are used for different types of item. e.g. Red for bugs, yellow for development tasks, making classes of item clearly visible.
Some columns are Queue columns. I usually put a queue in front of system test column up to a limit. Testing is usually a bottleneck in the system due to the developer/tester ratio. By adding a queue developers can start on the next development task while testers are completing its current batch. However, once the ready for test queue reaches its WIP limit, developers may switch to Kaizen tasks.
Paper tabs are used to denote who is working on a task (blue) and which tasks are blocked (red). Using tabs in this way is a great way to impose a physical limit that a certain name can only be assigned to a single task at a time. Red tabs are used to visualize items that are blocked and can be initialled to denote who raised the blocked issue.
An expedite queue is included at the top and is for emergency items the team may be working on. The policy is usually that an expedite item freezes all other work on the board until it is complete.

Electronic vs Physical Boards

Some of the electronic work tracking tools available today, such as Atlassian Jira or Visual Studio Team Services are really feature rich and the Kanban boards they provide do a great job. Benefits include automated reporting of team metrics (e.g. for calculating lead times) and integration with other systems such as email. However, there are also limitations. For example, Kanban works best when work is always visible to the team. With electronic boards, this is not always the case. Not all teams have large screen displays in every meeting room and having to pull up the board on laptops or projectors before progress can be discussed can be a real time sink. Also, some of these electronic tools can be overcomplicated for smaller teams.

On the other hand, the benefits of a physical board are:

Work is visible always to everyone in the office.
Mobile whiteboards can be easily carted around the office and in to meetings.
The board imposes physical limits on the number of cards that can be added to the columns. This acts as a physical WIP limit for the team.

The downsides of a physical board are:

The board is only visible locally to the team. This can make it hard to share information with a wider audience, say, international teams.
No automated reporting of lead times is available and it must be done manually.

One strategy when considering a physical board is to run a backup electronic board for calculating statistics and reports, which the scrum master keeps in sync with the physical board.

Anatomy of a Kanban Card

For Kanban cards a standard index card can be used. Printing the index cards is a great way to make them legible and tidy. The card should make visible any pertinent process related info that helps discussing and helps it flow through the process.

Here's a sample similar to a card layout I've used before:

The number records the electronic boards number if one is being used as a backup.
To generate metrics, key dates are recorded in the top right of the card. It's important to record at a minimum the card creation date, when the team started working on it, and when they completed it.
A clear description of the item appears in the centre of the card.
A definition of done is included in the bottom left of the card. This gives a quick visual indicator to the team of the "done-ness" of the item and when it can pulled downstream. If certain activities are not required they can be crossed out. Having a definition of done on the card is an alternative to adding a done column for each phase on the board and is a good space saving technique.
Notes are maintained on the bottom right. For example, the team might note the due date for the item as requested by the stakeholder, or if collaboration with an external team is required before the item is "done done"

This card layout is an example only. Kanban encourages that a team tailors the visual system (board and cards) to suit the process of the team. This is an evolutionary process, where the team tries something and keeps it if it improves the process, and discards it if it doesn’t. Changes are evaluated in the regularly held retrospective meetings.

WIP Limits and Flow

Batch size (or the number of items in a release) has a direct impact on lead time and quality. What this means is that the more items a team is working on at the same time, the longer it will take for items to make their way through the development process and be delivered to the business. At the same time the rate of defects will increase. This is due to the following factors:

The time for each type of activity (analysis, development, testing) that needs to be completed within the release increases, and this impacts the lead time for items in that release.
The potential for complex dependencies between items increases if the scope of a release increases.
An increase in complexity requires an increase in the amount of management and planning required for the release.
Defect rates increase as the scope increases and the time between analysis, development and testing increases. This can simply be due to the fact that there is a greater scope to discuss, implement, test and more potential for things to get lost in the mix.
The time required to sign off a release increases as there is more functionality for business users to consider.
The risk associated with a release increases as more code has changed in a large release, also meaning that it’s harder to find issues when something goes wrong.

So keeping a small number of items per release and releasing more regularly promotes team agility by reducing lead time, reducing management overhead and reducing defect rates.

Kanban maximizes flow through the development pipeline by keeping the number of items in a release at an optimum level, known as the Work In Progress limit (WIP). Small batch sizes are achieved by setting work in process limits for each phase of development within the development process. These WIP limits are usually set based on the capacity of resources within that development phase.

This WIP limit ensures that bottlenecks within the system are exploited at 100% capacity. For example a development team may have 4 developers available. The WIP limit for the development phase of the pipeline may be a maximum of 4 items being worked on at any time (or 2 if pair programming is being used). Typically WIP limits are displayed on the Kanban board at the top of each column.

There is a great Yow conference video by Don Reinertsen on the effect of batch size on throughput here:

https://yow.eventer.com/yow-2012-1012/the-practical-science-of-batch-size-by-don-reinertsen-1269

Pull System & Kaizen Culture

Kanban implements a pull system, which means that cards pulled from left to right on a board as resources become available. e.g. Developers will pull tasks from the "Ready For Dev" column when they have finished development on their current item. Testers will pull development tasks into the System Test column when they are out of items to test and there are development items ready to test.

Since it is only the bottlenecks in the development process that are working at 100%, the pull system can create slack within the development process. For example, a developer cannot pull a card unless their current card has been pulled into system test by the testers. This means the developer must wait before starting new work.

With Kanban this slack is considered to be a good thing and gives time to develop a self-improvement culture within the team, referred to as a "Kaizen" culture. Waiting resources focus on "Kaizen" tasks that are aimed at improving the development process or team resources in some way. This might be optimizing the builds, training, blogging etc, etc. Developing the Kaizen culture is the core of how Kanban enables team agility.

In terms of managing the Kaizen tasks, by their nature they should be tasked that can paused easily if the bottleneck has cleared in the process. I've seen Kaizen work best when the team maintains a much simpler "To Do, Doing, Done" style Kaizen board separate to the main delivery board.

More on Kaizen can be found here: http://en.wikipedia.org/wiki/Kaizen

Outcomes and challenges using Kanban

Using Kanban I have seen the following positive outcomes for a team:

Everyone on the team gets a clear understanding of the development process and the state of items in the team as all work is visible.
Delivery becomes more efficient as Kaizen tasks are being completed to improve the development process (e.g. automating manual tasks)
Team members work at a sustainable rate of development while delivering continuously.
Meetings (scrums and retrospectives) become very focused on delivery and take the minimal amount of time necessary. Scrums become focused on flow of work and discussions are about coordination and blocked issues on the board.
Team morale increases due to the team working effectively together.

Kanban works well if items being requested are of a consistent size. If they are, then meeting SLA's for upstream requesters is easy. Care needs to be taken at the input queue (our Sandbox) to slice things up in a consistent way (e.g. a small number of acceptance criteria per story). This may not always be possible though, and Kanban requires that any variability in size is managed accordingly.

Kanban also works on an SLA basis for delivery instead of upfront planning. The rationale for this is that if items being delivered are of a fairly consistent size, and a small batch size is used per release, then over time the delivery frequency of items averages out.

Team Metrics

The key to successfully using Kanban to improve flow is to monitor team throughput metrics. You can go as complex as you like with this, but at a minimum I like to track the following items:

Number of Items in Flight
Number of Items "Done Done"
Average Lead Time (From requested to done done) +/- Std Deviation
Average Dev Time (From dev start to done done) +/- Std Deviation
Max Lead Time
Max Dev Time

A simple monthly report on these figures might look like the following, and is very simple to create in Excel. This kind of report can be used in retrospective and for high level planning with stakeholders.

Resources

The "Kanban" book by David J. Anderson is a really great resource on the topic, highly recommended if you are interested in using Kanban. You can get a copy on Amazon.com

Terraforming a Wordpress blog on AWS

Sun, 01 Jan 2017 00:00:00 GMT

Terraform by Hashicorp is an awesome tool that allows you to define your system architecture as code using a json language variant.

With Terraform you can:

Define your architecture as code in simple text files rather than manually tweaking things in your cloud provider
Store your architecture definition in a source control repo and have a complete revision history over changes made
Deploy resources to multiple cloud providers
Preview changes to your architecture before deploying
Work in an iterative way, deploying and tearing down complex cloud architectures in minutes

In this blog post we'll use terraform to define a WordPress website and deploy it to AWS. We are going to create:

A standalone EC2 instance with Bitnami WordPress Multisite installed
A security group to restrict access to the server and allow SSH from our local machine
An elastic IP so that DNS mappings work if the server is rebooted
A root device that won't get deleted with the instance, so that if the server dies, we still have the website data

We'll be using Atom to edit terraform files and doing all of this on windows. I usually use Linux these days, but I'm using a surface tablet to write this post so I'll use git bash prompt for the examples.

This post assumes that you already have already signed up for an AWS account, but will cover configuring your local system for use with Terraform.

AWS SDK Install

Install the AWS SDK for your local system. For windows that can be installed from http://docs.aws.amazon.com/cli/latest/userguide/installing.html#install-msi-on-windows. Once its installed open a prompt and check that the aws cli is accessible by typing aws --version.

Now execute "aws configure" to configure your local environment to access aws. You will be promoted for the access key, secret, region and output format. You can generate a user account in AWS IAM and generate the access key and secret. The user should have adequate permissions to create EC2 resources. For this article I created a user with full EC2 access.

I usually use text output which is easier for bash scripting.

To check configuration was successful, ask aws to kindly list running ec2 instances in your default region. If this is a brand new account, or you have no instances in that region, you should see output similar to the following...

Okay, now AWS is good to go. Lets install terraform...

Install terraform

We are going to install to a directory on the C drive. So let's create a directory…

mkdir c:\terraform

Download terraform from https://www.terraform.io/downloads.html and extract terraform.exe into c:\terraform

To make it easy to use terraform from any prompt. Let's add c:\terraform to your path. Go to System > Advanced System Settings > Environment Variables and add c:\terraform; to the start of your path.

Verify the installation by typing terraform --version at the prompt.

Atom

Install Atom from https://atom.io. You may also want to install the terraform language which adds syntax highlighting for terraform files. https://atom.io/packages/language-terraform

apm install language-terraform

Also, to support reformatting terraform files on save we can use the terraform executable and a plugin called terraform-fmt. First install it.

apm install terraform-fmt

Then configure it by editing the atom config file and add the following lines (File > Config):

Now we are ready to terraform our architecture. At the prompt change to your source code directory and open atom (usually just type atom .)

Lets Terraform

Add the following files to your terraform project.

config.tf - This is where we will configure our aws provider
main.tf - This is the file where we will define the main resources used in the architecture, including the EC2 instances, elastic IP's etc.
security.tf - This is where we will define security groups to allow access to our server over http and to restrict SSH access to our local IP
outputs.tf - This is where we will define the outputs of our terraform architecture once its complete. For example this is handy if you output the public DNS of any server created.
variables.tf - Here we list any variables used in our architecture definition. For example, this can be used to provide region specific ami ids.

Note, I'm using multiple files for code organization. It's perfectly fine to define everything in a single terraform file. Terraform is smart enough to process all terraform files it finds in the current directory and consider them as part of your solution when planning or deploying the architecture.

Let's first setup our variables...

Variables.tf

Here we define the variables that are in use in our architecture. This will cause Terraform to expect these variables when it is executed, either to be passed at the command line with the -var option, or in a .tfvars file using the -var-file option.

We will define the following variables:

local_ip - Later on we will create a security group to allow SSH access to the server. We will define a variable for the local ip address so that it should be provided when running terraform at the command line
region - This variable will allow us to pass the region as a variable. This allows us to conveniently pass it in the tfvars file.
wordpress-images - This is a map or lookup variable allowing us to use specific images for a certain region. We will only be deploying to N.Virginia in this article, but I'll include this kind of variable for illustration purposes.

The image id for bitnami wordpress multi-site can be found here: https://aws.amazon.com/marketplace/fulfillment?productId=2f1d4d67-324b-41d7-8af9-b7860d269c6d.

To launch in Virginia well be using the image ami-a42d75b3. You must subscribe to this ami in the amazon market place before using it. To do this, go to the Bitnami WP multisite page in the marketplace, select the Manual Launch tab and then hit the "Accept Terms" button on the right of the page.

Here is our completed variables file...

I will use a local tfvars file to provide these values so that I don't have to pass them at the command line, or check them into source control. The format of this file is shown below….

Config.tf

Here we can configure some defaults for our provider. Here we could put our access key and secret, however, it's better to let terraform use the locally configured aws profile and prompt for the region during deployment. For now we will tell terraform to use the default aws profile configured on the local system and deploy to the region specified in the region parameter. Note the user of the interpolation ${} syntax here to reference the variable we defined earlier.

Okay, lets first make sure we've considered security...

Security.tf

Add a security group resource to the security.tf file and give it an id, name and description. Our security group will contain a number of rules, so we will "outline" rather than "inline" its rules.

Now we will add rules to the security group. We will use the Terraform interpolation syntax to specify the security group the rule belongs to e.g. ${aws_security_group.wordpress_security_group.id}

We'll allow ingress on http port 80 and 443 to allow web traffic to hit the server. (I won't set up SSL just yet, but it's good to have the port open for when I do)

We'll also allow ingress reply traffic from the internet. This will allow replies on ephemeral ports to requests made by the server out to the internet.

We'll allow egress traffic from the server to the internet. This will allow the server to connect to the internet on all ports.

We'll also allow SSH access from our local IP to the server. Note that we are using the local_ip variable here to define which IP has access.

The cool thing about terraform is that now we can go ahead and create those security groups in AWS, i.e. work in an iterative fashion. In a prompt, change to the source code directory and execute terraform plan.

Note the out parameter is used to save the plan file. This is important because you want to apply exactly what was listed in the plan phase. If you don't save your plan, when you apply your changes the underlying architecture may have changed. Be careful however, as the saved plan has some security issues and may store some sensitive information so don't check that into source control.

Now we can apply the plan to create our security groups. Execute "terraform apply terraform.plan" at the prompt. You should eventually get output similar to the following.

Let's go to the AWS console and verify that terraform is awesome. In the EC2 console for your selected region, verify that the SG is created and has the rules defined.

Confirmed, it's awesome. Now let's create our wordpress instance.

Main.tf

We'll create a t2 micro instance that’s free tier eligible, with a 40GB harddrive. This means it should cost about $1 per month on the free tier (https://calculator.s3.amazonaws.com/index.html). We'll also assign our security group we created earlier to the server. We'll also assign a public IP and configure the root_block_device to not delete the volume when the instance is deleted.

Finally, we'll create an elastic IP so that in the event the server is rebooted, the server will maintain its public IP address.

Before creating the server, lets create some outputs so Terraform will tell let us know what our new server DNS and public IP is. In outputs.tf, define the following output values.

As before, we can now iteratively update our architecture. First execute terraform plan.

Terraform should report that it will create 2 new resources. Now apply the plan to create the server.

It will take a number of minutes to create the server, but you can check in AWS console if the server creation is in progress, again confirming Terraforms awesomeness. Once TF has finished creating the instances you should see something like:

Outputs: Wordpres_DNS = ec2-[your elastic ip address].compute-1.amazonaws.com Wordpress_Public_IP = [your elastic IP address]

Let's confirm that TF has created the instance successfully in the AWS console.

We can see here that the instance is created and running. We can also see that our security group is mapped to the instance and that its using our elastic IP address.

And now we can successfully navigate to the blog using the DNS output by terraform, sweet!

Conclusion

Terraform rocks. You can download the source code for this terraform solution here: https://github.com/ginocoates/terraform-aws-hosted-wordpress

Learn Terraform...

Image PanView - Objective-C

Sat, 18 Aug 2012 00:00:00 GMT

In my current IOS project I want to display an image on screen, but make it a little more interesting.

The FlipBoard iPad application has a nice start screen which pans the images it displays, quite an eye catching effect. I wanted to do something similar in my application.

To achieve this I created a UIView subclass with the following structure:

UIView
--UIImageView

I then used [UIView animateWithDuration] to move the UIImageView whichever way I want.
In the sample code the view selects a random animation from the following set:

Pan Left
Pan Right
Zoom In
Zoom Out

The end result is quite nice!

Sample code can be downloaded here: https://code.google.com/p/objective-c-panview

Screenshot from the sample is below. Clicking "Animate" will select a random animation.

Objective C State Machine

Sun, 12 Jun 2011 00:00:00 GMT

Im currently working on a game in which the user interface has quite a number of possible states, with certain sprites being enabled, disabled, visible or invisible depending on the what the user touches on screen.

Rather than coding all of the UI state logic into a view controller or helper class, I decided to put together a simple state machine instead. It seemed to work for my current app quite well, and I reckon its something that will come in handly in the future.

You can find the source code here: http://code.google.com/p/objective-c-state-machine/

So How do you use it?

Each state is represented by a StateInfo object, which has a name, a set of transitions, and a selector to execute when the state is entered.

@interface StateInfo : NSObject
{
  NSString\* \_stateName;
  NSMutableArray\* \_transitions;
  SEL \_selector;
  }
  @property (nonatomic, retain) NSMutableArray\* transitions;
  @property (nonatomic, retain) NSString\* stateName;
  @property (nonatomic) SEL selector;
  -(id) initWithStateName:(NSString\*)stateName;
  -(StateInfo\*) executeTransition:(id)anObject;
  -(BOOL) canExecuteTransition:(id)anObject;
  -(void) registerNextState:(StateInfo\*)nextState withCondition:(NSString\*)condition;
@end

The collection of transitions is made up of Transition objects, each with a condition and a StateInfo object pointing to the next state. Underneath, the condition uses an NSPredicate object, so the format for specifying the condition should follow the NSPredicate syntax.

@interface Transition : NSObject
  {
    NSString* _condition;
    StateInfo* _nextState;
  }
  @property (nonatomic,retain) NSString* condition;
  @property (nonatomic, retain) StateInfo* nextState;
  -(id)initWithCondition:(NSString*)condition andState:(StateInfo*)state;
  -(StateInfo*)execute:(id)anObject;
  @end

  The StateMachine object is then a simple set of states and a pointer to the current state.

  @interface StateMachine : NSObject {
    NSMutableArray* _states;
    StateInfo* _currentState;
  }
  @property (nonatomic, assign) StateInfo* currentState;
  @property (nonatomic, retain) NSMutableArray* states;
  -(id)initWithInitialState:(StateInfo*)state;
  -(void) registerState:(StateInfo*)state;
  -(StateInfo*) nextState:(id)anObject;
  -(void) registerInitialState:(StateInfo*)state;
  -(BOOL) canTransition:(id)anObject;
  -(BOOL) isCurrentState:(NSString*)stateName;
@end

The main method for the StateMachine is called nextState, and its called by passing a target object to test the transitions against, e.g.

StateMachine* stateMachine = [[StateMachine alloc] initWithInitialState:initialState];
...
[stateMachine nextState:targetObject];

When its called, the StateMachine tests the condition of each transition in the currentState against targetObject.

If a transitions condition is true, Transition.nextState becomes StateMachine.currentState, and the currentState selector is performed in the targetObject.

To setup a statemachine its pretty simple, create a statemachine object, then register the states one by one....

StateInfo* initialState = [[StateInfo alloc] initWithStateName:@"initialState"];
StateMachine* stateMachine = [[StateMachine alloc] initWithInitialState:initialState];
StateInfo* state1 = [[StateInfo alloc] initWithStateName:@"state1"];
state1.selector = @selector(state1Method);

//--register the next state for the initial state
[initialState registerNextState:state1 withCondition:@"TRUEPREDICATE"];

The project on google code contains a sample objective-c application that uses the state machine. To get the code, use svn at the command line.... svn checkout http://objective-c-state-machine.googlecode.com/svn/trunk/ objective-c-state-machine-read-only