Pics or it didn’t happen!

Amazon Web Services - SimpleDB using the PHP SDK

Long time no post… here’s a nerdy one which I am writing since I have found no information on it anywhere on the internet.

Amazon Web Services has recently released their PHP SDK. It’s not the most well documented library out there but it does make doing a lot of things on AWS a lot easier. I’ve been working on a few small projects as proof-of-concepts for scalable web applications that can run off a regular web server but leverage the cloud for storage and extra compute capacity.

One component I’ve spent a few hours on working out has been a monitoring component. Basically, a script runs regularly and collects a few statistics (e.g. S3 bucket size), storing all the information into a SimpleDB domain. Anyway, what it does isn’t relevant - the point is that it stores a load of data roughly resembling a logfile using SimpleDB. That bit is all very easy to architect and develop.

The hard part is retrieving this information. I’m planning to use the Google Visualisation API to generate some cool graphs, and I was planning to create a simple PHP script that would basically extract some log data and output it to CSV. This is where the roadblock is. Amazon SimpleDB result sets are restricted to the lesser of 100 rows or 1 MB. Queries with larger result sets are paged, and each result set will contain a token which can be supplied on a subsequent request. Sounds simple.

But the SDK has no method to retrieve the token from the request. The samples don’t mention it and the API reference is just a JavaDoc which doesn’t include much detail. (Although it at least does tell you how to pass the token on the next request). Without a way around this, the PHP SDK is just about useless for SimpleDB in most use cases where you’d use SimpleDB over a traditional RDBMS like MySQL.

It took some analysis and reverse engineering and eventually I found out how to get to the NextToken value in the response. Here’s a code snippet. I just realised I set this blog up for photography so I never bothered to put in a decent code highlighter so bear with me.

$sdb = new AmazonSDB();

$opt = array();
$query = “SELECT * FROM `opengal_monitor`”;

do {
// Send query to SimpleDB
$results = @$sdb->select($query,$opt);

// The value for NextToken is found here
$opt['NextToken'] = $results->body->SelectResult->NextToken;

// Remove all line breaks or subsequent queries will fail
$opt['NextToken'] =
ereg_replace(”/\r|\n/”, “”, $opt['NextToken']);

// Put the items in an array
$items = $results->body->Item();

// Do Stuff

} while ($opt['NextToken']);

And that’s it! Based on this, it would be a trivial task to overload the CFResponse class for SimpleDB Select operations and provide a method that would return the NextToken value in its correct format (i.e. no linebreaks).

No Comments | Posted in » Coding