More GAE datastore resources

There’s an interesting interview with the creators of Twig, Objectify-Appengine, and SimpleDS, which are all ORM-like frameworks built for the GAE that facilitate working with the underlying datastore (an abstraction of Bigtable). If you haven’t worked with GAE, you need to know that the exposed hip datastore isn’t relational — it’s schema-less and more like a key/value store; consequently, the JDO features exposed by default tend to leave people a bit distressed (especially when it comes to relationships).

The questions asked and the answers these developers provide are quite helpful in understanding both the pains and the beauties of the Bigtable abstraction. And while these developers are obviously biased towards their respective framework and the details of the interview are focused on the datastore itself, this conversation is a worthwhile read for anyone new or considering using the GAE.

What’s more, the interview provides a link to a GAE forum where both the creators of Twig and Objectify-Appengine square off regarding their respective frameworks implementations. Both frameworks have a distinctly different mechanism for dealing with relationships and while I tend to prefer Twig’s relaxed syntax, Jeff Schnitzer, the creator of Objectify-Appengine makes a cogent case for why dealing directly with GAE keys is safer.

Interestingly enough, a full-stack framework targeting the GAE has yet (to my knowledge) fully emerge — indeed, Grails works on the GAE but this is basically an afterthought (i.e. Grails wasn’t built for the GAE). Moreover, Gaelyk is specifically built for the GAE, but lacks a fully fleshed out ORM implementation, preferring to expose an enhanced version the low-level Entity API. The Play framework, which is somewhat like Grails but without a lot of Groovy, has a GAE module (aka plugin) along with an Objectify-Appengine one. It should be interesting to see how things play out (no pun intended).

In the clouds with Amazon’s SimpleDB

development 2.0As part of the Amazon Web Services family, Amazon’s SimpleDB is a massively scalable and reliable key/value datastore, which is exposed via a web interface and can be accessed using any language you’d like — from Java to Ruby to Perl to C#. In fact, Amazon has recently released a standardized SDK for both the .NET and Java platforms.

Check out IBM DeveloperWorks’ newest article entitled “Cloud storage with Amazon’s SimpleDB, Part 1” — in this article, you’ll see firsthand how to leverage Amazon’s Java SDK to work with SimpleDB. In fact, this is the first of two articles exploring SimpleDB’s unique approach to schemaless data storage, including a demonstration of one of the datastore’s most unusual features: lexicographic searching.

Stay tuned for part 2, where I’ll cover using JPA to work with SimpleDB. Until then, happy reading!

Grails hip tip: LinkageError with SAXParseException

Recently a bogue error popped up with an existing Grails project at a client site. In an effort to abstract core domain logic (i.e. business rules) for a financial application, we decided to leverage Drools; consequently, after some prototyping of rules in a non-Grails project, we decided to start evaluating the Grails Drools plugin.

After installing said plugin, however, Grails failed to start up, instead issuing the error:

java.lang.LinkageError: loader constraint violation: loader (instance of ) previously initiated loading for a different type with name “org/xml/sax/SAXParseException”

followed by an equally nefarious stack trace.

It turns out that differing versions of the xml-apis jar will force this issue; accordingly, if this happens to you, you can fix the situation by simply locating the offending jar file (in this case, it was the Drools plugin which includes a differing version: xml-apis-1.0.b2.jar) and removing it.

Effortless Groovy

Does the prospect of learning a new language daunt you? Does the effort it takes to download a jar file and place it in your classpath overwhelm you? If you answered “yes” (or even “maybe”) to either of these questions, have no fear, baby! The bright folks behind the Gaelyk framework (and not to mention Groovy itself) have lowered the bar as low as it can possibly get for checking out Groovy. The Groovy web console is one of the hippest websites around for seeing firsthand how Groovy works.

Via the web console, you can enter in valid (or invalid for that matter) Groovy code and execute it — all that’s required is a browser. That’s it! Interestingly, the website even allows you to save your scripts and view and comment on other people’s scripts — this provides a great way to learn more about Groovy (think of it as an IDE-blog combination). So what are you waiting for? Check it out today (unless the effort to click a link is just too arduous for you).

Schemaless data modeling with Bigtable and Groovy

development 2.0NoSQL datastores like Bigtable and CouchDB are becoming more and more hip these days because they solve the problem of scalability on a massive scale. Google, Twitter, Digg and Facebook are just a few of the big names that have adopted NoSQL, and we’re in early days yet. Schemaless datastores, however, are fundamentally different from traditional relational databases. Leveraging them is easier than you might think, especially if you start with a domain model, rather than a relational one.

Check out IBM DeveloperWorks‘ “Schemaless data modeling with Bigtable and Groovy’s Gaelyk” and see for yourself how refreshing schemaless datastore modeling combined with the power of Groovy really is!

GAE datastore resources

While I’m a big fan of leveraging Google’s low-level API Entity class (as decorated by Gaelyk) for persisting data into GAE’s underlying datastore, there are some up-and-coming frameworks that are worth examining. Both frameworks note the challenges with JDO and like me, prefer to work with Entity types. Each framework has a good deal of documentation too. Without further ado, check out:

It appears that at the moment, Twig is a bit less verbose (with respect to coding) but each framework supports a wide range of hip features.

Don’t forget to check out Gaelyk (which is a super slick Groovy framework for GAE) and while you’re at it, read “Schemaless data modeling with Bigtable and Groovy’s Gaelyk” for a layman’s approach to data modeling with Entity objects.

A recent Java platform roundtable

I had the pleasure of joining a number of colleagues in a panel discussion regarding the Java platform recently. The resulting dialog can be found on IBM DeveloperWorks entitled “Java platform roundtable, Spring 2010” — in this discussion with the likes of Dan Allen, J. Steven Perry, Alex Miller, and Richard Hightower (just to name a few, man!), we had the opportunity to converse about “most important challenge facing Java-based developers today” and what we see in regards to Oracle’s acquisition of Sun. Moreover, we examined Java 7, alternate JVM languages, Spring, and the cloud.

All in all, I enjoyed reading the varied opinions and views my Java platform compatriots shared. I can’t say I agree with all of them; nevertheless, diversity in opinions makes things interesting. Have a read and see for yourself!

Concurrency with Kilim

The basic programming model of languages, like Java, is thread based and while multithreaded applications aren’t terribly hard to write, there are challenges to writing them correctly. What’s difficult about concurrent programming is thinking in terms of concurrency with threads. Alternate concurrency models have arisen along these lines — one that is particularly interesting, and gaining mindshare in the Java community, is the actor model.

The actor model facilitates concurrent programming by allowing a safer mechanism for message-passing between processes (or actors). Implementations of this model vary between languages and frameworks. Luckily, there are a number of choices for leveraging this model on the Java platform.

The latest article in IBM DeveloperWorks’ series Java Development 2.0 entitled “Introducing Kilim“, introduces Kilim, an actor-based message-passing framework that weaves together concurrent and distributed programming.

If you want to leverage “plain Jane” Java actors, then your best bet might be Kilim or a similar framework — as you’ll find, an actor-based framework does make concurrency programming, and leveraging multicore processes, much easier.

To find out more regarding Kilim and the actor model, read the article and while you’re at it, check out the other articles in this series that explore the spectrum of technologies and tools that make Java development these days quite exciting!

easyb 0.9.7 is on the streets

The easyb team has recently released version 0.9.7, which contains a scattering of features and fixes including one of my favorites: behavior tagging. Briefly, behavior tagging permits the isolation of behavior groups. Much like TestNG’s group annotation, tags in easyb allow you to selectively execute a group of behaviors — this is a handy feature on a number of fronts, not the least of which is test categorization.

What’s more, version 0.9.7 now supports the ability to generate a report of all behaviors without having to run the suite itself; that is, you can generate a listing of all behaviors, which will be marked as “in review” — this is distinctly different than a “pending” state as “in review” behaviors could be either implemented or not; what’s more, they could even be failing. The intent of “in review” is for reviewing the natural language behaviors (i.e. goals) with team members, stakeholder, etc.

Version 0.9.7 also leverages Groovy 1.7.2; plus, there are a few DSL changes, such as but is now supported (works just like and) along with shared_behavior can now be written without the underscore (i.e. shared behavior — this is much like narrative‘s as a and i want).

For more information about what’s in 0.9.7 check out the release page. You can find the 0.9.7 download on easyb’s project page. Thanks to everyone in the community for your support!

Actor style messaging and honey do lists

free lunchAs I previously mentioned in “Free lunches, mousetraps and the Actor model“, Edward A. Lee wrote an interesting article entitled “The Problem with Threads” in which he advocates leveraging the actor model in popular languages (such as in Java) as opposed to adopting an entire new paradigm (like Erlang). He states:

We should not replace established languages. We should instead build on them.

It appears that more than a few hip people agree with his line of thinking. It turns out there are quite a few options available for leveraging the actor model in Java. That is, aside from alternative languages like Scala, which supports actors and Groovy with GPars, there’s framework’s like Kilim, ActorFoundry, Actors Guild, and jetlang.

I ended up employing Kilim at a client side over a year ago to replace a thread based computational model. At the time, GPars was in its early stages and I was specifically looking for a speed up in application performance. The multi-threaded application was taking roughly 5 hours to complete.

In the end, the speed up attributed to Kilim (or indirectly leveraging its actor model) was hardly noticeable (some aspects of Kilim were noticeably faster though — such as spawning a task was quite fast as opposed to spawning a normal threads) as the real performance gain was leveraged by reducing and improving database queries (as usual, performance issues were essentially IO bound); nevertheless, the prime benefit of Kilim, which at the time I had overlooked, was the notion of a mailbox. That is, in the actor model, processes can share data more safely.

There are quite a few different implementations and ways to facilitate message passing in various languages and platforms, but to me, the actor model’s mailbox notion is quite intuitive. In Kilim’s actor model, messages are passed between processes via a Mailbox — in many ways, you can think of it as a queue. Processes can put items into a mailbox and also pull items from a mailbox in both a blocking and non-blocking manner (blocking of the underlying process not a thread).

As an example of leveraging mailboxes in Kilim, I wrote two actors (a Husband and a Wife) that extend from Kilim’s Task type. Previous versions of Kilim had an Actor type; however, as of version 0.6, Task is the way to go.

import kilim.Mailbox;
import kilim.Pausable;
import kilim.Task;

public class Husband extends Task {
 private Mailbox<Message> mailbox;

 public Husband(Mailbox<Message> mailbox) {
  super();
  this.mailbox = mailbox;
 }

 @Override
 public void execute() throws Pausable, Exception {
  while (true) {
   System.out.println("Husband listening...");
   Message msg = mailbox.get(); // blocks
   if (msg.getReceipient() == Message.HUSBAND) {
    System.out.println("Husband hears: " + msg.getMessage());
    Message reply = new Message(Message.WIFE, "Yes, dear");
    mailbox.putnb(reply);
   }
   Task.sleep(1000);
  }
 }
}

Under the covers, Kilim works by weaving bytecode so as to control Task types and facilitate their safe interaction — specifically, Kilim’s weaver is looking for methods that throw the Pausable type (previous versions used the @Pausable annotation).

In the Husband class, a few things are going on — first, the instance waits for a message from a shared mailbox. In this case, I used a blocking get call (as in reality, a husband really doesn’t do anything else but waits for orders (I mean requests) from his wife). When a message is picked up, the instance checks to see if it was intended for it (in many cases, wife instances can communicate with children types sending messages like “clean your room” or “brush your hair”).

Once a Message type (which, in this case, is not a Kilim type) is determined to be sent to a Husband instance, the Husband type replies appropriately by creating a Message and placing it into the mailbox instance. Finally, the sleep call is just placed to facilitate reading console output.

The Wife class is similar (expect that she doesn’t wait to listen…). Like the Husband instance, this class creates a Message (in the form of a Honey Do) and sends it off via the shared MailBox; however, she doesn’t wait around — that is, the instance uses the putnb call, which is non-blocking. What’s more, she’ll also attempt to see if anything is in the mailbox for her, but in her case, she also uses a non-blocking call (getnb) like so:

import kilim.Mailbox;
import kilim.Pausable;
import kilim.Task;

public class Wife extends Task {

 private Mailbox<Message> mailbox;

 public Wife(Mailbox<Message> mailbox) {
  super();
  this.mailbox = mailbox;
 }

 @Override
 public void execute() throws Pausable, Exception {
  while (true) {
   Message request = new Message(Message.HUSBAND,
    "Please do x, y, and z today, husband.");

   mailbox.putnb(request);
   Task.sleep(1000);
   Message msg = mailbox.getnb(); // no block

   if (msg != null && msg.getReceipient() == Message.WIFE) {
    System.out.println("Wife hears: " + msg.getMessage());
   }
  }
 }
}

As you can see, if there is a message waiting for her, she’ll hear it, otherwise, she moves on and requests her husband to do something else.

Lastly, everything is coordinated by a simple driver class containing a main method like so:

import kilim.Mailbox;
import kilim.Task;

public class HoneyDo {

 public static void main(String[] args) {

  Mailbox<Message> sharedMailbox = new Mailbox<Message>();

  Task wife = new Wife(sharedMailbox);
  Task husband = new Husband(sharedMailbox);

  husband.start();
  wife.start();
 }
}

Note how a Mailbox instance is created for my custom Message type; what’s more, the sharedMailbox is then shared between both the wife and husband instances. Lastly, things are started via the start method.

Running this hip example yields the following output (remember, your exact output will most likely look different; however, the logic sequence of activities will line up. That is, the wife requests things be done and the husband responds with “yes, dear”, which the wife hears).

 Husband listening...
 Husband hears: Please do x, y, and z today, husband.
 Husband says: Yes, dear
 Husband listening...
 Wife hears: Yes, dear
 Husband hears: Please do x, y, and z today, husband.
 Husband says: Yes, dear
 Husband listening...
 Wife hears: Yes, dear
 Husband hears: Please do x, y, and z today, husband.
 Husband says: Yes, dear
 Wife hears: Yes, dear
 Husband listening...
 Husband hears: Please do x, y, and z today, husband.
 Husband says: Yes, dear
 Husband listening...
 Wife hears: Yes, dear
 Husband hears: Please do x, y, and z today, husband.
 Husband says: Yes, dear
 Wife hears: Yes, dear
 Husband listening...
 Husband hears: Please do x, y, and z today, husband.
 Husband says: Yes, dear

The actor model facilitates concurrent programming by allowing a safer mechanism for message passing between processes (or actors). Implementations of this model vary between languages and frameworks — I suggest checking out Erlang‘s actors followed by Scala‘s as each implementation is quite neat given their respective syntax.

In the end, if you want to leverage plain Jane Java actors, then have a look at Kilim (or one of the other frameworks available) — just make sure you’ve finished your honey do list first, man.

Next »