The Disco Blog

Can you dig it?

Elasticsearch in a Box

Are you looking to get going with Elasticsearch as quickly as possible without having to worry about installing Java or Elasticsearch itself? Are you looking for a repeatable and automated mechanism for bringing up Elasticsearch instances for developmental and or testing purposes? While there’s certainly a number of Elasticsearch-as-a-platform service providers out there, there’s one other option: use Elasticsearch-in-a-box.

Elasticsearch-in-a-box is a freely available Vagrant base box. What that means is that you can quickly fire up and tear down an Elasticsearch environment with simple commands like vagrant up and vagrant destroy.

Provisioning Ubuntu With Java in 3 Steps

As I’ve written about before, Vagrant is handy tool for creating localized VMs. It’s a lot like firing up EC2 images, but, for the most part, things are localized (you can, by the way, use Vagrant to fire up EC2 images). If you’ve ever used VMWare before, its the same thing, except Vagrant is free. You can create VMs of various operating systems, fire them up, and tear them down all with ease.

Vagrant plays nicely with hip DevOps frameworks like Chef and Puppet and if your installations require a number of components, then these tools are defiantly the way to go. Sometimes, however, a simple Bash script is good enough as in the case for auto-installing some base component, like Java, Node.js or Ruby.

Using Vagrant’s configuration file, aptly dubbed Vagrantfile, you can instruct a VM instance to run a series of steps – these steps can be simple shell scripts, Chef cookbooks, or the Puppet equivalent.

SSH & Vagrant

Vagrant is a handy tool for creating VMs. It’s a lot like firing up an EC2 instance, but in Vagrant’s case, everything is localized. And best of all, it’s free.

I tend to favor Ubuntu as my preferred flavor of linux; consequently, all production EC2 instances use a customized Ubuntu AMI. Testing various aspects of this system with various software libraries, however, is initially tested locally using Vagrant VMs. What’s more, you can install localized VMs of other operating systems ranging from Debian to OpenSuse to Heroku’s Cedalon.

AWS EBS in 4 Steps

When you fire up an AWS AMI, you are given a small partition of disk space that survives reboots. For example, the base Ubuntu AMI I tend to favor comes with an 8GB primary partition; however, 8GB is often not enough, especially if you’re running a database or something that requires a lot of disk space.

If you poke around on an AMI instance, you’ll notice some AMI instances will have additional partitions and in many cases, these partitions will be huge; nevertheless, they’re transient and any data on those disks will disappear after a reboot.

Accordingly, if you need to gain some more permanent space on an AMI instance, you’ll need to leverage an Elastic Block Store (or EBS), which is basically a permanent hard disk that you can attach to a running AMI instance. The data on an EBS will survive a reboot.

All Other Metrics Are Useless

When it comes to queues, whether they’re implemented as JMS, database tables (i.e. what Ruby’s Delayed::Job uses for a queue), or even Amazon’s SQS, the most common metric used to evaluate the state of a queue is its length. In essence, one derives an efficiency metric based upon how many messages are residing in a queue at any given time. If there are just a few messages, the queue is operating efficiently. If there are numerous messages, things are inefficient and alarms must be sounded.

The Significance of HTML5

HTML5 is important for three reasons. And its importance starts with the end of browser plugins. That’s right. With HTML5, rich media aspects that were formally handled by plugins (think Flash), are now built-in. That’s why there are new media tags like <audio> and <video>.

Think about it for a second – when’s the last time you visited a site on your tablet that asked you to install a plugin? Never.

That’s also why some older plugin laden sites do not work on your mobile device. Remember the whole ”Steve Jobs no flash” kerfuffle years ago? Yep, Mr. Jobs was adamant that the iPhone would support HTML5 and not fall into the plugin trap. Incidentally, Google and other major vendors have since followed suit. HTML5 has the support of all major browser vendors now – Apple, Google, Firefox, Opera, and yes, even Microsoft.

Ahoy There Callbacks!

Because it’s my bag, I like JavaScript. In fact, I’ve grown to love JavaScritp’s asynchronous callback oriented style of programming. Consequently, when I find myself in a non-JavaScript environment, say, like Java, I tend to miss using callbacks.

The good news is that you can emulate asynchronous callbacks in Java. In fact, I did just that recently with a library I’ve dubbed Ahoy!, which is an asynchronous SQS adapter for AWS’s Java SQS library.

App Signing: iOS Versus Android

Apple’s app signing process can be a real pain-in-the-neck; nevertheless, it works to keep apps trusted. You know when you download a Bank of America app from iTunes that it’s the real thing. You know that the app comes from the Bank of America.

On the other hand, Android allows apps to be self-signed. This has a fundamental flaw: an Android developer can claim to be anyone they want, including Bank of America. Thus, when you download an app from an Android App store, there’s a real possibility that the app was submitted by a charlatan. What’s more, those charlatan apps can actually be malicious!

I Like My ElasticSearch a La Node.js

While ElasticSearch is easy enough to work with via its RESTful HTTP API, there are myriad client libraries available in almost every conceivable programming language. If Node.js is your language of choice, then there’s at least two actively supported libraries available.

My favorite is dubbed, albeit rather dully, ”Elastic Search Client”, but don’t let the library’s unimaginative name fool you: this is a handy library that allows you to do everything you could do via cURL with the added benefit of JavaScript callbacks. Best of all, you can use the Node Elastic Search Client in Coffeescript, which is a handy language that makes JavaScript less verbose and that ultimately compiles into JavaScript.