December 19, 2014
Hot Topics:

A Rails Cloud Implementation Using CouchDB and Heroku

  • January 27, 2010
  • By Mark Watson
  • Send Email »
  • More Articles »

CouchDB is an interesting implementation of a schema-less data store. It supports client applications through HTTP and a REST-style API. I don't use CouchDB's support for replication, using it instead to store structured data. While I sometimes run CouchDB locally during development, I like to keep CouchDB running on a low-cost VPS instance that I access interactively and from client applications. (I will refer to data instances as "documents" in this article.)

When you have mastered how to use the Heroku platform to deploy and manage Rails web applications, you can choose CouchDB to use on the backend. Using a simple Rails app, Note Taker with Search (see the previous article in this series, "Deploying a Rails Application to Heroku"), I will demonstrate how to use CouchDB, based on my own use of this data storage and management tool. (The code download for this article contains all the examples in the directory note_taker_couchdb, and you should extract them and work along with me through every example.) I will use a combination of the APIs in the couchrest gem with direct REST-style calls using the simplehttp gem.

Because CouchDB is built so well on top of HTTP and REST, it often seems most natural to me to simply make direct REST calls to a CouchDB service and use the json gem to process the returned JSON data—"close to the metal" instead of using the APIs provided by the couchrest gem.

A particularly interesting CouchDB attribute is its versioning system. CouchDB never discards old versions after adding new data. Rather, it creates new versions of documents by reusing the ID of a document and updating the document's version number. Old versions are left intact. If you are concerned about wasted disk space, don't be: CouchDB also uses a lot of disk storage for indexes, and disk space is inexpensive.

You create indexes on documents by writing map/reduce functions in JavaScript and adding them to databases. The map/reduce functions that you write define what data can be searched for efficiently. The general topic of writing CouchDB map/reduce functions is beyond the scope of this article, but I will walk you through the function I defined for the next example. The database for this example is notes. I have only one type of data document in the notes database, and the document type is also called notes. In all further discussions, whenever I refer to notes I mean documents.

I write map/reduce functions for two types of views on the notes documents:

  1. words: used in note titles and content
  2. users: defined by user IDs in notes that specified who wrote the note

In this example, you are allowed to see only notes that have the same user ID as that set in a session when you login to this web application. CouchDB uses JSON to store data, so your notes documents will be stored internally as JSON. Map/reduce functions are also expressed as JSON with the JavaScript code in embedded strings. I don't much like this notation, but it is only a minor annoyance. Document IDs are specified by the hash key _id, and documents containing map/reduce JavaScript functions for defining views have ID names starting with _design; for example:

{
   "_id": "_design/notes",
   "language": "javascript",
   "views": {
       "words": {
           "map": "function(doc) { var s = doc.title + doc.content; 
var words = s.replace(/[0123456789!.,;]+/g,' ').toLowerCase().split(' ');
for (var word in words) { emit(words[word], doc._id); } }" } "users": { "map": "function(doc) { if (doc.user_id) { emit(doc.user_id, null); }}" } } }


Neither of these views required a reduce function. The function emit writes a key/value pair. It is fairly common to see null for either the key or value. In the view users, I only need all user IDs as keys because I specify a null value for each key/value pair; I only need the keys. Interestingly, the user IDs for the view are culled from the notes documents and there is no separate document type for users.

To help you understand the views created by these JavaScript functions, take a look at some examples of REST calls to access the two views I just created (note that %22 is a " (quotation mark) character in URL encoding):

  1. To get all words: http://localhost:5984/notes/_view/notes/words
  2. To search for documents containing a specific word: http://localhost:5984/notes/_view/notes/words/?key=%22java%22
  3. To list the first 11 docs (including views): http://localhost:5984/notes/_all_docs?limit=11
  4. To get note docs by user ID = "1": http://localhost:5984/notes/_view/notes/users/?key=%221%22

Numbers 2 and 4 are the most interesting, because they filter on specific key values. Also, notice in example number 3 that although the query would return all documents of type notes, I set a limit of returning 11 documents.

Author's Note: Using CouchDB seems natural to me because it is built with tools and concepts that I know, such as REST-style calls and JSON storage. I have been using CouchDB for almost a year, and unlike simpler key/value stores like memcached, Tokyo Cabinet, and Redis (which does offer some structure like lists and sets), document-oriented data stores like CouchDB are a more natural fit for most of my work. That said, I try to choose the best tools for each specific job and you obviously should too.

In all these examples, the returned data is in JSON format. CouchDB provides a web interface called Futon (see Figure 1 for a screenshot of me inspecting the document that defined the map/reduce functions for the two views I need in this example).


Figure 1. Using Futon to Inspect Two JavaScript Views:
Here is a screenshot of me inspecting the document that defined the map/reduce functions for the two views.

At the bottom of the screenshot, I have nine versions of the implementations of these views. Futon makes it easy to go back and review changes in old versions. The screenshot in Figure 2 shows an edit view in Futon that allows you to modify a document and save it as a new version:


Figure 2. Using Futon to Edit One JavaScript View:
Here is an edit view in Futon that allows you to modify a document and save it as a new version.

The screenshot in Figure 3 shows me using Futon to view a note. Notice that there are no data items for "words." Those are defined in an index and show themselves only when the user performs a search.


Figure 3. Inspecting a Note Document:
Here is a screenshot of me using Futon to view a note.

I seldom use Futon for editing or creating documents, although I did use Futon to define my views. I write almost all of my CouchDB client code in Ruby.

Now you can look at the changes you need to make to the MongoDB-based web application (from the previous article in this series) to use CouchDB instead.

Require three gems in your environment.rb file:

  config.gem 'postgres'
  config.gem 'couchrest'
  config.gem 'simplehttp'


Also set two global variables at the end on your environment.rb file:

# setup for CouchDB
COUCHDB_HOST = ENV['COUCHDB_RUBY_DRIVER_HOST'] || 'localhost'
COUCHDB_PORT = ENV['COUCHDB_RUBY_DRIVER_PORT'] || 5984


Most of the code changes are in the Notes model class. First, notice that this Notes class is not derived from ActiveRecord:

class Note
  attr_accessor :user_id, :title, :content
  def to_s
    "note: #{title} content: #{content[0..20]}..."
  end


Using mostly low-level, REST-style calls to CouchDB, I will manually implement the behavior in the ActiveRecord version from the PostgreSQL-backed example (Part I) and the MongoRecord::Base version from the MongoDB-backed example (Part II).

The next method is used to create a new note document. This code is simpler than the MongoDB article (where I had to create a document attribute that was a list of words in the document), but you pay for some of this simplicity by having to write the JavaScript view functions. Here, I use the higher-level save_doc API from the couchrest gem:

  def Note.make user_id, title, content
    @db ||= CouchRest.database("http://#{COUCHDB_HOST}:#{COUCHDB_PORT}/notes")
    @db.save_doc({'user_id' => user_id.to_s, 'title' => title, 'content' => content})['id']
  end


The next method implements a search function. I tokenize the search string and for each token make a REST-style call to get all of the document IDs that contain the word. These results are stored in the hash table score_hash (keys are the document IDs, and the values are counts of how many times a search token is found in the corresponding document). I sort the hash table by value and return the documents in JSON hash table format in sort order:

  def Note.search query
    @db ||= CouchRest.database("http://#{COUCHDB_HOST}:#{COUCHDB_PORT}/notes")
    tokens = query.downcase.split
    score_hash = Hash.new(0)
    tokens.each {|token|
      uri = "http://localhost:5984/notes/_view/notes/words/?key=%22#{token}%22"
      JSON.parse(SimpleHttp.get(uri))['rows'].each {|row| score_hash[row['value']] += 1}
    }
    score_hash.sort {|a,b| a[1] <=> b[1]}
    score_hash.keys.collect {|key| @db.get(key)}
  end


Note: This implementation of method search would be very inefficient for search strings with many words, because a REST call would be made for each search word. Compare this to the MongoDB version of method search, where a single call is made and the entire query is performed on the server (in fast C++ code).

The next method returns all notes in the data store with a given user ID. I build a GET request URI and then use the simplehttp and json gems to get the documents as an array of JSON hash tables:

  def Note.all user_id
    @db ||= CouchRest.database("http://#{COUCHDB_HOST}:#{COUCHDB_PORT}/notes")
    uri = "http://localhost:5984/notes/_view/notes/users/?key=%22#{user_id}%22"
    JSON.parse(SimpleHttp.get(uri))['rows'].collect {|hash| @db.get(hash['id'])}
  end


The following method returns a note with a specific ID. In contrast to the last method, I use a low-level API from the couchrest gem instead of building a request URI and manually performing the REST call:

  def Note.find id
    puts "** Note.find id=#{id}"
    @db ||= CouchRest.database("http://#{COUCHDB_HOST}:#{COUCHDB_PORT}/notes")
    @db.get(id)
  end
end


The controller code is almost identical to the first two Rails examples in this article. Calling the search method you just saw performs the search:

  notes = Note.search(params[:search])


All notes with a specific user ID are found and passed to the scaffold view:

  @notes = Note.all(session['user_id'])



Tags: Ruby on Rails, Ruby, Cloud, Heroku, CouchDB



Page 1 of 2



Comment and Contribute

 


(Maximum characters: 1200). You have characters left.

 

 


Enterprise Development Update

Don't miss an article. Subscribe to our newsletter below.

Sitemap | Contact Us

Rocket Fuel