Read my latest article: Launching Rails projects, an open call for lessons learned (posted Tue, 23 Jun 2009 17:33:00 GMT)

Master/Slave Databases with Ruby on Rails

Posted by Robby Russell Thu, 15 Nov 2007 22:02:00 GMT

6 comments Latest by ahuang@azukisystem.com Mon, 29 Dec 2008 16:15:16 GMT

Not terribly long ago, I announced Active Delegate, which was a really lightweight plugin that I developed to allow models to talk to multiple databases for specific methods. The plugin worked great for really simple situations, like individual models.. but when it came time to test with associations it fell apart. I haven’t had a chance to work on any updates and knew that it was going to take more work to get it going.

Earlier this week, we helped one of our bigger clients launch their new web site1. For the deployment, we needed to send all writes to a master database and a reads to slaves (initial deployment is talking to almost 10 slaves spread around the globe!). We needed something to get integrated quickly and decided to ditch Active Delegate for the time being and began looking at the following options.

I spoke with Rick Olson2 and he pointed me to a new plugin that he hasn’t really released yet. So, I’m going to do him a favor and announce it for him. Of course… I got his permission first… ;-)

Announcing Masochism!

Masochism3 is a new plugin for Ruby on Rails that allows you to delegate all writes to a master database and reads to a slave database. The configuration process is just a few lines in your environment file and the plugin takes care of the rest.

Installing Masochism

With piston, you can import Masochism with:


  $ cd vendor/plugins
  $ piston import http://ar-code.svn.engineyard.com/plugins/masochism/

You can also install it with the old-fashioned way:


  $ ./script/plugin install -x http://ar-code.svn.engineyard.com/plugins/masochism/

Configuring Masochism

The first thing that you’ll need to do is add another database connection in config/database.yml for master_database. By default, Masochism expects you to have a production database, which will be the read-only/slave database. The master_database will be the connection details for your (you guessed it…) master database.


# config/database.yml  
production:
  database: masochism_slave_database
  adapter: postgresql
  host: slavedb1.hostname.tld
  ...

master_database:
  database: masochism_master_database
  adapter: postgresql
  host: masterdb.hostname.tld
  ...

The idea here is that replication will be handled elsewhere and your application can reap the benefits of talking to the slave database for all of it’s read-only operations and let the master database(s) spend their time writing data.

The next step is to set this up in your environment file. In our scenario, this was config/environments/production.rb.



# Add this to config/environments/production.rb
config.after_initialize do 
  ActiveReload::ConnectionProxy.setup!    
end



Voila, you should be good to go now. As I mentioned, we’ve only been using this for this past week and we’ve had to address a few problems that the initial version of the plugin didn’t address. One of our developers, Andy Delcambre, just posted an article to show how we had a problem with using ActiveRecord observers and masochism, which we’re sending over a patch for now.

As we continue to monitor how this solution works, we’ll report any findings on our blog. In the meantime, I’d be interested in knowing what you’re using to solve this problem. :-)

1 Contiki, a cool travel company we’re working with

2 Rick just moved to Portland… welcome to stump town!

3 The Masochism plugin README

Subscribe to my RSS feed Enjoying the content? Be sure to subscribe to my RSS feed.
Comments

Leave a response

  1. Avatar
    texxas Fri, 16 Nov 2007 20:57:39 GMT

    I would like to see, that write / read are done on master, only asynchronous reads are done from slave. In PG you can use “update / returning” SQL statement to stay in sync with DB. With MySQL – how to get last id or anything like that? From slave? What abut replication lag time? Nice to know that there is something like that, but be careful about data integrity.

  2. Avatar
    Marston A. Sat, 17 Nov 2007 14:29:27 GMT

    Hey Robby,

    This looks really great. Quick question: Is this something that works along side projects like lbpool (http://code.tailrank.com/lbpool) or sqlrelay or does it replace most that functionality all by itself in the application layer?

    For example, if you had a 2 server multi-master setup and say a slave pool of 6 read servers could this plugin handle all the load balancing of writes to the 2 masters and reads to the 6 slaves?

    What about fail-over? Will it simply skip servers that are down and can’t be reached (for whatever reason).

  3. Avatar
    Jonathan Conway Sat, 17 Nov 2007 18:25:47 GMT

    Good to see more work done in this area. I’ll be sure to check out Masochism as soon as I have some time.

    I’ve been using Mysql Proxy(http://forge.mysql.com/wiki/MySQL_Proxy) for the past month in production and have found it great.

    It’s a very flexible and powerful solution if you’re able to use MySQL, as it allows me to do things like transparently proxy all writes to a master and reads to a slave. It also does load balancing and failover.

  4. Avatar
    Chu Yeow Fri, 23 Nov 2007 03:04:02 GMT

    I was using MySQL Replication Adapter before I saw this blog post. The main problem I had with it that it only works on gem Rails due to a change in AR on edge and I was having problems getting it to reconnect to the slave servers should the connection time out. But other than that it worked perfectly.

    Anyway, I tried out Masochism and it’s really much friendlier integration-wise (and look at the conciseness of the plugin code).

    I did encounter a bug though where it was still using the slave database to execute an INSERT for a habtm association. If you look at the habtm code you can see around line 102 where it’s using AR::B#execute to perform an INSERT. This bypasses the delegation specified in connection_proxy.rb and tries to INSERT into the slave database.

    I’m gonna see if I can convince Rick to delegate #execute as well as a safety net.

  5. Avatar
    Chu Yeow Fri, 23 Nov 2007 03:13:00 GMT

    You know, I have no idea how to contact Rick about Masochism other than emailing him directly or posting on his blog. Is there a bug tracker somewhere? Or could you do me a favor and point this out to Rick? Thanks :)

  6. Avatar
    ahuang@azukisystem.com Mon, 29 Dec 2008 16:15:16 GMT

    when I try to install the plugin, it needs username/password for Authentication realm: http://ar-code.svn.engineyard.com:80> Engine Yard SVN Cluster: ar-code. How to get that info?

Share your thoughts... (really...I want to hear them)

Comments