Robby on Rails: Tag activerecord

Master/Slave Databases with Ruby on Rails

Robby Russell — Thu, 15 Nov 2007 16:02:00 -0600

Not terribly long ago, I announced Active Delegate, which was a really lightweight plugin that I developed to allow models to talk to multiple databases for specific methods. The plugin worked great for really simple situations, like individual models.. but when it came time to test with associations it fell apart. I haven’t had a chance to work on any updates and knew that it was going to take more work to get it going.

Earlier this week, we helped one of our bigger clients launch their new web site¹. For the deployment, we needed to send all writes to a master database and a reads to slaves (initial deployment is talking to almost 10 slaves spread around the globe!). We needed something to get integrated quickly and decided to ditch Active Delegate for the time being and began looking at the following options.

I spoke with Rick Olson² and he pointed me to a new plugin that he hasn’t really released yet. So, I’m going to do him a favor and announce it for him. Of course… I got his permission first… ;-)

Announcing Masochism!

Masochism³ is a new plugin for Ruby on Rails that allows you to delegate all writes to a master database and reads to a slave database. The configuration process is just a few lines in your environment file and the plugin takes care of the rest.

Installing Masochism

With piston, you can import Masochism with:


  $ cd vendor/plugins
  $ piston import http://ar-code.svn.engineyard.com/plugins/masochism/

To learn more about piston, read Every Second Counts with a Piston in your trunk

You can also install it with the old-fashioned way:


  $ ./script/plugin install -x http://ar-code.svn.engineyard.com/plugins/masochism/

Configuring Masochism

The first thing that you’ll need to do is add another database connection in config/database.yml for master_database. By default, Masochism expects you to have a production database, which will be the read-only/slave database. The master_database will be the connection details for your (you guessed it…) master database.


# config/database.yml  
production:
  database: masochism_slave_database
  adapter: postgresql
  host: slavedb1.hostname.tld
  ...

master_database:
  database: masochism_master_database
  adapter: postgresql
  host: masterdb.hostname.tld
  ...

The idea here is that replication will be handled elsewhere and your application can reap the benefits of talking to the slave database for all of it’s read-only operations and let the master database(s) spend their time writing data.

The next step is to set this up in your environment file. In our scenario, this was config/environments/production.rb.


# Add this to config/environments/production.rb
config.after_initialize do 
  ActiveReload::ConnectionProxy.setup!    
end

Voila, you should be good to go now. As I mentioned, we’ve only been using this for this past week and we’ve had to address a few problems that the initial version of the plugin didn’t address. One of our developers, Andy Delcambre, just posted an article to show how we had a problem with using ActiveRecord observers and masochism, which we’re sending over a patch for now.

As we continue to monitor how this solution works, we’ll report any findings on our blog. In the meantime, I’d be interested in knowing what you’re using to solve this problem. :-)

¹ Contiki, a cool travel company we’re working with

² Rick just moved to Portland… welcome to stump town!

³ The Masochism plugin README

Multiple Database Connections in Ruby on Rails

Robby Russell — Fri, 05 Oct 2007 17:54:00 -0500

We have a client that already has some database replication going on in their deployment and needed to have most of their Ruby on Rails application pull from slave servers, but the few writes would go to the master, which would then end up in their slaves.

So, I was able to quickly extend ActiveRecord with just two methods to achieve this. Anyhow, earlier today, someone in #caboose asked if there was any solutions to this and it prompted me to finally package this up into a quick and dirty Rails plugin.

Introducing… Active Delegate!

To install, do the following:


cd vendor/plugins;
piston import http://svn.planetargon.org/rails/plugins/active_delegate

Next, you’ll need to create another database entry in your database.yml.


login: &login
  adapter: postgresql
  host: localhost
  port: 5432

development:
  database: rubyurl_development
  <<: *login

test:
  database: rubyurl_test
  <<: *login

production:
  database: rubyurl_servant
  <<: *login

# NOTICE THE NEXT ENTRY/KEY
master_database:
  database: rubyurl_master
  <<: *login

At this point, your Rails application won’t talk to the master_database, because nothing is being told to connect to it. So, the current solution with Active Delegate is to create an ActiveRecord model that will act as a connection handler.


  # app/models/master_database.rb
  class MasterDatabase < ActiveRecord::Base
    handles_connection_for :master_database # <-- this matches the key from our database.yml
  end

Now, in the model(s) that we’ll want to have talk to this database, we’ll do add the following.


  # app/models/animal.rb
  class Animal < ActiveRecord::Base
     delegates_connection_to :master_database, :on => [:create, :save, :destroy]
  end

Now, when your application performs a create, save, or destroy, it’ll talk to the master database and your find calls will retrieve data from your servant database.

It’s late on a Friday afternoon and I felt compelled to toss this up for everyone. I think that this could be improved quite a bit, but it’s working great for the original problem that needed to be solved.

If you have feedback and/or bugs, please send us tickets.

Q&A: ActiveRecord Observers and You

Robby Russell — Sat, 28 Apr 2007 17:28:00 -0500

Yesterday, I wrote a short post titled, Observers Big and Small, about using Observers in your Rails applications.

The following questions were raised in the comments.

When should I use an Observer?

Eric Allam asks…

“Why not just use ActiveRecord callback hooks instead of Observers? Are Observers more powerful or is it just a matter of preference?”

Eric, this is an excellent question. I’d say that a majority of the time, using the ActiveRecord callbacks in your models is going to work for your situation. However, there are times that you want the same methods to be called through callbacks. For example, let’s take a recent problem that we used an observer to solve.

Graeme is working on implementing Ferret into a project that we’re developing for a client. With the use of Ferret, we can index and later search through content over several objects into a format that makes sense for our implementation goals. Each time an object is created and updated, we have to update our Ferret indexes to reflect these changes. The most obvious location that we can call our indexing methods is in each models’ callbacks, but this violates the DRY[1] principle. So, we created an Observer, which observes each of the models that need these methods to be called. In fact, as far as we’re concerned, the fact that we’re indexing some of its data, is none of its business. We only want our models to be concerned with that they’re designed to be concerned about. We may opt to change our indexing solution in the future and we’d just need to rethink that at the Observer level and not change anything about the business logic in our models.

This is the sort of scenario when using an Observer makes great sense in your application.

Logging from an Observer

Adam R. asks…

“I’d also like the ability to use the logger from within an observer, but that’s another issue.”

I assume that you are referring to the logger method? I always forget to even use that method. I do know that the following works just fine in an Observer.


class IndexObserver < ActiveRecord::Observer
  observer Article, Editorial, BlogPost, ClassifiedAd

  def after_save(model)
    RAILS_DEFAULT_LOGGER.warn("Every single day. Every word you say. Every game you play. Every night you stay. I'll be watching you.")
    # execute something fun
  end
end

This will output to your log file without any problem.

This reminded me of when I used to want to log from Unit Tests.

(few minutes later)

Okay, I just attempted to use logger from an Observer and you’re right… it doesn’t currently work. There is a simple fix though, just extend ActiveRecord::Observer to add a logger method like so and require it in config/environment.rb (much like I did in with unit tests).


# lib/observer_extensions.rb
class ActiveRecord::Observer
  def logger
    RAILS_DEFAULT_LOGGER
  end
end

This will give you a solution to that problem.


class FooObserver < ActiveRecord::Observer
  observer Foo

  def after_save(model)
    logger.warn("I wonder if the #{address.class} knows that I've been watching it all along?")
  end
end

Observers Spy for Us

Most often, I look at Observers as being the guys that I hire to spy on my models. I don’t want my models to know that they’re being spied on and I’d like to keep it that way. They don’t solve all of our problems and it’s easy to overuse them. However, I have found several cases that they made a lot of sense and most of those cases have been where we’ve had the same things occurring in our model’s callbacks.

If you have other questions related to Observers, feel free to let me know. If you’re already using Observers, perhaps you could post a comment and/or blog post response with an example of when and how you use Observers in your Rails applications.

¹ Don’t Repeat Yourself

Observers Big and Small

Robby Russell — Fri, 27 Apr 2007 13:15:00 -0500

My colleague, Gary, keeps a stack of Ruby and Rails books on his desk and was implementing an Observer into a client project. It appears that the Agile Web Development with Rails book is still encouraging people to do the following in order to load an Observer.


# app/models/flower_observer.rb
class FlowerObserver < ActiveRecord::Observer
  observe Flower

  def after_create(model)
    # model.do_something!
  end
end

# controller(s)
class FlowerController < ApplicationController
  observer :flower_observer
end

What is wrong with this approach?

Well, in order for your Observer to be used, the model(s) callbacks that it is observing need to be triggered through a controller. If you end up writing any scheduled rake tasks, your observer will not be called. In my opinion, the controller shouldn’t know this much about the model. In fact, the model doesn’t even really know about it’s observer… so why should a controller?

This was actually changed a long time ago (I previously blogged about a different solution here) and the Rails docs for ActiveRecord::Observer are currently correct.

Observers in the Environment

If you open up a recent version of config/environment.rb, you notice in the comments the following.


  # Activate observers that should always be running
  # config.active_record.observers = :cacher, :garbage_collector

Take a moment to go ahead and specify which observer(s) you’d like to load into your Rails environment.

config.active_record.observers = :flower_observer

Then you can remove your observer calls in all your controllers, because that’s not where you should be defining them.

Also, if you’re not using Observers yet, I’d really encourage you to consider reading up on them and giving them a try.

Where did my observer go?

Robby Russell — Mon, 27 Feb 2006 23:05:00 -0600

I was playing around with an application from script/console and all of a sudden… my Observers just stopped working. It took me a while to figure out that when I would call Dispatcher.reset_application!... it wouldn’t reload config/environment.rb where I have defined the observers.

I really don’t like that this is the current implementation for invoking Observers and adding them to the controllers is not kosher with my (do-it-in-script-console) approach.

The current solution?


# app/models/foo_bar.rb
class FooBar < ActiveRecord::Base
end

FooBarObserver.instance

... there needs to be a better way!

Rails Migrations and PostgreSQL Constraints

Robby Russell — Fri, 11 Nov 2005 10:42:00 -0600

A question was posed on the Rails mailing list concerning how one would go about adding CONSTRAINTs to the database tables with ActiveRecord::Migration.

One argument was raised stating that it is easier to handle these in plain SQL schema files. I disagree. :-)

Migrations to the Rescue

Databases evolve and I have recently found the Migration structure to be perfect for handling iterations and schema changes. Using the #execute method has helped move more of my code into the Ruby/Rails framework… and that just makes things easier to manage in the long-run. This is the approach that we are using at PLANET ARGON with some of our current client projects.

# db/migrate/6_add_foreign_key.rb
class AddForeignKey < ActiveRecord::Migration
  def self.up
    execute "ALTER TABLE bees ADD CONSTRAINT beehive_id_fkey FOREIGN KEY
(beehive_id) REFERENCES beehives (id);"
  end

  def self.down
    execute "ALTER TABLE bees DROP CONSTRAINT beehive_id_fkey;"
  end
end

This gives us an easy way to use the standard, #create_table syntax for building our tables with Ruby… and then we can slap these constraints on later.

This would add the constraints…


rake migrate VERSION=6

...run tests…


rake

...roll back…


rake migrate VERSION=5

I have found that this approach is really useful with testing in Rails. When I think that I have everything working great (without CONSTRAINTS in PostgreSQL), I run another migration to add a bunch of foreign key and data constraints to the tables and… run my tests again.

Let’s give Active Record a Hug

This has helped me gain some trust in Active Record while still giving me that comforting feeling that PostgreSQL is acting as the body guard for my data.

Even if you don’t end up using Migrations to handle these types of database schema changes, I would highly suggest that you model your implementation after this. I’ve worked with many database schemas and this just makes it easy to add your new change and run one command to commit it to the database.

...and now I go play with beehives…

Einstein on Rails

Robby Russell — Fri, 04 Nov 2005 18:59:00 -0600

Einstein would like Active Record… I think…

Have a great weekend… !

Migrating from MySQL to PostgreSQL in 60 seconds (with Rails)

Robby Russell — Sat, 29 Oct 2005 14:23:00 -0500

I do quite a few MySQL to PostgreSQL migrations for clients of PLANET ARGON and over the years I have gone through different methods of handling this procedure. Typically, it involved exporting data, re-importing it through some shell script that does its best to take into consideration the slight differences between the two databases.

While working on a recent project that was to be migrated from MySQL to PostgreSQL, but after some Refactoring had been done to the code base. I had been meaning to spend a few hours coming up with some simplified process of taking data from MySQL and replicating it to a new PostgreSQL database, while maintaining referential integrity.

My original idea was to build a script that referenced two seperate databases and then copied data out and inserted it into the new db after performing a few data changes. This seemed like too much work and I knew that I should be able to harness the power of Ruby and Rails in my process. My next thought? RailsFS.

In a perfect world, I would mount one instance of the application with a mysql database and another with a postgresql database… and just copy the yaml files to the postgresql and bam, it’d work, right? Well, it doesn’t quite work that way. So, I ruled out that idea.

Then I had another idea. I would build a task in my Rakefile that… imported each of those YAML files into PostgreSQL!

We can now type in the following commands:

This first creates my PostgreSQL database tables using Migrations.

rake migrate VERSION=1

then…

rake mysql2pgsql

mysql2pgsql runs…and approx 60 seconds later, I had an exact copy of the MySQL database in PostgreSQL. But wait! PostgreSQL has SEQUENCES… so I then run:

rake migrate VERSION=2

...and my SEQUENCES are ALTER’d.

Yes, I will post some code in the near future. But all I am doing is basically running through all the YAML files that are available from RailsFS and adding them into the new database. It automatically figures out the model name…and is fairly generic at the moment. However, I’d like to spend a bit more time stress-testing it before I post it. :-)

I want to try to work on a RailsFS-less version as well so that I can run this on my PowerBook.

Until next time… have fun!

Using Named Placeholders in Ruby

Robby Russell — Fri, 21 Oct 2005 00:53:00 -0500

Insert Hip Quote Here:

“In ancient times, hundreds of years before the dawn of history, an ancient race of people… the Druids. No one knows who they were or what they were doing… “ – Nigel Tufnel, Spinal Tap

Story Time…

Earlier, I was giving a customer of ours, Jared from CommunityWalk.com a quick tutorial on some of the features script/console… which lead to helping him with a SQL query. When I provided him with some working code he was curious about what I had done in the SQL query string that I was passing to find_by_sql.

WARNING

If you have ANY SQL queries that resembles the following, PLEASE READ THE REST OF THIS. :-)

values = params[:search]
RockLegend.find( :all, :conditions => "first_name = '#{values['first_name']}'" )

If you are doing that… then you are opening yourself up to some security problems. Let’s take a few minutes and discuss how you can make this more secure and still keep your code readable. (the best of both worlds!)

The ? Placeholder

Many of you are probably familiar with this approach…

RockLegend.find( :all, :conditions => ['first_name = ?', 'Nigel'] )

RockLegend.find( :all, :conditions => ['first_name = ? AND last_name = ?', 'Nigel', 'Tufnel'] )

You can pass it a hash as well.. and as long as you put everything in the same order as the ?s are placed… then all is well.

My only real problem with this approach is that it requires you to keep things in a specific sequential order… and who wants to keep track of that? So, I would like to recommend that you use named placeholders. Aside from that, it looks magical and I don’t like magical-looking code. I like easy to read code. :-)

Named Placeholders

If you already use these… you know how useful they can be in your SQL queries. If you haven’t seen them before… it’s because the Rails docs don’t really mention it and is something that comes from the underlying database library in Ruby. So what is so great about these?

Let’s first replace the above code with named parameters…

RockLegend.find( :all, :conditions => ['first_name = :first_name', { :first_name => 'Nigel' } ] )

We are passing a hash with a matching key to the conditions option. Pretty neat, right? In this case with just one placeholder we just increased the amount of code to do the same thing. So, it might always be the best solution… but it is easier to read.

Let’s try another with multiple keys in our hash… infact, we’ll build the hash prior to calling find.

values = { :first_name => 'Nigel', :last_name => 'Tufnel'}
RockLegend.find( :all, :conditions => ['first_name = :first_name AND last_name = :last_name', values ] )

It will happily match the hash keys to the named placeholders in the conditions string. Again, nothing terribly exciting…but it is easier to read.

Who cares about order? Not named placeholders!

Okay, let’s mix things up a bit…

values = { :last_name => 'Tufnel', :first_name => 'Nigel' }
RockLegend.find( :all, :conditions => ['first_name = :first_name AND last_name = :last_name', values ] )

The hash keys were not added in the same order… but it still works!

Okay, now for one last quick example (it’s late and I am tired…).

I have a search mechanism on a site that allows you to search for a string of text across multiple fields. So, I have one string… but several fields to compare against.

Here is a string that I will pass to the find method.

conditions = "role = :role AND (first_name ~* :str OR last_name ~* :str OR nick_name ~* :str)"

Note: this string is using PostgreSQL regular expressions... (~*).

Here is a hash that with that matches the keys, :str and :role

values = { :str => '^(Nigel|Tufnel)$', :role => 'Guitar' }

If you look above, you’ll see that the conditions string contains four named placeholders… but the hash only has two keys. With the ? placeholder, we would have to pass the same vaule three times… which isn’t any fun to read or maintain. ;-/

So, with our new friends, named parameters, we can call find (or any find-like method) using this technique for placeholders.

RockLegend.find( :all, :conditions => [ conditions, values ] )

...and hopefully this is useful to you. :-)

Have fun!

New Active Record Options for Associations

Robby Russell — Wed, 19 Oct 2005 18:00:00 -0500

Two months ago today I posted about a bug in Active Record. A bug that reminded me to remain cautious about how much trust I put into a database abstraction layer. I am happy to now say that this particular bug has been fixed, and I got to help! In the process, I also got to add some new features. (see my original rant, Active Record, I love U but I still trust my database server a tiny bit more.

I discovered this bug when I was working on a chapter in my book on Active Record. I’m known to gladly take advantage of database constraints and triggers, and it was when I decided to test my code without these constraints, I discovered the bug. “Hey, Active Record isn’t doing what it’s supposed to!” I’ve since had a number of people ask me if what a more pragmatic way to work around this issue is, rather than go my route by adding a constraint/trigger.

You can now DO AWAY with ON DELETE CASCADE! (sort of)

The new release of Rails 1.0 Release Candidate includes some new options for the has_many and has_one declarations.

Previously, you could do the following:

class Customer < ActiveRecord::Base
  has_many :orders, :dependent => true
end

This was supposed to nullify the dependent records, but it didn’t!

My patch not only fixes this, but also gives more control with what :dependent does. Now, you can pass the :dependent option to the has_many and has_one declarations with either :nullify or :destroy. This has a similar affect as ON DELETE CASCADE in those fancy RDBMs like PostgreSQL.

Let’s take a closer look at these new options:

has_one

A spider has_one web, and the web belongs_to one spider. If you destroy the spider, you would most likely want to destroy the web as well.

class Spider < ActiveRecord::Base
  has_one :web, :dependent => :destroy
end

On the other hand, in the case of a snail that has_one shell (and the one shell belongs_to the snail), if you destroy the snail, you may want to keep the shell. Remember, your crazy Aunt Ruby collects snail shells.

class Snail < ActiveRecord::Base
  has_one :shell, :dependent => :nullify
end

Now, for every snail we destroy, the shells, though once dependent on the snail, are now available for Aunt Ruby.

has_many

The same rules apply to the has_many association. Most people assume that if you destory a beehive that you would destroy all the bees. I’d like to think that they wander around until they find a new beehive to join. So, we can :nullify their relationship when the beehive is destroyed, thus making them homeless, but available for future hives.

class Beehive < ActiveRecord::Base
  has_many :bees, :dependent => :nullify
end

Sadly, some people might want to destroy the bees along with the destruction of the beehive. So, those people can pass the :dependent option, :destroy.

class Beehive < ActiveRecord::Base
  has_many :bees, :dependent => :destroy
end

Don’t worry, your usage of :dependent => true will now work, even though it wasn’t working before.

I’m going to try to put more trust into Active Record now, and I hope that this new addition to the library finds itself useful for you. :-)

Take a peak at the cool new features in the latest version of Rails and see the Active Record CHANGELOG for more information.

Enjoy!

Rails meets FuseFS

Robby Russell — Wed, 21 Sep 2005 15:28:00 -0500

_why has posted an entry on his blog which shows how to use the Ruby FuseFS library to… mount Active Record.


railsmnt]$ cat contact_type/1
--- !ruby/object:ContactType
attributes:
  name: Client
  id: "1" 

 railsmnt]$ for i in `ls`; do echo $i; done
contact
contact_type

Umm… I am… I dunno… (wow)

I’ve used FuseFS for mounting SSH… but found myself using SSHFS as a better alternative.

But… to mount Rails? This could get interesting.

UPDATE

A screenshot while browsing the directory with Konqueror…and opening the record with Kedit.

UPDATE #2 If you’re hosting at PLANET ARGON, you can try this out with your account. Both, FUSE and Ruby-FuseFS are installed on all the servers.

PostgreSQL sequences in Rails

Robby Russell — Sat, 20 Aug 2005 02:05:00 -0500

Rails doesn’t support legacy or custom named sequences at the moment. (as far as I am aware). It’s kind of tricky to have it detect the SEQUENCE name automatically (every time).

In PHP, I used this big ugly query to detect this info:


$sql = $db->prepare("SELECT seq.relname::text
                        FROM pg_class src, pg_class seq, pg_namespace, pg_attribute,
                        pg_depend
                        WHERE
                            pg_depend.refobjsubid = pg_attribute.attnum AND
                            pg_depend.refobjid = src.oid AND
                            seq.oid = pg_depend.objid AND
                            src.relnamespace = pg_namespace.oid AND
                            pg_attribute.attrelid = src.oid AND
                            pg_namespace.nspname = ? AND
                            src.relname = ? AND
                            pg_attribute.attname = ?");

I used this to mimmick the mysql_insert_id function in PHP for PostgreSQL… ( pg_insert_id )

Well, with Rails, I thought that I would build a similar patch, as the current code just assumes the value would be {column}_id_seq.

After hours of playing around and thinking that I figured it all out ... I decided to run a quick test with a non standard sequence name… like this one:


testingdb=# \d legacy.foobar
                                     Table "legacy.foobar" 
  Column   |         Type          |                         Modifiers                          
-----------+-----------------------+------------------------------------------------------------
 foobar_id | integer               | not null default nextval('legacy.old_sequence_name'::text)
 name      | character varying(40) | 
Indexes:
    "foobar_pkey" PRIMARY KEY, btree (foobar_id)

testingdb=# INSERT INTO legacy.foobar (name) VALUES ('abc')
testingdb-# ;
INSERT 17514 1
testingdb=# SELECT * FROM legacy.foobar ;
 foobar_id | name 
-----------+------
       106 | abc
(1 row)

My patch wouldn’t figure that out because the sequence was not created by SERIAL. So, my patch started to feel lame and a total waste of time, because I thought that it was fixing a problem.. that works pretty much as effectively as assuming it is _seq.. but without needing to run a SQL query to determine that. We all (should) know that the field will be named like that when working with SERIAL. So, my patch didn’t buy us anything.

However, Active Record still doesn’t support those funky sequence names. So, I found this ticket #1273.

Their approach was very similar to what caused me to use my long SQL query in the first place because this was suggested to me well over a year ago and I found it to not work in the following situation.

If I have two seperate schemas with the same table name in each like so:


=# \d legacy.people
                                       Table "legacy.people" 
  Column   |         Type          |                           Modifiers                           
-----------+-----------------------+---------------------------------------------------------------
 people_id | integer               | not null default nextval('legacy.people_people_id_seq'::text)
 name      | character varying(50) | 
Indexes:
    "people_pkey" PRIMARY KEY, btree (people_id)

=# \d foo.people
                                       Table "foo.people" 
  Column   |         Type          |                         Modifiers                          
-----------+-----------------------+------------------------------------------------------------
 people_id | integer               | not null default nextval('foo.people_people_id_seq'::text)
 name      | character varying(50) | 
Indexes:
    "people_pkey" PRIMARY KEY, btree (people_id)

That patch will not work because you can’t call the following query:


# SELECT adsrc FROM pg_attrdef WHERE adrelid = (SELECT oid FROM pg_class WHERE relname = 'people');
ERROR:  more than one row returned by a subquery used as an expression

... because there are two tables with the same name! (fun, huh?)

Mine would work… but why bother with that huge query? So, I took my ticket out of [PATCH] and decided that I thought it would be best to just assume that sequences are generated with SERIAL ( link ) by default in AR.

Okay, so what can we do about custom SEQUENCE names?

Well, I am proposing the following (and mentioned this in the ticket #2016)...


class LegacyTable < ActiveRecord::Base
  def self.table_name() "legacy.foobar" end

  # new option for this
  set_primary_key "foobar_id", :sequence => "legacy.old_sequence_name" 
end

(or something along those lines)

With this, I can work around these legacy database scenarios with a quick option. Thoughts/opinions?

I decided to post this on my blog as well, because I do know that there are a few skeptical PostgreSQL people out there who read my blog… I want you to know that I am looking out for you. ;-)

I am sick and tired.. and going to sleep now.

Are you a console master?

Robby Russell — Thu, 18 Aug 2005 22:06:00 -0500

I have a few questions.

1.) Do you know what ./script/console does?

2.) If not, why not?

3.) If so, do you have any fun tips and tricks to share with the masses?

It occured to me earlier that many people, who might have came from the PHP camp, may have never really tested their object-oriented code from some sort of interactive program. (irb) If you are coming from the Python, Java, etc worlds, interactive testing isn’t anything new. Rails is nice enough to bundle a console script right within it!

I meet people online who have never even tried to run it. There are not many tutorials on the wiki that show console… and in my opinion, its one of the coolest things about Ruby and Rails. (but, I come from the php world…)

So, if you aren’t using it… why not? got a moment? try this from the root path of your Rails application.


./script/console

It start up okay? If so, what is the name of one of your models? Let’s say that I have a model structure like:

class Customer < ActiveRecord::Base
  has_many :orders, :dependent => true
end

class Order < ActiveRecord::Base
  belongs_to :customer
end

From console, you can access your models and do all sorts of fun things.


>> y = Customer.find(16)
=> #<Customer:0x2743ea4 @attributes={"name"=>"Robby", "id"=>"16"}>
>> y.orders
=> [#<Order:0x27416b8 @attributes={"id"=>"18", "amount"=>"12.00", "customer_id"=>"16"}>, #<Order:0x274167c @attributes={"id"=>"19", "amount"=>"12.50", "customer_id"=>"16"}>]

Pretty neat, huh?


>> o = Order.find(18)
=> #<Order:0x273da68 @attributes={"id"=>"18", "amount"=>"12.00", "customer_id"=>"16"}>
>> o.customer.name
=> "Robby"

If you are remotely a console wizard, please share some tips and tricks for those who are not sure what to do with it. I personally find myself in console all the time that I am working with Rails, testing stuff out with my models, before I move any of the code to my application.

It sure beats, hitting refresh in your browser all day. :-)

Active Record, I <3 U but I still trust my database server (a tiny bit more)

Robby Russell — Thu, 18 Aug 2005 21:20:00 -0500

While working on a portion of my book, I found myself in ./script/console and was seeing some weird issues when I would use has_many and belongs_to.

Let’s take two simple models.

<pre>
class Order < ActiveRecord::Base
  belongs_to :customer
end

class Customer < ActiveRecord::Base
  has_many :orders
end
</pre>
</code>

After a few test records...

<code>
<pre>
test_dev=# SELECT * FROM customers;  
 id |      name      
----+----------------
  1 | Robby
  2 | Nigel
  3 | Linus
(3 rows)

test_dev=# SELECT * FROM orders;
 id | customer_id | amount 
----+-------------+--------
  1 |           1 |  12.00
  2 |           3 |  12.00
(2 rows)
</pre>
</code>

Nothing completely crazy going on, right?

<typo:code lang="ruby">
Loading development environment.
>> Customer.destroy(3)
=> {"name"=>"Linus", "id"=>"3"}
>>     

=# SELECT * FROM orders;
 id | customer_id | amount 
----+-------------+--------
  1 |           1 |  12.00
  3 |           3 |  12.00
(2 rows)

Wait a minute! I just deleted a customer with an id of 3!

So, what is wrong with this scenario? Can you think of any potential problems that could occur from data like this? The record has a customer_id for a customer that does not exist. This is why we have relational databases in the first place, right? :-)

Here is something that I learned today that I was unaware of. Active Record allows you to pass the has_method declaration the option :dependent.

class Customer < ActiveRecord::Base
  has_many :orders, :dependent => true
end

What is this option? Well, according to the AR documentation, “:dependent – if set to true all the associated object are destroyed alongside this object. May not be set if :exclusively_dependent is also set.”

In a nutshell, this works like ON DELETE CASCADE does in PostgreSQL. So, it will through and delete the orders associated with the customer that I was attempting to destroy.

Up until today, I hadn’t broken myself out of the habit of using the built-in constraints/triggers of PostgreSQL. So, as soon as I did, this issue came up and I learned about :dependent.


test_dev# \d orders
                                Table "public.orders" 
   Column    |     Type      |                       Modifiers                        
-------------+---------------+--------------------------------------------------------
 id          | integer       | not null default nextval('public.orders_id_seq'::text)
 customer_id | integer       | 
 amount      | numeric(10,2) | 
Indexes:
    "orders_pkey" PRIMARY KEY, btree (id)
Foreign-key constraints:
    "orders_customer_id_fkey" FOREIGN KEY (customer_id) REFERENCES customers(id)

test_dev=# ALTER TABLE orders DROP CONSTRAINT orders_customer_id_fkey;
ALTER TABLE 

RobbyOnRails:~/Programming/footest robbyrussell$ ./script/console 
Loading development environment.
>> cust = Customer.create(:name => 'Jim')
=> #<Customer:0x275373c @new_record_before_save=true, @new_record=false, @attributes={"name"=>"Jim", "id"=>5}, @errors=#<ActiveRecord::Errors:0x274fa88 @base=#<Customer:0x275373c ...>, @errors={}>>
>> cust.orders.create(:amount => '25.00')
=> #<Order:0x274991c @new_record=false, @attributes={"id"=>4, "amount"=>"25.00", "customer_id"=>5}, @errors=#<ActiveRecord::Errors:0x2746dfc @base=#<Order:0x274991c ...>, @errors={}>>
>>                

test_dev=# SELECT * FROM orders;
 id | customer_id | amount 
----+-------------+--------
  1 |           1 |  12.00
  3 |           4 |  29.00
  4 |           5 |  25.00
(3 rows)

As you can see, I put myself into the hands of Active Record when I ran the DROP CONSTRAINT. Then I tried running the code at the top of the post… and it didn’t work.

According to the docs, if you use :dependent => true, it should delete the foreign table records. If not, it should set the value of the foriegn key field to NULL in the foreign table.

Basically, perform these two SQL queries:


UPDATE orders SET customer_id = NULL WHERE customer_id = 17; 

DELETE FROM customers  WHERE id = 17;

Then, the records are still in the database for those orders, but the customer is deleted. There are arguments for and against doing this sort of thing… but the ability to have the option is always nice. In any event, Active Record would not run the first query,it was just deleting from the customers table. Without my constraint, no error would be returned from PostgreSQL and I started to get some bad data.

Imagine showing a list of orders and trying to display the customer name associated with an order that has no linking customer. Doh! If Active Record sets the customer_id to NULL we can at least have some logic to work with this without having to run some fun SQL queries to figure out which orders do and dont have customers. (we want our applications to have clean data!)

Anyhow… Was this a bug? Should Active Record know to update the records to NULL in this case? I figured that is should be handling this task, especially since it was handling cascading deletes when you passed :dependent => true.

However, I didn’t want to prematurely post a bug report, so I began asking around on #rubyonrails (irc.freenode.net). People made a bunch of suggestions as to how to work around it. I could add a before_destroy method in my model, track the bug down, (re)add an ON DELETE trigger to my table (hah), etc. So, I decided that I would see if I could track down what happens when has_many is used for a model upon #destroy.

After a while of digging and making some tests, I posted a patch and a bug report. (please disregard my first patch… it did not work! heh)

Now that I figured this out, I am going to happy add my constraints back to my tables and go back to playing around. This reminded me of a post I had a few months ago when I mentioned that I thought it was best to put some constraints and logic in the database. I also agree that constraints should be put in the abstraction layer, but we cannot always put all faith in our code either. A few levels of checks doesn’t hurt. :-)

This was a fun little riddle that I took on today. The moral of the story? If you have the ability to use the builtin referential integrity features of PostgreSQL and those other databases, it might be a good idea to do so. Things get overlooked, people login to the database in many ways, and from different programs.

UPDATE: DHH responded to this post and provided a link which discusses Application Database versus Integrated Database

It should be noted that there is an important distinction between the two methods. When I said, “Things get overlooked, people login to the database in many ways, and from different programs” I was basically describing Integration Database. However, I was also thinking of the possibility of someone opening up their MySQL or PostgreSQL GUI and manually removing a record in plain SQL. According to Application Database, the moment that you do that, you basically break this model and cannot expect your application to be fully responsible for the problems that may or may not occur. At this point, you would need to look at your application in terms of Integration Database. Please do correct me if I am wrong on this. :-)

However, with this scenario, my first attempt to move to relying on AR had a minor hiccup, but it was an easy enough fix.

By performing the following command, I am moving towards an Integration Database pattern and that should be recognized when taking this into consideration.


ALTER TABLE orders ADD CONSTRAINT orders_customers_id_fkey 
    FOREIGN KEY (customer_id) REFERENCES customers (id) MATCH FULL;

Okay, back to work!

Once again. Use constraints! (if you can)

... and thanks to DHH for providing the link and motivating me to make a note of this in my entry.