Robby on Rails: Active Record, I <3 U but I still trust my database server (a tiny bit more)thoughts.sort_by{|t| t[:topic]}.collect tag:www.robbyonrails.com,2005:TypoTypo2006-09-05T22:12:44-04:00Robby Russellurn:uuid:025a8f69575ed0fcc3b30cce680425682005-08-18T21:20:00-04:002006-09-05T22:12:44-04:00Active Record, I <3 U but I still trust my database server (a tiny bit more)<p>While working on a portion of my book, I found myself in <code>./script/console</code> and was seeing some weird issues when I would use <code>has_many</code> and <code>belongs_to</code>.</p>
<p>Let’s take two simple models.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="punct"><</span><span class="ident">pre</span><span class="punct">></span>
<span class="keyword">class </span><span class="class">Order</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">belongs_to</span> <span class="symbol">:customer</span>
<span class="keyword">end</span>
<span class="keyword">class </span><span class="class">Customer</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">has_many</span> <span class="symbol">:orders</span>
<span class="keyword">end</span>
<span class="punct"></</span><span class="regex">pre>
<</span><span class="punct">/</span><span class="ident">code</span><span class="punct">></span>
<span class="constant">After</span> <span class="ident">a</span> <span class="ident">few</span> <span class="ident">test</span> <span class="ident">records</span><span class="punct">...</span>
<span class="punct"><</span><span class="ident">code</span><span class="punct">></span>
<span class="punct"><</span><span class="ident">pre</span><span class="punct">></span>
<span class="ident">test_dev</span><span class="punct">=</span><span class="comment"># SELECT * FROM customers; </span>
<span class="ident">id</span> <span class="punct">|</span> <span class="ident">name</span>
<span class="punct">----+----------------</span>
<span class="number">1</span> <span class="punct">|</span> <span class="constant">Robby</span>
<span class="number">2</span> <span class="punct">|</span> <span class="constant">Nigel</span>
<span class="number">3</span> <span class="punct">|</span> <span class="constant">Linus</span>
<span class="punct">(</span><span class="number">3</span> <span class="ident">rows</span><span class="punct">)</span>
<span class="ident">test_dev</span><span class="punct">=</span><span class="comment"># SELECT * FROM orders;</span>
<span class="ident">id</span> <span class="punct">|</span> <span class="ident">customer_id</span> <span class="punct">|</span> <span class="ident">amount</span>
<span class="punct">----+-------------+--------</span>
<span class="number">1</span> <span class="punct">|</span> <span class="number">1</span> <span class="punct">|</span> <span class="number">12.00</span>
<span class="number">2</span> <span class="punct">|</span> <span class="number">3</span> <span class="punct">|</span> <span class="number">12.00</span>
<span class="punct">(</span><span class="number">2</span> <span class="ident">rows</span><span class="punct">)</span>
<span class="punct"></</span><span class="regex">pre>
<</span><span class="punct">/</span><span class="ident">code</span><span class="punct">></span>
<span class="constant">Nothing</span> <span class="ident">completely</span> <span class="ident">crazy</span> <span class="ident">going</span> <span class="ident">on</span><span class="punct">,</span> <span class="ident">right?</span>
<span class="punct"><</span><span class="ident">typo</span><span class="symbol">:code</span> <span class="ident">lang</span><span class="punct">="</span><span class="string">ruby</span><span class="punct">"></span>
<span class="constant">Loading</span> <span class="ident">development</span> <span class="ident">environment</span><span class="punct">.</span>
<span class="punct">>></span> <span class="ident">Customer</span><span class="punct">.</span><span class="ident">destroy</span><span class="punct">(</span><span class="number">3</span><span class="punct">)</span>
<span class="punct">=></span> <span class="punct">{"</span><span class="string">name</span><span class="punct">"=>"</span><span class="string">Linus</span><span class="punct">",</span> <span class="punct">"</span><span class="string">id</span><span class="punct">"=>"</span><span class="string">3</span><span class="punct">"}</span>
<span class="punct">>></span>
<span class="punct">=</span><span class="comment"># SELECT * FROM orders;</span>
<span class="ident">id</span> <span class="punct">|</span> <span class="ident">customer_id</span> <span class="punct">|</span> <span class="ident">amount</span>
<span class="punct">----+-------------+--------</span>
<span class="number">1</span> <span class="punct">|</span> <span class="number">1</span> <span class="punct">|</span> <span class="number">12.00</span>
<span class="number">3</span> <span class="punct">|</span> <span class="number">3</span> <span class="punct">|</span> <span class="number">12.00</span>
<span class="punct">(</span><span class="number">2</span> <span class="ident">rows</span><span class="punct">)</span>
</code></pre></div>
<p>Wait a minute! I just deleted a customer with an <code>id</code> of <code>3</code>!</p>
<p>So, what is wrong with this scenario? Can you think of any potential problems that could occur from data like this? The record has a <code>customer_id</code> for a customer that does not exist. This is why we have relational databases in the first place, right? :-)</p>
<p>Here is something that I learned today that I was unaware of. Active Record allows you to pass the <code>has_method</code> declaration the option <code>:dependent</code>.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">Customer</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">has_many</span> <span class="symbol">:orders</span><span class="punct">,</span> <span class="symbol">:dependent</span> <span class="punct">=></span> <span class="constant">true</span>
<span class="keyword">end</span></code></pre></div>
<p>What is this option? Well, according to the AR documentation, “<code>:dependent</code> – if set to true all the associated object are destroyed alongside this object. May not be set if <code>:exclusively_dependent</code> is also set.”</p>
<p>In a nutshell, this works like <code>ON DELETE CASCADE</code> does in PostgreSQL. So, it will through and delete the <em>orders</em> associated with the <em>customer</em> that I was attempting to <code>destroy</code>.</p>
<p>Up until today, I hadn’t broken myself out of the habit of using the built-in constraints/triggers of PostgreSQL. So, as soon as I did, this issue came up and I learned about <code>:dependent</code>.</p>
<code>
<pre>
test_dev# \d orders
Table "public.orders"
Column | Type | Modifiers
-------------+---------------+--------------------------------------------------------
id | integer | not null default nextval('public.orders_id_seq'::text)
customer_id | integer |
amount | numeric(10,2) |
Indexes:
"orders_pkey" PRIMARY KEY, btree (id)
Foreign-key constraints:
"orders_customer_id_fkey" FOREIGN KEY (customer_id) REFERENCES customers(id)
test_dev=# ALTER TABLE orders DROP CONSTRAINT orders_customer_id_fkey;
ALTER TABLE
RobbyOnRails:~/Programming/footest robbyrussell$ ./script/console
Loading development environment.
>> cust = Customer.create(:name => 'Jim')
=> #<Customer:0x275373c @new_record_before_save=true, @new_record=false, @attributes={"name"=>"Jim", "id"=>5}, @errors=#<ActiveRecord::Errors:0x274fa88 @base=#<Customer:0x275373c ...>, @errors={}>>
>> cust.orders.create(:amount => '25.00')
=> #<Order:0x274991c @new_record=false, @attributes={"id"=>4, "amount"=>"25.00", "customer_id"=>5}, @errors=#<ActiveRecord::Errors:0x2746dfc @base=#<Order:0x274991c ...>, @errors={}>>
>>
test_dev=# SELECT * FROM orders;
id | customer_id | amount
----+-------------+--------
1 | 1 | 12.00
3 | 4 | 29.00
4 | 5 | 25.00
(3 rows)
</pre>
</code>
<p>As you can see, I put myself into the hands of Active Record when I ran the <code>DROP CONSTRAINT</code>. Then I tried running the code at the top of the post… and it didn’t work.</p>
<p>According to the docs, if you use <code>:dependent => true</code>, it should delete the foreign table records. If not, it should set the value of the foriegn key field to <code>NULL</code> in the foreign table.</p>
Basically, perform these two <span class="caps">SQL</span> queries:
<code>
<pre>
UPDATE orders SET customer_id = NULL WHERE customer_id = 17;
DELETE FROM customers WHERE id = 17;
</pre>
</code>
<p>Then, the records are still in the database for those orders, but the customer is deleted. There are arguments for and against doing this sort of thing… but the ability to have the option is always nice. In any event, Active Record would not run the first query,it was just deleting from the customers table. Without my constraint, no error would be returned from PostgreSQL and I started to get some bad data.</p>
<p>Imagine showing a list of orders and trying to display the customer name associated with an order that has no linking customer. Doh! If Active Record sets the <code>customer_id</code> to <code>NULL</code> we can at least have some logic to work with this without having to run some fun <span class="caps">SQL</span> queries to figure out which orders do and dont have customers. (we want our applications to have clean data!)</p>
<p>Anyhow… Was this a bug? Should Active Record know to update the records to <code>NULL</code> in this case? I figured that is should be handling this task, especially since it was handling cascading deletes when you passed <code>:dependent => true</code>.</p>
<p>However, I didn’t want to prematurely post a bug report, so I began asking around on <b>#rubyonrails</b> (irc.freenode.net). People made a bunch of suggestions as to how to work around it. I could add a <code>before_destroy</code> method in my model, track the bug down, (re)add an <code>ON DELETE</code> trigger to my table (hah), etc. So, I decided that I would see if I could track down what happens when has_many is used for a model upon <code>#destroy</code>.</p>
<p>After a while of digging and making some tests, I posted a <a href="http://rubyurl.com/Lgv">patch</a> and a <a href="http://dev.rubyonrails.com/ticket/2009">bug report</a>. (please disregard my first patch… it did not work! heh)</p>
<p>Now that I figured this out, I am going to happy add my constraints back to my tables and go back to playing around. This reminded me of a post I had a few months ago when I mentioned that I thought it was best to put some constraints and logic in the database. I also agree that constraints should be put in the abstraction layer, but we cannot always put all faith in our code either. A few levels of checks doesn’t hurt. :-)</p>
<p>This was a fun little riddle that I took on today. The moral of the story? If you have the ability to use the builtin referential integrity features of PostgreSQL and those <strong>other</strong> databases, it might be a good idea to do so. Things get overlooked, people login to the database in many ways, and from different programs.</p>
<p><strong><span class="caps">UPDATE</span></strong>: <span class="caps">DHH</span> responded to this post and provided <a href="http://martinfowler.com/bliki/DatabaseStyles.html">a link</a> which discusses <a href="http://martinfowler.com/bliki/ApplicationDatabase.html">Application Database</a> versus <a href="http://martinfowler.com/bliki/IntegrationDatabase.html">Integrated Database</a></p>
<p>It should be noted that there is an important distinction between the two methods. When I said, “Things get overlooked, people login to the database in many ways, and from different programs” I was basically describing Integration Database. However, I was also thinking of the possibility of someone opening up their MySQL or PostgreSQL <span class="caps">GUI</span> and manually removing a record in plain <span class="caps">SQL</span>. According to Application Database, the moment that you do that, you basically break this model and cannot expect your application to be fully responsible for the problems that may or may not occur. At this point, you would need to look at your application in terms of Integration Database. Please do correct me if I am wrong on this. :-)</p>
<p>However, with this scenario, my first attempt to move to relying on AR had a minor hiccup, but it was an easy enough fix.</p>
<p>By performing the following command, I am moving towards an Integration Database pattern and that should be recognized when taking this into consideration.</p>
<code>
<pre>
ALTER TABLE orders ADD CONSTRAINT orders_customers_id_fkey
FOREIGN KEY (customer_id) REFERENCES customers (id) MATCH FULL;
</pre>
</code>
<p>Okay, back to work!</p>
<p>Once again. Use constraints! (if you can)</p>
<p>... and thanks to <a href="http://www.loudthinking.com/"><span class="caps">DHH</span></a> for providing the link and motivating me to make a note of this in my entry.</p><p>While working on a portion of my book, I found myself in <code>./script/console</code> and was seeing some weird issues when I would use <code>has_many</code> and <code>belongs_to</code>.</p>
<p>Let’s take two simple models.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="punct"><</span><span class="ident">pre</span><span class="punct">></span>
<span class="keyword">class </span><span class="class">Order</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">belongs_to</span> <span class="symbol">:customer</span>
<span class="keyword">end</span>
<span class="keyword">class </span><span class="class">Customer</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">has_many</span> <span class="symbol">:orders</span>
<span class="keyword">end</span>
<span class="punct"></</span><span class="regex">pre>
<</span><span class="punct">/</span><span class="ident">code</span><span class="punct">></span>
<span class="constant">After</span> <span class="ident">a</span> <span class="ident">few</span> <span class="ident">test</span> <span class="ident">records</span><span class="punct">...</span>
<span class="punct"><</span><span class="ident">code</span><span class="punct">></span>
<span class="punct"><</span><span class="ident">pre</span><span class="punct">></span>
<span class="ident">test_dev</span><span class="punct">=</span><span class="comment"># SELECT * FROM customers; </span>
<span class="ident">id</span> <span class="punct">|</span> <span class="ident">name</span>
<span class="punct">----+----------------</span>
<span class="number">1</span> <span class="punct">|</span> <span class="constant">Robby</span>
<span class="number">2</span> <span class="punct">|</span> <span class="constant">Nigel</span>
<span class="number">3</span> <span class="punct">|</span> <span class="constant">Linus</span>
<span class="punct">(</span><span class="number">3</span> <span class="ident">rows</span><span class="punct">)</span>
<span class="ident">test_dev</span><span class="punct">=</span><span class="comment"># SELECT * FROM orders;</span>
<span class="ident">id</span> <span class="punct">|</span> <span class="ident">customer_id</span> <span class="punct">|</span> <span class="ident">amount</span>
<span class="punct">----+-------------+--------</span>
<span class="number">1</span> <span class="punct">|</span> <span class="number">1</span> <span class="punct">|</span> <span class="number">12.00</span>
<span class="number">2</span> <span class="punct">|</span> <span class="number">3</span> <span class="punct">|</span> <span class="number">12.00</span>
<span class="punct">(</span><span class="number">2</span> <span class="ident">rows</span><span class="punct">)</span>
<span class="punct"></</span><span class="regex">pre>
<</span><span class="punct">/</span><span class="ident">code</span><span class="punct">></span>
<span class="constant">Nothing</span> <span class="ident">completely</span> <span class="ident">crazy</span> <span class="ident">going</span> <span class="ident">on</span><span class="punct">,</span> <span class="ident">right?</span>
<span class="punct"><</span><span class="ident">typo</span><span class="symbol">:code</span> <span class="ident">lang</span><span class="punct">="</span><span class="string">ruby</span><span class="punct">"></span>
<span class="constant">Loading</span> <span class="ident">development</span> <span class="ident">environment</span><span class="punct">.</span>
<span class="punct">>></span> <span class="ident">Customer</span><span class="punct">.</span><span class="ident">destroy</span><span class="punct">(</span><span class="number">3</span><span class="punct">)</span>
<span class="punct">=></span> <span class="punct">{"</span><span class="string">name</span><span class="punct">"=>"</span><span class="string">Linus</span><span class="punct">",</span> <span class="punct">"</span><span class="string">id</span><span class="punct">"=>"</span><span class="string">3</span><span class="punct">"}</span>
<span class="punct">>></span>
<span class="punct">=</span><span class="comment"># SELECT * FROM orders;</span>
<span class="ident">id</span> <span class="punct">|</span> <span class="ident">customer_id</span> <span class="punct">|</span> <span class="ident">amount</span>
<span class="punct">----+-------------+--------</span>
<span class="number">1</span> <span class="punct">|</span> <span class="number">1</span> <span class="punct">|</span> <span class="number">12.00</span>
<span class="number">3</span> <span class="punct">|</span> <span class="number">3</span> <span class="punct">|</span> <span class="number">12.00</span>
<span class="punct">(</span><span class="number">2</span> <span class="ident">rows</span><span class="punct">)</span>
</code></pre></div>
<p>Wait a minute! I just deleted a customer with an <code>id</code> of <code>3</code>!</p>
<p>So, what is wrong with this scenario? Can you think of any potential problems that could occur from data like this? The record has a <code>customer_id</code> for a customer that does not exist. This is why we have relational databases in the first place, right? :-)</p>
<p>Here is something that I learned today that I was unaware of. Active Record allows you to pass the <code>has_method</code> declaration the option <code>:dependent</code>.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">Customer</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">has_many</span> <span class="symbol">:orders</span><span class="punct">,</span> <span class="symbol">:dependent</span> <span class="punct">=></span> <span class="constant">true</span>
<span class="keyword">end</span></code></pre></div>
<p>What is this option? Well, according to the AR documentation, “<code>:dependent</code> – if set to true all the associated object are destroyed alongside this object. May not be set if <code>:exclusively_dependent</code> is also set.”</p>
<p>In a nutshell, this works like <code>ON DELETE CASCADE</code> does in PostgreSQL. So, it will through and delete the <em>orders</em> associated with the <em>customer</em> that I was attempting to <code>destroy</code>.</p>
<p>Up until today, I hadn’t broken myself out of the habit of using the built-in constraints/triggers of PostgreSQL. So, as soon as I did, this issue came up and I learned about <code>:dependent</code>.</p>
<code>
<pre>
test_dev# \d orders
Table "public.orders"
Column | Type | Modifiers
-------------+---------------+--------------------------------------------------------
id | integer | not null default nextval('public.orders_id_seq'::text)
customer_id | integer |
amount | numeric(10,2) |
Indexes:
"orders_pkey" PRIMARY KEY, btree (id)
Foreign-key constraints:
"orders_customer_id_fkey" FOREIGN KEY (customer_id) REFERENCES customers(id)
test_dev=# ALTER TABLE orders DROP CONSTRAINT orders_customer_id_fkey;
ALTER TABLE
RobbyOnRails:~/Programming/footest robbyrussell$ ./script/console
Loading development environment.
>> cust = Customer.create(:name => 'Jim')
=> #<Customer:0x275373c @new_record_before_save=true, @new_record=false, @attributes={"name"=>"Jim", "id"=>5}, @errors=#<ActiveRecord::Errors:0x274fa88 @base=#<Customer:0x275373c ...>, @errors={}>>
>> cust.orders.create(:amount => '25.00')
=> #<Order:0x274991c @new_record=false, @attributes={"id"=>4, "amount"=>"25.00", "customer_id"=>5}, @errors=#<ActiveRecord::Errors:0x2746dfc @base=#<Order:0x274991c ...>, @errors={}>>
>>
test_dev=# SELECT * FROM orders;
id | customer_id | amount
----+-------------+--------
1 | 1 | 12.00
3 | 4 | 29.00
4 | 5 | 25.00
(3 rows)
</pre>
</code>
<p>As you can see, I put myself into the hands of Active Record when I ran the <code>DROP CONSTRAINT</code>. Then I tried running the code at the top of the post… and it didn’t work.</p>
<p>According to the docs, if you use <code>:dependent => true</code>, it should delete the foreign table records. If not, it should set the value of the foriegn key field to <code>NULL</code> in the foreign table.</p>
Basically, perform these two <span class="caps">SQL</span> queries:
<code>
<pre>
UPDATE orders SET customer_id = NULL WHERE customer_id = 17;
DELETE FROM customers WHERE id = 17;
</pre>
</code>
<p>Then, the records are still in the database for those orders, but the customer is deleted. There are arguments for and against doing this sort of thing… but the ability to have the option is always nice. In any event, Active Record would not run the first query,it was just deleting from the customers table. Without my constraint, no error would be returned from PostgreSQL and I started to get some bad data.</p>
<p>Imagine showing a list of orders and trying to display the customer name associated with an order that has no linking customer. Doh! If Active Record sets the <code>customer_id</code> to <code>NULL</code> we can at least have some logic to work with this without having to run some fun <span class="caps">SQL</span> queries to figure out which orders do and dont have customers. (we want our applications to have clean data!)</p>
<p>Anyhow… Was this a bug? Should Active Record know to update the records to <code>NULL</code> in this case? I figured that is should be handling this task, especially since it was handling cascading deletes when you passed <code>:dependent => true</code>.</p>
<p>However, I didn’t want to prematurely post a bug report, so I began asking around on <b>#rubyonrails</b> (irc.freenode.net). People made a bunch of suggestions as to how to work around it. I could add a <code>before_destroy</code> method in my model, track the bug down, (re)add an <code>ON DELETE</code> trigger to my table (hah), etc. So, I decided that I would see if I could track down what happens when has_many is used for a model upon <code>#destroy</code>.</p>
<p>After a while of digging and making some tests, I posted a <a href="http://rubyurl.com/Lgv">patch</a> and a <a href="http://dev.rubyonrails.com/ticket/2009">bug report</a>. (please disregard my first patch… it did not work! heh)</p>
<p>Now that I figured this out, I am going to happy add my constraints back to my tables and go back to playing around. This reminded me of a post I had a few months ago when I mentioned that I thought it was best to put some constraints and logic in the database. I also agree that constraints should be put in the abstraction layer, but we cannot always put all faith in our code either. A few levels of checks doesn’t hurt. :-)</p>
<p>This was a fun little riddle that I took on today. The moral of the story? If you have the ability to use the builtin referential integrity features of PostgreSQL and those <strong>other</strong> databases, it might be a good idea to do so. Things get overlooked, people login to the database in many ways, and from different programs.</p>
<p><strong><span class="caps">UPDATE</span></strong>: <span class="caps">DHH</span> responded to this post and provided <a href="http://martinfowler.com/bliki/DatabaseStyles.html">a link</a> which discusses <a href="http://martinfowler.com/bliki/ApplicationDatabase.html">Application Database</a> versus <a href="http://martinfowler.com/bliki/IntegrationDatabase.html">Integrated Database</a></p>
<p>It should be noted that there is an important distinction between the two methods. When I said, “Things get overlooked, people login to the database in many ways, and from different programs” I was basically describing Integration Database. However, I was also thinking of the possibility of someone opening up their MySQL or PostgreSQL <span class="caps">GUI</span> and manually removing a record in plain <span class="caps">SQL</span>. According to Application Database, the moment that you do that, you basically break this model and cannot expect your application to be fully responsible for the problems that may or may not occur. At this point, you would need to look at your application in terms of Integration Database. Please do correct me if I am wrong on this. :-)</p>
<p>However, with this scenario, my first attempt to move to relying on AR had a minor hiccup, but it was an easy enough fix.</p>
<p>By performing the following command, I am moving towards an Integration Database pattern and that should be recognized when taking this into consideration.</p>
<code>
<pre>
ALTER TABLE orders ADD CONSTRAINT orders_customers_id_fkey
FOREIGN KEY (customer_id) REFERENCES customers (id) MATCH FULL;
</pre>
</code>
<p>Okay, back to work!</p>
<p>Once again. Use constraints! (if you can)</p>
<p>... and thanks to <a href="http://www.loudthinking.com/"><span class="caps">DHH</span></a> for providing the link and motivating me to make a note of this in my entry.</p>