InquiryLabs

Politics, Programming and Possibilities

Archive for the ‘Software Engineering’ Category

Condition Builder 1.0 Released

I released version 0.5 of this Ruby on Rails plugin a few months ago and have since updated it to include ‘and’ and ‘or’ methods, as well as the ability to use sub-conditions.

Here are some code samples:

  Condition.block { |c|
    c.and "one", 1
    c.and "two", 2
    c.and { |d|
      d.or "three", 3
      d.or "four", 4
    }
  }

# => ["one = ? AND two = ? AND (three = ? OR four = ?)", 1, 2, 3, 4]

Since this uses Ruby’s blocks to construct a Rails condition array, you can use all of Ruby’s familiar logical constructs to achieve cleaner code in some cases. For example:

  Condition.block { |c|
    c.and "user_id", @user_id
    c.and { |d|
      d.or "admin", true
      d.or "joined_at", "<", Date.new(2007, 1, 1)
    } unless @dont_override_user_id
  }

# If @dont_override_user_id is false, then
# => ["user_id = ? AND (admin = ? OR joined_at < ?)", 10, true, Date.new(2007, 1, 1)]
#
# If @dont_override_user_id is true, then
# => ["user_id = ?", 10]

I use this directly in my ActiveRecord queries, like this:

  Book.find(:all,
      :include => {:content_pointer => :invitees},
      :conditions => Condition.block { |c|
        c.and "invitees.user_id", self.id
        c.and "content_pointers.created_by_id", self.id
        c.and "content_pointers.company_id", company.id if company
      })

Note: You may get some unexpected results if you try mixing ‘and’ and ‘or’ within the same block. The current implementation assumes that you will be consistent, and use a new block whenever you wish to switch logical operators. As an example, don’t do this:

Condition.block { |c|
  c.and "user_id", 1
  c.and "book_id", 2
  c.or "company_id", 3
}
# BAD
# Output will be: ["user_id = ? OR book_id = ? OR company_id = ?", 1, 2, 3]

# Do this instead:
Condition.block { |c|
  c.and "user_id", 1
  c.and { |d|
    d.or "book_id", 2
    d.or "company_id", 3
  }
}
# OK
# Output will be: ["user_id = ? AND (book_id = ? OR company_id = ?)", 1, 2, 3]

The Condition Builder plugin is available as a tar/gz file. Download it here.

This was a weird Ruby gotcha that my coworker, Paul Jones, discovered. Here’s a test class to show the point:

class T
  private
  def l=(val); puts val end
  def l; 1 end

  public
  def m=(val); self.l=val end
  def m; self.l end
end

irb> t.m=1
1
=> 1

irb> t.m
NoMethodError: private method `l' called for #
        from (irb):11:in `m'
        from (irb):18

Anyone know why this is the case?

Update: Perhaps I should clarify. I was expecting the “m=” method to work as it did, but I did not expect the NoMethodError when calling “m”. A better question might be, “Why the discrepency?”

Getting the _session_id from SWFUpload

SWFUpload is an open-source Flash 8 embedded application that lets users upload multiple files in a web application. It looked like an easy answer to all of our single-file-uploading woes until we discovered that we couldn’t maintain session state with it.

The problem is that Flash 8 has no way of sending meta data with the uploaded file–for example, who the file belongs to! This is extremely important for us in our iMemoryBook application, since we want uploaded files to be associated with the user’s account–not just a public file repository.

So the culprit (inasmuch as we can’t fix Flash 8 ) appears to be in Ruby’s CGI::Session class: no matter what we tried, it would not pick up the _session_id in the query string when the HTTP request was a POST.

Normally the work-around is quite simple. In a POST-type request, a form is submitted to the server (e.g. a Rails application). In this scenario, we could simply send the _session_id as a hidden field and all would be well.

With Flash 8, however, there is no way to add a ‘hidden field’ to the multi-part form data, thus Rails will fail to recognize the _session_id in the query string portion of our request.

Tracking things down was the hard part. The hackish work-around wasn’t so bad:


# The following code is a work-around for the
# Flash 8 bug that prevents our multiple file uploader
# from sending the _session_id.  Here, we hack the
# Session#initialize method and force the session_id
# to load from the query string via the request uri.
# (Tested on Lighttpd)

class CGI::Session
  alias original_initialize initialize
  def initialize(request, option = {})
    session_key = option['session_key'] || '_session_id'
    option['session_id'] =
      request.env_table["REQUEST_URI"][0..-1].
      scan(/#{session_key}=(.*?)(&.*?)*$/).
      flatten.first
    original_initialize(request, option)
  end
end

Put the above code in your environment.rb file, or separate it out as a file in your lib directory and include it.

This code is also available at my favorite snippet place, BigBold’s snippets directory.

Update: The above code doesn’t work on Mongrel/Apache, so I’ve modified it and uploaded the new code here.

Column Comments 1.3

This is a minor update to the Column Comments plugin. We discovered this morning that some changes to edge rails had caused the plugin to fail. Note: This will not work on Rails 1.x. Use the older version 1.2 of the plugin (linked below) for non-edge Rails.

Download Column Comments v. 1.3

What is the Column Comments Plugin?

The ColumnComments plugin makes the necessary modifications to ActiveRecord to allow for an optional :comment in your table migrations. In addition, it includes the same feature provided by the AnnotateModels plugin (rake annotate_models), but also includes the database-stored comments in the auto-generated header.

See my original post for a full explanation. Download version 1.2 here.

Some of our users at FamilyLearn have wondered about what Amazon has to do with FamilyLearn, and how our new software system is going to hold up under high demand. While most people usually associate Amazon.com with books, software engineers like myself are starting to think of Amazon as a bag of tricks and solutions to common web development challenges. For example, here is a brief overview of our plan, using Amazon:

  • Using Amazon’s S3 storage servers, we have unlimited space for all of the pictures and stories that FamilyLearn users upload and send while working on their projects. The data is stored in a fail-safe system so that we’ll never lose anyone’s pictures or books. The final, press-ready PDF files that our system generates are also stored on Amazon S3 so that users can download their finished product.
  • We’ve also started using Amazon’s EC2 computing servers. This gives us a way to increase our computing power on-demand when traffic gets heavier. For now, we haven’t implemented the “on-demand” part of the plan, but we can turn on more servers manually when we see the need.

The technology behind iMemoryBook and Pyxlin is a fairly complicated process. Being able to see a book “published as you go” is a wonderful thing, although it comes with some cost. As it turns out, converting what you see on the internet in to a printed and bound book is a fairly intensive computational task—and in our case, we’re doing it on-the-fly. For those of you familiar with such conversion tools as laTeX, ImageMagick and pdftk, you can imagine what it must take to respond to each web action by converting it and presenting it back to the user as a visible image in the browser. We look forward to optimizing the system so that it will be more responsive in the future; but, for now, we’re quite happy with the way it’s turned out on EC2.

Happy

Looks like we’ve got a little gift from the TextMate art crew today:

TextMate Halloween Icon

Pretty nice, eh?

While hanging around the TextMate IRC channel yesterday, there were quite a few different reactions to this, few of which I expected!

Most people seemed to be pretty impressed that their favorite editor would pull out a treat like this on Hallowe’en. But some people thought it was a trick (Did some Rails-hater create a virus??) and others were offended by it.

I suppose it could be taken as such, but in the end you have to try to receive things with the intent they were given–and this was all for the fun of it (ask Soryu!) :)

Thanks, TextMate, for raising the bar for editors (once again)!

Update: Here’s a link to Soryu’s blog about it.

TextMate Halloween Bundle Editor
TextMate Spider Web

Security Vulnerability in

Word has it that Ruby 1.8.5 and below (i.e. all versions of Ruby) have a security flaw in its cgi.rb file that will allow a remote hacker to cause your application to max out your CPU usage and essentially launch a denial-of-service attack with a single request.

There’s a nice write-up of it on Evan Weaver’s blog. According to Evan and Zed Shaw, Rails apps using mongrel and Litespeed are also affected. The original announcement from Zed is on the mailing list here.

Thanks to Pat Eyler for the tip-off.

Simplified Condition Class

So I’ve tried Ezra’s “ez_where” and InVisible’s “where” plugins, but in the end I just couldn’t get past the hackish feeling I got when I used them.

I liked the concept that Ezra built—trying to make a domain-specific language for a where clause in Ruby—but the result felt like a heavy-weight addition to a small but common problem. In addition, I had trouble memorizing which Ruby operators mapped to which SQL operators in fringe cases.

Using InVisible’s “where” plugin was a bit of a relief, as it was lighter weight and the mapping was more direct; however, in order to test for a condition in the case where the column is in another table, I had to resort to a “send” method, like this:

    c = InVisible::Cond.new do
      send("permissions.value", params[:role])
    end

Which isn’t all that bad, but the inelegant solution gnawed at me for a while. Finally, I decided to make a “simplest-case” Condition class by re-using Ezra’s Cond class. The final outcome looks like this, and feels more Rubyesque to me:

    @invitees = @book.permitted_users \
      :conditions => Condition.block { |c|
        c << [ "permissions.value", params[:role] ] if params[:role]
      }

As with the InVisible::Cond class, you can do things like this:

cond = Condition.new do |c|
  c << [ "first_name", "like", "#{prefix}%" ] if prefix
  c << [ "verified", true ]
  c << [ "created_at", ">", 5.days.ago ]
end

cond.where
# => ["first_name like ? AND verified = ? AND created_at > ?", "dua%", true, "2006-10-18"]

users = User.find(:all, :conditions => cond.where)

My simple version of the plugin is available for download as a tar/gz file here.

Thanks to both Ezra and InVisible for their pioneer work!

Summary of Some Recent Changes to Edge Rails

There have been some noteworthy changes to the Rails code base recently. I thought I’d list some of them here to familiarize myself (and others) with them:

  • You can now specify which plugins you want loaded, instead of having Rails assume you want to automatically load all plugins in the vendor/plugins folder. In config/environment.rb:
    config.plugins = %w[ textmate_footnotes acts_as_taggable ]
    

  • Namespaced Models! It’s now possible (and has been for some time, I believe) to create models like this:
    class Admin::Tools < ActiveRecord::Base; end
    


    The above example would reside in app/models/admin/tools.rb.

    Update: This feature is in Rails 1.1—thanks to Matthias for pointing out how out of date I am :P

    I think I’m going to find that it’s a lot easier to keep large applications organized, given that good applications often break concepts up in to small and manageable classes in the domain.

    Note that you can also use this ability with non-active-record classes as well–it’s just as valid to create a model that connects with your LDAP server or loads text files from disk. Namespacing is good :)

  • You can run your own script files on Unix-like systems like a real script, using script/runner. All it takes is adding the following shebang line (first line) to a ruby file:
    #!/usr/bin/env /path/to/my/app/script/runner

    This is similar to adding

    #!/usr/bin/env ruby

    to ruby files, but it gives you the whole Rails environment to work in as well.

  • The content_tag and form_tag helper methods now support blocks. This means you can enclose a bunch of HTML or more eRB in a block, and it’ll close the tags automatically for you at the end of your block.
    <% form_tag :action => "create" do %>
    HTML and Text goes here.
    <%= submit_tag "Done" %>
    <% end %>
    


    Functionality is similar for content_tag:

    <% content_tag :span, :class => "title" do %>
      <%= @names.join(", ") %>
      <%= image_tag "link.gif" %>
    <% end %>
    


    Produces:

    George, Tom, Lisa 
    

  • Some helpers have become deprecated:
    • start_form_tag / end_form_tag
    • link_to_image
    • link_to :post => true
    • others?

Amazon S3 &amp;quot;max-keys&amp;quot;

We (at FamilyLearn) have been using Amazon S3 for photo storage in our new system. The Ruby libraries are still a little young, so we run in to trouble occasionally.

For example, today I was unable to limit the number of keys returned from a bucket with the :max_keys option. As it turns out, the Amazon API expects “max-keys” even though Ruby syntax does not allow dashes in its symbols. This turned out to be a simple problem to fix:

@files = @@s3_connection.list_bucket(
  @current_bucket,
  :prefix => @prefix,
  :"max-keys" => 15).entries