Peppy Heppy

Tonight I attended a presentation at Google that given by Jason Hoffman (the CTO at Joyent) about Ruby, and scaling web architecture. Although none of the actual information was new to me it did remind me of the basic points to scaling a web application. Here they are.

• Scalability is language, performance, and throughput independent.

• Test each piece of your architecture and find out what the maximums of each service are.

• Find the real bottlenecks and remove them efficiently. (Use DTrace) An example was how Apache’s proxy module will limit the number of requests per second to 140 per Apache instance, and by using virtualization you could have eight instances running Apache on the same server with the capacity of 1120 rps.

Overall it was worth while especially when you factor in the snacks and dinner that Google provided.

Scale away!

In one of my last posts I showed how I was able to create completely custom urls for SEO, but there is an issue that sometimes comes up when creating custom urls or when migrating urls, etc.

Here is a simple way to ensure that urls that are being requested are valid. Google and Yahoo! (and others) crawl your sites links and can on occasion come across an incorrect ink from someone else’s site that may be old or mistyped. There are some stiff penalties associated with having two different urls pointing to the same page. There may also be a need to retire certain urls or to change the way they are formated.

Here is an example, the URL:

http://domain.com/d-123456-mountain_viewering

Should be redirected to:

http://domain.com/d-123456-mountain_view

Here is the simple solution:

I created a module that looked like the following in the lib directory and included it into the ActionController class.

include ActiveRecord

module MY
  module URL

    def page_code_object_map
      { 
        'd' => Destination, 'p' => Photo
      }
    end

    def execute_url_post_process
      canonicalize if params[:canonicalize]
    end

    def canonicalize

      whole_url   = request.request_uri().split('?')[0].split('#')[0]
      url_pieces  = current_url.split('-')
      page_type   = url_pieces[0].gsub(/\//, '')
      type_id     = url_pieces[1]

      begin
        object = page_code_object_map[page_type].find(type_id)
        canonical_url = send "custom_#{page_type}_path", object, params
      rescue RecordNotFound => e
        render :file => File.join(RAILS_ROOT, 'public', '404.html'), :status => 404
        return
      end

      if canonical_url and canonical_url != whole_url
        headers['Status'] = '301 Moved Permanently'
        redirect_to("#{http_base}#{canonical_url}", :status => 301)
      end

    end

  end
end

ActionController::Base.send :include, MY::URL
ActionView::Base.send :include, MY::URL

In the route below, notice that I am passing a parameter named :canonicalize with the value of true. This parameter is passed through to the controller as a request parameter and can be accessed in the params hash.

map.d '/d-:destination_global_id-:name*other_params', :controller => 'destinations', :action => 'show', :canonicalize => true, :destination_global_id => /\d{1,20}/, :name => /[^-]+/

How does this all work you say? Simple. In your application controller (controllers/application.rb) you need to include something like this:

before_filter :execute_url_post_process

This will start the checking process by calling the execute_url_post_process() method defined above in my module. If the route that matches passes the :cononicalize parameter, the conanicalize() method will get the current url and certain important pieces. Then depending on the object that is mapped to the page code (d) it will reconstruct the url of the destination object that should match the existing url. If it matches then were golden, if it doesn’t then we redirect to the new/correct url ensuring that we do not loose page rank or be counted as spam (duplicate content).

There are many things that you can do within this code. Some of them include managing authorization, hiding pages, etc.

I hope you enjoyed this tip. If you have any suggestions, please post them, I am sure some genius will have something to add. :)

I needed to build urls that were packed with keywords for SEO. I needed to make sure that the url more fully described the contents of the page.

This default rails url does not cut it.

/destinations/12345

This does cut it.

/d-12345-mountain_view

So here is the hack that I did to get the desired affect. (Suggestions or insults on my approach are welcomed!)

First, I added this code into a plugin that I was using for our custom routes stuff. You can probably add this to the environment.rb file or better yet to a a file within lib and just make sure that you require the file from within environment.rb. I really needed to add the ’-’ as a delimiter.

This is step is important because by default rails uses slashes (/) as a dilimeter for parts of the url, but by adding a dash (-) to the array things work the way they should.

module ActionController
  module Routing
    SEPARATORS = %w( / ; . , ? -)  
  end
end

Then I added a named route (config/routes.rb) that looked something like this:

map.d '/d-:destination_global_id-:name*other_params', :controller => 'destinations', :action => 'show', :canonicalize => true, :destination_global_id => /\d{1,20}/, :name => /[^-]+/

Now we can create helper methods that take all of these wonderful parameters.

def custom_d_path(destination, params={})
  d_path(
    destination.global_id, 
    string_for_url(destination.name)
  ) + (params.size > 0 ? create_other_parameters(params) : "")
end                                                

The method string_for_url() just replaced spaces with underscored and removed illegal characters.

The create_other_parameters() appended parameters in a subtle way that ensured that Google and Yahoo! wouldn’t get prejudice about dynamic pages with parameters. (This is another topic for another time.)

In short, now we can simply call custom_d_path(destination) from any view (or controller if we included the helper in both ActionView and ActionController classes).

I realize that there may be a better way to do this to make it simpler to code, but this is a simple example of a way to solve this problem.

Now for a couple of caveats:

1. For those who have OCRD (obsesive compulsive REST disorder) the urls may not suite your style. I use them for the read only pages of a site.
2. You may not need to go to this extreme to keyword pack your urls… there are many other approaches that may be more robust and easier to implement.

Hopefully this example helps someone. :)

A half hour ago I released a game that I designed with the help of a friend and the input from my team members. The game took just under three weeks from inception to completion, and I also moved with my family to a new apartment as well.

It was fun to work on this project and I hope that it is successful.

Oh, on a more technical note, I wrote the game using Rails 2.0 with no plugins and only a handful of tables. It was fun on every level.

Now here comes the real question… can you beat me to where ever it is you want to go? If you are reading this, please invite me to your race (Paul Hepworth/ dderby[at]omniop[dot]com.)

Here is the link: http://realtravel.com/ddracing

You can also get to the game by going to any RealTravel destination page and click the “play” button when prompted.

Thanks again Phil for your creative flare major contribution.

Almost a year ago I was asked to work on a project with a designer to build a rails site that lists designers and their portfolios. it was a fun project overall and was during my nights and weekends over the period of a couple of months. Although my work was done for quite sometime the site didn’t technically release until a short time ago. Anyways, I coded it up and although the project was a little too large for my side-project taste it was fun.

You can check it out by going to: http://indiefashionhouse.com

Recently I have been configuring Mongrel and Capistrano for a few apps that I have been working on. I use FastCGI on shared hosts, but if I have a bit more control on a VPS or a dedicated server I prefer to use Mongrel since it is easier to install and configure and it is also more stable. Here is what I do to get a Mongrel cluster up and running using Capistrano and the extension that Jamis Buck also wrote named Multistage.

Make sure you have ruby, rails, capistrano, capistrano-ext, mongrel, palmtree and mongrel_cluster gems installed on your system.

To generate your cluster files you can run the following commands with your specific details of course. ;)

~ $ cd rails_app
~/rails_app $ mongrel_rails cluster::configure -e development -c "/home/www/apps/web2.0app.com" -p 8000 -N 4 -C config/mongrel_development.yaml
~/rails_app $ mongrel_rails cluster::configure -e staging -c "/home/www/apps/web2.0app.com" -p 8010 -N 4 -C config/mongrel_staging.yaml
~/rails_app $ mongrel_rails cluster::configure -e production -c "/home/www/apps/web2.0app.com" -p 8020 -N 4 -C config/mongrel_production.yaml

Now all you need to do is configure Capistrano config/deploy.rb file to make use of these cluster files and the Multistage extension for Capistrano. If you don't how to setup Multistage, you can do a search for it or read my other post in it.

require 'palmtree/recipes/mongrel_cluster'
set :stages, %w(staging production development)
set :default_stage, "development"
require 'capistrano/ext/multistage'
...
set(:mongrel_conf) { "#{current_path}/config/mongrel_cluster.yml" }
...
deploy.task :after_update_code, :roles => [:web] do
  desc "Copying the right mongrel cluster config for the current stage environment."
  run "cp -f #{release_path}/config/mongrel_#{stage}.yml #{release_path}/config/mongrel_cluster.yml"
end

In order for the correct mongrel cluster to start you will need to have a recipe in your deploy that ensures that the correct cluster file is renamed to the config/mongrel_cluster.yml and then it is used by Capistranos' mongrel:cluster:restart task.

Admittedly there are many more things that need to be done before and after this little tip, but I thought that this may help someone who may need to bridge their understanding of how easy it is to solve the Mongrel cluster on different stages problem.

A few months ago I was working on a project that required me to setup credit card processing. My client already had a merchant account using the Verifi gateway and instead of talking my client into using a different gateway or doing something from scratch, i figured i would just add support for Verifi for the Active Merchant Rails plug-in and then give back the code to the plug-in project. So if you are using the Verifi payment gateway you have a great rails plug-in with support for it.

Lets be open about this, Rich McIver sent me an email informing me that he had written an article on "74 Quality Ruby on Rails Resources and Tutorials." My immediate thought was "what? who? why?" but then I went to the article and after reading through it I found myself interested in the articles that he had put together. Good job Rich for the great article and for the 74 handpicked resources -- even if the number 74 is one article shy of 75. Okay, who is going to write the 75th resource now?

Oh, yeah! before I forget, here is the link to Richs article.

74 Quality Ruby on Rails Resources and Tutorials

With all of the resources out there on Rails, its nice to have a quality list.

I have been attending the Silicon Valley Ruby on Rails Meetup since the first meeting a year ago this next May (or so.) I am surprised at the number of people that show up and their purpose in coming. it started out with just a few computer geeks who love Ruby and Rails and now I think half of the attendees tonight were VCs, companies looking for Rails engineers, or people looking for partners for their next big idea for a web app.

I think its great but I have to admit it must be a sign of the times.

Ruby on Rails is here to stay and even companies like Spock (spock.com) are building large apps with it with 9M$ in VC.

now all I have to do is figure out a way to use Rails more since I don’t get the pleasure currently at my day job. :)

On November 9th I attended the Ruby on Rail Camp unconference that was help at the IBM research labs on San Jose. Not only was it fun to attend but great to see so many rails enthusiasts. I attended various unconference style workshops that covered everything from MemCacheD to Test-driven development. The unconference started out with a discussion on Rubys' (and Rail) performance and asked the question: Is Ruby on Rails ready for business? I was impressed with how level headed everyone seemed to be -- no one was so blind to Ruby on Rails weaknesses (performance) that the conversation became irrational. I concluded personally that it was ready for business but that there is a lot of room for performance improvements and that Rails, although really powerful and agile, is not the best tool for every project. Ruby on Rails, like so many other language frameworks is good at certain types of projects in certain types of teams. Personally I like the creating and being apart of the businesses, projects, and teams that would see Ruby on Rails as the most agile and versatile development framework. Rails Camp was a blast! I look forward to attending the next one.

Yesterday evening I was pleased to be able to attend the Silicon Valley Ruby on Rails Meetup. There were two presenters, Graham Glass and Jonathan Palley, who showed us all the ins-and-outs of their rails apps. Graham's application, EDU 2.0 (edu20.com), was an online education site that allowed collaboration for students and teachers or just individuals who want to learn more about specific subjects. I look forward to seeing it go live in the coming months while he releases a public beta. Jonathan Palley mainly showed us how to setup a rails app to interface with a telephone using VoIP. He was using the Astricks open source PBX server and rails in his demo. It was very cool. He was kind enough to show us his code even though he told us all that it was not finished with his implementation of responds_to: voice -- none the less it was impressive.

Last Thursday I attended a Ruby on Rails workshop in San Francisco put on by Carson Workshops and presented by Geoffrey Grosenbach (topfunky). It was a good workshop for those of us who have played with Rails and are starting to get a little bit more serious about it. It was nice to get a more general exposure to the framework that what I had from reading the first edition of Agile Web Development with Rails. One thing a came away from the workshop completely exited about was that ROR is not only a framework, but a complete turnkey solutions for developing and deploying web apps that scales from a one man development team to as many developers that it may take to develop the app you are working on (usually not more than four or five.) It was also fun to meet some other cool guys the night before at an informal little get-together. (click for pic) BTW, I am the guy who is standing between Geoff and Josh in the back fighting to be seen in the pic. ;)

I just started playing with Ruby on Rails a little while ago and find it very interesting to say the least. I was a little skeptical when I heard David Heinemeier Hansson on many blog posts and podcasts talk about how enjoyable coding web apps with Ruby on Rails was. One thing that bothered me was that in all of the press and comparisons of ROR to other frameworks done on the ROR podcast and other podcasts and blogs rarely would compare PHP to Ruby. Finally last week I was listening to an interview of David on the Web 2.0 podcast, I learned that David used to use PHP and then found Ruby and fell in love with the language. That was insightful to me. I am no expert (yet, give me a month or so) but as I have played with both the language and the framework, I too am starting to believe. :) I look forward to creating some web apps.