Testing Javascript d3 visualisations with jasmine and jasmine-jquery

There’s lot of d3 examples on the web but they mainly very simple tutorial examples, which is great for getting started but if you plan to have a complex set of visualisations you want to avoid duplicating JS in your different visualisations and do automated testing.

I’ve created an example of how to write testable d3 javascript with inheritance so multiple visualisations can share common functionality. Having a base object for your visualisations means you can write common code once and automated testing allows you to validate changes without manual testing. Obviously there is a limit to how much you can or should test your visualisations without looking at them in browser, but this allows you to write testable code, do TDD coding and encourage re-use in your javascript.

D3 Testing

Using jasmine to test the Javascript, extending jasmine with jasmine-jquery for jquery selectors and ruby (1.9.2+) to install dependencies (not required but makes life easier). I’ll also use jasmine-headless to run tests without opening the browser.

Jasmine suite

Links:

Advertisements

Capistrano for Ruby/Sinatra deployments onto server without source access

I put off using Capistrano as I thought it would take as much effort to learn as msbuild or ant, but after trying it I regret not doing it sooner. It’s really simple to use, has great source integration and offers a very reliable process for deployments using ssh onto unix boxes. It works well with other languages/frameworks, so you can also deploy your Java/Scala/php projects. The focus of this post is deploying a simple Ruby/Sinatra app from git onto a server which doesn’t have direct access to source (e.g. private git host).

To use you will need SSH access to your servers

First install Capistrano:

gem install capistrano

Prepare your project (in project dir, will generate default deployment files):

capify .

Ignore the Capfile created, it’s pretty standard and you shouldn’t have to touch it. Edit the config/deploy.rb file and clear down, this contains some standard Rails deployment stuff which you won’t need.

set :application, “My Sinatra Application”

set :scm, :git

set :repository,  “git@github.com:mygit/myproject.git”

set :deploy_via, :copy # makes capistrano clone and copy source locally rather than on server

set :deploy_to, “/var/www/mysite.com” # current release will be symlinked in ‘/var/www/mysite.com/current’

set :user, “deploy” # user used to ssh onto server and perform tasks

server “mysite.com”, :app, :web, :db, :primary => true # server to deploy to, must be able to “ssh deploy@mysite.com”

set :use_sudo, false # whether to use sudo on deploy actions

namespace :deploy do

  task :restart, :roles => :web do

    run “echo this is where the actions necessary to restart the application should go”

  end

  task :finalize_update, :roles => :web do

    # Overwrite default rails action and perform any steps before symlink

    run “echo this is the path to the release folder #{ current_release }”

  end

end

Run setup before deployment to create deployment folders:

cap deploy:setup

Check:

cap deploy:check

Deploy!

cap deploy

Now you should have your application copied over to the server under the :deploy_to ‘current’ folder. Old releases (default 5) will be stored in the ‘release’ folder. You can use the restart task to call any commands necessary to restart the application. You can do a lot more complex things, like setup dbs, stage multiple servers simultaneously, tag release in source etc.

Useful links:

Sinatra SSL redirection via nginx unicorn – forbidden error

Recently banged my head against this issue for ages until someone with nginx knowledge pointed out it wasn’t nginx but a sinatra/unicorn issue.

Basically the symtoms were after setting up my Sinatra application, with Unicorn as the webserver and Nginx as the SSL proxy (redirecting http to https for the domain) anytime the application received a post request on my login screen it responded with “forbidden” (403). Get requests for the login screen worked fine.

Initially I thought the error came from Nginx, but it turned out it came from Rack (running via Unicorn). The issue was as part of the login post sinatra-authentication was redirecting to http, not https. Nginx then redirected the client to https but lost the auth details, so Rack gave the forbidden response (with no helpful debug).

To fix this all I had to do was add the following to the nginx conf at the proxy settings:

proxy_set_header X-Forwarded-Ssl on;

 

That told Rack it was working via an ssl proxy and changed the redirects to use https. No clue why I needed to do this, but in future I’ll use something like rack-ssl-enforcer and handle the redirects in the application itself rather than nginx conf.

Links:

 

Unicorn – "’raise_if_conflicts’ can’t activate rack" error

If you’ve been trying to start a Unicorn app and seeing something like the following error:

“‘raise_if_conflicts’: can’t activate rack-1.4.1, already activated rack-1.5”

 

It’s down the versions of rack gem which you installed. In your applications gemfile you should specify the version of rack you want to use, e.g.

gem “rack”, “1.4.1”

Run “bundle update rack” and see if that fixes the issue.

If not, run “gem uninstall rack”, you may see that you have two versions of rack installed, uninstall the wrong one and try again.

Ruby Sinatra Unicorn – Errno::EIO Input/output error

This has caught me out twice now, so typing it up to remember. By default Unicorn outputs stderr/stdout to terminal, which is great if you have one running, but if you don’t (i.e. you logged off) you get a massive error screen were your app was “Errno::EIO Input/output error” in “lint.rb” or something similar.

To fix this just add the following lines to your unicorn.conf file you create for your application, which redirect the output to log files instead:

# By default, the Unicorn logger will write to stderr.# Additionally, some applications/frameworks log to stderr or stdout,# So prevent them from going to /dev/null when daemonized here:stderr_path "/srv/www/logs/unicorn.stderr.log"stdout_path "/srv/www/logs/unicorn.stdout.log"

Ruby CI on Jenkins – Installation steps

Steps for setting up Ruby CI on Jenkins running on EC2 micro instance, Ubuntu 12.04.

  1. sudo apt-get update
  2. Install jre/jdk
    • sudo apt-get install openjdk-6-jre openjdk-6-jdk
  3. Install jenkins
  4. Switch user to jenkins user
    • sudo su – jenkins
  5. Install git
    • sudo apt-get git-core
  6. Set jenkins git user/email
    • git config –global user.email “jenkins@none.com
    • git config –global user.name “jenkins”
  7. Install rvm and install/use ruby 1.9.2 as jenkins user (takes ages…)
  8. Setup jenkins project and add bash script build step to call rake (needs all lines to set paths, install bundler, get gems, run rake)
    • #!/bin/bash
      source ~/.bash_profile
      rvm use 1.9.2
      gem install bundler
      bundle install
      bundle exec rake

Call explicit rake tasks to change build steps to perform build/deploy tasks.

LINUX – Setup Ruby/Sinatra/Unicorn/NGINX server

Quick intro in angry split personality QA format:

Ya What? Unicorns? Sinatra?? Nuggex??? You’re just stringing random words together! Badly!

Ok, it’s a lot of tech words, here’s the breakdown:

  • Ruby – Lovely scripting language that makes it easy to write clear concise code
  • Sinatra – Framework for creating web applications in Ruby, very simple to use
  • Unicorn – Quick and reliable HTTP server for Ruby
  • NGINX – (n-gin-x) HTTP server and reverse proxy, ultra quick for static files

Why? You could be drinking beer at home?

Using Unicorn and NGINX together we get the best of both for hosting our site, Unicorn handles the requests to the Ruby app and NGINX handles the static public files (Images/CSS/JS). GitHub posted a blog why they switched to Unicorn and the benefits.

Fine! Lets get this over with then!

Steps in this sample done in AWS EC2 Ubuntu 11.10 image:

  • Install NGINX
  • Start NGINX and test – “/etc/init.d/nginx start” but location of nginx depends on distro
  • Install Ruby – “sudo apt-get install ruby1.9.1-dev” (actually installs 1.9.2), need dev version for gems
  • Install Ruby gems
    • Install make – “sudo apt-get install make” (may already be installed)
    • Install gcc – “sudo apt-get install gcc” (may already be installed)
    • Install gems – “sudo gem install unicorn sinatra”
  • Check unicorn is working “unicorn -version” (if not check Ruby gems installed and are in path)
  • Make simple sinatra app
    • make a folder for your app e.g. ruby1
    • In the folder write a file “app.rb” with the following content:
    • require ‘rubygems’
      require ‘sinatra’

      get ‘/’ do
        “hello world!”
      end

    • Test site starts with “ruby app.rb“, you can wget the page to check content
  • In same directory write config.ru for unicorn
    • require ‘sinatra’

      set :env, :production
      disable :run

      require ‘./app.rb’

      run Sinatra::Application

  • In same directory write unicorn.conf for unicorn
    • worker_processes 8
      working_directory “/home/USER/APPDIR”
      listen ‘unix:/tmp/basic.sock’, :backlog => 512
      timeout 120
      pid “/var/run/unicorn/basic_unicorn.pid”

      preload_app true
      if GC.respond_to?(:copy_on_write_friendly=)
        GC.copy_on_write_friendly = true
      end

      before_fork do |server, worker|
        old_pid = “#{server.config[:pid]}.oldbin”
        if File.exists?(old_pid) && server.pid != old_pid
          begin
            Process.kill(“QUIT”, File.read(old_pid).to_i)
          rescue Errno::ENOENT, Errno::ESRCH
            # someone else did our job for us
          end
        end
      end

  • Create unicorn running dir
    • “mkdir /var/run/unicorn”
    • Set perms so unicorn running user can write to this dir
  • Start unicorn
    • “unicorn -c unicorn.conf &” should show worker threading starting
  • Add new NGINX conf
    • In /etc/nginx/sites-enabled/, write a file unicorn
    • upstream app_server {
        server unix:/tmp/basic.sock fail_timeout=0;
      }

      server {
        listen 80 default deferred; # for Linux

        client_max_body_size 4G;
        server_name _;

        keepalive_timeout 5;

        # path for static files
        root /home/USERNAME/basic/public;

        # Prefer to serve static files directly from nginx to avoid unnecessary
        # data copies from the application server.
        try_files $uri/index.html $uri.html $uri @app;

        location @app {
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header Host $http_host;
          proxy_redirect off;
          proxy_pass http://app_server;
        }
      }

  • Restart nginx – “sudo service nginx restart”
  • Test your site!