ParsaLabs | Blog

A publication about the web and more.

Concurrency in Ruby, Simplified by Celluloid Gem

| Comments

Here is a basic example of concurrent programming in Ruby. I’ll be making use of the awesome Celluloid gem to significantly simplify the process. In this snippet, we are going to process 20 (relatively lengthy) jobs in parallel. Note that MRI (or also called cRuby implementation) does not support true concurrency due to Global Interpreter Lock (GIL). For that reason, you will need to switch to another implementation of ruby. I ran and tested the following code sample on jRuby, so that’s what I suggest you use as well. It’s trivial to install and switch to jRuby with rbenv.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
require "benchmark"
require "celluloid/autostart"

class MyWorker
  include Celluloid

  def do_lengthy_operation(id)
    sleep 2
    puts "worked on id: #{id}"
  end
end

pool = MyWorker.pool(size: 20)

time = Benchmark.measure do
  futures = []
  (1..100).each {|i| futures << pool.future.do_lengthy_operation(i)}
  futures.map(&:value)
end

puts time

We start by defining a worker class that performs an operation that takes at least 2 seconds to complete, and include the Celluloid goodness into it. Now, if we were to do this 100 times, sequentially, it would take 200 seconds to complete all of the jobs. But thanks to Celluloid we can create a pool of threads, and execute 20 jobs at the same time. That would reduce the processing time down to around ~ 10 seconds. To prove that, we make use of benchmark to measure time taken to run this code. Sure enough, running the script produces the following results/output for me in the console:

1
1.960000   0.130000   2.090000 ( 10.236000)

Feel free, to change the pool size, play with other API such as async or even randomize the sleep time (for example, using rand*10) and see for yourself how each change affects the program execution. Also, don’t forget to checkout Celluloid’s Github page and tutorials for more info.

Comments