How to Benchmark Ruby Code

The Benchmark module in the Ruby standard library helps you measure and report the time used to execute your code. This post explores three useful methods in this module, which allow you to measure and compare the execution times of code as well as warm up the code to stabilize the runtime environment.

Measure Time

measure returns the time used to execute the given block as a Benchmark::Tms object. Takes label option.

require 'benchmark'

puts Benchmark.measure { 1_00_000 * 45 }

# 0.000009   0.000008   0.000017 (  0.000006)

The output indicates the user CPU time, system CPU time, total time, and the elapsed real time in seconds.

Compare Time

bm compares various operations sequentially.

require 'benchmark'

n = 5_000_000
Benchmark.bm(5) do |x|
  x.report("for") { for i in 1..n; a = "1"; end }
  x.report("times") { n.times do   ; a = "1"; end }
  x.report("upto") { 1.upto(n) do ; a = "1"; end }
end

#            user     system      total        real
# for     0.309944   0.001038   0.310982 (  0.311954)
# times   0.323748   0.004644   0.328392 (  0.336111)
# upto    0.332247   0.001191   0.333438 (  0.335128)

Warm Up

bmbm performs a rehearsal before the real benchmarking starts to eliminate the costs associated with memory allocation and garbage collection. According to the docs,

Sometimes benchmark results are skewed because code executed earlier encounters different garbage collection overheads than that run later. bmbm attempts to minimize this effect by running the tests twice, the first time as a rehearsal in order to get the runtime environment stable, the second time for real.

Here’s an example from the ‘Programming Ruby’ book. I’ve refactored it slightly.

require 'Benchmark'

DICT = "/usr/share/dict/words"

def read_all
  str = File.read(DICT)
  words = str.scan(/[-\w']+/)
end

def read_lines
  words = []
  File.foreach(DICT) do |line|
    words << line.chomp
  end
end

Benchmark.bmbm(6) do |x|
  x.report("all") { read_all }
  x.report("lines") { read_lines }
end

# Rehearsal ------------------------------------------
# all      0.063806   0.005232   0.069038 (  0.069187)
# lines    0.065963   0.003346   0.069309 (  0.069376)
# --------------------------------- total: 0.138347sec
#
# user     system      total        real
# all      0.048083   0.000751   0.048834 (  0.048901)
# lines    0.075965   0.002674   0.078639 (  0.078782)

I hope this post was useful and you learned something new. I sure did.