Metaprogramming in Ruby: Advanced Level

This post is the third in a series focused on the application of Ruby metaprogramming. If you're just starting to learn about metaprogramming, "Metaprogramming in Ruby: Beginner Level" is a great place to get started. For those seeking a more practical example of metaprogramming, check out "Metaprogramming in Ruby: Intermediate Level". In this article, we'll discuss how a few popular Ruby gems make use of metaprogramming to solve everyday programming problems.

When you go out for a walk, do you bring a paddle with you? Of course not! A paddle would be pointless. It's only when you're up the creek that a paddle is invaluable.

So too it goes with metaprogramming.

Welcome to the final installment of our Metaprogramming in Ruby series. Up to this point, we've been going through relatively simple, handcrafted Ruby metaprogramming examples. In the beginner level post, we discussed how to use Ruby define_method and send to dynamically create methods. And in the intermediate level post, we put those concepts into practice with a slightly more practical example.

This time, we're going to look at how metaprogramming is used in the wild by diving into the code underlying popular Ruby gems. The libraries we'll be looking at are:

• devise: An authentication library designed for Rails
• factory_bot: A fixtures replacement
• rspec: A testing library

Ruby metaprogramming in the devise gem#

Of the gems we're analyzing today, this is probably the most straightforward example of metaprogramming in Ruby.

When initially setting up devise in your project, you generally run rails generate devise MODEL, where MODEL is the name of your devise model. So if you wanted your users to have a model name of User, you'd run rails generate devise User.

To make this example a little clearer, let's assume you run rails generate devise Gnarnian, which will make your devise model Gnarnian (that's what we call ourselves at Gnar).

After migrating, you'll get access to a slew of helpers created by the generator, including some nifty url and path helpers like new_gnarnian_session_path, gnarnian_session_path, and destroy_gnarnian_session_path.

But that's not all! As you add third-party sign in methods to your app/models/gnarnian.rb file, you additionally get access to those path helpers! For instance, you can add :omniauthable, omniauth_providers: [:google_oauth2] to your devise modules to set up Google sign in; that should provide you with user_google_oauth2_omniauth_authorize_path for redirecting users to Google sign in.

At its core, this is made possible by lib/devise/controllers/url_helpers.rb, which uses define_method to create the path helpers, like so:

# lib/devise/controllers/url_helpers.rb

def self.generate_helpers!(routes = nil)
  routes ||= begin
    mappings = Devise.mappings.values.map(&:used_helpers).flatten.uniq
    Devise::URL_HELPERS.slice(*mappings)
  end

  routes.each do |module_name, actions|
    [:path, :url].each do |path_or_url|
      actions.each do |action|
        action = action ? "#{action}_" : ""
        method = :"#{action}#{module_name}_#{path_or_url}"

        define_method method do |resource_or_scope, *args|
          scope = Devise::Mapping.find_scope!(resource_or_scope)
          router_name = Devise.mappings[scope].router_name
          context = router_name ? send(router_name) : _devise_route_context
          context.send("#{action}#{scope}_#{module_name}_#{path_or_url}", *args)
        end
      end
    end
  end
end

Ruby metaprogramming in the factory_bot gem#

Factory Bot is considered by many to be a must-have tool for testing Rails projects. The factory_bot gem actually uses metaprogramming concepts in a couple of places, but here we'll focus on the use of traits, callbacks, and the evaluator argument.

For the sake of example, let's say you have articles and authors tables. Each Article belongs to a specific Author and each Author can have many Articles. You could create a basic factory like so:

# spec/factories/authors.rb

FactoryBot.define do
  factory :article do
    title { "MyString" }
    body { "MyText" }
    published { false }
    author
  end
end

But Factory Bot also provides callbacks that allow you to execute code for a given strategy. So if you'd like to run code after calling create, then you could write something like after(:create) do <something>.

Now let's assume an Author has written Books that you need to access in your tests; to generate Books written by your Author, you could make something like:

# spec/factories/authors.rb

FactoryBot.define do
  factory :author do
    transient do
      book_names { ['The Color of Magic'] }
    end

    name { 'Terry Pratchett' }
  end

  trait :has_books do
    after(:create) do |author, evaluator|
      Array(evaluator.book_names).each do |book_name|
        create(:book, name: book_name, author: )
      end
    end
  end
end

Notice that you can use evaluator to access transient properties. So in your test, you could write create(:author, :has_books, book_names: ['Equal Rites', 'Mort']), which would create two books with the provided names and the same Author.

This flexible and powerful behavior is facilitated by the Ruby define_method and method_missing blocks found in factory_bot/lib/factory_bot/evaluator.rb.

# lib/factory_bot/evaluator.rb (selected snippets)

def method_missing(method_name, *args, &block)
  if @instance.respond_to?(method_name)
    @instance.send(method_name, *args, &block)
  else
    SyntaxRunner.new.send(method_name, *args, &block)
  end
end

def self.define_attribute(name, &block)
  if instance_methods(false).include?(name) || private_instance_methods(false).include?(name)
    undef_method(name)
  end

  define_method(name) do
    if @cached_attributes.key?(name)
      @cached_attributes[name]
    else
      @cached_attributes[name] = instance_exec(&block)
    end
  end
end

For more detailed examples, check out the getting started docs!

How do you pair method_missing with respond_to_missing? in Ruby?#

Pair them by giving both methods the same rule for which names they handle. Ruby calls method_missing when an object is sent a message it cannot handle, and by default that raises an error. Ruby's respond_to? calls respond_to_missing? when a method is not defined, so that hook should say yes to the same names. For a name you do not handle, call super so an ancestor can pick it up.

factory_bot's evaluator.rb also defines respond_to_missing?, right below its method_missing. Both look at the same two objects: @instance first, then a new SyntaxRunner.

The class below uses the same pair to read names from a hash. It runs on plain Ruby, with no gems.

class Settings
  def initialize(values)
    @values = values
  end

  def method_missing(name, *args)
    if @values.key?(name)
      @values[name]
    else
      super
    end
  end

  def respond_to_missing?(name, include_private = false)
    @values.key?(name) || super
  end
end

settings = Settings.new(timeout: 30)
p settings.timeout
p settings.respond_to?(:timeout)
p settings.method(:timeout).call
p Settings.instance_methods(false).include?(:timeout)

begin
  settings.retries
rescue NoMethodError
  puts "super raised NoMethodError for retries"
end

Running it prints these five lines:

30
true
30
false
super raised NoMethodError for retries

settings.timeout returns 30 through method_missing. respond_to? and method find timeout too, because respond_to_missing? says yes. The fourth line is false: instance_methods(false) lists the public and protected methods the class defines itself, and timeout is not one of them. The last line is super at work: retries is not in the hash, so Ruby raises NoMethodError.

Delete respond_to_missing? and run it again. settings.timeout still returns 30, but respond_to? returns false and method raises NameError.

Ruby metaprogramming in the rspec-core gem#

At long last, we've arrived at our final Ruby metaprogramming example: the 800-lb gorilla that is rspec and its many associated gems. Of the three examples discussed here, rspec uses metaprogramming most extensively. This makes some intuitive sense given that we're talking about a full-fledged testing framework.

That being said, the code and documentation for rspec is generally clear and descriptive, making it a fantastic repository for learning about metaprogramming techniques. In rspec-core/lib/rspec/core/dsl.rb, for example, the implementation is spelled out in code comments.

Let's create an example for illustration:

RSpec.describe "posts/index", type: :view do
  it "renders a list of posts" do
    # ...something
  end
end

# Creates RSpec::ExampleGroups::PostsIndex
# $ RSpec::ExampleGroups::PostsIndex.describe
# $  => RSpec::ExampleGroups::ArticlesIndex::Anonymous
# $ RSpec::ExampleGroups::PostsIndex.examples
# $  => [#<RSpec::Core::Example "renders a list of posts">]

For each spec, RSpec will construct the RSpec::ExampleGroups::Example subclass. Note as well that RSpec::ExampleGroups::Examples::Anonymous.describe will increment the id appended to the end of RSpec::ExampleGroups::Examples::Anonymous (e.g., Anonymous_2, Anonymous_3, Anonymous_4, etc.).

When you create your spec file with RSpec.describe, behind the scenes your arguments are being passed to dsl.rb:

# rspec-core/lib/rspec/core/dsl.rb
# In this case, `args` are `["posts/index", {:type=>:view}]`
# and `example_group_block` is the block of code representing your spec 

(class << RSpec; self; end).__send__(:define_method, name) do |*args, &example_group_block|
  group = RSpec::Core::ExampleGroup.__send__(name, *args, &example_group_block)
  RSpec.world.record(group)
  group
end

Those arguments are then passed to RSpec::Core::ExampleGroup, which determines if the block is top-level and, if so, defines a new ExampleGroup subclass.

# rspec-core/lib/rspec/core/example_group.rb (selected snippets)
# In this case, `name` is `:describe`

def self.define_example_group_method(name, metadata={})
  idempotently_define_singleton_method(name) do |*args, &example_group_block|
    thread_data = RSpec::Support.thread_local_data
    top_level   = self == ExampleGroup

    registration_collection =
      if top_level
        if thread_data[:in_example_group]
          raise "Creating an isolated context from within a context is " \
                "not allowed. Change `RSpec.#{name}` to `#{name}` or " \
                "move this to a top-level scope."
        end

        thread_data[:in_example_group] = true
        RSpec.world.example_groups
      else
        children
      end

    begin
      description = args.shift
      combined_metadata = metadata.dup
      combined_metadata.merge!(args.pop) if args.last.is_a? Hash
      args << combined_metadata

      subclass(self, description, args, registration_collection, &example_group_block)
    ensure
      thread_data.delete(:in_example_group) if top_level
    end
  end

  RSpec::Core::DSL.expose_example_group_alias(name)
end

def self.subclass(parent, description, args, registration_collection, &example_group_block)
  subclass = Class.new(parent)
  subclass.set_it_up(description, args, registration_collection, &example_group_block)
  subclass.module_exec(&example_group_block) if example_group_block

  # The LetDefinitions module must be included _after_ other modules
  # to ensure that it takes precedence when there are name collisions.
  # Thus, we delay including it until after the example group block
  # has been eval'd.
  MemoizedHelpers.define_helpers_on(subclass)

  subclass
end

The returned subclass, in this case, would be RSpec::ExampleGroups::PostsIndex. Calling subclass.module_exec will step through the example_group_block and define Examples (i.e., tests) for the RSpec::ExampleGroups::PostsIndex subclass:

# rspec-core/lib/rspec/core/example_group.rb
# In this case, `name` is `:it`
# and `all_args` is `renders a list of posts`

def self.define_example_method(name, extra_options={})
  idempotently_define_singleton_method(name) do |*all_args, &block|
    desc, *args = *all_args

    options = Metadata.build_hash_from(args)
    options.update(:skip => RSpec::Core::Pending::NOT_YET_IMPLEMENTED) unless block
    options.update(extra_options)

    RSpec::Core::Example.new(self, desc, options, block)
  end
end

Now that the ExampleGroups subclass (i.e., the spec) and its Examples (i.e., the tests within the spec) have been defined, the runner is called to execute the tests!

# rspec-core/lib/rspec/core/runner.rb
# If we're running only this spec, `example_groups` would be
# a single-member array containing `RSpec::ExampleGroups::PostsIndex`

def run_specs(example_groups)
  examples_count = @world.example_count(example_groups)
  examples_passed = @configuration.reporter.report(examples_count) do |reporter|
    @configuration.with_suite_hooks do
      if examples_count == 0 && @configuration.fail_if_no_examples
        return @configuration.failure_exit_code
      end

      example_groups.map { |g| g.run(reporter) }.all?
    end
  end

  exit_code(examples_passed)
end

When does metaprogramming hurt readability and debugging?#

Metaprogramming hurts when a reader cannot find where a method comes from. A method made by define_method is still a real method, so Ruby lists it and can point to the line where its block is defined. A name that only method_missing answers has no definition to point to. That gap is the cost to weigh before you reach for either one.

One way to choose between them:

Devise's url_helpers.rb, shown earlier in this post, is that middle row: it builds its route helpers with define_method from a list it knows.

The class below makes two methods with define_method. It answers any name ending in _total through method_missing.

class Report
  %w[title owner].each do |field|
    define_method(field) { "#{field} value" }
  end

  def method_missing(name, *args)
    name.to_s.end_with?("_total") ? 0 : super
  end

  def respond_to_missing?(name, include_private = false)
    name.to_s.end_with?("_total") || super
  end
end

report = Report.new
p Report.instance_method(:title).source_location
p report.method(:sales_total).source_location
p Report.instance_methods(false).sort

Saved as report.rb and run, it prints:

["report.rb", 3]
nil
[:method_missing, :owner, :title]

source_location returns the file name and line number where a method is defined. For title, that is line 3, the define_method call. For sales_total, which only method_missing answers, it returns nil. The last line lists title and owner, plus method_missing itself, but no sales_total. The beginner post walks through define_method from the start.

A rule of thumb follows from this. If you can write the list of names down, use def for a few and define_method for many. Keep method_missing for names you cannot list, and pair it with respond_to_missing? every time.

Is metaprogramming in Ruby pointless?#

If you've read all three installments in this series, then firstly, thank you! We hope we adequately addressed your Ruby metaprogramming questions, or at least inspired curiosity.

Secondly, you've probably noticed we reiterated ad nauseum that metaprogramming is a "break glass in case of emergency" kind of tool.

So let's wrap this up by addressing the elephant in the room one last time: is Ruby metaprogramming useful to the average developer? Our hope is that these three articles have proven to you that the answer should be an emphatic, "YES"....

…followed by a cautiously muttered, "under the right circumstances and for specific use-cases."

Learn more about how The Gnar builds Ruby on Rails applications.