I’ve sometimes wanted to release a gem with an optional dependency. In one case I can recall, it was an optional dependency on Celluloid (although I’d update that to use concurrent-ruby if I did it over again now) in bento_search.

I didn’t want to include (eg) Celluloid in the gemspec, because not all or even most uses of bento_search use Celluloid. Including it in the gemspec, bundler/rubygems would insist on installing Celluloid for all users of my gem — and in some setups the app would also actually require Celluloid on boot too. Requiring celluloid on boot will also do some somewhat expensive setup code, run some background threads, and possibly give you strange warnings on app exit (all of those things at least in some versions of Celluloid, like the one I was developing against at the time; not sure if it’s still true). I didn’t want any of those things to happen for most people who didn’t need Celluloid with bento_search.

But rubygems/bundler has no way to specify an optional gem dependency. So I resorted to not including the desired optional dependency in my gemspec, but just providing documentation saying “If you want to use feature X, which uses Celluloid, you must add Celluloid to your Gemfile yourself.”

What I didn’t like was there was no way, other than documentation, to include a version specification for what versions of Celluloid my own gem demanded, as an optional dependency. You don’t need to use Celluloid at all, but if you do, then it must be a certain version we know we work with. Not too old (lacking features or having bugs), but also not too new (may have backwards breaking changes; assuming the optional dependency uses semver so that’s predictable on version number).

I thought there was no way to include a version specification for this kind of optional dependency. To be sure, an optional gem dependency is not a good idea. Don’t do it unless you really have to, it complicates things. But I think sometimes it really does make sense to do so, and if you have to, it turns out there is one way to deal with specifying version requirements too.

Because it turns out Rails agrees with me that sometimes an optional dependency really is the lesser evil. The ActiveRecord database adapters are included with Rails, but they often depend on a lower-level database-specific gem, which is not included as an actual gemspec dependency.

They provide a best-of-dealing-with-a-bad-situation pattern to specify version constraints too: use the runtime (not Bundler) `gem` method that rubygems provides, as at:

https://github.com/rails/rails/blob/661731c4c83f7d60f6b97c77f008e2f08441e1a1/activerecord/lib/active_record/connection_adapters/mysql2_adapter.rb#L3

This will get executed only if you are requiring the mysql2_adapter. If you are, it’ll try to load the `mysql2` gem with that version spec, in that version of mysql2_adapter `‘~> 0.3.13‘` If the “optional dependency” is not loaded at all (because you didn’t include it in your Gemfile), or a version is loaded that doesn’t match those version requirements, it’ll raise a Gem::LoadError, which Rails catches and re-raises with a somewhat better message:

https://github.com/rails/rails/blob/dfb89c9ff2628da9edda7d95fba8657d2fc16d3b/activerecord/lib/active_record/connection_adapters/connection_specification.rb#L175

Of course, this leads to a problem many of us have run into over the past two days since mysql2 was released. The generated (or recommended) Gemfile for a Rails app using mysql2 includes an unconstrained `gem “mysql2″`. So Bundler is willing to install and use the newly released mysql2 0.4.0. But then the mysql2_adapter is not willing to use 0.4.0, and ends up raising the somewhat confusing error message:

Specified ‘mysql2’ for database adapter, but the gem is not loaded. Add `gem ‘mysql2’` to your Gemfile (and ensure its version is at the minimum required by ActiveRecord).

In this case, mysql2 0.4.0 would in fact work fine, but mysql2_adapter isn’t willing to use it. (As an aside, why the heck isn’t the mysql2 gem at 1.0 yet and using semver?) As another aside, if you run into this, until Rails fixes things up, you need to modify your app Gemfile to say `gem ‘mysql2’, “< 0.4.0″`, since Rails 4.2.4 won’t use 0.4.0.

The error message is confusing, because the problem was not a minimum specified by ActiveRecord, but a maximum. And why not have the error message more clearly tell you exactly what you need?

Leaving aside the complexities of what Rails is trying to do and the right fix on Rails’ end, if I need an optional dependency in the future, I think I’d follow Rails lead, but improve upon the error message:

begin gem 'some_gem', "~> 1.4.5" rescue Gem::LoadError => e raise Gem::LoadError, "You are using functionality requiring the optional gem dependency `#{e.name}`, but the gem is not loaded, or is not using an acceptable version. Add `gem '#{e.name}'` to your Gemfile. Version #{MyGem::VERSION} of my_gem_name requires #{e.name} that matches #{e.requirement}" end

Note that the Gem::LoadError includes a requirement attribute that tells you exactly what the version requirements were that failed. Why not include this in the message too, somewhat less confusing?

Except I realize we’re still creating a new Gem::LoadError, without those super useful `name` and `requirement` fields filled out. Our newly raised exception probably ought to copy those over properly too. Left as an exersize to the reader.

I may try to submit a PR to Rails to include a better error message here.

Optional dependencies are still not a good idea. They lead to confusingness like Rails ran into here. But sometimes you really do want to do it anyway, it’s not as bad as the alternatives. Doing what Rails does seems like the least worst pattern available for this kind of optional dependency: Use the runtime `gem` method to specify version constraints for the optional dependency; catch the `Gem::LoadError`; and provide a better error message for it (either re-raising or writing to log or other place developer will see an error).