Yuki & Moto Press

Gem Developer's Guide (Book Edition)

by Nick Quaranto, Eric Hodel, et al

1. What is a gem? Structure of a Gem The Gemspec

2. Make your own gem Introduction Your first gem Requiring more files Adding an executable Writing tests Documenting your code Wrapup

3. Gems with extensions Gem layout extconf.rb C Extension rake-compiler Gem specification Extension Naming

4. Name your gem Use underscores for multiple words Use dashes for extensions Mix underscores and dashes appropriately Don't use UPPERCASE letters

5. Publishing your gem Introduction Sharing Source Code Serving Your Own Gems Publishing to RubyGems.org Push Permissions on RubyGems.org Gem Security

6. Patterns Consistent naming Semantic versioning Declaring dependencies Loading code Prerelease gems

A1. Security General Using Gems Building Gems Reporting Security Vulnerabilities

A2. Resources Tutorials Presentations Philosophy Patterns Creating Monitoring Hosting and Serving Utilities

A3. Credits Contributors Acknowledgments Hosting

Contents

Notes

This is the original source reformatted in a single-page book edition (using the Manuscripts format).

See the source repo for how the book gets auto-built with "plain" Jekyll - of course - and hosted on GitHub Pages.

Onwards.

1. What is a gem? Contents Structure of a Gem

The Gemspec Contents Unpack the mystery behind what’s in a RubyGem. Structure of a Gem Each gem has a name, version, and platform. For example, the rake gem has a 0.8.7 version (from May, 2009). Rake’s platform is ruby , which means it works on any platform Ruby runs on. Platforms are based on the CPU architecture, operating system type and sometimes the operating system version. Examples include “x86-mingw32” or “java”. The platform indicates the gem only works with a ruby built for the same platform. RubyGems will automatically download the correct version for your platform. See gem help platform for full details. Inside a gems are the following components: Code (including tests and supporting utilities)

Documentation

gemspec Each gem follows the same standard structure of code organization: % tree freewill freewill/ ├── bin/ │ └── freewill ├── lib/ │ └── freewill.rb ├── test/ │ └── test_freewill.rb ├── README ├── Rakefile └── freewill.gemspec Here, you can see the major components of a gem: The lib directory contains the code for the gem

directory contains the code for the gem The test or spec directory contains tests, depending on which test framework the developer uses

or directory contains tests, depending on which test framework the developer uses A gem usually has a Rakefile , which the rake program uses to automate tests, generate code, and perform other tasks.

, which the rake program uses to automate tests, generate code, and perform other tasks. This gem also includes an executable file in the bin directory, which will be loaded into the user’s PATH when the gem is installed.

directory, which will be loaded into the user’s when the gem is installed. Documentation is usually included in the README and inline with the code. When you install a gem, documentation is generated automatically for you. Most gems include RDoc documentation, but some use YARD docs instead.

and inline with the code. When you install a gem, documentation is generated automatically for you. Most gems include RDoc documentation, but some use YARD docs instead. The final piece is the gemspec, which contains information about the gem. The gem’s files, test information, platform, version number and more are all laid out here along with the author’s email and name. More information on the gemspec file Building your own gem The Gemspec Your application, your gem’s users, and you 6 months from now will want to know who wrote a gem, when, and what it does. The gemspec contains this information. Here’s an example of a gemspec file. You can learn more in how to make a gem. % cat freewill.gemspec Gem::Specification.new do |s| s.name = 'freewill' s.version = '1.0.0' s.summary = "Freewill!" s.description = "I will choose Freewill!" s.authors = ["Nick Quaranto"] s.email = 'nick@quaran.to' s.homepage = 'http://example.com/freewill' s.files = ["lib/freewill.rb", ...] end For more information on the gemspec, please check out the full Specification Reference which goes over each metadata field in detail. Credits This guide was adapted from Gonçalo Silva’s original tutorial on docs.rubygems.org and from Gem Sawyer, Modern Day Ruby Warrior.

2. Make your own gem Contents Introduction

Your first gem

Requiring more files

Adding an executable

Writing tests

Documenting your code

Wrapup Contents From start to finish, learn how to package your Ruby code in a gem. Introduction Creating and publishing your own gem is simple thanks to the tools baked right into RubyGems. Let’s make a simple “hello world” gem, and feel free to play along at home! The code for the gem we’re going to make here is up on GitHub. Your first gem I started with just one Ruby file for my hola gem, and the gemspec. You’ll need a new name for yours (maybe hola_yourusername ) to publish it. Check the Patterns guide for basic recommendations to follow when naming a gem. % tree . ├── hola.gemspec └── lib └── hola.rb Code for your package is placed within the lib directory. The convention is to have one Ruby file with the same name as your gem, since that gets loaded when require 'hola' is run. That one file is in charge of setting up your gem’s code and API. The code inside of lib/hola.rb is pretty bare bones. It just makes sure that you can see some output from the gem: % cat lib/hola.rb class Hola def self.hi puts "Hello world!" end end The gemspec defines what’s in the gem, who made it, and the version of the gem. It’s also your interface to RubyGems.org. All of the information you see on a gem page (like jekyll’s) comes from the gemspec. % cat hola.gemspec Gem::Specification.new do |s| s.name = 'hola' s.version = '0.0.0' s.date = '2010-04-28' s.summary = "Hola!" s.description = "A simple hello world gem" s.authors = ["Nick Quaranto"] s.email = 'nick@quaran.to' s.files = ["lib/hola.rb"] s.homepage = 'http://rubygems.org/gems/hola' s.license = 'MIT' end The description member can be much longer than you see in this example. If it matches /^== [A-Z]/ then the description will be run through RDoc’s markup formatter for display on the RubyGems web site. Be aware though that other consumers of the data might not understand this markup. Look familiar? The gemspec is also Ruby, so you can wrap scripts to generate the file names and bump the version number. There are lots of fields the gemspec can contain. To see them all check out the full reference. After you have created a gemspec, you can build a gem from it. Then you can install the generated gem locally to test it out. % gem build hola.gemspec Successfully built RubyGem Name: hola Version: 0.0.0 File: hola-0.0.0.gem % gem install ./hola-0.0.0.gem Successfully installed hola-0.0.0 1 gem installed Of course, the smoke test isn’t over yet: the final step is to require the gem and use it: % irb >> require 'hola' => true >> Hola.hi Hello world! If you’re using an earlier Ruby than 1.9.2, you need to start the session with irb -rubygems or require the rubygems library after you launch irb. Now you can share hola with the rest of the Ruby community. Publishing your gem out to RubyGems.org only takes one command, provided that you have an account on the site. To setup your computer with your RubyGems account: $ curl -u qrush https://rubygems.org/api/v1/api_key.yaml > ~/.gem/credentials; chmod 0600 ~/.gem/credentials Enter host password for user 'qrush': If you’re having problems with curl, OpenSSL, or certificates, you might want to simply try entering the above URL in your browser’s address bar. Your browser will ask you to login to RubyGems.org. Enter your username and password. Your browser will now try to download the file api_key.yaml. Save it in ~/.gem and call it ‘credentials’ Once this has been setup, you can push out the gem: % gem push hola-0.0.0.gem Pushing gem to RubyGems.org... Successfully registered gem: hola (0.0.0) In just a short time (usually less than a minute), your gem will be available for installation by anyone. You can see it on the RubyGems.org site or grab it from any computer with RubyGems installed: % gem list -r hola *** REMOTE GEMS *** hola (0.0.0) % gem install hola Successfully installed hola-0.0.0 1 gem installed It’s really that easy to share code with Ruby and RubyGems. Requiring more files Having everything in one file doesn’t scale well. Let’s add some more code to this gem. % cat lib/hola.rb class Hola def self.hi(language = "english") translator = Translator.new(language) translator.hi end end class Hola::Translator def initialize(language) @language = language end def hi case @language when "spanish" "hola mundo" else "hello world" end end end This file is getting pretty crowded. Let’s break out the Translator into a separate file. As mentioned before, the gem’s root file is in charge of loading code for the gem. The other files for a gem are usually placed in a directory of the same name of the gem inside of lib . We can split this gem out like so: % tree . ├── hola.gemspec └── lib ├── hola │ └── translator.rb └── hola.rb The Translator is now in lib/hola , which can easily be picked up with a require statement from lib/hola.rb . The code for the Translator did not change much: % cat lib/hola/translator.rb class Hola::Translator def initialize(language) @language = language end def hi case @language when "spanish" "hola mundo" else "hello world" end end end But now the hola.rb file has some code to load the Translator : % cat lib/hola.rb class Hola def self.hi(language = "english") translator = Translator.new(language) translator.hi end end require 'hola/translator' Gotcha: For newly created folder/file, do not forget to add one entry in hola.gemspec file, as shown- % cat hola.gemspec Gem::Specification.new do |s| ... s.files = ["lib/hola.rb", "lib/hola/translator.rb"] ... end without the above change, new folder would not be included into the installed gem. Let’s try this out. First, fire up irb : % irb -Ilib -rhola irb(main):001:0> Hola.hi("english") => "hello world" irb(main):002:0> Hola.hi("spanish") => "hola mundo" We need to use a strange command line flag here: -Ilib . Usually RubyGems includes the lib directory for you, so end users don’t need to worry about configuring their load paths. However, if you’re running the code outside of RubyGems, you have to configure things yourself. It’s possible to manipulate the $LOAD_PATH from within the code itself, but that’s considered an anti-pattern in most cases. There are many more anti-patterns (and good patterns!) for gems, explained in this guide. If you’ve added more files to your gem, make sure to remember to add them to your gemspec’s files array before publishing a new gem! For this reason (among others), many developers automate this with Hoe, Jeweler, Rake, Bundler, or just a dynamic gemspec . Adding more directories with more code from here is pretty much the same process. Split your Ruby files up when it makes sense! Making a sane order for your project will help you and your future maintainers from headaches down the line. Adding an executable In addition to providing libraries of Ruby code, gems can also expose one or many executable files to your shell’s PATH . Probably the best known example of this is rake . Another very useful one is prettify_json.rb , included with the JSON gem, which formats JSON in a readable manner (and is included with Ruby 1.9). Here’s an example: % curl -s http://jsonip.com/ | \ prettify_json.rb { "ip": "24.60.248.134" } Adding an executable to a gem is a simple process. You just need to place the file in your gem’s bin directory, and then add it to the list of executables in the gemspec. Let’s add one for the Hola gem. First create the file and make it executable: % mkdir bin % touch bin/hola % chmod a+x bin/hola The executable file itself just needs a shebang in order to figure out what program to run it with. Here’s what Hola’s executable looks like: % cat bin/hola #!/usr/bin/env ruby require 'hola' puts Hola.hi(ARGV[0]) All it’s doing is loading up the gem, and passing the first command line argument as the language to say hello with. Here’s an example of running it: % ruby -Ilib ./bin/hola hello world % ruby -Ilib ./bin/hola spanish hola mundo Finally, to get Hola’s executable included when you push the gem, you’ll need to add it in the gemspec. % head -4 hola.gemspec Gem::Specification.new do |s| s.name = 'hola' s.version = '0.0.1' s.executables << 'hola' Push up that new gem, and you’ll have your own command line utility published! You can add more executables as well in the bin directory if you need to, there’s an executables array field on the gemspec. Note that you should change the gem’s version when pushing up a new release. For more information on gem versioning, see the Patterns Guide Writing tests Testing your gem is extremely important. Not only does it help assure you that your code works, but it helps others know that your gem does its job. When evaluating a gem, Ruby developers tend to view a solid test suite (or lack thereof) as one of the main reasons for trusting that piece of code. Gems support adding test files into the package itself so tests can be run when a gem is downloaded. In short: TEST YOUR GEM! Please! Test::Unit is Ruby’s built-in test framework. There are lots of tutorials for using it online. There are many other test frameworks available for Ruby as well. RSpec is a popular choice. At the end of the day, it doesn’t matter what you use, just TEST! Let’s add some tests to Hola. This requires adding a few more files, namely a Rakefile and a brand new test directory: % tree . ├── Rakefile ├── bin │ └── hola ├── hola.gemspec ├── lib │ ├── hola │ │ └── translator.rb │ └── hola.rb └── test └── test_hola.rb The Rakefile gives you some simple automation for running tests: % cat Rakefile require 'rake/testtask' Rake::TestTask.new do |t| t.libs << 'test' end desc "Run tests" task :default => :test Now you can run rake test or simply just rake to run tests. Woot! Here’s a basic test file for hola: % cat test/test_hola.rb require 'minitest/autorun' require 'hola' class HolaTest < Minitest::Test def test_english_hello assert_equal "hello world", Hola.hi("english") end def test_any_hello assert_equal "hello world", Hola.hi("ruby") end def test_spanish_hello assert_equal "hola mundo", Hola.hi("spanish") end end Finally, to run the tests: % rake test (in /Users/qrush/Dev/ruby/hola) Loaded suite Started ... Finished in 0.000736 seconds. 3 tests, 3 assertions, 0 failures, 0 errors, 0 skips Test run options: --seed 15331 It’s green! Well, depending on your shell colors. For more great examples, the best thing you can do is hunt around GitHub and read some code. Documenting your code By default most gems use RDoc to generate docs. There are plenty of great tutorials for learning how to mark up your code with RDoc. Here’s a simple example: # The main Hola driver class Hola # Say hi to the world! # # Example: # >> Hola.hi("spanish") # => hola mundo # # Arguments: # language: (String) def self.hi(language = "english") translator = Translator.new(language) puts translator.hi end end Another great option for documentation is YARD, since when you push a gem, RubyDoc.info generates YARDocs automatically from your gem. YARD is backwards compatible with RDoc, and it has a good introduction on what’s different and how to use it. Wrapup With this basic understanding of building your own RubyGem, we hope you’ll be on your way to making your own! The next few guides cover patterns in making a gem and the other capabilities of the RubyGems system. Credits This tutorial was adapted from Gem Sawyer, Modern Day Ruby Warrior. The code for this gem can be found on GitHub.

3. Gems with extensions Contents Gem layout

extconf.rb

C Extension

rake-compiler

Gem specification

Extension Naming Contents Creating a gem that includes an extension that is built at install time. Many gems use extensions to wrap libraries that are written in C with a ruby wrapper. Examples include nokogiri which wraps libxml2 and libxslt, pg which is an interface to the PostgreSQL database and the mysql and mysql2 gems which provide an interface to the MySQL database. Creating a gem that uses an extension involves several steps. This guide will focus on what you should put in your gem specification to make this as easy and maintainable as possible. The extension in this guide will wrap malloc() and free() from the C standard library. Gem layout Every gem should start with a Rakefile which contains the tasks needed by developers to work on the gem. The files for the extension should go in the ext/ directory in a directory matching the extension’s name. For this example we’ll use “my_malloc” for the name. Some extensions will be partially written in C and partially written in ruby. If you are going to support multiple languages, such as C and Java extensions, you should put the C-specific ruby files under the ext/ directory as well in a lib/ directory. Rakefile ext/my_malloc/extconf.rb # extension configuration ext/my_malloc/my_malloc.c # extension source lib/my_malloc.rb # generic features When the extension is built the files in ext/my_malloc/lib/ will be installed into the lib/ directory for you. extconf.rb The extconf.rb configures a Makefile that will build your extension based. The extconf.rb must check for the necessary functions, macros and shared libraries your extension depends upon. The extconf.rb must exit with an error if any of these are missing. Here is an extconf.rb that checks for malloc() and free() and creates a Makefile that will install the built extension at lib/my_malloc/my_malloc.so : require "mkmf" abort "missing malloc()" unless have_func "malloc" abort "missing free()" unless have_func "free" create_makefile "my_malloc/my_malloc" See the mkmf documentation and extension.rdoc for further information about creating an extconf.rb and for documentation on these methods. C Extension The C extension that wraps malloc() and free() goes in ext/my_malloc/my_malloc.c . Here’s the listing: #include <ruby.h> struct my_malloc { size_t size; void *ptr; }; static void my_malloc_free(void *p) { struct my_malloc *ptr = p; if (ptr->size > 0) free(ptr->ptr); } static VALUE my_malloc_alloc(VALUE klass) { VALUE obj; struct my_malloc *ptr; obj = Data_Make_Struct(klass, struct my_malloc, NULL, my_malloc_free, ptr); ptr->size = 0; ptr->ptr = NULL; return obj; } static VALUE my_malloc_init(VALUE self, VALUE size) { struct my_malloc *ptr; size_t requested = NUM2SIZET(size); if (0 == requested) rb_raise(rb_eArgError, "unable to allocate 0 bytes"); Data_Get_Struct(self, struct my_malloc, ptr); ptr->ptr = malloc(requested); if (NULL == ptr->ptr) rb_raise(rb_eNoMemError, "unable to allocate %ld bytes", requested); ptr->size = requested; return self; } static VALUE my_malloc_release(VALUE self) { struct my_malloc *ptr; Data_Get_Struct(self, struct my_malloc, ptr); if (0 == ptr->size) return self; ptr->size = 0; free(ptr->ptr); return self; } void Init_my_malloc(void) { VALUE cMyMalloc; cMyMalloc = rb_const_get(rb_cObject, rb_intern("MyMalloc")); rb_define_alloc_func(cMyMalloc, my_malloc_alloc); rb_define_method(cMyMalloc, "initialize", my_malloc_init, 1); rb_define_method(cMyMalloc, "free", my_malloc_release, 0); } This extension is simple with just a few parts: struct my_malloc to hold the allocated memory

to hold the allocated memory my_malloc_free() to free the allocated memory after garbage collection

to free the allocated memory after garbage collection my_malloc_alloc() to create the ruby wrapper object

to create the ruby wrapper object my_malloc_init() to allocate memory from ruby

to allocate memory from ruby my_malloc_release() to free memory from ruby

to free memory from ruby Init_my_malloc() to register the functions in the MyMalloc class. You can test building the extension as follows: $ cd ext/my_malloc $ ruby extconf.rb checking for malloc()... yes checking for free()... yes creating Makefile $ make compiling my_malloc.c linking shared-object my_malloc.bundle $ cd ../.. $ ruby -Ilib:ext -r my_malloc -e "p MyMalloc.new(5).free" #<MyMalloc:0x007fed838addb0> But this will get tedious after a while. Let’s automate it! rake-compiler rake-compiler is a set of rake tasks for automating extension building. rake-compiler can be used with C or Java extensions in the same project (nokogiri uses it this way). Adding rake-compiler is very simple: require "rake/extensiontask" Rake::ExtensionTask.new "my_malloc" do |ext| ext.lib_dir = "lib/my_malloc" end Now you can build the extension with rake compile and hook the compile task into other tasks (such as tests). Setting lib_dir places the shared library in lib/my_malloc/my_malloc.so (or .bundle or .dll ). This allows the top-level file for the gem to be a ruby file. This allows you to write the parts that are best suited to ruby in ruby. For example: class MyMalloc VERSION = "1.0" end require "my_malloc/my_malloc" Setting the lib_dir also allows you to build a gem that contains pre-built extensions for multiple versions of ruby. (An extension for Ruby 1.9.3 cannot be used with an extension for Ruby 2.0.0). lib/my_malloc.rb can pick the correct shared library to install. Gem specification The final step to building the gem is adding the extconf.rb to the extensions list in the gemspec: Gem::Specification.new "my_malloc", "1.0" do |s| # [...] s.extensions = %w[ext/my_malloc/extconf.rb] end Now you can build and release the gem! Extension Naming To avoid unintended interactions between gems, it’s a good idea for each gem to keep all of its files in a single directory. Here are the recommendations for a gem with the name <name> : ext/<name> is the directory that contains the source files and extconf.rb ext/<name>/<name>.c is the main source file (there may be others) ext/<name>/<name>.c contains a function Init_<name> . (The name following Init_ function must exactly match the name of the extension for it to be loadable by require.) ext/<name>/extconf.rb calls create_makefile('<name>/<name>') only when the all the pieces needed to compile the extension are present. The gemspec sets extensions = ['ext/<name>/extconf.rb'] and includes any of the necessary extension source files in the files list. lib/<name>.rb contains require '<name>/<name>' which loads the C extension Further Reading my_malloc contains the source for this extension with some additional comments.

extension.rdoc describes in greater detail how to build extensions in ruby

MakeMakefile contains documentation for mkmf.rb, the library extconf.rb uses to detect ruby and C library features

rake-compiler integrates building C and Java extensions into your Rakefile in a smooth manner.

Writing C extensions part 1 and part 2) by Aaron Patterson

Interfaces to C libraries can be written using ruby and fiddle (part of the standard library) or ruby-ffi

4. Name your gem Contents Use underscores for multiple words

Use dashes for extensions

Mix underscores and dashes appropriately

Don't use UPPERCASE letters Contents Our recommendation on the use of “_” and “-“ in your gem’s name. Here are some examples of our recommendations for naming gems: Gem name Require statement Main class or module ruby_parser require 'ruby_parser' RubyParser rdoc-data require 'rdoc/data' RDoc::Data net-http-persistent require 'net/http/persistent' Net::HTTP::Persistent net-http-digest_auth require 'net/http/digest_auth' Net::HTTP::DigestAuth The main goal of these recommendations is to give the user some clue about how to require the files in your gem. Following these conventions also lets Bundler require your gem with no extra configuration. If you publish a gem on rubygems.org it may be removed if the name is objectionable, violates intellectual property or the contents of the gem meet these criteria. You can report such a gem on the RubyGems Support site. Use underscores for multiple words If a class or module has multiple words, use underscores to separate them. This matches the file the user will require, making it easier for the user to start using your gem. Use dashes for extensions If you’re adding functionality to another gem, use a dash. This usually corresponds to a / in the require statement (and therefore your gem’s directory structure) and a :: in the name of your main class or module. Mix underscores and dashes appropriately If your class or module has multiple words and you’re also adding functionality to another gem, follow both of the rules above. For example, net-http-digest_auth adds HTTP digest authentication to net/http . The user will require 'net/http/digest_auth' to use the extension (in class Net::HTTP::DigestAuth ). Don’t use UPPERCASE letters OS X and Windows have case-insensitive filesystems by default. Users may mistakenly require files from a gem using uppercase letters which will be non-portable if they move it to a non-windows or OS X system. While this will mostly be a newbie mistake we don’t need to be confusing them more than necessary. Credits This guide was expanded from How to Name Gems by Eric Hodel.

5. Publishing your gem Contents Introduction

Sharing Source Code

Serving Your Own Gems

Publishing to RubyGems.org

Push Permissions on RubyGems.org

Gem Security Contents Start with an idea, end with a distributable package of Ruby code. Ways to share your gem code with other users. Introduction Now that you’ve created your gem, you’re probably ready to share it. While it is perfectly reasonable to create private gems solely to organize the code in large private projects, it’s more common to build gems so that they can be used by multiple projects. This guide discusses the various ways that you can share your gem with the world. Sharing Source Code The simplest way (from the author’s perspective) to share a gem for other developers’ use is to distribute it in source code form. If you place the full source code for your gem on a public git repository (often, though not always, this means sharing it via GitHub), then other users can install it with Bundler’s git functionality. For example, you can install the latest code for the wicked_pdf gem in a project by including this line in your Gemfile: gem "wicked_pdf", :git => "git://github.com/mileszs/wicked_pdf.git" Installing a gem directly from a git repository is a feature of Bundler, not a feature of RubyGems. Gems installed this way will not show up when you run gem list . Serving Your Own Gems If you want to control who can install a gem, or directly track the activity surrounding a gem, then you’ll want to set up a private gem server. You can set up your own gem server or use a commercial service such as Gemfury. RubyGems 2.2.0 and newer support the allowed_push_host metadata value to restrict gem pushes to a single host. If you are publishing private gems you should set this value to prevent accidental pushes to rubygems.org: Gem::Specification.new 'my_gem', '1.0' do |s| # ... s.metadata['allowed_push_host'] = 'https://gems.my-company.example' end See the Resources guide for an up-to-date listing of options for private gem servers. Publishing to RubyGems.org The simplest way to distribute a gem for public consumption is to use RubyGems.org. Gems that are published to RubyGems.org can be installed via the gem install command or through the use of tools such as Isolate or Bundler. To begin, you’ll need to create an account on RubyGems.org. Visit the sign up page and supply an email address that you control, a handle (username) and a password. After creating the account, use your email and password when pushing the gem. (RubyGems saves the credentials in ~/.gem/credentials for you so you only need to log in once.) To publish version 0.1.0 of a new gem named ‘squid-utils’: $ gem push squid-utils-0.1.0.gem Enter your RubyGems.org credentials. Don't have an account yet? Create one at https://rubygems.org/sign_up Email: gem_author@example Password: Signed in. Pushing gem to RubyGems.org... Successfully registered gem: squid-utils (0.1.0) Congratulations! Your new gem is now ready for any ruby user in the world to install! Push Permissions on RubyGems.org If you have multiple maintainers for your gem you can give your fellow maintainers permission to push the gem to rubygems.org through the gem owner command. Gem Security See Security page.