tl'dr: sbt Managed dependencies using git submodules on heroku. It can be done.

A post about source compatibility recently stirred up some commotion in the Scala community.

I too have felt some of the compatibility pain, but even though I have only been dancing the Odersky waltz for only a fraction of the time it takes to get on scala’s Dancing with the Stars, I believe I have seen hope. And I believe that source is the answer.

The source of the problem

In recent times, the common tech wisdom says that after 6-7 years, you are pretty much old news. The unfortunate news I have for you is that I think this is actually the time it takes to mature into something both dependable and beautiful. Younger, possibly cuter, programming languages will eventually learn that they will need to evolve and mature too, assuming they stick around for that long.

Evolution is in Scala’s blood. Some languages resist evolution in order to honor dead ancestors by carrying around spirits of the past. I don’t see Scala as a language that attends that same summer camp. As such, during its youth, Scala traded dead weight for a better design. The compromise was backwards binary compatibility. This obviously will cause issues with dependencies, hence the great debacle.

Simple, the way nature intended it

It wasn’t long before Sbt hit the Scala scene lookin’ all hipster-like from the frames of xml-configured eye glasses and roaming ants. Sbt gave us a number of things, one of which was the ability to build Scala software using Scala source code. The issues with source compatibility were known at that time too. Sbt’s the author couldn’t have made is simpler for library authors to wrangle the the issue of publishing binaries for their libraries across any combination of Scala versions you can think of.

The hat trick was to take something like

publish

and turn into something like

+ publish

To this day, it baffles me that some library authors haven’t caught on to this and even complain about it. I can’t see it getting any more simple that adding a single operator. Although I can see some talking issue with it being an operator. ( Just kidding guys :) )

But there was something even more beautiful that came out of those early versions of Sbt: source dependencies, dependencies that could be managed for you and be compiled right along side your code using the same version of Scala.

Fast forward, Let’s git to the point

The latest versions of Sbt also support source dependencies, but through a much more powerful tool, git .

In addition to resolving dependencies through ivy/maven identifiers, Sbt now provides a simple way for declaring managed source dependencies through git uris, another under-rated, and probably under used featured among library authors.

lazy val mahSourceDep = uri("git://github.com/the/awesome.git")

Here’s the nice thing about git. It’s all about the versions!

lazy val mahSourceDep = uri("git://github.com/the/awesome.git#mahbranch") lazy val mahSourceDep2 = uri("git://github.com/the/awesome2.git#mahtag") lazy val mahSourceDep3 = uri("git://github.com/the/awesome3.git#mahsha")

While I agree mvn published-versions of libraries work well for binary dependencies, I can’t really see authors publishing a binary for every bug fix/commit. But you know where I can see them publishing them to often? git .

As such, for agile development and a community-driven workflow, the more I look to a system mostly defined in terms of xml and reversed domain names that don’t really mean anything anymore, the more it feels a bit, antiquated? Don’t quote me on that! I know you’ve been doing this since before I was in short pants. I’m just sayin’.

Heroku me loco

That’s not the only way to play the source fiddle. Once you have your highly-scalable, fault- tolerant web 4.8 online pet store ready to go, the next step is to get it hosted.

My recent love affair with hosting has been with Heroku. Why is Heroku different than other hosting providers? Heroku gets developers. Heroku gets developer workflows. Heroku gets git.

How do you deploy am app on Heroku?

git push heroku master

KA-POW!

Don’t worry Scala coders, Heroku speaks many dialects of sbt.version .

So now that I’ve got you all PUMPED about source dependencies and Heroku. I’m going to tell you a stort fail story. And then I’m going to tell you another story. About not failing. About source. And git. And not failing.

I recently refitted a GAE armored app with the scantily garb of nothingness Heroku requires and tripped on a bump in the cloud.

The way Sbt git source dependencies work is that it clones them into a staging area, outside of your project, before compilation. So what’s the problem with that? When you push to Heroku you are actually triggering the compilation of something called a slug. Heroku is smart enough to know how to detect your flavor of slug (language), but there is one thing you should be aware of.

There ain’t no writing to disk in a cloud an their ain’t no special treatment for staging a clone out side of your slugs play pen!

insert crazy angry face here

Well that’s just great. All this show and dance and no payout. Well, that’s not the end of the story my friends.

If there is once thing for certain with Sbt’s new awesome design, it’s that you’ve got options™.

A good friend of mine pointed out that you can actually use git as unintended with Sbt for profit.

Here are the secret ingredients.

1) Git will, by default, allow you to create submodules in any subdirectory of a repo.

2) Sbt will, by default, resolve “unmanaged dependencies” under your projects ‘lib’ directory.

3) Heroku will allow you enable submodules

So if you declare a submodule for your awesome dependency

$ git submodule add awesome git://github.com/the/awesome.git#mahbranch lib/awesome $ git submodule init $ git submodule update

And tell sbt to depend on it

lazy val mahAwesomeDep = file("lib/awesome")

and enable gitsubmodules on Heroku

$ heroku labs:enable git_submodules

You may just get your wings after all

git push heroku master

Don’t believe me? I’m doing it now.