Dear Contributors,

The newest updates on Scala Center projects are here!

Enjoy reading (o:

Darja

At a glance

Scalameta

Ólafur Páll Geirsson @olafurpg

Released Scalameta v2.1.0, see changelog.

Highlights:

#1143 Fix incompatibility with the semanticdb-scalac compiler plugin

and scalamacros/paradise macro annotation.

and scalamacros/paradise macro annotation. #1144 Upgrade semanticdb-scalac to 2.12.4. Due to large internal refactorings

in scalac typer, this upgrade turned out to be unusually complicated.

in scalac typer, this upgrade turned out to be unusually complicated. #1145 Fix bug in handling of by-name and repeated parameters.

#1148 Add Denotation.members . This opens the possibility to reproduce

lexical scoping at a position enabling rules like 1) ExplicitResultTypes

to safely insert unqualified named with accompanying imports or 2) an

OrganizeImports rule to safely.

. This opens the possibility to reproduce lexical scoping at a position enabling rules like 1) to safely insert unqualified named with accompanying imports or 2) an rule to safely. #1154 Fix bug where Foo in new Foo {} resolved to an anonymous class

instead of the Foo symbol.

in resolved to an anonymous class instead of the symbol. #1158 Amongst many things, significantly reduces memory pressure while compiling

sources with the semanticdb-scalac compiler plugin, which is required by Scalafix.

Scalafix

Ólafur Páll Geirsson @olafurpg and Guillaume Massé @MasseGuillaume

Guillaume joins Ólafur on the Scalafix project. He will work on a wide range of topics, including adding new rules and polish the general user experience, which is quite rough at the moment.

Released Scalafix v0.5.4, see changelog.

Highlights:

#412 Make it easy to enable sbt-scalafix for custom configurations like

IntegrationTest. (@olafurpg)

IntegrationTest. (@olafurpg) #415 Suppress false linter errors with comments. This PR makes

it practical to use Scalafix for linting, since previously there was no way

to silence Scalafix for exceptional cases where the linter reports a bogus

diagnostic. (@olafurpg, @MasseGuillaume)

it practical to use Scalafix for linting, since previously there was no way to silence Scalafix for exceptional cases where the linter reports a bogus diagnostic. (@olafurpg, @MasseGuillaume) #418 Added detailed documentation comparing scalafix with alternative tools

like Scala Refactoring, WartRemover, ScalaStyle and IntelliJ Scala plugin. (@olafurpg

like Scala Refactoring, WartRemover, ScalaStyle and IntelliJ Scala plugin. (@olafurpg #419 Simplify installation of sbt-scalafix. Based on feedback from users,

this PR adds a scalafixEnable command that can automatically setup all the

correct scala compiler settings from an sbt shell session, regardless of

existing settings in build.sbt. Previously, it was necessary to manually

update build.sbt with fairly tricky corner cases. (@olafurpg)

this PR adds a command that can automatically setup all the correct scala compiler settings from an sbt shell session, regardless of existing settings in build.sbt. Previously, it was necessary to manually update build.sbt with fairly tricky corner cases. (@olafurpg) #420 Reviewed PR adding documentation for how to use scalafix for Maven builds. (@olafurpg)

#431 Implement a Syntaxtic rule to disable Scala keyword, xml, tabs, carriage return and semicolons (@MasseGuillaume)

Macros

Ólafur Páll Geirsson @olafurpg

A lot of failed experiments trying to automatically expanded trees from

scala-reflect macros that that mix untyped and typed trees.

scala-reflect macros that that mix untyped and typed trees. Studying @odersky’s ideas about a new design for macros that unifies LMS

and macros, see discussion

and macros, see discussion Orienting Ryan Culpepper, the author of Racket macros, on the design challenges in this new macro rework. Ryan has expressed interest in contributing to this project.

Most recently, reading up on Squid, a

metaprogramming framework based on type-safe and hygienic quasiquotes,

developed at the DATA lab at EPFL.

Squid seems to address a lot of the main issues in scala-reflect.

Squid also builds on top of scala-reflect, which means it could maybe provide

a smooth migration story for the existing ecosystem of macros.

I only recently found out about Squid, and I am very excited to look into

it deeper. Tomorrow, I am meeting the author of Squid to get a hands-on

tour.

Compiler Profiling

Jorge Vicente Cantero @jvican

scalac-profiling has been significantly improved to be battle-tested in big,

well-known open source libraries of the Scala community. This work started as

part of my writing of a technical guide to let people in the Scala community

use the plugin to profile their projects. This technical guide will be

published towards the end of November.

Improvements to the project:

Add flamegraph generation for implicit searches (!).

Add dot graph generation for implicit searches.

Add more measurements of the compiler to get more data for

users.

users. Profile the cost of macros and implicit search in all the integration builds.

Add shapeless , scalatest , better-files and scalac as new integration

builds of the

scalac-profiling build (complementing our previous integration builds

circe and monocle ). Every merge in the project is tested against these

projects in the CI to ensure there’s bad interaction.

For a sneak preview, head to this page that explains

the current status of the visualizations of implicit search. If you’re too

busy, check the Flamegraph and dot graph for the example in

the Circe website.

The plan for the technical guide is the following:

Explain the overall idea of scalac-profiling and why it allows people to

fix their compile times.

and why it allows people to fix their compile times. Write the guide by showing how to profile compile times for several use

cases. Every use case will be for some of the integration builds that we

have: shapeless, scalatest, and circe.

Zinc

Jorge Vicente Cantero @jvican

(Read the bolg article)

My work on Zinc has focused on several areas:

Improving classpath hashing for incremental

compilations. The classpath hashing

was added before the Scala Center got involved in the project and created

a severe regression in incremental compiles for users of Zinc 1.0 –

incremental compiles were much slower because it had to hash all the

classpath entries. This regression was measured to be in the range of 15 to

40 seconds in medium to big projects. Fix #127: Use unexpanded names. Reviewed around 8 PRs on our queue; some have been merged, some are still

under work. Fix #436: Remove annoying log4j scripted exception. Work on a better build and test workflow for Zinc. The scripted integration

was slow, and that put off contributions and made changes to Zinc slow to

happen. The rework of the build is splited into several PRs (#438,

#428, #429 and #440) that add: A Zinc scripted mechanism that is reproducible (isolated from users’

cache) and fast (parallel execution of tests). Note that Zinc uses an

independent scripted than sbt. On-merge releases (so that users of Zinc can depend on versions of pull

requests that have been merged right away instead of being blocked on

maintainers to release them). A publishLocal implementation that is cached and avoids publishing when

the same version is already in the cache. Better UI for both the build and the test execution (colors,

highlighting, log collection in an independent tmp folder, etc). Documentation for all the steps of the build and removal of dead code.

Scala Platform

Jorge Vicente Cantero @jvican

Installed the new Drone CLI version ( 0.8 ) in

platform.scala-lang.org for the benefit of the Scala Platform maintainers

and the Scala Center engineers. Coordinated two pull requests for Scala JSON and Better files that

use the new Scala Platform infrastructure. Made several significant improvements to

the Scala Platform plugin, which now fully integrates with

the sbt-release-early and has a detailed test infrastructure.

Scala Improvement Process

Jorge Vicente Cantero @jvican

Coordination with the Committee on several topics. Organisation of a voting in the next SIP meeting to vote for the admission

of a Typelevel representative: Miles Sabin.

SBT 1.x Plugin Migration

As part of SCP-013, we migrated two of the most starred open source sbt plugins that had not been

ported to sbt 1.0 yet:

Guillaume Massé @MasseGuillaume and Martin Duhem @Duhemm

Based on the sbt 1.x migration wiki. We

migrated non-commercial plugins that have an impact on the community:

scala-native is a new target for the Scala programming

language. It uses llvm to link an run Scala code to any platform. It has a seamless integration with sbt. You

can run and test just like you do it on the JVM. PR #1039 (@MasseGuillaume)

Dotty is a research platform for new language concepts and compiler technologies for Scala. It’s tightly

coupled with sbt and Zinc and the migration to 1.X is still in progress. There is more detail on the Dotty

section (@Duhemm)

sbt-dependency-graph is a tool to understand how your

external dependencies are wired togetter. It’s essential to debug anomalies on the classpath. PR #140 PR #139 (@MasseGuillaume)

g8 in an sbt plugin that powers the sbt new command. It allows

the creation of template to quickly create new projects. PR #350 (@MasseGuillaume)

sbt-aws-lambda is an sbt plugin that helps developer in

deploying code to AWS lambda. PR #39 (@Duhemm)

sbt-spark-submit is an sbt plugin that makes it

easy to submit jobs to spark, using spark-submit under the hood.

PR #9. (@Duhemm)

Thanks to the authors of these plugins for the quick reviews!

As far as we can tell, most of the open source, non-commercially owned sbt plugin now support sbt

1.0. Please let us know if we missed something important!

Dotty

Martin Duhem @Duhemm

Specializing functions

The work I started last month (from

an initial PR by Felix Mulder) to specialize

functions in the dotc compiler is working. Our tests show that the code that is produced by dotty

allocates fewer boxes to hold primitive values. However, it turns out that it has a negative impact

on the performance of the compiler and increases compilation

time.

I am working on reducing the overhead of this optimization so that compilation times are not

affected.

The work is happening on this pull request.

Embedding documentation in TASTY

I started working on embedding documentation in TASTY, so that it is always available with the

classfiles of a compiled project. The goal is to be able to easily query the documentation from the

REPL, or an IDE:

scala> object Foo { /** hello! */ def bar = 123 } scala> object Random { | /** Returns a random number | * @return A number chosen by fair dice roll, | * guaranteed to be random. | */ | def getRandomNumber(): Int = 4 | } defined object Random scala> :? Random.getRandomNumber /** Returns a random number * @return A number chosen by fair dice roll, * guaranteed to be random. */ def getRandomNumber(): Int

At the moment, my work in progress can store the documentation strings in a new section of

the TASTY information and the REPL starts to support queries but needs more work.

Improving the Dotty Community Build

The Dotty Community Build is a corpus of open

source Scala projects that we use to detect regression in Dotty. At the time of writing, eight projects

are part of the Dotty Community Build.

Every night, we use the latest nightly release of Dotty to compile those eight projects, and can

therefore quickly detect any regression. However, we didn’t have a simple way to test the Dotty

community build against a pull request, a tag or a specific commit.

I took some time to improve on that and added a new script that allows us, given a reference inside

the Dotty repository, to locally build, publish and test Dotty

against the community build. This way, we can merge with confidence pull requests that may cause a

regression that would be hard to catch with smaller tests.

Here is the pull request.

We do not have integration with our bot at the moment, be I hope to add that soon.

Porting Dotty to sbt 1

The dotty sbt plugin can currently only be used with sbt 0.13. However, now that sbt 1.0 is out, it

makes sense to upgrade to the latest versions of sbt. Future releases of Dotty will drop support for

sbt 0.13.

I started upgrading our build, plugin and compiler

interface to sbt 1.0 and the latest release of

Zinc. Upgrading to the latest version of sbt lets us benefit from the recent improvements that were

made in sbt, and most notably it allows us to enjoy faster incremental builds, thanks to the new

class based name hashing featured in Zinc 1.0.

Because of the many changes that happened in Zinc, especially regarding how a Scala compiler must

communicate API and dependency analyses with the incremental compiler, upgrading our compiler

interface was a large task. We also discovered several bugs in Zinc in that process.

After I got our build to work and all our tests to pass, I needed to upgrade all the projects that

appear in our community build to use sbt 1.0. The changes that were made to upgrade the build

definitions to sbt 1.0 have been proposed for inclusion the upstream projects, and I am waiting to

hear back from them.

The work is happening on this pull request.

Fixing a bug in generic java signatures

My first task with Dotty was to add support for generic java

signatures. See my description in our previous Scala

Center

update

to know what this work is all about.

We discovered a bug with the generic signatures,

where additional parameter lists may be missing.

I fixed this issue in a subsequent pull request.

MOOCs

Julien Richard-Foy @julienrf

Added XQueue support to our grading infrastructure.

Scala.js Bundler

Julien Richard-Foy @julienrf

Made it possible to release scalajs-bundler by pushing a Git tag #184, so that any contributor that has write access to the

scalajs-bundler GitHub repository can cut a release (they don’t need to deal with Sonatype

access rights)

scalajs-bundler GitHub repository can cut a release (they don’t need to deal with Sonatype access rights) Released scalajs-bundler 0.9.0

Collections

Julien Richard-Foy @julienrf

Fixed undefined references in API documentation #274

Ported scalacheck tests of the current collections to the strawman #276

Reviewed @marcelocenerine ’s work on lazyZip . The goal of this operation is to provide

a more discoverable alternative to (xs, ys).zipped . The new syntax is xs lazyZip ys .

This operation lazily zips collections together (the zipped collection is not evaluated

unless it is iterated on) and provides transformation operations (such as map and

filter ) that don’t create intermediate tuples. #223

’s work on . The goal of this operation is to provide a more discoverable alternative to . The new syntax is . This operation lazily zips collections together (the zipped collection is not evaluated unless it is iterated on) and provides transformation operations (such as and ) that don’t create intermediate tuples. #223 Tried an alternative approach to implement MultiMap : create its branch in the

hierarchy. #269

: create its branch in the hierarchy. #269 Fixed performance regressions #270,

#271.

#271. Added an updateWith / updatedWith operation to Map collection types (in collections-contrib)

#268.

Zinc standalone

Martin Duhem and Jorge Vicente Cantero are

collaborating together in a Zinc standalone tool that allows users to consume the incremental compiler via the CLI

Presentations and Workshops

Scala.io

Julien Richard-Foy @julienrf

Presented the ongoing work on the standard collections (slides)

Ran a FLOSS Spree with @MasseGuillaume

(meetup event).

About 25 people hacked on the following projects:

hamsters,

twitter4s,

coursier,

scaladex,

collection-strawman and

cats.

Lambda World

Martin Duhem @Duhemm and Jorge Vicente Cantero @jvican

Organised the Spree at Lambda World in which

around 20 people attended the event. More than 15 pull request was merged in

well-known open source projects, and we had special help from Daniela Sfregola @DanielaSfregola!