Pony 0.21.0 is a recommended update. It's a huge release for us as it contains many stability fixes as well as a variety of other improvements. It contains a breaking change related to as and match statements. Updating existing codebases should be relatively easy.

Generalised runtime backpressure

There's a ton of significant changes in 0.21.0. Our headliner is generalised runtime backpressure. The Pony scheduler has been updated to apply backpressure between actors. I'll be writing a more in-depth blog post on the topic.

Before the addition of runtime backpressure, it was possible to create Pony programs that would be able to cause runaway memory growth due to a producer/consumer imbalance in message sending. There are a variety of actor topologies that could cause the problem.

Because Pony actor queues are unbounded, runaway memory growth is possible. This commit contains a program that demonstrates this. examples/overload has a large number of actors sending to a single actor. Under the original scheduler algorithm, each of these actors would receive a fairly equivalent number of chances to process messages. For each time an actor is given access to the scheduler, it is allowed to process up to batch size number of messages. The default for batch size is 100. The overload example many many actors sending to a single actor where it can't keep up with the send.

With this feature, the Pony scheduler can now apply backpressure. The basic idea is:

Pony message queues are unbounded

Memory can grow without end if an actor isn't able to keep up with the incoming messages

We need a way to detect if an actor is overloaded and if it is, apply backpressure

We apply backpressure according to the following rules:

If an actor processes batch size application messages then it is overloaded. It wasn't able to drain its message queue during a scheduler run.

Sending to an overloaded actor will result in the sender being "muted."

Muting means that an actor won't be scheduled for a period of time allowing overloaded actors to catch up.

Particular details on this

Sending to an overloaded or muted actor will result in the sender being muted unless the sender is overloaded.

Muted actors will remain unscheduled until any actors that they sent to that were muted/overloaded are no longer muted/overloaded

The basics of backpressure are in place. Still to come:

Backpressure isn't currently applied to the cycle detector so its queue can still grow in an unbounded fashion. More work/thought needs to go into addressing that problem.

It's possible that due to implementation bugs that this commit results in deadlocks for some actor topologies. I found some implementation issues that had to be fixed after my first pass. The basic algorithm though should be fine.

There are some additional work items that could be added on to the basic scheme. Some might turn out to be actual improvements; some might turn out to not make sense.

Allow for notification of senders when they send to a muted/overloaded actor. This would allow application level decisions on possible load shedding or other means to address the underlying imbalance.

Allow an actor to know that it has become overloaded so it can take application level

Allow actors to have different batch sizes that might result in better performance for some actor topologies

This work was performance tested at Wallaroo Labs and was found under heavy loads to have no noticeable impact on performance.

Runtime stability improvements

This release address many edge cases that could lead to a lack of Pony runtime stability. Some issues related to possible runaway memory growth have been closed as well as a variety of smaller issues.

Add DTrace probes for all message push and pop operations

Scott Fritchie, while helping diagnose several of the runtime stability issues that are fixed in this release, realized Pony's DTrace coverage was lacking. Some message sends between actors where being traced but other paths were not. Scott updated and refactored DTrace support for message sends so that all messages will now be accounted for. If you are using DTrace for observability of Pony programs, this is a huge moment for you. Things just got much easier. Thanks, Scott!

Experimental support for LLVM 4.0.1 and 5.0.0

LLVM 4 and 5 can now be used to build Pony. However, support is experimental. To work around a bug that caused Pony to segfault, we had to introduce a performance degradation in Pony when building with LLVM 4 and 5. Additionally, it's possible that your programs might crash. If you can build with LLVM 4 and 5, we encourage you to do so and give us feedback. We do not recommend LLVM 4, and 5 be used to build Pony programs that will be used in a production environment. To follow our progress with making LLVM 4 and 5 officially supported, please see issue #2371.

Error on unreachable cases in match expressions and illegal as expressions

This change is adding some sanity checks to usages of match and as expressions.

It does two things:

Match Expressions

It validates that there is no unreachable code left in your match expressions. This was a subtle source of bugs and left dead code to linger in your codebase. Unreachable cases or else clauses are triggering a compiler error now.

Example:

class Foo fun ex_match (a: ( String | Bool )): String => match a | let a: Stringable => a. string () | let b: Bool => if b then " true " else " false " end // unreachable already else " unreachable " end

The second branch and the else clause are both unreachable and thus can (and should) be safely removed. In some cases it might instead make sense to rewrite the match , reordering the cases from more specific checks to less specific ones.

As Expressions

It also validates the correct use of as , which should only be used to safely increase the specificity of an object's type, that is casting. Previously it was possible to cast to the type of the expression to be cast or to one of its subtypes, which can be achieved by simple assignment. Using as here introduces a false positive partiality. Those incorrect or unnecessary usages of as trigger a compiler error with this change.

Example:

class Foo fun subtype_cast (a: String ): Stringable ? => a as Stringable // useless as, is actually not partial fun eq_cast (a: Array [ String ]) => let tmp = a as Array [ String ] // no as needed ...

This error can easily be fixed by removing the as as it is not necessary in both cases.

[0.21.0] - 2017-12-17

Fixed

Added

Changed