All notable changes to this project will be documented in this file.

1.9.0 – 2020-06-08¶ Added¶ A custom Redis connection can now be passed to the Redis broker via the new client keyword argument. (#274, @davidt99)

keyword argument. (#274, @davidt99) Message priority can now be changed in before_enqueue hooks. (#313, @thomazthz)

hooks. (#313, @thomazthz) Support for storing actor exceptions. (#156)

Support for silent Retries . (#295)

. (#295) Support for expected exceptions via the throws actor option. (#303, @takhs91)

actor option. (#303, @takhs91) Support for changing the consumer class in the RabbitMQ and Redis brokers. (#316, @AndreCimander) Changed¶ Worker processes are no longer daemons. (#289, #294, @takhs91) Fixed¶ A race condition during command line startup where the wrong exit codes could be returned when subprocesses failed. (#286)

A race condition between worker processes and fork processes during boot. (#297)

A logging race condition on Linux. (#171, #286)

fileno has been added to StreamablePipe . (#291, @takhs91)

1.8.1 – 2020-02-02¶ Fixed¶ An issue where an IndexError would be raised when multiple middlewre containing fork functions were defined. (#271)

1.8.0 – 2020-02-02¶ Added¶ Support for forking and running arbitrary functions (so-called “fork functions”). (#127, #230)

The --fork-function flag.

flag. The --skip-logging flag. (#263, @whalesalad) Fixed¶ An issue where the max_age parameter to Callbacks was being ignored. (#240, @evstratbg)

parameter to was being ignored. (#240, @evstratbg) An issue with delaying pipelines. (#264, @synweap15)

An issue where the master process would sometimes hang when stopped. (#260, @asavoy)

An issue where the RedisBroker could sometimes prefetch more messages than it was configured to. (#262, @benekastah)

could sometimes prefetch more messages than it was configured to. (#262, @benekastah) The StubBroker now flushes its dead letter queue when its flush_all method is called. (#247, @CapedHero)

now flushes its dead letter queue when its method is called. (#247, @CapedHero) The RedisBroker now takes the max lua stack size into account. This should fix certain heisenbugs that folks have encountered with that broker. (#259, @benekastah) Changed¶ The RabbitmqBroker now creates its queues lazily. (#163, #270, @timdrijvers)

now creates its queues lazily. (#163, #270, @timdrijvers) The Prometheus middleware no longer depends on file locking to start its exposition server. Instead, it uses the new fork functions functionality to start the server in a separate, unique process. The middleware no longer takes any parameters. While this would normally be a breaking change, it appears those parameters were previously ignored anyway. (#127, #230)

1.7.0 – 2019-09-22¶ Added¶ Generic actors can now be passed custom actor registires. (#223, @jonathanlintott)

--use-spawn command line argument. (#227, #228, @jrusso1020) Changed¶ Uncaught exceptions within workers are now logged as errors rather than warnings. (#221, @th0th)

1.6.1 – 2019-07-24¶ Added¶ RabbitmqBroker now supports multiple connection uris to be passed in via its url parameter. (#216, @wsantos) Changed¶ Updated allowed version range for prometheus-client. (#219, @robinro)

1.6.0 – 2019-05-02¶ Added¶ dramatiq_queue_prefetch environment variable to control the number of messages to prefetch per worker thread. (#183, #184, @xelhark)

environment variable to control the number of messages to prefetch per worker thread. (#183, #184, @xelhark) The RabbitMQ broker now retries the queue declaration process if an error occurs. (#179, @davidt99)

Support for accessing nested broker instances from the CLI. (#191, @bersace)

Support for eagerly raising actor exceptions in the joining thread with the StubBroker . (#195, #203)

. (#195, #203) Support for accessing the current message from an actor via CurrentMessage . (#208) Changed¶ Pinned pika version >1.0,<2.0 . (#202) Fixed¶ An issue where workers would fail and never recover after the connection to Redis was severed. (#207)

pipe_ignore has been fixed to apply to the next message in line within a pipeline. (#194, @metheoryt)

1.5.0 – 2019-02-18¶ Added¶ The RabbitMQ broker now supports native message priorities. (#157, @davidt99)

Support for specifying the actor class to actor . (#169, @gilbsgilbs) Changed¶ Pika 0.13 is now required. Fixed¶ Consumers are now stopped after workers finish running their tasks. (#160, @brownan)

Worker logging on Python 3.7 is no longer delayed.

1.4.3 – 2019-01-08¶ Fixed¶ Changed license classifier to the correct license. This is why you shouldn’t publish changed before you’ve had coffee, folks!

1.4.2 – 2019-01-08¶ Fixed¶ License classifier in PyPI package. There were no source code changes for this release.

1.4.1 – 2018-12-30¶ Added¶ Support for redis-py 3.x. (#142, @maerteijn) Fixed¶ Workers wait for RMQ messages to be acked upon shutdown. (#148)

Pipelines no longer continue when a message is failed. (#151, @davidt99)

Log files now work under Windows. (#141, @ryansm1)

1.4.0 – 2018-11-25¶ Added¶ Barrires . Changed¶ cli.main now takes an optional argument namespace so that users may define their own entrypoints. (#140, @maerteijn)

now takes an optional argument namespace so that users may define their own entrypoints. (#140, @maerteijn) Actor “message received” and “completed in x ms” log messages are now logged with the DEBUG level instead of INFO level. This improves throughput and makes logging much less verbose.

level instead of level. This improves throughput and makes logging much less verbose. The TimeLimit middleware no longer uses signals to trigger time limit handling. Instead it uses a background thread per worker process.

middleware no longer uses signals to trigger time limit handling. Instead it uses a background thread per worker process. Dramatiq now shuts itself down if any of the workers die unexpectedly (for example, if one of them is killed by the OOM killer).

Windows is now supported (with some caveats)! (#119, @ryansm1) Fixed¶ Allow pipe_ignore option to be set at the actor level. (#100)

option to be set at the actor level. (#100) Result encoder now defaults to the global encoder. (#108, @xdmiodz)

Dot characters are now allowed in queue names. (#111)

Tests are now run on Windows. (#113, @ryansm1)

1.3.0 – 2018-07-05¶ Changed¶ Upgraded prometheus_client to 0.2.x.

Bumped pika to version 0.12. Because of this change, the interrupt method on Broker and its usages within Worker have been dropped.

method on and its usages within have been dropped. There is no longer a max message delay. Fixed¶ Brokers can now be passed an empty list of middleware. (#90)

can now be passed an empty list of middleware. (#90) Potential stack overflow when restarting Consumer threads. (#89)

1.2.0 – 2018-05-24¶ Added¶ Support for worker heartbeats to RedisBroker .

. maintenance_chance and heartbeat_timeout parameters to RedisBroker .

and parameters to . Interrupt base class for thread-interrupting exceptions. (@rpkilby)

base class for thread-interrupting exceptions. (@rpkilby) ShutdownNotifications middleware. (@rpkilby) Changed¶ TimeLimitExceeded is now a subclass of Interrupt . Fixed¶ StubBroker.join and Worker.join are now more reliable.

and are now more reliable. Module import path is now prepended to search path rather than appended. This fixes an issue where importing modules with the same name as modules from site-packages would end up importing the modules from site-packages. (#88)

Prometheus middleware no longer wipes the prometheus data directory on startup. This fixes an issue with exporting application metrics along with worker metrics. Deprecated¶ requeue_{deadline,interval} parameters to RedisBroker . These two parameters no longer have any effect.

1.1.0 – 2018-04-17¶ Added¶ confirm_delivery parameter to RabbitmqBroker .

parameter to . dead_message_ttl , requeue_deadline and requeue_interval parameters to RedisBroker .

, and parameters to . url parameter to Redis rate limiter backend.

parameter to rate limiter backend. url parameter to Redis result backend.

parameter to result backend. timeout parameter to all the brokers’ join methods. (#57)

parameter to all the brokers’ methods. (#57) flush and flush_all methods to RedisBroker . (#62)

and methods to . (#62) flush and flush_all methods to RabbitmqBroker . (#62) Changed¶ Cleaned up command line argument descriptions. Deprecated¶ URLRabbitmqBroker is deprecated. The RabbitmqBroker takes a url parameter so use that instead. URLRabbitmqBroker will be removed in version 2.0. Fixed¶ rabbitmq and watch extra dependencies are only installed when they are explicitly required now. (#60, @rpkilby)

and extra dependencies are only installed when they are explicitly required now. (#60, @rpkilby) signal handling from the master process on FreeBSD 10.3. (#66)

reloading now uses sys.executable when exec’ing workers that were started with python -m dramatiq .

when exec’ing workers that were started with . an issue that caused logging to fail when non-utf-8 characters were printed to stdout/err. (#63)

an issue with potentially drifting keys in the WindowRateLimiter . (#69, @gdvalle)

1.0.0 – 2018-03-31¶ Added¶ --log-file command line argument. (#43, @najamansari)

command line argument. (#43, @najamansari) --pid-file command line argument. (#43, @najamansari) Changed¶ Dramatiq is now licensed under the LGPL. Fixed¶ Passing time_limit in send_with_options . (#44)

0.20.0 – 2018-03-17¶ Added¶ --queues CLI argument. (#35) Changed¶ Unhandled errors within workers now print the full stack trace. (#42)

0.19.0 – 2018-01-17¶ Added¶ group and pipeline .

and . retry_when parameter to Retries . Changed¶ RateLimitExceeded errors no longer log the full stack trace when raised within workers.

errors no longer log the full stack trace when raised within workers. Consumer connection errors no longer dump a stack trace.

Consumers now wait exactly 3 seconds between retries after a connection error, rather than using exponential backoff.

0.18.0 – 2018-01-06¶ Added¶ pip install dramatiq[all] installs all deps.

installs all deps. --path command line argument. (#27) Changed¶ pip install dramatiq now installs RabbitMQ and watch deps.

0.17.0 – 2017-12-30¶ Added¶ Callbacks middleware.

middleware. asdict method to Messages . Fixed¶ Pinned pika version 0.11 to avoid an issue where passing heartbeat to RabbitmqBroker in get_broker would raise a TypeError . (#23, @chen2aaron)

0.16.0 – 2017-12-25¶ Added¶ long_running example.

example. scheduling example.

example. Messages now support pluggable Encoders .

now support pluggable . ResultBackends now support pluggable Encoders . Changed¶ Redis result backend is now considerably more resource-efficient (it no longer polls).

result backend is now considerably more resource-efficient (it no longer polls). sys.std{err,out} are now redirected to stderr and line-buffered. Fixed¶ TimeLimit middleware now uses a monotonic clock.

0.15.1 – 2017-12-08¶ Fixed¶ Autoreload now works under gevent.

0.15.0 – 2017-11-24¶ Added¶ Support for Results .

. pool parameter to the Memcached rate limiter backend.

parameter to the rate limiter backend. client parameter to the Redis rate limiter backend.

parameter to the rate limiter backend. --watch-use-polling command line argument. Fixed¶ Fixed bad file descriptor issue during RMQ broker shutdown under gevent.

0.14.0 – 2017-11-21¶ Added¶ dramatiq.Actor.logger .

. Logging statements before and after an actor is called. Fixed¶ class-based actors behave more like normal Python classes now. (#15)

0.13.1 – 2017-11-17¶ Changed¶ Connection and import errors that occur during process boot now log stack traces (@rakanalh).

Added support for Python 3.5 (#7 by @jssuzanne).

0.12.1 – 2017-11-15¶ Fixed¶ An AssertionError after starting the consumer if RabbitMQ is not running (#10).

0.12.0 – 2017-11-14¶ Added¶ Worker.pause and Worker.resume .

and . url parameter to RedisBroker . Fixed¶ Pending interrupt messages are now removed from pika’s queue before cancel is called. This fixes an issue where an AtrributeError was sometimes raised on worker shutdown.

was sometimes raised on worker shutdown. Pika connection reset logs from the main thread are now hidden.

Distribution of dramatiq-gevent (#2).

0.11.0 – 2017-11-09¶ Added¶ SkipMessage middleware error.

middleware error. after_skip_message middleware hook.

middleware hook. join now takes optional min_successes and idle_time parameters. Changed¶ Consumer reconnect backoff factor has been lowered from 10s to 100ms.

URLRabbitmqBroker is now a factory function that creates instances of RabbitmqBroker . Fixed¶ Worker processes no longer use a spinlock to consume messages.

Consumers now use the same idle timeout as workers.

StubBroker no longer declares dead letter queues.

0.10.2 – 2017-11-06¶ Changed¶ pika is now pinned to >=0.10,<0.12 .

0.10.1 – 2017-11-04¶ Added¶ More benchmarks. Fixed¶ StubBroker.flush_all now flushes delay queues.

0.10.0 – 2017-10-30¶ Added¶ URLRabbitmqBroker (@whalesalad).

(@whalesalad). StubBroker StubBroker.flush and StubBroker.flush_all .

and . before_consumer_thread_shutdown middleware hook.

middleware hook. before_worker_thread_shutdown middleware hook. Changed¶ Implementation of the window rate limiter has been streamlined.

Redis requeue is now more efficient.

RabbitMQ enqueue is now resilient to disconnects. Fixed¶ dramatiq-gevent packaging (@bendemaree).

0.9.0 – 2017-10-20¶ Changed¶ Messages are no longer assigned new ids when they are re-enqueued. This makes tracking messages using middleware significantly easier.

The RedisBroker now assigns its own internal message ids.

0.8.0 – 2017-10-19¶ Changed¶ RabbitmqBroker no longer takes a ConnectionParameters param as input. Instead, it builds one based on kwargs.

exec is now used to reload the main process on source code changes when the --watch flag is enabled.

0.7.1 – 2017-10-08¶ Fixed¶ Lua files are now properly distributed with the package.

0.7.0 – 2017-09-13¶ Changed¶ Reworked scheduled messages to improve fairness. Messages are now re-enqueued on the broker once they hit their eta.

prometheus-client has been pinned to version 0.0.20 .

0.6.1 – 2017-07-20¶ Fixed¶ A race condition with calls to cas in the memcached rate limiter backend.

0.6.0 – 2017-07-09¶ Added¶ before and after arguments to add_middleware .

and arguments to . Support for RateLimiters .

0.5.2 – 2017-06-29¶ Changed¶ Changed the default max retries value from None to 20 , meaning tasks are now retried for up to about 30 days before they’re dead-lettered by default.

0.5.1 – 2017-06-28¶ Removed¶ Dropped RabbitMQ heartbeat to avoid spurious disconnects.