Jonathan Ricketson

I had a need for an application that I am writing on Google App Engine for a way to store jobs and then process them all at once. I found the idea for a Memcached lockless queue and created an implementation of it: queue.py

To write to it (I do this from a view where I want to store some stats about the data that was shown in the view):

thisQueue=queue. Queue ( QUEUE_NAME ) def method ( ) : thisQueue. write ( data )

and then later on to read from it. I created a cron job that is executed often.

thisQueue=queue. Queue ( QUEUE_NAME ) def cronMethod ( ) : msg = thisQueue. read ( ) while msg: processMessage ( msg ) msg = thisQueue. read ( )

If you are using this on Google App Engine, you should also be careful that you don’t run out of time to execute. If you get a DeadlineExceededError after you have read, but before you have finished processing, then the message might get lost.