Overview

One objective is avoid blocking the main loop for extended periods (in order to improve interactivity and reduce latency), and kaa.base provides two convenient approaches to accomplish this with asynchronous programming: coroutines, and threads.

Coroutines are used to break up large and computationally expensive tasks into smaller tasks, where control is relinquished to the main loop after each smaller task. Coroutines are also very useful in constructing state machines. In the event where blocking is unavoidable, and the duration of the block is unknown (for example, connecting to a remote host, or scaling a very large image), threads can be used. These two different approaches are unified with a very similar API.

InProgress Objects

Throughout Kaa, when a function executes asynchronously, it returns an InProgress object. The InProgress object is a signal that can be connected to in order to handle its return value or any exception raised during the asynchronous execution. When the InProgress object is emitted, we say that it is "finished."

InProgress objects are emitted (they are Signal objects, remember) when finished, so handlers can retrieve the return value of the asynchronous task. There is also an exception member, which is itself a Signal, and is emitted when he asynchronous task raises an exception. Exception handlers must accept three arguments: exception class, exception instance, and traceback object. InProgress objects have the following methods:

connect: connects a callback to be invoked when the InProgress has returned normally (no exception raised)
connect_both: connects a single callback to both normal and exception handlers. (The callback must accept either one argument, for the result, or three arguments, for an exception.)
get_result: returns the result of the asynchronous task. If the InProgress is not yet finished, it raises a RuntimeError exception. If the asynchronous task is finished but raised an exception, then get_result() will re-raise that exception.
wait: blocks until the InProgress is finished, and returns its result or raises its exception. The main loop is kept alive while waiting. Or, if being called from a thread, the thread is blocked until the InProgress finishes. Use of this method is discouraged, but there are some occasions where it is useful. If you use this method, it's good form to document why.
finish: finishes the InProgress object with the return value of the asynchronous task. (You're not likely to call this function directly.)
throw: finishes the InProgress with the exception raised inside the asynchronous task. (Again, you're not likely to call this function directly.)
execute: execute a function with the given parameter and store the result or the exception in the InProgress object.

And the following properties:

finished: True if the InProgress is finished, and False otherwise.
failed: True if an exception was thrown to the InProgress, and False if it is finished without error or if it is not finished.
result: the result of a finished InProgress. Like the get_result() method, accessing this property if the InProgress is not finished will raise a RuntimeError exception. If an exception was thrown to the InProgress, accessing this property will raise that exception.

InProgressAny

InProgressAny objects represent multiple InProgress objects, and is finished when any one of the underlying InProgress objects finishes. The object is finished with a 2-tuple (idx, result) where idx is the index of the underlying finished InProgress (offset from 0 and in the order of the InProgress objects as passed to the InProgressAny constructor), and where result is the result the underlying InProgress finished with. If that InProgress was finished by an exception, then result is a 3-tuple of (type, value, traceback) representing the exception.

InProgressAll

InProgressAll objects represent multiple InProgress objects, and is finished when all of the underlying InProgress objects are finished. The InProgressAll object is always finished with itself (that is in_progress_all.result == in_progress_all). The object is an iterable, and will iterate over all of the InProgress objects passed to its constructor.

inprogress

Similar to __len__ and len(), objects that implement the __inprogress__ method (which takes no arguments) return an InProgress object that represents the progress of the original object. There is a method kaa.inprogress() which accepts an object and simply calls its __inprogress__ method.

A practical demonstration of this protocol is in the Signal object, which implements the __inprogress__ method. The returned InProgress in that case is finished with the signal is next emitted. Any object implementing the __inprogress__ protocol can be passed directly to the constructor of InProgressAny or InProgressAll.

Coroutines

A function or method is designated a coroutine by using the coroutine decorator. Any function decorated with coroutine will return an InProgress object, and the caller can connect a callback to the InProgress object in order to be notified of its return value or any exception.

When a coroutine yields kaa.NotFinished, control is returned to the main thread, and the coroutine will resume after the yield statement at the next main loop iteration or if an interval is provided with the decorator after this time time interval. When a coroutine yields any value other than kaa.NotFinished (including None), the coroutine is considered finished and the InProgress returned to the caller will be emitted (i.e. it is finished). There is a single exception to this rule: if the coroutine yields an InProgress object, the coroutine will be resumed when the InProgress object is finished.

Here is a simple example that breaks up a loop into smaller tasks:

import kaa

@kaa.coroutine()
def do_something():
   for i in range(10):
      do_something_expensive()
      yield kaa.NotFinished
   yield True

def handle_result(result):
   print "do_something() finished with result:", result

do_something().connect(handle_result)
kaa.main.run()

A coroutine can yield other coroutines (or rather, the InProgress object the other coroutine returns):

@kaa.coroutine()
def do_something_else():
   progress = do_something()
   yield progress
   try:
      result = progress.get_result()
   except:
      print "do_something failed"
      yield

   if result == True:
      yield True
   yield False

In Python 2.5, it is possible for the yield statement itself to return a value or raise an exception. This is supported as well:

@kaa.coroutine()
def do_something_else():
   try:
      result = yield do_something()
   except:
      print "do_something failed"
      yield

   yield True if result else False

Note that if you elect to use this idiom, your code will not run on Python 2.4. (kaa.base itself supports Python 2.4, however, as this syntax is not used internally.) Because of this idiom, in Python 2.5 the yield statement will raise an exception if there is one while Python 2.4 continues and raises the exception when calling get_result. That also means that none of the above two variants will work perfectly with both Python versions, but one would have to wrap the yield in the try..except block:

@kaa.coroutine()
def do_something_else():
   progress = do_something()
   try:
      yield progress                 # may throw in python 2.5
      result = progress.get_result() # may throw in python 2.4
   except:
      print "do_something failed"
      yield

   if result == True:
      yield True
   yield False

Note that the code becomes much more elegant if Python 2.4 compatibility is sacrificed.

With the help of InProgress objects, it is possible to construct non-trivial state machines, whose state is modified by asynchronous events, using a single coroutine.

Threads

Kaa provides a ThreadCallback class which can be used to invoke a callback in a new thread every time the ThreadCallback object is invoked.

With the NamedThreadCallback class, invocations are queued and each executed in the same thread. A priority may also be specified, and NamedThreadCallback objects with the highest priority is first in the queue (and hence executed first). This allows you to create a priority-based job queue that executes asynchronously.

Instances of the two classes above are callable, and they return InProgress objects.

def handle_result(result):
   print "Thread returned with", result

kaa.ThreadCallback(do_blocking_task)(arg1, arg2).connect(handle_result)

Any function or method may be decorated with @kaa.threaded() which takes two optional arguments: a thread name, and a priority. If a thread name is specified, the decorated function is wrapped in NamedThreadCallback, and invocations of that function are queued to be executed in a single thread. If the thread name is kaa.MAINTHREAD the decorated function is invoked from the main thread. If no thread name is specified, the function is wrapped in ThreadCallback so that each invocation is executed in a separate thread. Because these callbacks returns InProgress objects, they may be yielded from coroutines.

(This example uses Python 2.5 syntax.)

@kaa.threaded()
def do_blocking_task():
   [...]
   return 42

@kaa.coroutine()
def do_something_else():
   try:
      result = yield do_blocking_task()
   except:
      print "Exception raised in thread"
 
   print "Thread returned", result

The @kaa.threaded decorator also supports a async kwarg, which is by default True. When True, the decorated function returns an InProgress object. When False, however, invocation of the function blocks until the decorated function completes, and its return value is passed back. This allows a threaded function to be used as a standard callback.

MainThreadCallback

The MainThreadCallback is a callback that will be executed from the main loop. The thread calling this function will return immediately after calling the object without waiting for the result. Invoking MainThreadCallbacks always returns an InProgress object.

def needs_to_be_called_from_main(param):
    print param
    return 5

# ... suppose we are in a thread here ...
cb = kaa.MainThreadCallback(needs_to_be_called_from_main)
print cb(3).wait()

As a rule of thumb, if you have a function that must always be called in the main thread, you would use @kaa.threaded(kaa.MAINTHREAD) as mentioned above, if you need to decide case-by-case, don't decorate it and use MainThreadCallback when needed.