Twisted is a framework for writing asynchronously networked applications. One of the greatest advantages of Twisted is that it allows developers to integrate many protocols into their applications, which it does by offering consistent APIs to these different protocols.
An overview of several of Twisted’s parts, along with design ideas and code samples, is provided.
In this installment, I’ll give a high-level explanation and rationale for Twisted, an overview of the way Twisted is structured, and some examples of the implementation of simple servers and clients.
Python Twisted Overview
Let’s break the expression down: “a framework for writing asynchronously networked applications”.
First: “a framework”.
Twisted isn’t a typical library. The simplest definition of “framework” would be “a library that calls your code as well as you calling it”.
When writing code that will use Twisted, you should expect to be implementing Twisted-defined interfaces or sub-classing Twisted-defined classes while implementing particular methods that are expected of you.
Twisted uses asynchronous interfaces wherever another library would typically block (and assume that you would use threads to multiplex).
When you’re writing code that uses Twisted, it should never block. There is an event loop, otherwise known as the “reactor”. At the beginning of your program, you’ll do some things that cause the reactor to call your code, and start the reactor.
When events like “reactor started”, “connection made”, or “data received” happen, your code will be called if you’ve registered handlers for these events.
A bit more on “asynchronous”: It’s not always the reactor that receives and triggers the events; sometimes framework code needs to expose an interface for others to call that is asynchronous.
This is what Deferreds are for. A Deferred is basically an abstracted callback, supporting error handling and chaining of callbacks. More on this later.
Twisted provides various levels of abstraction for writing networked applications. Protocols can be implemented independently of their transports.
This means that if you have, say, an IRC client protocol and a SOCKS transport implementation, you don’t need to touch the IRC client protocol implementation to get it to run on SOCKS; you only need to change the part where they’re glued together.
If you’re not working with already defined protocols, and you control both ends of the connection, there is an efficient remote object and method call system called Perspective Broker that lets you deal with APIs instead of byte-streams.
The rationale of Twisted is two-fold: there is the typical reason of enabling the programmer to avoid wasting time on grunt work like implementing protocols and frameworks to build her application on.
The second rationale, though, is integration. We want to ease the integration of various systems into one application. In an email server, for example, one will often want SMTP, IMAP4, POP3, and even a Web interface, in addition to allowing each of these protocols to optionally use encryption (SSL/TLS). It is our hope that Twisted makes it relatively easy to implement systems that integrate so many protocols.
Twisted has many packages; here are the interesting or important ones.
High Level Framework/Application:
Like all properly designed systems, Twisted layers its abstractions so that the developer may use whatever is appropriate for her task.
At the bottom, in twisted.internet, there are the low-level platform-specific reactor implementations that implement an event loop, as well as networking, threading, and other services.
These implementations provide a common, platform-agnostic API, which is the reactor interface defined in twisted.internet.interfaces.
Protocols are implemented on top of this. They are separated from the transport level so that they can run on TCP, SSL, SOCKS, and so on, without changes in their actual implementation (in most cases). Protocols such as HTTP, FTP, DNS, IMAP4, and SMTP are included.
Atop the protocols are the frameworks that help in the writing of applications that use the protocols; for HTTP, there is twisted.web, which exposes an “object publishing” system. twisted.names exposes a DNS framework and twisted.news a netnews framework. These frameworks often contain stand-alone functionality; e.g., you can run a simple static file-serving Web server without writing any code, using twisted.web.
Let’s dive into some code, shall we? Now, I’m not going to take the approach that some other pedagogical articles take. They’ll often have code using lower-level APIs that I wouldn’t honestly recommend for those programs, in order to build up to higher-level concepts.
Instead, I’ll show examples of actual best practices, using our various abstractions, and then, after the examples, explain how things under those abstractions fit together
Python Twisted Example
We always start out with an example of an Echo server, so let’s do that. Run the following snippet with:
$ twistd -y echo.py
from twisted.internet.protocol import Protocol, ServerFactory class Echo(Protocol): def dataReceived(self, data): #As soon as any data is received, write it back self.transport.write(data) factory = ServerFactory() factory.protocol = Echo from twisted.application import service, internet application = service.Application(“echoserver”) internet.TCPServer(1025, factory).setServiceParent(application)