Thanks for the comment. Sorry to have misinterpreted your second comment, but I’m not used to be called a “very skilled coder” and was wondering who that guy Gene might be ;-)

| You don’t seem to understand Twisted’s design, or
| the features it provides, or the motivations behind
| various aspects of its design.

The fact that Gene, like many very skilled coders, can’t make sense of much of Twisted, is kind of the point.

-Ken

Florian - If you want something that is “simpler” and almost, but not quite, exactly like Twisted, http://eblong.com/zarf/zymb/

At first, I didn’t really like the interface stuff (which isn’t really Twisted’s, from what I understand) but it is much nicer to have the methods required defined by code, not just documentation.

Cheers,
mike

Ya and Glyph, you’re a scary man! what else is there to say… ^^

Twisted is slow serving for http. As in requests per second compared to other servers. It’s also lacking some performance features of a fast http server. A really simple benchmarking test to show this is to use the ab tool. Having said that… twisted.web2 is fast enough for me and I use it with love.

“Blasting divmod’s new web client”

web2.client wasn’t written by Divmod; we haven’t even commented extensively on its implementation yet. I wish we had more resources to devote to Twisted’s web infrastructure but it’s not Divmod’s.

“I’m not sure that Zope 2.x users on Windows and BitTorrent users everywhere else have ever noticed that reliability disaster.”

[sarcasm]I’m sure not. It’s just a coincidence that both Zope and BitTorrent have migrated away from asyncore and a homebrew mainloop respectively to use Twisted.[/sarcasm]

In fact, here is a changelog message from BitTorrent:

“2005-10-11: TCP Stack flaking out bug fixed, using Twisted”

Do you think that “TCP stack flaking out” is a “reliability problem”?

“I’m quite convinced that a “monolithic” channel aggregating transport and protocol state is a much more practical interface for applications developers.”

It’s quite clear you’re convinced of this :). You’re also mistaken. Some googling lead me to someone’s masters’ thesis in CS circa 1995, with a nice one-page summary of why composition is generally superior to inheritance:

http://brighton.ncsa.uiuc.edu/~prajlich/T/node14.html

Now, it’s always a judgement call whether to use inheritance or composition (and Twisted uses utility superclasses for all of its composeable bits, so I wouldn’t say that inheritance is always wrong), but replacing inheritance with composition is sort of a basic object-oriented design technique to achieve separation of concerns. Moving bytes around and deciding what bytes mean *are* separate concerns, even if they’re related.

“Glyph seems to have forgotten all lessons about API design for Python. … what I see in Twisted is a Java pattern.”

If that’s what you see, you have overlooked a huge amount of the code in Twisted.

It’s fantastic that Python provides direct bindings to the OS and doesn’t attempt to obscure it. Twisted makes heavy use of this. For example, Twisted provides a GTK+ reactor and a Win32 event reactor specifically so that you can write incredibly platform-specific user interfaces.

However, there is a difference between exposing platform *features* and exposing platform *flaws*. Python does both, but your application or framework is going to have to paper over the flaws to get things done.

What Twisted does is expose a portable API for the things that can be done portably, but expose enough of the implementation that you can tell what’s going on. This provides the benefit of the “Java” approach (portability) but simultaneously the benefit of the “Python” approach (platform-specific APIs, “native”-feeling applications). It avoids the pitfalls of the “Java” approach (a restricted “least common denominator” API that performs poorly everywhere) and also of the “Python” approach (having to rewrite basic portable tools in application code).

In other words, with the Python standard library, you have to be aware of portability issues *all the time*. With Twisted, you can choose to only be aware of portability issues when you actually care about them.

Someone familiar with GTK would realize that the GTK reactor must be implemented with io_add_watch; someone familiar with Win32 would understand that the event reactor must be implemented with WaitForMultipleObjects (or one of its variants). These reactors then expose the platform-specific features you want to use through the normal Python mechanisms for accessing them (the ‘gtk’ module, the ‘win32gui’ module).

“Having all transport and protocol states in one namespace means that they are faster to access and update. Having fewer distinct namespaces to instanciate [sic] produces faster applications too.”

This is simply wrong. I can’t say with confidence that Twisted will be faster, because it doesn’t have the performance-monitoring discipline that I wish it did, but then, neither does Allegra. If one or the other is faster at the moment it is probably an accident. Where is your automated performance tests measuring performance vs. revision? Where is your profiling data? What is your benchmarking methodology? Unless you can quickly answer all of those questions, arguing about hypothetical performance is worse than useless - it detracts from real issues and wastes everyone’s time.

Then you say “namespaces are one honking great idea’, in support of your argument for using fewer, flatter namespaces? What? I don’t necessarily think that the Zen of Python is a good way to win an argument, but you’re not even agreeing with it.

“I’m not sure there is even [an optimization] possible for Twisted.”

FUD. There are many ways to optimize Twisted; one way analagous to the module you posted is Itamar Shtull-Trauring’s “Fusion”, a C++ extension for Twisted to reduce memory copying overhead, and also do partial protocol implementations straight in C++.

http://twistedmatrix.com/pipermail/twisted-python/2004-December/009226.html

It’s definitely also possible to implement the whole reactor in C. This is one of the reasons that the API is well-defined, in terms of interfaces. There have been a few abortive attempts, but it turns out that this is not really worth the trouble, since Twisted can already go wire-speed if you are interested in optimizing as much as possible. Here’s a project which does just that:

http://pythondirector.sourceforge.net/

(Which, by the way, says “twisted is recommended, and asyncore support will be removed in a future version”. Hmm, where have I heard that before?)

The nice thing about Twisted’s separation of transports and protocols is that your entire multiplexing / event loop / scheduling infrastructure could be re-implemented in C, and your application code *wouldn’t even be able to tell*.

As I’ve pointed out, your conclusion is based on incorrect assumptions about a number of things in Twisted, but it includes some other errors that don’t appear to be related to the points you made in the post at all.

“configuration over convention”?

Wrong. Twisted doesn’t even have a configuration-file format. You can load “configuration” which is serialized Python objects, or Python programs which build Twisted objects, and then run them. Twisted almost by definition does not support configuration of any kind.

So, Twisted does not “favor configuration”, since it doesn’t even *have* configuration. You have to implement your own configuration, if you want configuration when you use Twisted.

“purity over practicality”

Wrong. Here are some practical uses of Twisted: http://twistedmatrix.com/trac/wiki/SuccessStories

“little applications”

Wrong. Let me give you the beneift of the doubt for a moment: maybe Twisted is wrong for your *particular* project for some reason that I don’t understand. I don’t really think so, but even if that’s true, take a look here for a number of other applications: http://twistedmatrix.com/trac/wiki/ProjectsUsingTwisted

These are a wide variety of applications which use Twisted, many of which (as I have mentioned repeatedly) originally used some other mainloop - including Medusa - and replaced it with Twisted.

So here’s my conclusion.

You don’t seem to understand Twisted’s design, or the features it provides, or the motivations behind various aspects of its design. I’ve commented on various posts here to try to explain it, but after this post I am starting to think that you are deliberately misunderstanding or misrepresenting Twisted in order to make Allegra look better by comparison. Obviously you might fool some people, but is that really what you want to do?

I would like to finish with something nice, but it’s hard for me to comment positively on Allegra because you are intent on making design errors that I have dedicated a disproportionate amount of my life trying to eliminate from Python programs. The advantages to your approach that you keep citing - simplicity, fewer lines of code - don’t actually exist. The Twisted echo server is less than half as long as the asyncore, or “allegra” equivalent. Not only that, but you are attempting to convince others that your approach is better because of these imagined advantages.

What could I write here that would make you stop making up problems with Twisted that aren’t real, and take a look at the many good features it has? It’s far from perfect, but we could use the *help* of people like you with novel application ideas to improve it with real code, rather than just spreading FUD on blogs.

For example, we really could use some more developers for the HTTP client.