Getting it

Currently, z3c.tutorialtest is only available from Assembla:

svn co http://svn.assembla.com/svn/z3c_tutorialtest/trunk z3c.tutorialtest

If other people use it I will move it to svn.zope.org and make a release.

Using it

z3c.tutorialtest defines the http://namespaces.zope.org/tutorialtest zcml namespace with a single directive, test. The test directive registers an existing doctest as a TutorialTest, for example:

<tutorialtest:test
    id="businessdevelopment"
    title="Business development introduction"
    test_path="web/ftests/businessdev.txt"
    />

The TutorialTest test runner will set a browser object in the test namespace very similar to an instance of zc.testbrowser.real.Browser. To run the same test as part of your test suite, you'll therefore likely to want to create an instance a different browser class in your setUp code:

from zope.testbrowser.testing import Browser
from dsrscheduler.testing import FunctionalDocFileSuite

def setUp(test):
    test.globs = {'browser':Browser()}

def test_suite():
    return FunctionalDocFileSuite('web/ftests/businessdev.txt', setUp=setUp)

This package creates a new traversal namespace for tests. To view an index of all the available tests, visit /++tutorialtest++index Individual tests can be accessed at /++tutorialtest++<testid>.

To run a test, you will first need to start MozRepl (MozLab 0.1.9 on Firefox 3 works) on port 4242, and open a browser window. If the web application you are testing is running on a different machine, you will need to select "Allow outside connections" before starting MozRepl. Be aware of the security implications of this: if you're not behind a firewall, don't do it!

Then, navigate to ++tutorialtest++index and choose the test you want. The test should now play in your browser. Comments from the doctest are turned into Javascript alert boxes. (Obviously there's room for improvement there!)

Because z3c.tutorialtest runs tests against a live application, it uses the live ZODB and modifies real data. So far I've got round this by creating a new instance of the application, but that's only easy since I'm using Grok. I'm not sure what might be considered good practice here; it certainly requires some thought.

Showing the current branch in the prompt

To show the current git branch in your prompt, edit it to look something like this:

export PS1='\[\033[1;36m\]\u@\h:\w\[\033[0m\]$(__git_ps1 "(%s)")$ '

Autocompletion

git has proper command completion which you'll need to enable, like this:

ln -s /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_devel_git-core/work/git-1.5.3.7/contrib/completion/git-completion.bash ~/
. ~/.git-completion.bash

And whilst we're on the completion theme, here's how to get sudo to do something sensible:

complete -c -f command sudo

Desktop development

Google are providing an SDK which enables you to run the same platform on your desktop in order to develop your app. I haven't looked into this is detail yet. Can you download you DB to test new changes? How do you set up a staging environment? What about running tests?

The datastore

I'm glad to see Google using something which is basically an Object database with simple queries on object types and back-references. Hopefully this will start to spread the concept of object databases into the mainstream, though I suspect that a lot of people are going to complain about the lack of an RDB. In fact, quite a few already are doing.

Google's stated reason for not using an RDB (my feeling is that 'this is better' would be good enough) is that relational databases can't be made to scale in the same way as the datastore. I can't find any reference to how the datastore deals with conflicts, so this doesn't really make any sense. If a user in Australia and a user in the UK make changes to the data, something still needs to check that those changes are not in conflict before both parties can see the results of the changes they made. Maybe instead of a having a single master server App Engine allows for multiple master servers each responsible for part of the database. For a discussion of the issues of distributed RDBs, I highly recommend Scalable Internet Architectures (and the chapter on databases is available for preview), which does what it says on the tin, though it also covers reliability and redundancy, whilst wisely giving optimisation a wide berth.

Effect on the industry

There's an interesting question as to how big an effect App Engine will have on the industry. I can imagine that this changes the landscape significantly for Paul Graham-style web start-ups (without even considering AppJet, who I think it leaves in a significantly worse situation than they were already in). App Engine significantly reduces the complexity of launching a web start-up, and possibly removes the need for investment from the business model completely.

A few commentators are suggesting that this is going to be the start of the PaaS (Platform as a Service) wars. Having control of someone's database is a really good way to tie people to service.

I see that within a few hours of the launch, someone had already produced an OpenID provider application, using the authentication against Google accounts to produce an OpenID provider based on Google logins. Very neat.

Interestingly (to me), Google have included an yaml parser and use yml as the configuration language for applications. When I encountered yaml in my brief visit to the world of Ruby I quite liked it and thought it was quite Pythonic, and I continue to be surprised that it isn't supported by the standard library. Okay, I just looked the the yaml specs. Now I understand.

In summary, I think Google did everything right; including PIL in App Engine would make it really attractive, and overall I'm quite happy. As ever, though, I can't immediately think of any uses for it!

Zimki

About a year ago, a small firm called Fotango launched a server-side JavaScript web development environment called Zimki. At the time, they were offering a MacBook Pro for the best application developed using Zimki, so I had a play with it. It was missing some critical features including database transactions, but I managed to make some simple stuff work with it.

Part of the Fotango's plan was to open-source the code powering Zimki, so that other providers could also host applications developed with it, making it a safer choice for development. But whilst this was the CTO's wish, the owners eventually decided they did not want to release the code, and at this point the CTO left. Soon after Zimki announced that they would be shutting down their service.

Javascript

JavaScript as a server-side language is an interesting choice. It has the obvious advantage that it's well-known, standard and familiar to the vast majority of web developers. That's not true of any other language. And it's not as bad as you might think, since the latest version of JavaScript, as was being used for Zimki, is much better than the elderly versions which have to be used for client-side programming. But at the same time, that means that in many ways it's not the same language that developers know (and hate). In fact, the latest version of JavaScript feels like it's nearly half way to being Python from the early versions.

But the real weakness of using JavaScript is not the language itself but the selection of libraries available for it. Whereas Python and PHP have a vast set of libraries available for web development, JavaScript has relatively few. And writing web applications without libraries for common tasks such as form validation is a pain.

Web-based development

When Zimki first appeared, my first thought was that it was about time, and wasn't through-the-web the obvious way to do web development? I often spend about half my time on system administration, just to create another environment that's basically the same as that thousands of other developers have set up. And by having a hosted setup, data backups are automatically made, and the application can automatically scale across multiple servers. The advantages are certainly significant.

But then, the flexibility of configuring my own system started to become apparent. Installing image libraries, running a mail server, and many more things are going to cause problems if you can't do them. And, perhaps most critically, the development environment is probably never going to be as flexible as developing on your own machine.

After a little while, I wondered how Zimki could possibly be used for anything significant. The lack of libraries and configurability was too limiting. And no-one was going to choose a framework which they couldn't be sure was going to provide the flexibility they needed. So it wasn't a big surprise to me when Zimki announced that they were stopping their service.

AppJet

But now there's a new company in the same game, AppJet. Doomed to go the same way? Well, I'd have thought so, but they are funded by Paul Graham, so I thought I'd look a bit more closely before drawing any conclusions.

So, what does AppJet offer? Does it address Zimki's shortcomings in a lack of mail and image processing libraries? What about form handling and database transactions? Ability to host your site at your own domain?

Well, there's no sign of transactions, making the data storage of dubious value. Interestingly, there is no templating language in appjet. You can print strings, or you can generate tags like this:

print(UL(LI("Ice cream"),
        LI("The sound of the ocean"),
        LI("That song by Bobby McFerrin"),
        LI("The movie ",I("Gone with the Wind")),
        LI("Elegant code"),
        LI("The color ",SPAN({style:"color:green"},"green"))));

which looks a bit smelly. Seaside (Smalltalk) and Uncommon Web (LISP) both have XML tree generation like this, but it doesn't seem right here. There's also something similar in Python called Breve, which I've been intending to look at for a while.

Request dispatch is an interesting topic. The Pylons web framework uses a port of Rails' routes, which defaults to a /controller/action/id style scheme. Zope uses the URL for object traversal, and when that matches your application it's excellent. TurboGears uses a similar object traversal dispatch mechanism from CherryPy. Django uses regular expression to match incoming requests.

AppJet? Well, if you call the dispatch() function, then the url /name will call the function get_name. So there's a way to go there. Similarly the form library doesn't do validation.

So, does AppJet have anything that might make it attractive for some kind of web development? Well, there's a library for accessing the Facebook API. So it might make it easy to produce pointless Facebook applications. But I can't see them making their authors much money, so there won't be much to come back to AppJet.

Conclusion

I'm quite surprised that there isn't anything about AppJet that makes me optimistic. I'm still not convinced that the idea of a hosted application development environment can't work. But AppJet seems to be no nearer than Zimki was.

Roll-your-own

I've been wondering how hard it would be to set up a Python-based TTW environment. IMO the whole hosting environment must be installable on your local machine so that standard development tools can be used. Editor wars are still fun, after all! At this point, what's the point in a hosted system? Well, as soon as you move to a live server --- maybe by just running a script --- then you could have the scalability and backup issues automatically addressed. But then if you changed the data strucutres? What would happen to the database? Hmmm. I still think that there ought to be mileage in this general idea.

The primary reason that I think the system needs to be available locally is so that people can develop and test the libraries they want to use. Maybe if it is possible to install any eggs needed for your application on the server, that would be enough. That might start to address what in my opinion is the biggest issue facing hosted application development.

Hosted systems are always going to be more restricted than having full control over a system, but it is interesting to see how flexible they might manage to become.

Rails?

A few days after writing the above, I was watching a screen-cast on database migrations in Rails. It was quite impressive, but I was as ever intimidated by the need to remember all the conventions (which avoid the configurations) in Rails. But it made me think that maybe Rails is the obvious technology to be offering hosted development for. The vast majority of Rails developers don't try to use their own ORM/templating system/form library. In my opinion, there's much more mileage in hosted Rails development than in hosted JS development. As far as I can make out, no one is doing so as yet. I'm tempted. [Note: I'm not going to develop it in Rails, though. Rails is shockingly slow in some of the screen-casts.]