Adventures with buildout

Posted by Graham Stratton Wed, 27 Jun 2007 22:22:00 GMT

Recently I wanted to try some modifications to a Pylons project against the latest Pylons trunk without upgrading the system-installed Pylons, so I decided it was about time to try Buildout. Jim Fulton’s buildout (or zc.buildout, since Jim likes to put things in namespaces), is a package to create development and deployment versions of software. It’s not in any way Zope-specific, and whilst it is written in Python primarily for installing Python programs, it’s not really Python-specific either.

It is quite complex. Maybe very complex. But I think that this is only because deployment is a complex business. I can imagine using nearly all the features in the buildout docs, and a few more besides. But I did find a lack of really simple examples for us mortals.

The example I’m going to give here is this: suppose you have a project in a repository. You can check it out, but when you’ve done so you want a way to install all the dependencies, without affecting what’s already installed on your system.

Buildout makes this easy. You need to add two files to your repository:

  • bootstrap.py which installs the latest buildout and setuptools
  • buildout.cfg which contains the configuration for your install

You can get bootstrap.py from the Zope Corp subversion server at http://svn.zope.org/zc.buildout/trunk/bootstrap/

Here’s my buildout.cfg:

[buildout]
parts = foxtrot

[foxtrot]
recipe = zc.recipe.egg
interpreter = python2.5
eggs = python-cjson
       nose
       twill
       pastescript
       pylons == 0.9.5
       sqlalchemy
#       pymssql
find-links = http://python.cx.hu/python-cjson/python-cjson-1.0.3x5.tar.gz
             http://www.mirrorservice.org/sites/download.sourceforge.net/pub/sourceforge/p/py/pymssql/

So my buildout is simple. It only has one part, and the recipe for that is the simple egg recipe zc.recipe.egg. By specifying the interpreter option, I will get a python executable in my bin directory. When I run it, the environment will contain all the specified packages. I have commented out pymssql because it has too many system dependencies.

Packages which are explicitly specified (not installed as dependencies of other packages) and which define scripts to be installed will have those scripts written to ./bin/ instead of the system directory, and will work in the new environment. So I’ve explicitly specified nose and pastescript, so that I end up with ./bin/nosetests and ./bin/paster working with this new environment. Cool, eh?

The ‘find-links’ option is just a list of locations other than the cheeseshop in which to search for packages to install.

Once you have your configuration, setting up the environment is simple, if a little slow (mainly due to pypi being rather lethargic at the moment). First, run 

python bootstrap.py
, to install buildout itself. Then run ./bin/buildout, and everything should magically install. If you want to run it again, it will be much faster with the -N option, which tells buildout not to look for new versions of packages if it already has a version matching the requirements. You can specify version for packages as you would with easy_install.

Well, I thought it all worked. But actually buildout doesn’t seem to use the setuptools that it installs, despite it modifying the path:

$ ./bin/buildout -N
Uninstalling foxtrot.
Installing foxtrot.
Getting distribution for 'Routes>=1.6.3'.
The required version of setuptools (>=0.6c6) is not available, and
can't be installed while this script is running. Please install
 a more recent version first.

(Currently using setuptools 0.6c5 (/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg))
error: Setup script exited with 2

Which is really odd, given that bin/buildout begins with:

#!/usr/local/bin/python2.5

import sys
sys.path[0:0] = [
  '/Users/graham/development/pylons/foxtrot/eggs/zc.buildout-1.0.0b27-py2.5.egg',
  '/Users/graham/development/pylons/foxtrot/eggs/setuptools-0.6c6-py2.5.egg',
  ]

And I had another problem. I wanted to install the development version of pylons into my buildout. So I specified pylons==dev. But buildout gave an error since the version which got installed (0.9.6dev-r2373) wasn’t the version requested. But then specifying pylons >= 0.9.6dev-r2373 worked.

Once you have everything running nicely in your buildout environment, you probably want to then deploy it somewhere. Creating consistent environments between systems in buildout’s main purpose, so how do we do it? It’s described in detail in the buildout docs in the section controlling eggs used.

The configuration file buildout.cfg can contain sections specifying the versions of packages required. To find out what versions are being used, buildout can be run in verbose mode, using the -v option. Strangely, it seems that you then need to manually write these versions into the config file.

Hopefully this simple example will be enough to show the potential of buildout. The complete docs are very thorough and describe all the possible options.

Posted in , ,  | 46 comments

Comments

Leave a response

  1. Graham said 4 days later:

    Apparently the correct way to install a dev version without getting the problem mentioned above is to use

    Pylons==dev,>0.9.5

  2. ebook search said 743 days later:

    It is all so damn difficult to me (

  3. replica watches said 989 days later:

    thanks for the share

  4. May tinh xach tay said 998 days later:

    Great post, thank you for sharing.

  5. ac condensers said 1028 days later:

    thanks for sharing your programming know ledges.

  6. Tiffany silver jewelry said 1037 days later:

    That sound s good.I will try.And I am trying to search Tiffany Jewelry online store

  7. websites said 1049 days later:

    ‘zc.builtout’ is a library entity. Modifications in Pylons are implemented by modifying the library. This is done without upgrading the system-installed Pylons. The other library developed by Jim Fulton is a package that is used to create the development software versions and also so the deployment versions of the same software. This is neither Zope specific nor Phyton specific. Hence the modification to be brought out is quite complex. This is because of the complexity in the deployment stream. The modification is explained trough an example which features the library builtout docs. Builtout makes the modifications little loose by adding two files to the deployment. Hence the modification is compromised.

  8. miami limo said 1051 days later:

    Pylons is essentially a bare-bones wrapper around the WSGI specification that uses 3rd party modules for templating…..

  9. bridesmaid dresses said 1055 days later:

    GOOD POST.

  10. wifi repeater said 1056 days later:

    you have good point here. thanks

  11. Wicker Furniture Sofa said 1076 days later:

    Nice article. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your articles. Anyway I’ll be subscribing to your feed and I hope you post again soon.

  12. buy meridia online said 1096 days later:

    Thanks for your effort for this trick

  13. phentermine no prescription said 1096 days later:

    That’s great! You have made a very useful and informative post. Thanks a lot for sharing.

  14. buy adipex online said 1096 days later:

    Excellent blog post, I look forward to reading more.

  15. buy phentermine 37.5 said 1096 days later:

    Forwarded this to some friends, appreciate your advice

  16. auto insurance quote said 1096 days later:

    Nice tutorial. I understood everything. Thanks!

  17. compare car insurance said 1096 days later:

    I really find this post up to the mark without any silly unwanted elaboration. All the information has been covered in the right manner.

  18. auto insurance rates said 1096 days later:

    thanks for the share

  19. cheap auto insurance said 1096 days later:

    Anyway, thanks for your guidance, it is not easy though…

  20. hypnosis therapy to stop smoking said 1097 days later:

    No words in dictionary to appreciate the author of this blog….

  21. hummingbird food said 1100 days later:

    May be it was very old dictionary…. try to search in latest dictionary…. even though you not getting than search in internet….

  22. Web Design said 1100 days later:

    Nice article. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your articles. Anyway I’ll be subscribing to your feed and I hope you post again soon.

  23. placement argent said 1114 days later:

    Thanks for taking the time to discuss this, I feel strongly about information and love learning more on this. If possible, as you gain expertise, It is extremely helpful for me. would you mind updating your blog with more information?

  24. mens lacoste polo said 1125 days later:

    It is useful for me,thanks for your nice share. cheap ralph lauren polo shirts

  25. Wedding save the date photo magnets said 1134 days later:

    Thanks for the post buddy

  26. buy cheap ghd straighteners said 1135 days later:

    Recently I wanted to try some modifications to a Pylons buy cheap ghd straighteners project against the latest Pylons trunk without upgrading the system-installed Pylons, so I decided it was about time to try Buildout. Jim Fulton’s buildout (or zc.buildout, since Jim likes to put things in namespaces), is a package to create development and deployment versions of software. It’s cheap ghd hair straighteners not in any way Zope-specific, and whilst it is written in Python primarily for installing Python programs, it’s not really Python-specific either.

  27. Loi Scellier 2011 said 1135 days later:

    I wanted to thank you for this excellent read!! I definitely loved every little bit of it. I have you bookmarked your web site to check out the latest stuff you post.

  28. ink cartridges said 1145 days later:

    I provide information that is truly unique and the service they offer too have lived, I love this sevice so I request please visit the others.

  29. shipping vehicles said 1147 days later:

    Useful information in showing the potential of buildout. That is also a great example for me to clearly understand.

  30. farmville cheats said 1148 days later:

    So informative things are provided here,I really happy to read this post,I was just imagine about it and you provided me the correct information I really bookmark it,for further reading,So thanks for sharing the information.

  31. Boating Supplies said 1148 days later:

    I like your point of view.Could you explain more about this subject? I’m thrilled to found your site, and waiting for more updates from you.By the way keep up the good work.

  32. Moroccan furniture said 1149 days later:

    Moroccan furniture Your blog article is very intersting and fanstic,at the same time the blog theme is unique and perfect,great job.To your success.

  33. Locksmith Sacramento said 1149 days later:

    The presence of such high quality posts is very rare these days over the internet. I personally liked the information

  34. Personal Injury Lawyers Las Vegas said 1155 days later:

    I really like your blog content the way you put up the things…I’ve read the topic with great interest.

  35. Garage Doors Salt Lake City said 1155 days later:

    I was searching for info about this on Yahoo and chanced on your item. I found it to be well explained. Thanks

  36. Move 4 Less Las Vegas said 1155 days later:

    I like your point of view.Could you explain more about this subject? I’m thrilled to found your site, and waiting for more updates from you.By the way keep up the good work.

  37. Find a babysitter said 1155 days later:

    Cheers for the info. The internet is so full of junk its difficult to find what youre looking for these days. Do you have an rss feed on this site?

  38. Debt Consolidation Missouri said 1155 days later:

    Top experts for debt consolidation, bankruptcy alternatives and credit counseling in Missouri. No upfront fees! Personal approach, fast debt relief.

  39. Forex Managed Account said 1155 days later:

    Managed Forex trading account for savvy investors. Get up to 30% a month return on your investment with Getintoforex experts and our best automated trading system.

  40. replacement windows harrisburg pa said 1155 days later:

    I like the post lol and have bookmarked the website for the further reading great work.

  41. Computer Repair Service Las Vegas said 1156 days later:

    This is my first time i have visited here. I found many interesting stuffs

  42. <a rel="new tab" href="http://www.autoquake.com/used-Audi-A8/" rel="nofollow">buy used audi a4</a> said 1156 days later:

    Great idea, it very instructive for me. I have just got interested in blogging and hopefully i am able to do so.Thanks.

  43. tractors said 1159 days later:

    I really like the work that has gone into making the post. I will be sure to tell my blog buddies about your content keep up the good work. Thanks

  44. Atlanta bankruptcy lawyers said 1160 days later:

    I’ll be back down the track to check out other posts. I’m delighted I came across it. This is a really good site post. Atlanta bankruptcy lawyers

  45. coach knockoff handbags said 1162 days later:

    Thanks very much!

  46. speedingupregistry said 1163 days later:

    I really like your blog content the way you put up the things.

RSS feed for this post

(leave url/email »)

   Comment Markup Help Preview comment