developer’s creed

“I must create a system or be enslaved by another man’s; I will not reason and compare: my business is to create.”

- William Blake

In search of an editor/IDE

I’ve been a long time emacs user, my personal search for better development environments, has taken me across a host of tools, from eclipse, and nano to others. While i’ll continue to use emacs on occassion. I’ve finally settled on textmate for my standard development environment.

some editors and ide tools i’ve used along the way.

Editors

emacs

python-model, pdbtrack.el and loads of customizations of have suited me well for years. i have most of the keybindings for it wired into my fingers and reflex memory. it has served me well, in projects from large to small, and i’ve always come back to it, after trying out other development environments. the ability to drive multiple shells, and all the integration with standard unix development tools make it for me the gold standard. unfortunately, for all of that i’ve used emacs, i’m not a hard core emacs user. I’m not inclined to hack the lisp nesc, or for me to customize the environment on an ongoing basis to adapt to the way i work, because of my percieved cost of customization, which is what has lead me to explore alternatives.

eclipse and pydev

eclipse is a little strange to get into unless your familiar with the eccentricities of the full blown IDE mentality. whereby the ide sucks your code into some ‘workspace’, from wherever you point it to on the filesystem. customization beyond the builtin functionalities, requires programming the system in java.

for development of java projects, its hard to complain. if emacs is the editor os of the 80 and 90s, it seems that eclipse is destined to become the new os for the next 20 years.

wingide

has one of the best source completions and debuggers available for python. unfortunately its tied to the gtk toolkit, which is a great for single platform usage, but i find pales for anything approaching useable on platforms other linux.

eric3

the most comprehensive of the opensource python editors, its user interface has some notion of tossing in every feature under the sun into the user interface simulatenously. some creative editing of the preferences can mitigate this user interface nightmare, into something a bit more manageable. the source itself is fairly clean, owing i think both to good design and the elegance of the qt api.

supplemental tools

a collection of tools i commonly use when working on projects.

ipython

the python shell on steroids, useful tools for tab completion. personally i’ve typically use a lighter weight cousin of ipython, known as rlcompleter2 by holger krekel

buildout

a tool for creating and managing development and deployments. it allows for management of python eggs, as well as compilation of system environments.

ctags

a keyword extraction utility, that builds a symbol database for quick look up of symbols in files.

paster

a set of tools and libraries for wsgi application construction.

TextMate

i’ve heard about textmate for a while, but strongly resisted the notion of being tied to a propretiary platform specific tool, except its now also clear that its the most productive environment i’ve worked on because its also the most easy environment to customize and extend that i’ve seen. eclipse (including ria) and emacs are both probably superior in their overall customization capabilities, but the ease of doing customization in textmate without programming surpasses them both.

i wanted to get together a blog entry and collect some ideas on things for extensions on textmate to help me being productive in a python/zope3/plone environment. There is an existing Zope.tmBundle from the collective which i’ve used as a starting point. Its got some good features.

some ideas

- command results dialog
- buildout parts executor dialog
- class browser
- prefix based file search dialog
- needs a corresponding buildout part
- some of the more advanced would require running a code daemon synchronous to the project. something to index the code/symbols, run tests,
- indexed symbol search
- find search
- wrap code for post mortem debugging (done)
- restructured text mode
- automatically run pyflakes when saving python
- open zcml file
- open import definition

prefix based file search dialog

for indexed file search we need a per project corresponding data structure. let’s say etags.
it would needs a corresponding buildout part, to index the source.

ctags based completion
- found this one in the textmate svn bundles, required for zope developers

textmate restructured text mode
- also in the textmate svn bundles

Ubuntu virtual appliances

Canonical has just released a minimalist version of ubuntu server as a base for contructing virtual machine appliances. Now to get it on ec2.

• Release announcement
• LaunchPad Project
• Downloads

Scripting OSX with Unix

I’m currently using osx for a desktop, being able to write simple python applications which have interactive user interfaces, and which can retain OSX semantics (drag and drop), for scripting common tasks, would be great. I stumbled upon a set of tools today, that allows exactly that.

Cocoa Dialog
http://cocoadialog.sourceforge.net/

cocoadialog provides the ability for cocoa creating user interface dialogs

Playtpus
http://www.sveinbjorn.org/platypus

provides the ability to create osx applications from scripts… “Creating installers, maintenance applications, login items, launchers, automations and droplets is very easy using Platypus.”

Marco Polo
http://www.symonds.id.au/marcopolo/

slightly offbase, but also useful, this takes scripting osx to the next level, by providing a rule based framework that can respond to contextual events (network, devices, etc).

just as excellent, in fine gnu tradition… all of these tools are free software, released under the GPL

The Evils of GenericSetup

Summary

brain dump on generic setup and product bundles, ie. separately configuration management and package management. there are a couple of different issues here, configuration and state management of an instance, and distribution of software.

Use Cases

i think its important to preface discussion of these with a consideration of user stories

a user wants to

- install an addon from a remote repository, preferably installed without restart.

- try out an addon, decide against it, and remove it safely from the system, with zero application footprint. likely will need a restart.

a developer/integrator wants to have

- easy upgrading

an additional noteworthy case of upgrading, are
developers want to upgrade and deploy against scm
revisions.

regarding configuration and state management

generic setup is about managing configuration state for easy snapshotting and restoration of a system configuration. its good about managing and extending the whole system state, but its ability to deal with removal of partial states, and state differences states requires human intervention dealing with tool specific file configuration diffs.

modeling configuration as reversible changesets can accomplish these use cases. an installation/changeset would need to be composed of tasks, and each either associated with date or preferably revision, conflict discriminators (ala zcml), associated task dependencies, and install and uninstall methods. a library of common reversible tasks and a common configuration mechanism (zcml/yaml/etc) for usage and definition, ala tools like cpsinstaller, ploneinstaller, pymake.

an interesting issue is customization, its a hard issue and its an area where plone could use improvement. generic setup makes available raw file diffs, but with the ability to address task sets, we can compose much more user/integrator friendly with task conflicts noted by task, and optional installation of partial changesets (easy customized tool overides). conflicting configurations utilize the task discriminator, and can be presented to the user.

another interesting issue is working against svn with independent development, developers often want to use the latest and greatest features from svn while developing systems, but the confidence to know that going into production. an installation and configuration mechanism for plone should be able to intelligently update checkouts via always doing secondary sorting (post dependency sorting) on a task’s revision/date metadata, so we can track updates by order of application, instead of version number.

one doubt i have on this is that it continues the maintenance of configuration as instance data in the zodb, in addition to tool state the use of z3 components and achieving these benefits, relies on increased reliance on persistent component registries, when the move in z3 has been towards global configuration, i think at some level the growth of persistent data is inevitable for the degree of application instance specific customization of plone typically wants to offer. its unclear, though it would be interesting to see if stephan’s z3c.baseregistry is capable of dealing well with persistent utilities, and adapte registrations.

any solution needs to be at least interim compatible with quickinstaller to facilitate installation on existing versions of plone pre 3.0, which are likely to see widespread use for at least another year, a extensions install stub that fires off the task set installation would suffice.

distribution

distribution is mostly orthogonal to configuration, but the parts should play nice together in the ui to offer a good end user experience, good product distribution basic building blocks should also start to address the needs of micro distribution within the community, such as non profit distributions, intranet/collaboration distributions, in addition to plone’s adherence to being an easy to use cms product.

eggs are a good distribution means and are well supported in the python community.

we need packaging/development with a tool thats accomplished at doing source buildouts and construction of eggs, with examples/documentation on using it for plone. zc.buildout looks like a good tool though lacking in documentation or examples for plone at the moment.

but easy_install isn’t nesc. the appropriate management interface for plone eggs, part of being a being web app product, should also include the optional installation of components through the web. for a plone package manager, its not clear if reutilizing the easy install library is the best way to facilitate it, or if just dealing with raw eggs and using pkg_resources is. the easy_install internals are filled with tons of abstractions, that may or may not be easily used for such a manager construction. regardless, the use of eggs as a standard format for distribution is still a clear win.

for a global repository, there’s been talk of pypi enabling the software center, it not clear whats the best approach, outside of that it should be easy to upload globally such that software is discoverable, and the approach should be able to facilitate running various mini-distribution repositories.

Once more into the breach

yet another attempt to get a blog going.. hosted is easier.