Alex Clark - Python Web Developer

New Pyramid Site

2013-04-22T12:00:00-04:00

For the first time in 10 years, aclark.net is not powered by Plone. Nothing against Plone: it's still one of the greatest loves of my life (inasmuch as you can love a software and community, as I do).

Why

This was not the result of a revolutionary plan, rather more of an evolution. It happened like this:

As soon as Plone 4.3a1 was released (a year ago?) I deployed a new Plone site to aclark.net with it, featuring a Diazo (new Plone theming engine) theme.
Around the same time I became obsessed with deploying to Heroku, and also gained an interest in Python 3.
A few months ago, I got tired of paying $11/month to host my Plone site so I converted the site to static HTML and moved it to GitHub pages. But the result was flawed because maintenance involved editing duplicate copies of the website content (e.g. both clients.html and clients/foo.html contained the same text describing "foo").

So when it came time to do more than a few casual edits, I knew I had to find a new approach. That's when various elements of the Universe conspired to lead me in a new direction.

How

Pyramid

I spent a lot of time (~ 1 year) developing pythonpackages.com in Pyramid, but the result was a mess (code-wise). I'm in the process of rewriting and open sourcing it, but it's slow going. So what better way to get started than to do a small-ish site in Pyramid for fun?

about.me

I also recently gave in and created an about.me site. I was impressed by their content editing features, and my ability to create a page that looked OK using them.

In my about.me profile, I used a picture of me and a picture of DC I took in early 2012. When it came time to redo aclark.net I felt like I really wanted to capture the simplicity of the about.me site, so I used the same photo in the background.

Bootstrap

Bootstrap is old news at this point, but I really enjoy using it and I particularly like that they have added more example templates. So I combined my background photo with one of their example templates and a new site idea was born. As I'm not a particularly talented visual artist, my ability to produce something that looked OK (with code this time) was exciting.

What

Until I added a contact form, the site was entirely unremarkable. There are views and routes and templates, typical fare for a web framework. Here is the entire "main routine":

from pyramid.session import UnencryptedCookieSessionFactoryConfig
from pyramid.config import Configurator
from .redir import blog
from .redir import blog_entry
from .redir import blog_slash
from .views import contact
from .views import default
import deform_bootstrap


def main(global_config, **settings):
    """
    Oppan wsgi style! Configure and return WSGI application.
    """
    my_session_factory = UnencryptedCookieSessionFactoryConfig('itsaseekreet')
    config = Configurator(session_factory=my_session_factory)
    config.add_route('blog', '/blog')
    config.add_route('blog_entry', '/blog/{entry:.*}')
    config.add_route('blog_slash', '/blog/')
    config.add_route('contact', '/contact')
    config.add_route('clients', '/clients')
    config.add_route('projects', '/projects')
    config.add_route('services', '/services')
    config.add_route('team', '/team')
    config.add_route('testimonials', '/testimonials')
    config.add_route('root', '/')
    config.add_static_view(
        'static', 'aclarknet:static', cache_max_age=3600)
    config.add_view(blog, route_name='blog')
    config.add_view(blog_entry, route_name='blog_entry')
    config.add_view(blog_slash, route_name='blog_slash')
    config.add_view(
        default,
        renderer='aclarknet:templates/clients.mak',
        route_name='clients')
    config.add_view(
        contact,
        renderer='aclarknet:templates/contact.mak',
        route_name='contact')
    config.add_view(
        default,
        renderer='aclarknet:templates/projects.mak',
        route_name='projects')
    config.add_view(
        default,
        renderer='aclarknet:templates/root.mak',
        route_name='root')
    config.add_view(
        default,
        renderer='aclarknet:templates/services.mak',
        route_name='services')
    config.add_view(
        default,
        renderer='aclarknet:templates/testimonials.mak',
        route_name='testimonials')
    config.add_view(
        default,
        renderer='aclarknet:templates/team.mak',
        route_name='team')
    config.include(deform_bootstrap)
    return config.make_wsgi_app()

Contact form

But then I wanted a contact form. Which lead me to wanting an elegant way to send mail via Heroku. Which lead me to discover SendGrid. Which lead me create some primitive marketing features I am quite proud of and excited about.

I still ended up sending mail "the old way" via GMail. But now I send two mails: one to info@aclark.net to alert our staff about the lead (using GMail). And one to the lead acknowledging their submission (using SendGrid). SendGrid keeps a record of all the leads we've contacted, amongst other "fancy marketing features". Here's the relevant view code:

import deform
import smtplib

from email.mime.text import MIMEText

from .config import FORM_ERROR
from .config import FORM_SUCCESS

from .config import MIME_ONE_RECIPIENT
from .config import MIME_ONE_SUBJECT
from .config import MIME_TWO_MESSAGE
from .config import MIME_TWO_SUBJECT

from .config import GMAIL_HOSTNAME
from .config import GMAIL_PASSWORD
from .config import GMAIL_USERNAME

from .config import SENDGRID_HOSTNAME
from .config import SENDGRID_PASSWORD
from .config import SENDGRID_USERNAME

from .forms import ContactFormSchema


def contact(request):
    """
    Create and render deform form containing colander schema. Provide
    sendgrid integration for marketing.
    """
    button = deform.Button('Send', css_class='span9 btn-block btn-large')
    schema = ContactFormSchema().bind(request=request)
    form = deform.Form(schema, buttons=(button, ))
    if 'Send' in request.POST:
        items = request.POST.items()
        try:
            appstruct = form.validate(items)
        except deform.ValidationFailure:
            return {
                'form': form.render(),
                'request': request,
            }
        # This is the form contents
        email = appstruct['email']
        message = appstruct['message']

        # This is the mail to info@aclark.net
        mime_document_one = MIMEText(message)
        mime_document_one['Subject'] = MIME_ONE_SUBJECT
        mime_document_one['To'] = MIME_ONE_RECIPIENT
        mime_document_one['From'] = email
        mime_document_one = mime_document_one.as_string()

        # This is the mail to the new lead
        mime_document_two = MIMEText(MIME_TWO_MESSAGE)
        mime_document_two['Subject'] = MIME_TWO_SUBJECT
        mime_document_two['To'] = email
        mime_document_two['From'] = MIME_ONE_RECIPIENT
        mime_document_two = mime_document_two.as_string()

        try:
            # This is the mail to info@aclark.net
            smtp_server = smtplib.SMTP(GMAIL_HOSTNAME)
            smtp_server.starttls()
            smtp_server.login(GMAIL_USERNAME, GMAIL_PASSWORD)
            smtp_server.sendmail(email, MIME_ONE_RECIPIENT, mime_document_one)
            smtp_server.quit()

            # This is the mail to the new lead
            smtp_server = smtplib.SMTP(SENDGRID_HOSTNAME)
            smtp_server.starttls()
            smtp_server.login(SENDGRID_USERNAME, SENDGRID_PASSWORD)
            smtp_server.sendmail(MIME_ONE_RECIPIENT, email, mime_document_two)
            smtp_server.quit()
            request.session.flash(FORM_SUCCESS)
        except:
            request.session.flash(FORM_ERROR, 'errors')
        return {
            'form': form.render(appstruct={}),
            'request': request,
        }
    return {
        'form': form.render(),
        'request': request,
    }


def default(request):
    """
    This is the default view, to be used with most routes since we do not
    provide any content editing ability yet. Even then, maybe a default view
    would still be helpful.
    """
    return {}

Who cares

The best thing about all of this being able to run the site 100% for free on Heroku. Also:

Python 3 compat!
Free caching via CloudFlare
Free ping service from Pingdom keeps the site from "going to sleep" (HT: natea).
Updating the site fits my workflow. If I'm the content editor, I don't necessarily need or want to use Plone to edit my content. I can save Plone for my clients, and focus on what makes them happy with their CMS system.

The Story of Pillow

2013-03-15T17:00:00-04:00

On March 4, 2013 I got an email from the Python Software Foundation (PSF):

This email notification is being sent to you to inform you of the PSF Board’s decision to fund the facilitation of a Python 3 compatible release of the Python Imaging Library for the amount of $1,300 USD. Please see the resolution that was passed unanimously on March 4, 2013 via email below:

Yay! That was in response to my proposal asking for funding to finish a Pillow 2.0.0 release. In that proposal, I presented the "story of Pillow" which I'd like to present again now, for anyone interested:

Tired of seeing the proliferation of third party packagings of the Python Imaging Library, in 2010 Alex Clark took Hanno Schlicting's repackaging and used it to create a fork of the Python Imaging Library on GitHub. He subsequently released that fork to PyPI as Pillow 1.0. The ability to add additional development library paths to setup.py (e.g. 64 bit library and headers directories) and make releases quickly eventually led to widespread adoption of Pillow.

A little over a year later on 2011-09-08, Takayuki Shimizukawa uploaded the first Windows (win32) eggs. Since then, every Pillow release included Windows eggs thanks to Takayuki. And on 2013-02-02, the first 64-bit Windows eggs (amd64) were uploaded to PyPI by Takayuki.

For the first 3 years, the fork focused on packaging fixes only. Now a Python 3 compatible pull request from Brian Crowell has been merged, and the final stages of release preparation are underway. In early 2013, Barry Warsaw created an Ubuntu Personal Package Archive and tested it with Python 3. And the Fedora Project is now planning to include Pillow with their release of Fedora 19. Finally, the Pillow project has promised a Python 3 compatible release of Pillow by PyCon 2013.

That release is now done! And I have had an absolute blast working on Pillow full time over the course of the last week or so. So much so, that I'm going to propose you help me continue to have fun and be productive with Pillow… but more on that later. First, about the new release.

About 2.0.0

This release marks the first "serious effort" I have put into Pillow. While I've always enjoyed working on it in my spare time, I knew that there was no way I could finish it in time for PyCon 2013 (as I promised) without getting some financial assistance.

Enter: The PSF, who graciously offered to fund my work (and the work of one other contributor). Once I had funding in place, I knew exactly what needed to be done:

So much time has passed since the fork, no one seemed to care if we made image code changes (we were originally a packaging fork), so this release contains EVERY PATCH EVER SUBMITTED TO PIL but not released until now. I'm exaggerating, but we really crammed a lot of "goodness" in to the 2.0.0 release along with Python 3 support which is what prompted the release in the first place.
In order to fully test, I fine-tuned my tox environment on Mac OS X (tox is awesome) as well as fired up a new Windows VM (and used an already-existing Linux VM). Additionally, other folks joined in to test on their environments (most notably Christophe Gohlke and Eric Soroos). You'll find the results here: https://github.com/python-imaging/Pillow#platform-support. That means: THIS RELEASE HAS BEEN FULLY, EXTENSIVELY, AND EXHAUSTIVELY DEVELOPED AND TESTED. Aside from the possible inconvenience of Pillow 2.0.0 dropping 2.4 and 2.5 support (use Pillow 1.x if you need Python 2.4,2.5), you should not have any trouble with it. If you do, please open a ticket here: https://github.com/python-imaging/Pillow/issues.
As important as quality software is to me, quality presentation is almost equally important. (That's developer-eye quality, not designer-eye quality. ;-)) So I spent a lot of time cleaning up and enhancing the more subtle aspects of the release e.g. README.rst, docs/* (including the pythondoc documentation and HISTORY.txt and CONTRIBUTORS.txt). The pythondoc documentation has been converted to re-structured text and is hosted on ReadTheDocs: http://pillow.readthedocs.org/en/latest/. The website has been updated to include an actual image generated by PIL :-) http://python-imaging.github.com/.

I hope you enjoy and use Pillow 2.0.0. If you'd like to see me continue to devote significant time to maintaining Pillow, please consider a gittip! Thank you.

Trading Places Trading Scene Explained

2013-02-23T20:30:00-05:00

Every time I watch the movie Trading Places, I have to search the internet for an explanation of the climactic commodities trading scene. It usually takes me about 20 minutes to (re-)understand it, then another 10 minutes to explain it to my wife. Although there are plenty of great answers available, none of them really "speak" to me; so I decided to write my own. The goal for me is to be able to (re-)read my explanation and understand it enough to explain to someone else in less than 5 minutes.

This scene is tricky for non-stockbrokers to understand because you need to know a lot about trading, and you also need to know some important plot details. Rather than continue to look everything up on the internet each time, I figured I'd write this blog entry instead.

Important Plot Detail #1

The Crop Reports, real and fake. In the movie, the Dukes plan to cheat. But the "good guys" (Valentine and Winthrope) foil them by replacing their stolen, ill-gotten crop report with a fake. This allows the "good guys" to win. To understand the reverse-swindle, you must understand the crop reports (real and fake) and the market repercussions of each:

	Report (FAKE)	Report (REAL)
Weather	BAD	GOOD
Harvest (i.e. # oranges)	LOW	HIGH
Prices	HIGH	LOW

Important Plot Detail #2

Remember, the Dukes plan to cheat AKA "corner the market". And as Chris Carter says:

The (Duke's) agent wants to own as many contracts as possible before the crop report is revealed, since (he thinks) once it is, the price will go up and he can sell at a profit. Trading begins with a price of 102 cents per pound, which translates into $15,300 per contract. Once everyone sees that the Dukes' agent is trying to corner the market, they all want a piece of it, forcing the price up since more people are buying than selling.

I'll repeat that last part: when trading opens before the true crop report is revealed more people are buying than selling.

Important Plot Detail #3

The "good guys" know the truth i.e. that the current buying trend is based on the Duke's actions which are based on false information. That brings us to another important point: the "good guys" realize an opportunity to "sell short". Or as Chris Carter puts it:

They are betting they will be able to buy the contracts later at a lower price so they come out making money but not holding any contracts at the end of trading.

Light bulb

Ding! Ding! Ding! So after all this, here's what's really happening: the "good guys" successfully carry out a short sale, with the help of the "bad guys" who are trying to corner the market but acting on false information. Or as Chris Carter puts it, after the real crop report is published:

They (the "good guys") need to buy--a lot--to zero their position, and the crowd is more than willing to oblige.

Finally, the "good guys" are able to buy enough to contracts to fulfill the number of "short" sales they made. And they are able to purchase them for much less then they made the "short" sales for. Profit!

Does it make sense now? If not, let me know in the comments.

Built with Plone, Powered by GitHub Pages

2013-02-23T12:00:00-05:00

Is Plone the next great static website generator? Probably not, but it definitely could be

I am always looking for ways to improve, streamline and otherwise hack my e-life. And this post is about all of the above. https://raw.github.com/ACLARKNET/blog/gh-pages/images/httrack.png Plone in the cloud

Plone is still too "heavy" to easily run "in the cloud" (except via http://ploud.com, HT), but it's getting there. In particular, I find this effort by David Bain very inspiring:

https://github.com/pigeonflight/stack-python-plone

And based on my experiments and research on Heroku:

https://github.com/aclark4life/zope2-heroku

The only "real" remaining issue seems to be packaging; we need this PLIP to happen sooner, rather than later:

https://dev.plone.org/ticket/13283

Cheating

In the meantime, one way you can cheat is to export the contents of your Plone site with httrack then host the results for free on GitHub Pages (i.e. "the cloud")

This process if far from perfect [1], but in a pinch it can work. And it recently worked very well for me so I thought I'd share. See:

http://aclark.net https://github.com/ACLARKNET/aclarknet.github.com

These links are my website, and GitHub Pages repository respectively. The website was created with Plone (https://github.com/ACLARKNET/new_style) then exported via httrack, then uploaded to GitHub Pages (to the gh-pages branch of any repo except <{org,user}>>.github.com repos). [1] httrack got "stuck" on my @@search links, and I've got a lot of duplicate content now. Fortunately I don't mind editing lots and lots of text files :-). Conclusion

In doing this, I was able to turn off my $11/month "website and IRC server" which makes the CFO happy. And in a year or so with Plone 5, hopefully I'll be able to return to the cloud and edit content "live" again.

What do you think about Plone as a static site generator and/or Plone in the cloud? Let me know in the comments below.

I love checkoutmanager and dotfiles

2013-02-08T12:00:00-05:00

An ode to my OS X development workstation setup [1]

I am big on setting up my development environment, and enjoying the environment I work in. And I'm very thankful to the folks who make my life easier, including the authors of:

Python: Python Core Developers
dotfiles: Jon Bernard
checkoutmanager: Reinout Van Rees

I also love repetition. So picture if you will, a new Macbook Air or Pro ready to serve as my development workstation. I like to perform, and study, the steps required to turn a new laptop in to my development workstation. So here we go. In this article, I will walk through the steps required to turn a new machine in to my developer workstation. Do follow along!

Shell

One of the first things I do on a new system is change my shell to Zsh in System Preferences -> Users & Groups -> Current User -> Right Click -> Advanced Options.... Don't forget to Click the lock to make changes first.

Zsh

Why Zsh? One of my favorite features is shared history between open sessions. So I can run a command in one window, and then run the same command from another window by fetching it from the history (with CTRL-R).

XCode

After I take possession of my new laptop (running Mountain Lion, the newest OS X at the time of this writing), I head to the App Store to download XCode. [2] Among many other things, XCode gives me the GNU C Compiler and allows me to type "gcc" in my Applications -> Utilities -> Terminal.

Python

Now I need a Python interpreter [3]. For development I use the Collective Python Buildout but I also enjoy using Homebrew's Python 2.7. I use Homebrew for a variety of other things too (e.g. mobile-shell AKA mosh) so here we go:

$ ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"

… follow instructions …

$ brew install python

Git

I think OS X (or XCode) includes git, but just in case:

$ brew install git

Which reminds me, don't forget that installing the command line utilities in Mountain Lion's XCode requires an additional step in Preferences -> Downloads -> Command Line Tools -> Install:

Dotfiles

At this point, I can begin to get serious about turning this new machine in to my developer workstation. And that means: installing my private ssh key so I can check out code without typing a password, of course. Normally this would be tedious, but with git and dotfiles it's not so bad. This is what I do from my home directory:

$ git clone https://super_secret_url/dotfiles.git Dotfiles

I use https which requires a password for the first time only. Then I edit Dotfiles/.git/config and change the repository URL to git@super_secret_url/dotfiles.git. So every subsequent pull and push will require no password. And to "install" these dotfiles, I do [6]:

$ pip install dotfiles
$ dotfiles -s --force

Note: the dotfiles command finds my dotfiles in the default directory "Dotfiles" and create symbolic links to them.

Distribute & Pip

Homebrew's Python includes pip, but even if it didn't:

$ curl -O http://python-distribute.org/distribute_setup.py
$ /usr/local/bin/python distribute_setup.py
$ {easy_install, pip install} dotfiles

In other words, you can always install Distribute [4]. After which you can use easy_install or pip to install dotfiles. (You can read up on the differences. TL;DR: neither is "better" or "worse", it's just a question of which tradeoffs you are willing to make. I tend to use pip just because it's newer and I like its requirements.txt feature, but easy_install is still very well supported as part of the Distribute project.)

Checkoutmanager

Now I need some things to develop. Since I work on a bunch of different projects, I need a way to keep track of what should be checked out at any given time. So I do this [5]:

$ pip install checkoutmanager
$ checkoutmanager co

This creates and populates my ~/Developer directory with code. And it "just works" because I keep a .checkoutmanager.cfg in my Dotfiles repository. It currently looks like this:

[aclark]
basedir = /Users/aclark/Developer/aclark
checkouts =
    git@github.com:aclark4life/aclark4life.github.com.git resume
    git@github.com:aclark4life/desktops.git
    git@github.com:aclark4life/hireme.git
    git@github.com:aclark4life/projects.git
    git@github.com:aclark4life/reinstall.git
    git@github.com:ACLARKNET/tweets.git
    git@github.com:aclark4life/usesthis.git
vcs = git

[alt]
basedir = /Users/aclark/Developer/alt
checkouts =
    git@github.com:alt-aclark-net/alt-aclark-net.github.com.git
    git@github.com:alt-aclark-net/dexter.git
    git@github.com:alt-aclark-net/headstraight.git
vcs = git

[buildout]
basedir = /Users/aclark/Developer/buildout
checkouts =
    git@github.com:collective/buildout.bootstrap.git
    git@github.com:buildout/buildout.git
    git@github.com:buildout/buildout.github.com.git
vcs = git

[distribute]
basedir = /Users/aclark/Developer
checkouts = ssh://hg@bitbucket.org/tarek/distribute
vcs = hg

[clients]
basedir = /Users/aclark/Developer
checkouts =
; Bunch o client repos
vcs = git

[clients-hg]
basedir = /Users/aclark/Developer
checkouts =
; Bunch o client repos
vcs = hg

[dcpython]
basedir = /Users/aclark/Developer/dcpython
checkouts =
    git@github.com:DCPython/dcpython.github.com.git
    git@github.com:DCPython/pyramid-tutorials.git
vcs = git

[misc]
basedir = /Users/aclark/Developer
checkouts =
    git@github.com:ACLARKNET/aclarknet.github.com.git blog
    git@github.com:ACLARKNET/new_style.git
    git@github.com:aclark4life/binfiles.git
;    git@github.com:aclark4life/pyramid_python_3.git
    git@github.com:aclark4life/vanity.git
    git@github.com:aclark4life/zope2-heroku.git
    git@github.com:aclark4life/zope2_bootstrap.git
    git@github.com:codekoala/django-axes.git
    git@github.com:collective/buildout.python
vcs = git

[pillow]
basedir = /Users/aclark/Developer/pillow
checkouts =
    git@github.com:python-imaging/Pillow.git
    git@github.com:python-imaging/python-imaging.github.com.git
vcs = git

[plethorasociety]
basedir = /Users/aclark/Developer/plethorasociety
checkouts =
    git@github.com:plethorasociety/plethorasociety.github.com.git
vcs = git

[plone]
basedir = /Users/aclark/Developer/plone
checkouts =
    git@github.com:aclark4life/Plone-Debug-Assistant.git
    git@github.com:aclark4life/collective.recipe.bluebream.git
    git@github.com:aclark4life/event_days_indexer.git
    git@github.com:aclark4life/hello_plone.git
    git@github.com:aclark4life/mr_migrator_demo.git
    git@github.com:aclark4life/parse2plone.git
    git@github.com:aclark4life/plone_1_fun.git
    git@github.com:aclark4life/plone_addon_upgrade.git
    git@github.com:aclark4life/plone_guide.git
    git@github.com:aclark4life/plone_workflow_events.git
    git@github.com:aclark4life/schemaextender-facetednav-demo.git
    git@github.com:aclark4life/silly_content_import.git
    git@github.com:aclark4life/transmogrify.extract.git
    git@github.com:aclark4life/transmogrify.regexp.git
    git@github.com:aclark4life/viewlets_dont_suck.git
    git@github.com:aclark4life/wordpress2plone.git
    git@github.com:collective/Products.AttachmentField.git
    git@github.com:collective/Products.CalendarX.git
    git@github.com:collective/Products.EventRegistration.git
    git@github.com:collective/Products.PloneSoftwareCenter.git
    git@github.com:collective/Products.ifQuotes.git
    git@github.com:collective/Products.naked_plone.git
    git@github.com:collective/buildout.plonetest.git
    git@github.com:collective/collective.contacts.git
    git@github.com:collective/collective.controlpanel.edit_css.git
    git@github.com:collective/collective.developermanual.git
    git@github.com:collective/collective.formtoy.git
    git@github.com:collective/collective.github.com.git
    git@github.com:collective/collective.googleanalytics.git
    git@github.com:collective/collective.package.git
    git@github.com:collective/collective.project.git
    git@github.com:collective/collective.recaptcha.git
    git@github.com:collective/collective.recipe.grp.git
    git@github.com:collective/collective.recipe.rsync.git
    git@github.com:collective/collective.rip.git
    git@github.com:collective/collective.stats.git
    git@github.com:collective/funnelweb.git
    git@github.com:collective/github-collective.git
    git@github.com:collective/mr.migrator.git
    git@github.com:collective/plonecom-buildout.git
    git@github.com:collective/plonecom.theme.git
    git@github.com:collective/plonetheme.coolblue.git
    git@github.com:collective/plonetheme.freshpick.git
    git@github.com:collective/plonetheme.grungeera.git
    git@github.com:collective/plonetheme.keepitsimple.git
    git@github.com:collective/plonetheme.unilluminated.git
    git@github.com:collective/transmogrify.filesystem.git
    git@github.com:plone/Installers-OS-X.git
    git@github.com:plone/Products.PloneOrg.git
    git@github.com:plone/admin-docs.git
    git@github.com:plone/buildout.coredev.git
    git@github.com:plone/planet.plone.org.git
    git@github.com:plone/plone.api.git
    git@github.com:plone/plone.github.com.git
    git@github.com:plone/ploneorg.admin.git
    git@github.com:plone/plonetheme.ploneorg.git
vcs = git

[pythonpackages]
basedir = /Users/aclark/Developer/pythonpackages
checkouts =
    git@github.com:aclark4life/buildout-apache-mysql.git
    git@github.com:aclark4life/buildout-munin.git
    git@github.com:aclark4life/buildout-mysql.git
    git@github.com:aclark4life/buildout-nginx.git
    git@github.com:aclark4life/buildout-plone-haproxy.git
    git@github.com:aclark4life/buildout-plone-varnish.git
    git@github.com:aclark4life/buildout-zenoss.git
    git@bitbucket.org:pythonpackages/pythonpackages.com.git vanity_app
    git@github.com:pythonpackages/buildout-apache-modwsgi.git
    git@github.com:pythonpackages/buildout-bluebream.git
    git@github.com:pythonpackages/buildout-django.git
    git@github.com:pythonpackages/buildout-jenkins.git
    git@github.com:pythonpackages/buildout-plone-getpaid.git
    git@github.com:pythonpackages/buildout-plone.git
    git@github.com:pythonpackages/buildout-wordpress.git
    git@github.com:pythonpackages/buildout-zope2.git
    git@github.com:pythonpackages/experimental.pythonpackages.git
    git@github.com:pythonpackages/github-services.git pythonpackages-github-services
    git@github.com:pythonpackages/pyramidpypi.git pythonpackages-index
    git@github.com:pythonpackages/pythonpackages-blog.git
    git@github.com:pythonpackages/pythonpackages-docs.git
    git@github.com:pythonpackages/pythonpackages-graphs.git
    git@github.com:pythonpackages/pythonpackages-paste.git
    git@github.com:pythonpackages/pythonpackages-scaffolds.git
    git@github.com:pythonpackages/pythonpackages.sendpickedversions.git
    git@github.com:pythonpackages/pythonpackages-whiskers.git
    git@github.com:pythonpackages/pythonpackages.git
vcs = git

[toys]
basedir = /Users/aclark/Developer/toys
checkouts =
    git@github.com:aclark4life/basic_pyramid_zodb.git
    git@github.com:aclark4life/github_repos_cloner.git
    git@github.com:aclark4life/other.git
    git@github.com:aclark4life/python_study.git
    git@github.com:aclark4life/django-hello.git
vcs = git

Now it's time to bootstrap the Collective Python Buildout, which gives me all versions of Python, ever [7]. And off we go:

$ cd Developer/buildout.python
$ python bootstrap.py

Finally, there is some PATH configuration required to make all of this seemless. The Collective Python Buildout gets installed in /opt while brew's stuff is in /usr/local. My PATH config currently looks like this:

export PATH=/usr/local/bin:/usr/local/sbin:/opt/local/bin:/Users/aclark/Developer/buildout.python/python-2.7/bin:$PATH
export PATH=~/Developer/binfiles:/usr/local/share/npm/bin:$PATH

With the above configuration, I default to the Python 2.7 in the Collective Python Buildout. That means that is the "python" or "virtualenv" I get when I type those commands. I use the full path or expanded binary name when I need them e.g. /usr/local/bin/python or python3.3.

That's it! I hope you will check out dotfiles and checkoutmanager for all your development needs.

[1]	Not really an ode: http://en.wikipedia.org/wiki/Ode

[2]	I know about Kenneth Reitz's XCode Command line Tools only, but if I recall correctly there is some "gotcha" that has bitten me more than once if I use that instead of the full XCode. I wish I could remember what it was now, but it's not coming to me. If it works for you though, great!

[3]

I know about the system Python, and for small things like checkoutmanager and dotfiles I don't mind using it. But there is merit in avoiding it because Apple treats it like "their" Python and makes decisions for you that you may prefer to make yourself. E.g. I believe they use a crippled version of the readline library.

[4]	Distribute is a more actively maintained fork of the venerable setuptools library (which itself is built on top of the Python standard library's distutils). Are we having fun yet?

[5]	I also alias checkoutmanager to cm :-)

[6]	I force because I want to replace the newly created .ssh dir with the one I keep in my Dotfiles repository.

[7]	Well, 2.4 through 3.3 at last count.

Please Help Me Do Open Source Work

2013-01-31T13:00:00-05:00

From my 'mid-life-crisis' series of blog entries :-)

Open Source Work

Open Source Work has paid my bills for a long time. Starting with Plone in the early 2000s, to Django in the early 2010s, to now. And for this, I am very grateful. To be clear: it's not exactly the Open Source Work that has paid my bills, it's the consulting work I've been able to secure as a result of my dedication and devotion to open source software and communities. Which is great! (Even more clear: my dedication and devotion to open source software and communities has made learning new skills fun. And those skills have paid my bills for the past 8 years.)

TL;DR: Open Source Work is awesome. I'll likely spend my entire life doing it (in some capacity or another). And while Open Source Work does not pay, the experience is invaluable.

SaaS Offering

Along the way, I caught the "startup bug" and have been interested in building a SaaS offering that would at least pay my bills, if not make me fantastically wealthy. Enter 2012's: pythonpackages.com. The year I spent doing pythonpackages.com work was awesome. I learned a lot. Traveled. Met a lot of cool folks who seemed genuinely interested in my idea(s). But unfortunately, I never made a dime doing it. The lesson, among others, is that SaaS offerings are hard. I am now semi-focused on a "reboot" of the pythonpackages.com idea(s), but that's going to take a while to get going.

TL;DR: Consulting *was* the dream. Now building a SaaS offering *is* the dream (among others). So where does Open Source Work fit in?

Play as Work

I've been "working for a living" since age 14. Back then, I was a busboy and dishwasher in my family's Italian restaurant, where I first learned to work hard and have fun doing it. Since then, I've always worked hard, had fun, and taken few vacations (except for 1994-1998 when I was a full time CS student). I suspect I will always "work for a living" though as I get older, the line between work and play becomes even blurrier. Everyone wants to love or at least like their work. But very few are able to turn their play in to paid work. That's what I want to do. And I am very fortunate, I think, to know what I want to do in life. I know middle aged folk that still don't know the answer to that question.

TL;DR: liking my work *was* the dream. Now turning the things I like to do into paid work *is* the dream. In 2013, I would like to spend more time doing Open Source Work, working on projects like Buildout, DC Python, Pillow, Plone, django-axes, et al. If you have benefited from my work on any of these projects, directly or indirectly, please consider making a gittip donation to help me do more Open Source Work in 2013.

Python 3 Porting

2013-01-10T19:15:00-05:00

The 3 in 2013 is for Python 3

I tend to like projects that everyone else hates, e.g.:

Removing persistent Python objects associated with missing classes in ZODB.
Making new releases for old software that is still useful but unmaintained.
Running flake8 on 10s or 100s of source files and hand-fixing the results.

Part of this has to do with repetition. I know how to do these things, and I like doing them. Over and over. If someone says, "my Data.fs file is broken" I get excited. Anyway, I hope you get the idea.

I am your man

Which brings me to the subject of this post: Python 3 Porting. I've been helping out porting the Python Imaging Library to Python 3 (via the Pillow fork, where I am the release manager). And it strikes me as the type of thing I'd be very interested in doing professionally. So, if you or your company are in need of a "workhorse" to plow through old code and update it, I am your man. Please do get in touch.

What's in it for you

Now I know what you are thinking: porting Python 2 software to Python 3 is a lot of work for little gain. Maybe. Maybe not. I'm not going to try to convince you otherwise, however I will tell you this:

For me, 2013 is "the year of Python 3". I've now promised to deliver and maintain a Python 3 compatible PIL by PyCon 2013. I now care about Python 3 (this was not true before).
The Python 3 Wall of Shame is now the Python 3 Wall of Superpowers. We are over the hump.
Many popular Python web frameworks support or are about to support Python 3 e.g. CherryPy, Django, Pyramid.

So, there is no time like the present to take a serious look at Python 3.

Pillow Python 3

2013-01-10T12:15:00-05:00

PIL is on its way to Python 3 via Pillow. Support from Brian Crowell and others has been merged into master here:

https://github.com/python-imaging/Pillow/pull/35

And work continues toward a pre PyCon 2013 release! Please help if you can:

Ubuntu users, read through https://github.com/python-imaging/Pillow/issues/18 and provide assistance with testing this Ubuntu package: https://launchpad.net/~pythoneers/+archive/ppa.
Git experts, please comment on the merge issue described here: https://github.com/python-imaging/Pillow/pull/35.
Everyone else, please git clone the master branch for testing: git://github.com/python-imaging/Pillow.git and report issues as you find them: https://github.com/python-imaging/Pillow/issues.

Thank you!

About My Work

2013-01-10T11:45:00-05:00

Regarding some of the criticism or objection to some of the content of my blog entries, etc., here are my thoughts:

I hear you

No one is more critical of my work than me. If you have feedback, criticism, etc. then please do let me know. I consider very carefully every request I receive, and I make every attempt to be cordial and accommodating. The place to do that for this blog is currently:

https://github.com/ACLARKNET/aclarknet.github.com/issues

Or for folks without a GitHub account (i.e. anonymous):

https://bitbucket.org/aclarknet/blog/issues

Or just email me:

aclark@aclark.net

Or add a comment below.

Controversial Entries Explained

I'd also like to clarify my intentions with regard to two of my most recent and somewhat controversial entries:

Zen of Zope
Plone Kitty

They are intended to be pro Zope and pro Plone. Yes, Zope can be complex. But it's also useful. And full of "been there, done that" wisdom. Yes, the simple days of Plone Kitty are gone, but there are many great things ahead. Let's carry the best of the past forward into the future.

I don't always state my intentions this explicitly: because that is no where near as fun for me. But I don't want folks to get the wrong impression either, so please help me get the message right if you can.

As always, thanks for reading.

Hello Plone Again

2013-01-09T15:30:00-05:00

Regarding my recent "Goodbye Plone" post:

It's come to my attention that the style of my departure post, and associated actions, may be perceived poorly. I'm sorry. This was not my intention, and I apologize if I've upset anyone. The Plone Community deserves the best, so let me see what I can do to address this.
Several folks have reached out to express their concern about my departure, I appreciate this very much.
While I stand behind my need to distance myself from such conflicts, I don't want to give the Plone Community or the outside world the wrong impression.

So, let me say this: I'm thinking about what happened today, and I'm going to comment about it in the future. I.e. after 24 hours or so. Regardless of future updates, please consider me: "around in some capacity, but not as fully engaged for personal reasons" :-). This is an attempt to revert my abrubt departure, while at the same time not fully re-engaging until such time as I can determine the best course of action. Note that above all, and aside from my personal and professional goals, I don't want to be disruptive to the community if I can help it.

Thanks for reading.