evanculver.com

Notes on Upgrading Django

I recently pushed a bunch of code to my staging server for various projects after upgrading to Django 1.2.4, and things got a little frustrating, so I figured I’d share. So, obviously, I had upgraded in my local dev environment, tests pass and, to be completely honest, these are pretty small sites that I had assumed made little use of things that were backwards incompatible. Which, as it turns out, I was correct in assuming, but I was getting errors like:


[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] Traceback (most recent call last):
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] File "/usr/lib/pymodules/python2.6/django/core/handlers/wsgi.py", line 230, in __call__
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] self.load_middleware()
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] File "/usr/lib/pymodules/python2.6/django/core/handlers/base.py", line 42, in load_middleware
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] raise exceptions.ImproperlyConfigured, 'Error importing middleware %s: "%s"' % (mw_module, e)
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] ImproperlyConfigured: Error importing middleware django.middleware.csrf: "No module named csrf"
[Sat Jan 08 14:37:28 2011] [error] [client 192.168.2.102] mod_wsgi (pid=22250): Exception occurred processing WSGI script '/var/www/myproject/myproject/wsgi/app.py'.


It was obvious that by requiring a new feature (in this case CSRF), my app was parking because it was using an older version of Django.

Long story short, make sure that all previous versions of Django are gone before you re-install. Because I was a n00b when I set up my staging server, my Django install wasn’t sandboxed via a virtualenv and so I had artifacts of it in my PYTHONPATH which my apps were using. For me, I even found pieces in /usr/lib/pymodules/python2.6 on Ubuntu 10.04, weird considering global packages are usually installed in /usr/local/lib/python2.6/dist-packages. One way to get to the bottom of it all, is to grab your entire PYTHONPATH:


python -c 'import sys; print sys.path'


And make sure that Django is no where to be found before re-installing. Once I got all this cleared up, I went ahead and created virtualenvs for each of my projects so this wouldn’t happen again and so that I could isolate problems more effectively.

I guess this is just another little tidbit of being relatively new to python/django administration.

It’s all obvious in the end.

Hope this helps.

Tags: django, python

Filtering Special Characters Injected by Microsoft Word

I have been having trouble with users pasting content directly from M$ Word into admin fields, so I set out to create some sort of filter that replaces these little terds with something more HTML friendly. Here’s what I came up with, efficiency aside:

http://gist.github.com/370004

Please fork and enhance if you find more of these evil demon characters anywhere.

I guess it should also be noted that this can be called from any model save method on fields that you wish to filter.

UPDATE:

Just use SmartyPants: http://pypi.python.org/pypi/smartypants/

Tags: django, microsoft, microsoft-word, python, special-characters, string-filtering

Notes on PIL + virtualenv + Ubuntu

Just some quick notes on installing PIL in a virtualenv in Ubuntu:

Most forums or message boards suggest:

sudo apt-get install python-imaging


to install PIL, but that doesn’t exactly do when you’re trying to install into a virtualenv with –no-site-packages, so… you try to install from source by downloading the tarball and doing the usual:

wget http://effbot.org/downloads/Imaging-1.1.7.tar.gz
tar -xzf Imaging-1.1.7.tar.gz
cd Imaging
sudo python setup.py install


In some cases this works, but other times build will fail with this:

...
_imaging.c:3138: warning: return type defaults to ‘int’
_imaging.c: In function ‘DL_EXPORT’:
_imaging.c:3138: error: expected declaration specifiers before ‘init_imaging’
_imaging.c:3149: error: expected ‘{’ at end of input


A little known fact is that if you first install python developer tools, the build will go smoothly:

sudo apt-get install python-dev


This may install a few packages other than just python2.6-dev, but whatever. You should now be able to build and install as usual:

sudo python setup.py install


Took me some poking around, so I figured I’d share, maybe save someone some time.

Tags: PIL, python, ubuntu, virtualenv

Django 1.1 upgrade causes TypeError: __init__() takes exactly 1 argument (2 given)

Another Django 1.1 gotcha: So, I was just upgrading another project to Django 1.1 (actually 1.1.1) and it didn’t go smoothly. I installed and immediately tried to runserver but was promptly halted by this bad boy:

Traceback (most recent call last):
File "manage.py", line 11, in
execute_manager(settings)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/__init__.py", line 362, in execute_manager
utility.execute()

File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/contrib/admin/options.py", line 5, in
from django.contrib.contenttypes.models import ContentType
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/contrib/contenttypes/models.py", line 1, in
from django.db import models
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/__init__.py", line 57, in
'TIME_ZONE': settings.TIME_ZONE,
TypeError: __init__() takes exactly 1 argument (2 given)


So, I went right to the source to understand what was going on. After looking through the source and some Django bug reports/tickets, I was no nearer to a solution, in fact, I was more baffled than ever. Things just didn’t add up. After modifying some Django source (eeeek!) I got to point where I had to call bullshit: module references were gone from the source. Something was up. But, to make a long story short, I said screw it, I’m gonna blow away my entire Django installation and start from scratch. Aaaand…. voilà! Things worked.

My advice for those who said TL;DR: just re-install Django from scratch

That is all for now… good day.

Tags: django, django1.1, python, upgrade

URL encoding unicode characters in python+django

Let’s say you’re like me and you need to URL encode a user generated string to be sent across the pipe to a web service of some variety. Well what happens when that user’s copying stuff straight from Microsoft Word, or creating unicode characters somehow? You flex your 1337 python muscles that’s what. This deals with with Django and Python’s urllib and the exception that gets thrown when you try to url encode unicode characters such as:

'ascii' codec can't encode character u'\xb0' in position 96: ordinal not in range(128)


Let’s say your user is creating data that translates to this dictionary that you would normally pass to urllib.urlencode():


data = { 'param1': u'ßßß', 'param2': u'\u2044' }


Here’s a fancy one liner that I use to ensure my data dictionary is going to check out:

import types

data = dict([(k,v.encode('utf-8') if type(v) is types.UnicodeType else v) \
for (k,v) in data.items()])

This makes sure that for every item in your data dictionary that is of unicode type, it is first UTF-8 encoded before passing it in to urllib.urlencode()

How ’bout them apples?

Tags: django, python, tips, unicode, urllib

Django 1.1 and InMemoryUploadedFile

Just had a Django 1.1 gotcha after scratching my head in what I felt like was just python noob-ness. So, here’s the problem that is surfacing when trying to run a Django 1.0 app on Django 1.1:

'InMemoryUploadedFile' object has no attribute '_file'

DUBBYA.T.EFF.

The code is doing some things requiring request.FILES (after a file upload) to manipulate an uploaded file hence the use of the _file attribute. Nothing too crazy, but still, it worked and now it doesn’t.

On to solving the problem. After delving through the Django source at this so-called InMemoryUploadedFile class I saw a very meaningful revision. Yep, the _file attribute was changed to just file. Awesome. The Django release notes mention changes to the way file uploads are handled, but isn’t exactly concise to this degree.

And there you have it. My first Django 1.1 gotcha. I changed the _file reference to file and everything was peachy.

Hope this helps someone solve their problem a little more quickly than I did.

Tags: development, django, django1.1, python

Simple Django Age Verification Middleware

Anyone who has ever worked on a site for the liquor industry knows that law requires, in a lot of cases, users to be of age to view their site. Well, this is my solution to this little debacle with the help of our friend django. I saw it as something like an authentication filter of sorts and as a perfect chance to get my hands dirty with some django middleware. I was thinking that it could act just like django.contrib.sessions.middleware in adding a boolean attribute to the user object for every request based on the value of a session key. Then, to accompany it, a view decorator, just like django.contrib.auth.decorators.login_required to make sure that a given view redirects to the verify age page if need be. It’s really pretty simple, the code and instructions can be seen here.

The main reason why I published this code to djangosnippets.org is because it is one of many uses one can tailor this pattern to. It acts as a simple view-specific filter. For example, one might want to only show a specific alert to users once, simply flipping a flag in a database once the message has been shown. The fact that it is session-backed is arbitrary. Settings in session, cookie or database can easily fit into this pattern.

Cheers!

Tags: development, django, python

Django Standalone Scripts follow-up

I was recently in need of a way to run python scripts with django settings loaded, access to models, etc. This wasn’t exactly obvious to my n00b-python-skills-bearing self, so I insta-googled: “django standalone scripts” to find the wonderfully robust article Mr. James Bennett wrote in the ye ‘ol year of 2007. Some awesome information there that ultimately lead to my success in running django standalone scripts, just wanted to summarize what I did to get this going:

1. Make sure your project is in your PYTHONPATH. — This alluded me at first, but on Mac OSX it was as easy as creating a symlink from my PYTHONPATH to my project directory:

   sudo ln -s /path/to/mysite /Library/Python/2.5/site-packages/mysite

   This may not be the most glamorous way of adding your project to your PYTHONPATH, as when your path to your project changes, you break your script, but it worked for me in a quick-and-dirty type of way.

2. Put this at the top of your script:

   from django.core.management import setup_environ
from mysite import settings
setup_environ(settings)


With this in place, you can do all the standard stuff you would do in your views.py to interact with your models and settings. Still though, have a look at James Bennett’s article for more details and ways to run django standalone scripts.

UPDATE:

I also like one of the solutions that can be found in the comments of James’ article. Put this at the top of your script instead of creating the symlink:

import sys, os
sys.path.append(‘/home/username/django/django_projects’)


In my initial example, I was running into problems because in my django project, I load settings based on where the project lives (dev – /Users/evan/dev, production – /var/www/production, etc.). In the original case, settings would always load thinking that it was in PYTHONPATH, which isn’t good if you want to have this smart-loading of settings. I think I like this second solution more as it doesn’t break your script if your PYTHONPATH/project path changes over time.

Tags: django, python