python
Notes on Upgrading Django
I recently pushed a bunch of code to my staging server for various projects after upgrading to Django 1.2.4, and things got a little frustrating, so I figured I’d share. So, obviously, I had upgraded in my local dev environment, tests pass and, to be completely honest, these are pretty small sites that I had assumed made little use of things that were backwards incompatible. Which, as it turns out, I was correct in assuming, but I was getting errors like:
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] Traceback (most recent call last):
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] File "/usr/lib/pymodules/python2.6/django/core/handlers/wsgi.py", line 230, in __call__
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] self.load_middleware()
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] File "/usr/lib/pymodules/python2.6/django/core/handlers/base.py", line 42, in load_middleware
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] raise exceptions.ImproperlyConfigured, 'Error importing middleware %s: "%s"' % (mw_module, e)
[Fri Jan 07 20:56:54 2011] [error] [client 192.168.2.102] ImproperlyConfigured: Error importing middleware django.middleware.csrf: "No module named csrf"
[Sat Jan 08 14:37:28 2011] [error] [client 192.168.2.102] mod_wsgi (pid=22250): Exception occurred processing WSGI script '/var/www/myproject/myproject/wsgi/app.py'.
It was obvious that by requiring a new feature (in this case CSRF), my app was parking because it was using an older version of Django.
Long story short, make sure that all previous versions of Django are gone before you re-install. Because I was a n00b when I set up my staging server, my Django install wasn’t sandboxed via a virtualenv and so I had artifacts of it in my PYTHONPATH which my apps were using. For me, I even found pieces in /usr/lib/pymodules/python2.6 on Ubuntu 10.04, weird considering global packages are usually installed in /usr/local/lib/python2.6/dist-packages. One way to get to the bottom of it all, is to grab your entire PYTHONPATH:
python -c 'import sys; print sys.path'
And make sure that Django is no where to be found before re-installing. Once I got all this cleared up, I went ahead and created virtualenvs for each of my projects so this wouldn’t happen again and so that I could isolate problems more effectively.
I guess this is just another little tidbit of being relatively new to python/django administration.
It’s all obvious in the end.
Hope this helps.
Filtering Special Characters Injected by Microsoft Word
I have been having trouble with users pasting content directly from M$ Word into admin fields, so I set out to create some sort of filter that replaces these little terds with something more HTML friendly. Here’s what I came up with, efficiency aside:
Please fork and enhance if you find more of these evil demon characters anywhere.
I guess it should also be noted that this can be called from any model save method on fields that you wish to filter.
UPDATE:
Just use SmartyPants: http://pypi.python.org/pypi/smartypants/
Simple Fix for python-memcached MemcachedKeyLengthError and MemcachedKeyCharacterError
Here at work, we recently ripped out cmemcache and replaced it with the more apache+mod_wsgi friendly python-memcached, only to be innundated with a bunch of these:
MemcachedKeyCharacterError: Control characters not allowed
as well as these:
MemcachedKeyLengthError: Key length is > 250
The good news was that we knew it was definitely related to our new memcache library that Django was falling back on, but the bad news was that we didn’t want to go through all of our code and filter/hash our keys for every call to cache.get() or cache.set(). My simple solution was to write a set of overrides to the default Django memcached backend. I went ahead and gist’d it if anyone is interested:
This solution truncates and filters the keys as opposed to hashing them like some would suggest. I believe either method should work fine for the most part, hashing offering a far lesser chance of key collisions, but I was a little scared to start hashing keys after reading how it bit Reddit in the ass:
http://www.royans.net/arch/reddit-learning-from-mistakes/
This seems to work well at alleviating our python-memcached woes since our keys should be safely unique after truncating. One thing to note though, that took me for a sec (because I was referring to Django trunk) is that `set_many` and `delete_many` are added in Django 1.2, so they will not work unless using 1.2 or unless your backend already implements them. All others methods should work though.
Cheers.
Notes on PIL + virtualenv + Ubuntu
Just some quick notes on installing PIL in a virtualenv in Ubuntu:
Most forums or message boards suggest:
sudo apt-get install python-imaging
to install PIL, but that doesn’t exactly do when you’re trying to install into a virtualenv with –no-site-packages, so… you try to install from source by downloading the tarball and doing the usual:
wget http://effbot.org/downloads/Imaging-1.1.7.tar.gz
tar -xzf Imaging-1.1.7.tar.gz
cd Imaging
sudo python setup.py install
In some cases this works, but other times build will fail with this:
...
_imaging.c:3138: warning: return type defaults to ‘int’
_imaging.c: In function ‘DL_EXPORT’:
_imaging.c:3138: error: expected declaration specifiers before ‘init_imaging’
_imaging.c:3149: error: expected ‘{’ at end of input
A little known fact is that if you first install python developer tools, the build will go smoothly:
sudo apt-get install python-dev
This may install a few packages other than just python2.6-dev, but whatever. You should now be able to build and install as usual:
sudo python setup.py install
Took me some poking around, so I figured I’d share, maybe save someone some time.
Handy One-Liners
It’s been a while since I’ve posted, so I figured I’d just go with something that I found to have been pretty useful as of late. Peteris Krumins recently posted a great article about one-liners, so I figured I’d add a few of my own to that list:
- Python SMTP Server
python -m smtpd -n -c DebuggingServer localhost:1025This just starts an email consumer on your localhost on port 1025. When emails are sent to it, they are echoed to STDOUT instead of being routed. Handy when developing features that require email.
- Git Graph ‘Pretty Print’
git log --graph --abbrev-commit --pretty=oneline --decorateThis really just serves as a replacement to gitk or GitX in a (command-line) pinch.
- Remove .pyc
find . -name '*.pyc' -exec rm {} \;Does just what you’d think it does: removes all .pyc files from the current directory you are in.
These are all pretty much just great candidates for bash aliases, but I figured I’d share since I was inspired. Thanks Peteris, and if you haven’t checked out his article, do so IMMEDIATELY:
Django 1.1 upgrade causes TypeError: __init__() takes exactly 1 argument (2 given)
Another Django 1.1 gotcha: So, I was just upgrading another project to Django 1.1 (actually 1.1.1) and it didn’t go smoothly. I installed and immediately tried to runserver but was promptly halted by this bad boy:
Traceback (most recent call last):
File "manage.py", line 11, in
execute_manager(settings)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/__init__.py", line 362, in execute_manager
utility.execute()
...
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/contrib/admin/options.py", line 5, in
from django.contrib.contenttypes.models import ContentType
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/contrib/contenttypes/models.py", line 1, in
from django.db import models
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/__init__.py", line 57, in
'TIME_ZONE': settings.TIME_ZONE,
TypeError: __init__() takes exactly 1 argument (2 given)
So, I went right to the source to understand what was going on. After looking through the source and some Django bug reports/tickets, I was no nearer to a solution, in fact, I was more baffled than ever. Things just didn’t add up. After modifying some Django source (eeeek!) I got to point where I had to call bullshit: module references were gone from the source. Something was up. But, to make a long story short, I said screw it, I’m gonna blow away my entire Django installation and start from scratch. Aaaand…. voilà! Things worked.
My advice for those who said TL;DR: just re-install Django from scratch
That is all for now… good day.
django-remoteobjects
Just got done with our first LA Django meetup. It was awesome. I gave a lightning talk on django-remoteobjects, Jacob gave a talk on celery, and Jeff gave the token testing talk. It was awesome and I look forward to future meetups with people like James Tauber, Jacob Kaplan-Moss, Mike Malone in attendance. My slides could be better, put them together in about an hour after a hectic day of work, but whatever, had to start somewhere. I also wrote a Mahalo API client thanks to remoteobjects in about 20 minutes. Even though Mahalo doesn’t really have an open API, I think I’m going to gist it tomorrow once I’m back in the office, with API creds omitted an all. Will also do a follow up post too (hopefully) with more details once I’ve played around with this little jewel some more given the lack of some documentation on the project.
Cheers.
URL encoding unicode characters in python+django
Let’s say you’re like me and you need to URL encode a user generated string to be sent across the pipe to a web service of some variety. Well what happens when that user’s copying stuff straight from Microsoft Word, or creating unicode characters somehow? You flex your 1337 python muscles that’s what. This deals with with Django and Python’s urllib and the exception that gets thrown when you try to url encode unicode characters such as:
'ascii' codec can't encode character u'\xb0' in position 96: ordinal not in range(128)
Let’s say your user is creating data that translates to this dictionary that you would normally pass to urllib.urlencode():
data = { 'param1': u'ßßß', 'param2': u'\u2044' }
Here’s a fancy one liner that I use to ensure my data dictionary is going to check out:
import types
data = dict([(k,v.encode('utf-8') if type(v) is types.UnicodeType else v) \
for (k,v) in data.items()])
This makes sure that for every item in your data dictionary that is of unicode type, it is first UTF-8 encoded before passing it in to urllib.urlencode()
How ’bout them apples?
Django 1.1 and InMemoryUploadedFile
Just had a Django 1.1 gotcha after scratching my head in what I felt like was just python noob-ness. So, here’s the problem that is surfacing when trying to run a Django 1.0 app on Django 1.1:
'InMemoryUploadedFile' object has no attribute '_file'
DUBBYA.T.EFF.
The code is doing some things requiring request.FILES (after a file upload) to manipulate an uploaded file hence the use of the _file attribute. Nothing too crazy, but still, it worked and now it doesn’t.
On to solving the problem. After delving through the Django source at this so-called InMemoryUploadedFile class I saw a very meaningful revision. Yep, the _file attribute was changed to just file. Awesome. The Django release notes mention changes to the way file uploads are handled, but isn’t exactly concise to this degree.
And there you have it. My first Django 1.1 gotcha. I changed the _file reference to file and everything was peachy.
Hope this helps someone solve their problem a little more quickly than I did.
Simple Django Age Verification Middleware
Anyone who has ever worked on a site for the liquor industry knows that law requires, in a lot of cases, users to be of age to view their site. Well, this is my solution to this little debacle with the help of our friend django. I saw it as something like an authentication filter of sorts and as a perfect chance to get my hands dirty with some django middleware. I was thinking that it could act just like django.contrib.sessions.middleware in adding a boolean attribute to the user object for every request based on the value of a session key. Then, to accompany it, a view decorator, just like django.contrib.auth.decorators.login_required to make sure that a given view redirects to the verify age page if need be. It’s really pretty simple, the code and instructions can be seen here.
The main reason why I published this code to djangosnippets.org is because it is one of many uses one can tailor this pattern to. It acts as a simple view-specific filter. For example, one might want to only show a specific alert to users once, simply flipping a flag in a database once the message has been shown. The fact that it is session-backed is arbitrary. Settings in session, cookie or database can easily fit into this pattern.
Cheers!
Search
What I'm Doing...
- Everyone that considers themselves a Mizzou fan can... well, go fsck themselves. #checkyofilesystem 13 hrs ago
- Missouri sucks. Period. 13 hrs ago
- @aaronky seat's taken brah! You're not welcome in these parts. in reply to aaronky 15 hrs ago
- More updates...
Powered by Twitter Tools
del.icio.us
Tags
friends
Categories
Social Links