Using Django as a Pass Through Image Proxy An earlier post shows you how to setup Nginx as a pass through image proxy. This post shows you how to do it with just Django and nothing else. The Problem We've solving the same problem as the earlier post. However, I will repeat it here for clarity as there's been some confusion. You have a production DB with lots of images uploaded by users. For example, NationalGeographic.com has over 11gb of user uploaded images. When you download a data dump of the production database, it has links to all these images which you don't have. You either have to download and sync all the images locally every time you copy the database, live with broken images or point your static images to the prod server. Copying images locally is the brute force method and will work. If you have all day to sync up images. Pointing to the images in prod also works, but if you upload your own images to test functionality... You will not be able to see those new images. You have a prod site with gigabytes and gigabytes of user generated static content. Whenever the database from production is copied to some lesser environment, like your sandbox, you either need to copy all the images locally, point static to the production server or live with broken images. Solution Django allows you to serve static files through your sandbox. By replacing Django's standard static "serve" function with the one below, you will be able to serve images locally, but look for them on a remote URL if it's not found locally. Thanks to Johnny Dobbins for the idea. """ Views and functions for serving static files. These are only to be used during development, and SHOULD NOT be used in a production setting. file: static_fallback.py """ import mimetypes import os import posixpath import urllib import urllib2 import django from django.conf import settings from django.http import Http404 , HttpResponse , HttpResponseRedirect from django.views.static import serve as django_serve def serve ( request , path , document_root = None , show_indexes = False , cache = True , fallback_server = None ): """ Serve static files using Django's static file function but if it returns a 404, then attempt to find the file on the fallback_server. Optionally and by default, cache the file locally. To use, put a URL pattern such as:: (r'^(?P<path>.*)$', 'static_fallback.serve', {'document_root' : '/path/to/my/files/'}) in your URLconf. You must provide the ``document_root`` param (required by Django). You may also set ``show_indexes`` to ``True`` if you'd like to serve a basic index of the directory. These parameters are passed through directly to django.views.static.serve. You should see the doc_string there for details. Passing cache to True (default) copies the file locally. Be sure to set settings.FALLBACK_STATIC_URL to something like: FALLBACK_STATIC_URL = 'http://myprodsite.com' Alternatively, you can also tell it the fallback server as a parameter sent in the URLs. Author: Ed Menendez ( [email protected] Concept: Johnny Dobbins """ # This was mostly copied from Django's version. We need the filepath for # caching and it also serves as an optimization. If the file is not found # then there's no reason to go through the Django process. try : fallback_server = settings . FALLBACK_STATIC_URL except AttributeError : print u"You're using static_fallback.serve to serve static content " + \ "however settings.FALLBACK_STATIC_URL has not been set." # Save this for later to pass to Django. original_path = path path = posixpath . normpath ( urllib . unquote ( path )) path = path . lstrip ( '/' ) newpath = '' for part in path . split ( '/' ): if not part : # Strip empty path components. continue drive , part = os . path . splitdrive ( part ) head , part = os . path . split ( part ) if part in ( os . curdir , os . pardir ): # Strip '.' and '..' in path. continue newpath = os . path . join ( newpath , part ) . replace ( ' \\ ' , '/' ) if newpath and path != newpath : return HttpResponseRedirect ( newpath ) # RETURN fullpath = os . path . join ( document_root , newpath ) # End of the "mostly from Django" section. try : # Don't bother trying the Django function if the file isn't there. if not os . path . isdir ( fullpath ) and not os . path . exists ( fullpath ): raise Http404 , '" %s " does not exist' % fullpath # RAISE else : # Pass through cleanly to Django's verson return django_serve ( # RETURN request , original_path , document_root , show_indexes ) except Http404 : if fallback_server : # Attempt to find it on the remote server. fq_url = ' %s%s ' % ( fallback_server , request . path_info ) try : contents = urllib2 . urlopen ( fq_url ) . read () except urllib2 . HTTPError : # Naive to assume a 404 - ed raise Http404 , '" %s " does not exist' % fq_url # RAISE else : # Found the doc. Return it to response. mimetype = mimetypes . guess_type ( fq_url ) response = HttpResponse ( contents , mimetype = mimetype ) # Do we need to cache the file? if cache : if not os . path . exists ( os . path . split ( fullpath )[ 0 ]): os . makedirs ( os . path . split ( fullpath )[ 0 ]) f = open ( fullpath , 'wb+' ) f . write ( contents ) f . close () # Success! We have the file. Send it back. return response # RETURN else : # No fallback_server was defined. So, it's really a 404 now. raise Http404 # RAISE django, programming, python



View the discussion thread.