Upgrading Django (to 1.7)

Part I: Introduction and Django Releases Published September 24, 2014 by Andrew Pinkham Reviewed by Jacinda Shelly Edited by Amy Bekkerman Tweet

At 1:30 PM PDT on September 3, 2014, I presented a talk called "Upgrading Django (to 1.7)" to DjangoCon 2014. This article series presents and extends upon the content of that presentation. If you would prefer to watch my presentation or see the slides, that material may be found here.

Introduction

The goal of this series is to prepare developers for an upgrade to Django 1.7, released September 2, 2014. To that extent, I have identified four topics I think are helpful:

Django Versioning and Recent History An Introduction to Migrations in Django 1.6 and 1.7 Django 1.7's key new features Recommended Upgrade Methods and Strategies for Django

Due to the length of the article, I have decided to split the content into the four sections listed above. The series will be published on a weekly basis.

Part I is an introduction to the series, as well as an overview of Django's versioning scheme. The content will examine the meaning of the numbers associated with each Django release, ensuring that you understand the way each release is created and related. The article then covers Django's recent history in order to clarify changes made in Django 1.7. The article is for developers of all levels .

. Part II, to be published on October 1, 2014 , is an introduction to Migrations. In this part we will build a project first in Django 1.6 using South, and then in Django 1.7. The first two-thirds of this article is targeted at beginners , but developers of all levels should find the final third instructive, as it focuses solely on Django 1.7.

, is an introduction to Migrations. In this part we will build a project first in Django 1.6 using South, and then in Django 1.7. The first two-thirds of this article is targeted at , but developers of should find the final third instructive, as it focuses solely on Django 1.7. Part III, to be published October 8, 2014 , is organized by feature. Each section of the article will present the history of the new features covered in Part II and then examine the new tools. The first two features, Migrations and App Loading, will be available to beginners . The content presented for Systems Check, Custom QuerySets, and the Prefetch object is meant for intermediate developers.

, is organized by feature. Each section of the article will present the history of the new features covered in Part II and then examine the new tools. The first two features, Migrations and App Loading, will be available to . The content presented for Systems Check, Custom QuerySets, and the object is meant for developers. Part IV, to be published October 15, 2014, will begin by outlining practical steps to take for every Django upgrade. This part will then focus on key quirks of upgrading to Django 1.7 and finish with a discussion of Django's upcoming features as well as the planned release schedule. This section should be accessible to developers of all levels.

Please consider this series as an introduction, not a comprehensive study of each new feature. For reference materials, please refer to the official Django reference documentation.

Django Versioning

In the following sections we will examine the meaning of Django version numbers, establishing how Django releases are related, and how this affects you.

Release Number Meaning

James Bennett, the Django release manager, annotates each release with three numbers. Each number carries meaning and allows developers to infer information about changes made to the project. Prior to 1.7, the core development team presented these numbers in the form A.B.C and referred to each number as the major, minor, and micro release number. Django 1.6 thus has major number 1, minor number 6, and an implicit micro number 0.

Starting with Django 1.7, the names of these numbers have changed. The official Release Process guide groups A.B as the major number, while C has become the minor number.

In both Django 1.6 and Django 1.7, a new release requires that one of the numbers in the version identifier be incremented. Prior to Django 1.7, the release numbers were strictly defined according to the following rules:

Incrementing the micro number was a micro release (casually called a patch), which was for bug fixes and security releases.

Incrementing the minor number was a minor release, which was for new features and timed feature deprecation.

Incrementing the major number was a major release, which was (theoretically) for large, incompatible changes.

Django 1.7 removes the possibility for this last release type. There has never been a major release as defined above, and the core development team decided that there never would be. Furthermore, Django community members deemed that incrementing to 1.10 after 1.9 would be confusing, due to the similarity between 1.1 and 1.10. Django instead opted to follow version 1.9 with 2.0. This effectively means that the major release number is simply an extension of the minor release number and is why the Django 1.7 release process guide changes the names of the release numbers.

Despite the official mandate to refer to A.B as the major number, I find referring to two numbers as a single number confusing. Throughout the rest of the article, I will continue to refer to the three release numbers as major, minor, and micro. However, I will use the new names for releases. Renaming the releases makes sense, as there are now only two of them. Consider, however, that the purpose of these two releases has not changed; only the name of the releases has changed.

Minor releases (originally micro releases or patches) denote bug fixes and security releases. Minor releases increment the micro number.

Major releases (originally minor releases) denote new features and timed feature deprecation. A major release affects both the major number and minor number. The minor number may only be a single digit integer. If the minor number wraps (9 becomes 0), then then major number is incremented. As such, we expect 1.8 to lead to 1.9 to lead to 2.0 and then 2.1.

In direct opposition to minor release numbers, micro release numbers may be double digits. Django version 1.4.15 currently exists and is a valid release identifier.

Colloquially, a Django version refers to different major releases (A.B is incremented). Django 1.7 and Django 1.6 are different versions. However, Django 1.6.1 and Django 1.6.2 are different patches (minor releases) of the same version. You should always be using the latest minor release of any version. While upgrading from one version to another takes some effort, upgrading between minor versions should not.

Django also iterates through a pre-release process. Core developers will first release alpha versions, move into betas, and then provide release candidates. In alpha and beta versions, anything may change, and developers should not be using these pre-releases for actual development. Release candidates are releases that are almost ready for production use. API calls and features may not be modified, and changes are limited to bug and security fixes.

To separate pre-releases from other Django releases, core developers append a version letter and a version number to A.B.C for pre-releases. For clarity, I will now refer to A.B.C as M.m.μ. In the pre-release format, the letter a signifies alpha, b is for beta, and c denotes release candidates. The resulting format is thus M.m(.μ){a,b,c}#. As such, Django 1.7.0b2 is the second beta release of Django 1.7, while Django 1.7c1 is the first release candidate.

Release candidates are very useful, as they are a good place to begin the upgrade process for your site. If you run into a problem, you can submit a bug report on the Django project website, at which point a developer will fix the problem. A core developer will then include the fix in the next release candidate. That said, release candidates are not meant for production; wait until the full release of a new version before deploying an upgrade to a production server.

Version Support

The core Django team guarantees supports for two major releases of Django at a time. Officially, the latest version becomes the only version to receive critical bug fixes, while both supported versions will be eligible for security updates. In the case of documentation, core developers focus on the latest version, but allow for selective back-porting.

More concretely, when Django 1.6 was the current stable release, both Django 1.5 and Django 1.6 were supported. Both Django 1.6 and 1.5 were eligible for security updates, but only Django 1.6 received critical bug fixes and potential documentation improvements. The release of Django 1.7 signals the death of Django 1.5. The release of Django 1.7 further limits Django 1.6 to security updates in favor of improvements to Django 1.7.

The Django release process allows for support of more than two versions. The core developers reserve the right to mark versions for Long-Term Support (LTS), which guarantees security updates to these versions for a guaranteed period of time, usually 3+ years. At the moment, only Django 1.4 is marked for LTS and will be supported until at least March 2015.

With the release of Django 1.7, the core developers will thus provide support for Django 1.7, Django 1.6, and Django 1.4.

Feature Deprecation

The core development team deprecates features over a two-version period. Features and API calls that are deprecated in one version will continue to work for the next version and will be fully removed from the subsequent version. Essentially, features marked for deprecation in Django 1.5 will still work in version 1.6 but will be entirely removed from the 1.7 codebase. In this example, the feature would raise a silent warning in version 1.5 that you can opt in to viewing. The feature would then raise a default loud warning in Django 1.6 and fail entirely in 1.7.

Consider that the number two as Django's magic number. There are two guaranteed supported releases, and deprecations occur over two versions. Understanding the two versions prior to the one you're using is key to understanding whether existing code will continue to work.

Semantic Versioning, PEP Versioning, and Django Versioning

Many popular open-source projects release versions according to the rules of semantic versioning. Similar to Django's versioning system, semantic versioning annotates each release with three numbers, called major, minor, and patch numbers. These numbers may only increase. At first glance, it would seem that Django follows a semantic versioning scheme using slightly different number names, but this is not the case: Django does not adhere to semantic version rules.

Django versioning differs from semantic versioning because of feature deprecation. In semantic versioning, deprecation of features or the API may only occur when the major release number is incremented. Django, however, deprecates features when the minor release number is incremented, and it does so on a strict timeline. Furthermore, pre-releases are quite different in semantic versioning. Django 1.7a1 would be written as 1.7-alpha.1.

Django's release versions are much more in keeping with Python's recommended module versioning system, put forward in Python Enhancement Proposals (PEP) 440. The PEP provides a slew of flexible rules for writing version numbers. Django follows a subset of these rules and provides its own system for what the numbers mean, something with which PEP 440 does not concern itself.

In conclusion, Django explicitly defines it's own versioning rules, which is a subset of PEP 440 but different from semantic versioning.

Django Version History

The current active versions of Django are versions 1.7, 1.6, and 1.4. Because the core developers deprecate features according to a two-version timeline, establishing the changes made in Django 1.5 and 1.6 can facilitate an understanding of decisions made in 1.7. Furthermore, a look at the evolution of the framework may help reveal long-term objectives. As such, we will now summarize the key changes made in versions 1.4, 1.5, and 1.6. We will finish with a rapid overview of Django 1.7.

Changes in Django 1.4

The release of Django 1.4 in March 2012 saw a slew of new features. Notably, Django introduced timezone support, making upgrades tricky, but well worth the effort. Django also refined the Object Relational Manager (ORM), allowing for the use of SELECT FOR UPDATE and prefetch_related and improving the performance of bulk inserts. The user authentication contributed library gained the ability to store passwords using PBKDF2 and bcrypt, a massive improvement over the sole SHA1 option previously available. Importantly, sessions could now be stored in cryptographically signed cookies and were by default. The ability to use functional, in-browser testing tools such as Selenium became possible natively thanks to the addition of the django.test.LiveServerTestCase class. Django 1.4 also saw the addition of reverse_lazy , a new folder structure, and the ability to customize project creation.

There was no central feature that made Django 1.4 different from Django 1.2 or Django 1.3. However, the combination of features in Django 1.3 and Django 1.4 updated Django to a full-featured web framework. Django 1.4 is arguably the first modern version of the web framework, and likely the reason Django 1.4 was selected for LTS, although I have no proof of this.

Changes in Django 1.5

The release of Django 1.5 in February 2013 heralded experimental support for Python 3.2, and dropped support for Python 2.5, limiting Django to Python 2.6 and 2.7. Timezone support---introduced in 1.4---was improved and introduced fundamental changes in the structure of Django's User model that made it configurable and easier to extend. This eliminated the need for the old UserProfile extension pattern, which required a join to another table if users had additional data associated with their profiles.

Deprecations from version 1.3 came into full swing and included deprecation of generic function views in favor of generic, class-based views. Django 1.5 also saw the deprecation of the simplejson package in favor of the json package included in Python 2.6.

If you had avoided deprecated features and were already in version 1.4, the upgrade was simple, if a little tricky: some template tag syntax changed (such as the parameters for url ) and certain new settings---notably USE_TZ , ALLOWED_HOSTS , and SECRET_KEY (following an accelerated deprecation from 1.4)---became necessary for deployment, causing some confusion.

For developers seeing far fewer new features, Django 1.5 felt like a smaller update compared to Django 1.4, although the User model changes initially led to some confusion about best practices. The ability to run Django in Python 3 should not be underestimated, as the work done by the contributors was significant.

Changes in Django 1.6

Django 1.6, released November 2013, ushered in official support for Python 3.2 and maintained support for Python 2.6 and above. The biggest change to Django was the rewrite of database transactions, work led by core committer Aymeric Augustin. Under the hood, Django now auto-committed database changes, introduced the concept of atomicity in views via a decorator, and allowed for fine-grained rollbacks between save points (as long as the database supported these features). Django now also allowed for persistent database connections, significantly reducing the cost of transactions. As database transactions had remained largely unchanged since Django 0.9, the refactor of this code was big news, and a very positive change for Django. It largely overshadowed smaller but still significant changes like the new test runner.

Following in the steps of Django 1.4, Django 1.6 changed the anatomy of a new default project. The project settings file now defaults to best practice by providing a BASE_DIR variable to identify the path to the project, avoiding hardcoded absolute paths entirely. By further enabling contrib.admin in settings with autodiscover() enabled in the project URL configuration and providing a basic SQLite3 database configuration, the default Django 1.6 project has become far more approachable to beginners. What's more, it is also more secure and practical: clickjack prevention middleware comes out of the box, and the time zone is set to UTC.

A small but notable change for the user authentication contributed library was that user reset URLs were switched from base36 to base64 encoding. Django 1.6 provides a shim to ease the upgrade of the password reset system, allowing for uninterrupted service.

New apps created by startapp now included a simplified tests.py file and an admin.py file to reflect the addition of contrib.admin as a default. The django-admin.py check tool, which checks for basic errors in the project settings file, was introduced, easing the upgrade process from 1.5. While it was a larger release than version 1.5, Django 1.6 represented a fairly easy upgrade as long as you weren't using the old transactions API or relying on the comments, markdown, or local flavor contributed apps, which were all deprecated in this version.

Introducing Django 1.7

Django 1.7 is shaping up to be the biggest Django release since 1.0. Russell Keith-Magee on the Django Weblog

With the addition of native schema and data migrations, which we will investigate in depth in a moment, Django 1.7 has been lauded as a major new version. Django also upgrades the App Loading mechanism, untouched since version 0.9, and modularizes and upgrades the System Check. We will see these new features in Part II and Part III.

Importantly, Django drops support for Python 2.6 as of the new version. Codebases seeking to upgrade to Django 1.7 must run in Python 2.7 or Python 3.2 and above.

Conclusion

Django offers its own explicit versioning system, informing developers of the kinds of changes that occur with each release. Django versions are identified by three numbers that allow developers to distinguish between major and minor releases. Colloquially, a Django version refers to a release that has a different major or minor release number.

Django supports at least two versions, optionally allowing for certain releases to be marked for Long-Term Support and to have their lifetimes extended.

Django deprecates features over two major releases. A feature deprecated in 1.5 will work in both 1.5 and 1.6 but will cease to function in 1.7.

The last four versions of Django have seen a lot of work. There are a few trends worth noticing:

Emphasis on Python 3. Django developers with large projects should begin to plan their migration to Python 3. Starting in Django 1.6, third-party app developers were encouraged to make their apps compatible with Python 3, and the migration has largely been quite successful. Most of the key, maintained apps for Django now support Python 3. While there is no official word on the matter or an expected timeline, it is only a matter of time before support for Python 2 is dropped.

Slimming Django. Django's recent versions have seen extraneous contributed apps removed from Django in favor of new Python features or more specialized packages. Core features, by contrast, have been improved and extended. Django's long-term goal is to provide features necessary to a website and nothing more. Tools such as authentication and migrations are thus here to stay, as they are both difficult to code correctly and crucial for websites to get right. Other, less essential tools will be removed in upcoming versions. The webdesign app, for instance, will be removed in version 1.8, along with a number of utility functions and middleware functionality.

We will return to the topic of Django's trends and future in Part IV of the series. Before we do so, however, we will first investigate migrations and their utilities in Part II and focus on Django 1.7 features in Part III.