On the first day of sprints at US PyCon 2012 we had a long and fruitful discussion about PEP 382 and PEP 402 . We ended up rejecting both but a new PEP will be written to carry on in the spirit of PEP 402 . Martin von Löwis wrote up a summary: .

However, the proposed changes do NOT add any performance overhead to the importing of existing modules or packages, and performance for the new directory layout should be about the same as that of previous "namespace package" solutions (such as pkgutil.extend_path() ).

The proposed enhancements do not change the semantics of any currently-importable directory layouts, but make it possible for packages to use a simplified directory layout (that is not importable currently).

"Most packages are like modules. Their contents are highly interdependent and can't be pulled apart. [However,] some packages exist to provide a separate namespace. ... It should be possible to distribute sub-packages or submodules of these [namespace packages] independently." —Jim Fulton, shortly before the release of Python 2.3

When new users come to Python from other languages, they are often confused by Python's package import semantics. At Google, for example, Guido received complaints from "a large crowd with pitchforks" that the requirement for packages to contain an __init__ module was a "misfeature", and should be dropped.

In addition, users coming from languages like Java or Perl are sometimes confused by a difference in Python's import path searching.

In most other languages that have a similar path mechanism to Python's sys.path , a package is merely a namespace that contains modules or classes, and can thus be spread across multiple directories in the language's path. In Perl, for instance, a Foo::Bar module will be searched for in Foo/ subdirectories all along the module include path, not just in the first such subdirectory found.

Worse, this is not just a problem for new users: it prevents anyone from easily splitting a package into separately-installable components. In Perl terms, it would be as if every possible Net:: module on CPAN had to be bundled up and shipped in a single tarball!

For that reason, various workarounds for this latter limitation exist, circulated under the term "namespace packages". The Python standard library has provided one such workaround since Python 2.3 (via the pkgutil.extend_path() function), and the "setuptools" package provides another (via pkg_resources.declare_namespace() ).

The workarounds themselves, however, fall prey to a third issue with Python's way of laying out packages in the filesystem.

Because a package must contain an __init__ module, any attempt to distribute modules for that package must necessarily include that __init__ module, if those modules are to be importable.

However, the very fact that each distribution of modules for a package must contain this (duplicated) __init__ module, means that OS vendors who package up these module distributions must somehow handle the conflict caused by several module distributions installing that __init__ module to the same location in the filesystem.

This led to the proposing of PEP 382 ("Namespace Packages") - a way to signal to Python's import machinery that a directory was importable, using unique filenames per module distribution.

However, there was more than one downside to this approach. Performance for all import operations would be affected, and the process of designating a package became even more complex. New terminology had to be invented to explain the solution, and so on.

As terminology discussions continued on the Import-SIG, it soon became apparent that the main reason it was so difficult to explain the concepts related to "namespace packages" was because Python's current way of handling packages is somewhat underpowered, when compared to other languages.

That is, in other popular languages with package systems, no special term is needed to describe "namespace packages", because all packages generally behave in the desired fashion.

Rather than being an isolated single directory with a special marker module (as in Python), packages in other languages are typically just the union of appropriately-named directories across the entire import or inclusion path.

In Perl, for example, the module Foo is always found in a Foo.pm file, and a module Foo::Bar is always found in a Foo/Bar.pm file. (In other words, there is One Obvious Way to find the location of a particular module.)

This is because Perl considers a module to be different from a package: the package is purely a namespace in which other modules may reside, and is only coincidentally the name of a module as well.

In current versions of Python, however, the module and the package are more tightly bound together. Foo is always a module -- whether it is found in Foo.py or Foo/__init__.py -- and it is tightly linked to its submodules (if any), which must reside in the exact same directory where the __init__.py was found.

On the positive side, this design choice means that a package is quite self-contained, and can be installed, copied, etc. as a unit just by performing an operation on the package's root directory.

On the negative side, however, it is non-intuitive for beginners, and requires a more complex step to turn a module into a package. If Foo begins its life as Foo.py , then it must be moved and renamed to Foo/__init__.py .

Conversely, if you intend to create a Foo.Bar module from the start, but have no particular module contents to put in Foo itself, then you have to create an empty and seemingly-irrelevant Foo/__init__.py file, just so that Foo.Bar can be imported.

(And these issues don't just confuse newcomers to the language, either: they annoy many experienced developers as well.)

So, after some discussion on the Import-SIG, this PEP was created as an alternative to PEP 382, in an attempt to solve all of the above problems, not just the "namespace package" use cases.