If you've used Python for very long, you've at least heard of list comprehensions. They're a way to fit a for loop, an if statement, and an assignment all in one line. In other words, you can map and filter a list in one expression.

2.1.1 Mapping the List We'll start with something really simple. Say you're trying to square every element in a list. A freshly-initiated Python programmer might write code like this: # 's 1 numbers = [ 1 , 2 , 3 , 4 , 5 ]

2 squares = []

3 for number in numbers :

4 squares . append ( number * number )

5 # Now, squares should have [1,4,9,16,25]

You've effectively 'mapped' one list to another list. You could also use the map function, and do something like this: # 's 1 numbers = [ 1 , 2 , 3 , 4 , 5 ]

2 squares = map ( lambda x : x * x , numbers )

3 # Now, squares should have [1,4,9,16,25]

This code is definitely shorter (1 line instead of 3) but it's pretty ugly. It's hard to tell at a glance what the map function does (it accepts a function and a list, and applies the function to every element of that list). Plus, you have to give it a function of some sort which looks kind of messy. If only there were a cleaner way... perhaps a list comprehension: # 's 1 numbers = [ 1 , 2 , 3 , 4 , 5 ]

2 squares = [ number * number for number in numbers ]

3 # Now, squares should have [1,4,9,16,25]

This does the exact same thing as the previous two examples, but it's short (unlike the first example) and clean (unlike the second example). No one is going to have any problem determining what it does, even if they don't know Python.

2.1.2 Filtering the List What if you're more interested in filtering the list? Say you want to remove every element with a value equal to or greater than 4? (Okay, so the examples aren't very realistic. Whatever...) A Python neophyte might write: # 's 1 numbers = [ 1 , 2 , 3 , 4 , 5 ]

2 numbers_under_4 = []

3 for number in numbers :

4 if number < 4 :

5 numbers_under_4 . append ( number )

6 # Now, numbers_under_4 contains [1,4,9]

Pretty simple, right? But it took 4 lines, two degrees of nesting, and an append to do something completely trivial. You could reduce the size of the code with the filter function: # 's 1 numbers = [ 1 , 2 , 3 , 4 , 5 ]

2 numbers_under_4 = filter ( lambda x : x < 4 , numbers )

3 # Now, numbers_under_4 contains [1,2,3]

Similar to the map function we talked about above, this reduces code size but is really ugly. What the hell is going on? Like map , filter accepts a function and a list. It evaluates for every list element and if the function evaluates to true, that list element is included in the final list. Of course, we can do this with a list comprehension as well: # 's 1 numbers = [ 1 , 2 , 3 , 4 , 5 ]

2 numbers_under_4 = [ number for number in numbers if number < 4 ]

3 # Now, numbers_under_4 contains [1,2,3]

Again, using a list comprehension gives us shorter, cleaner, and easier to understand code.

2.1.3 Map and Filter at Once Now we get to the true power of list comprehensions. If I haven't yet convinced you that map and filter are generally a waste of your time, hopefully this will. Say I want to map and filter a list at the same time. In other words, I'd like to see the square of each element in the list where said element is under 4. Once more, the Python neophyte way: # 's 1 numbers = [ 1 , 2 , 3 , 4 , 5 ]

2 squares = []

3 for number in numbers :

4 if number < 4 :

5 squares . append ( number * number )

6 # squares is now [1,4,9]

The code is starting to expand in the horizontal direction now! Alas, what could we possibly do to simplify the code? We could try using map and filter , but I don't have a good feeling about this... # 's 1 numbers = [ 1 , 2 , 3 , 4 , 5 ]

2 squares = map ( lambda x : x * x , filter ( lambda x : x < 4 , numbers ))

3 # squares is now [1,4,9]

While map and filter were ugly before, now they're just unreadable. Obviously this isn't a good idea. Once more, list comprehensions save the day: # 's 1 numbers = [ 1 , 2 , 3 , 4 , 5 ]

2 squares = [ number * number for number in numbers if number < 4 ]

3 # square is now [1,4,9]

This is a bit longer than the earlier list comprehension examples, but in my opinion still very readable. It's definitely better than a for loop or using map and filter . As you can see, a list comprehension filters then maps. If you absoulutely need to map then filter, things can get more complicated. You might even have to use nested list comprehensions, the map and filter commands, or a regular old for loop, depending on what is cleanest. That discussion, though, is outside the scope of this article.

2.1.4 Generator Expressions There is a downside to list comprehensions: the entire list has to be stored in memory at once. This isn't a problem for small lists like the ones in the above examples, or even of lists several orders of magnitude larger. But eventually this becomes pretty inefficient. Generator expressions are newish in Python 2.4, and possibly the least publicized Cool Thing About Python ever. As in, I just found out about them. Generator expressions do not load the whole list into memory at once, but instead create a 'generator object' so only one list element has to be loaded at any time. Of course, if you actually need to use the entire list for something, this doesn't really help much. But if you're just passing it off to something that takes any iterable object -- like a for loop -- you might as well use a generator function. Generator expressions have the same syntax as list comprehensions, but with parentheses around the outside instead of brackets: # 's 1 numbers = ( 1 , 2 , 3 , 4 , 5 ) # Since we're going for efficiency, I'm using a tuple instead of a list ;)

2 squares_under_10 = ( number * number for number in numbers if number * number < 10 )

3 # squares_under_10 is now a generator object, from which each successive value can be gotten by calling .next()

4

5 for square in squares_under_10 :

6 print square ,

7 # prints '1 4 9'

This is ever so slightly more efficient than using a list comprehension. So, you want to use generator expressions for large numbers of items. You want to always use list comprehensions if you need the entire list at once for some reason. If neither of these is true, just do whatever you want. It's probably good practice to use generator expressions unless there's some reason not to, but you're not going to see any real difference in efficiency unless the list is very large. As a final note, generator expressions only need to be surrounded by one set of parentheses. So, if you're calling a function with only a generator expression, you only need one set of parentheses. This is valid Python: some_function(item for item in list) .

2.1.5 Nested 'for' Statements List comprehensions and generator expressions can be used for more than just mapping and filtering; you can create rather complex lists of lists with them . Not only can you map and filter, you can nest the for expressions. A python neophyte might write something like: # 's 1 for x in ( 0 , 1 , 2 , 3 ):

2 for y in ( 0 , 1 , 2 , 3 ):

3 if x < y :

4 print ( x , y , x * y ),

5

6 # prints (0, 1, 0) (0, 2, 0) (0, 3, 0) (1, 2, 2) (1, 3, 3) (2, 3, 6)

You can see that this code is pretty crazy. With a list comprehension, though, you can do this more quickly: # 's 1 print [( x , y , x * y ) for x in ( 0 , 1 , 2 , 3 ) for y in ( 0 , 1 , 2 , 3 ) if x < y ]

2 # prints [(0, 1, 0), (0, 2, 0), (0, 3, 0), (1, 2, 2), (1, 3, 3), (2, 3, 6)]

As you can see, this code iterates over four values of y , and for each of those values, iterates over four values of x and then filters and maps. Each list item then, is itself a list of x, y, x * y . Note that xrange(4) is a bit cleaner than (0,1,2,3) , especially for longer lists, but we haven't gotten there yet.