List comprehension vs map












608















Is there a reason to prefer using map() over list comprehension or vice versa? Is either of them generally more efficient or considered generally more pythonic than the other?










share|improve this question




















  • 7





    Note that PyLint warns if you use map instead of list comprehension, see message W0141.

    – lumbric
    Oct 31 '13 at 10:37








  • 1





    @lumbric, I'm not sure but it does only if lambda is used in map.

    – 0xc0de
    May 23 '17 at 4:53
















608















Is there a reason to prefer using map() over list comprehension or vice versa? Is either of them generally more efficient or considered generally more pythonic than the other?










share|improve this question




















  • 7





    Note that PyLint warns if you use map instead of list comprehension, see message W0141.

    – lumbric
    Oct 31 '13 at 10:37








  • 1





    @lumbric, I'm not sure but it does only if lambda is used in map.

    – 0xc0de
    May 23 '17 at 4:53














608












608








608


236






Is there a reason to prefer using map() over list comprehension or vice versa? Is either of them generally more efficient or considered generally more pythonic than the other?










share|improve this question
















Is there a reason to prefer using map() over list comprehension or vice versa? Is either of them generally more efficient or considered generally more pythonic than the other?







python list-comprehension map-function






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 18 '18 at 18:56









codeforester

18.5k84267




18.5k84267










asked Aug 7 '09 at 23:43









TimothyAWisemanTimothyAWiseman

5,72992944




5,72992944








  • 7





    Note that PyLint warns if you use map instead of list comprehension, see message W0141.

    – lumbric
    Oct 31 '13 at 10:37








  • 1





    @lumbric, I'm not sure but it does only if lambda is used in map.

    – 0xc0de
    May 23 '17 at 4:53














  • 7





    Note that PyLint warns if you use map instead of list comprehension, see message W0141.

    – lumbric
    Oct 31 '13 at 10:37








  • 1





    @lumbric, I'm not sure but it does only if lambda is used in map.

    – 0xc0de
    May 23 '17 at 4:53








7




7





Note that PyLint warns if you use map instead of list comprehension, see message W0141.

– lumbric
Oct 31 '13 at 10:37







Note that PyLint warns if you use map instead of list comprehension, see message W0141.

– lumbric
Oct 31 '13 at 10:37






1




1





@lumbric, I'm not sure but it does only if lambda is used in map.

– 0xc0de
May 23 '17 at 4:53





@lumbric, I'm not sure but it does only if lambda is used in map.

– 0xc0de
May 23 '17 at 4:53












9 Answers
9






active

oldest

votes


















564














map may be microscopically faster in some cases (when you're NOT making a lambda for the purpose, but using the same function in map and a listcomp). List comprehensions may be faster in other cases and most (not all) pythonistas consider them more direct and clearer.



An example of the tiny speed advantage of map when using exactly the same function:



$ python -mtimeit -s'xs=range(10)' 'map(hex, xs)'
100000 loops, best of 3: 4.86 usec per loop
$ python -mtimeit -s'xs=range(10)' '[hex(x) for x in xs]'
100000 loops, best of 3: 5.58 usec per loop


An example of how performance comparison gets completely reversed when map needs a lambda:



$ python -mtimeit -s'xs=range(10)' 'map(lambda x: x+2, xs)'
100000 loops, best of 3: 4.24 usec per loop
$ python -mtimeit -s'xs=range(10)' '[x+2 for x in xs]'
100000 loops, best of 3: 2.32 usec per loop





share|improve this answer



















  • 29





    Yep, indeed our internal Python style guide at work explicitly recomments listcomps against map and filter (not even mentioning the tiny but measurable performance improvement map can give in some cases;-).

    – Alex Martelli
    Aug 8 '09 at 3:55






  • 38





    Not to kibash on Alex's infinite style points, but sometimes map seems easier to read to me: data = map(str, some_list_of_objects). Some other ones... operator.attrgetter, operator.itemgetter, etc.

    – Gregg Lind
    Aug 8 '09 at 16:06






  • 48





    map(operator.attrgetter('foo'), objs) easier to read than [o.foo for foo in objs] ?!

    – Alex Martelli
    Aug 8 '09 at 18:42






  • 46





    @Alex: I prefer not to introduce unnecessary names, like o here, and your examples show why.

    – Reid Barton
    Jan 22 '10 at 20:38






  • 25





    I think that @GreggLind has a point, with his str() example, though.

    – Eric O Lebigot
    Oct 5 '11 at 7:55



















403














Cases





  • Common case: Almost always, you will want to use a list comprehension in python because it will be more obvious what you're doing to novice programmers reading your code. (This does not apply to other languages, where other idioms may apply.) It will even be more obvious what you're doing to python programmers, since list comprehensions are the de-facto standard in python for iteration; they are expected.


  • Less-common case: However if you already have a function defined, it is often reasonable to use map, though it is considered 'unpythonic'. For example, map(sum, myLists) is more elegant/terse than [sum(x) for x in myLists]. You gain the elegance of not having to make up a dummy variable (e.g. sum(x) for x... or sum(_) for _... or sum(readableName) for readableName...) which you have to type twice, just to iterate. The same argument holds for filter and reduce and anything from the itertools module: if you already have a function handy, you could go ahead and do some functional programming. This gains readability in some situations, and loses it in others (e.g. novice programmers, multiple arguments)... but the readability of your code highly depends on your comments anyway.


  • Almost never: You may want to use the map function as a pure abstract function while doing functional programming, where you're mapping map, or currying map, or otherwise benefit from talking about map as a function. In Haskell for example, a functor interface called fmap generalizes mapping over any data structure. This is very uncommon in python because the python grammar compels you to use generator-style to talk about iteration; you can't generalize it easily. (This is sometimes good and sometimes bad.) You can probably come up with rare python examples where map(f, *lists) is a reasonable thing to do. The closest example I can come up with would be sumEach = partial(map,sum), which is a one-liner that is very roughly equivalent to:




def sumEach(myLists):
return [sum(_) for _ in myLists]




  • Just using a for-loop: You can also of course just use a for-loop. While not as elegant from a functional-programming viewpoint, sometimes non-local variables make code clearer in imperative programming languages such as python, because people are very used to reading code that way. For-loops are also, generally, the most efficient when you are merely doing any complex operation that is not building a list like list-comprehensions and map are optimized for (e.g. summing, or making a tree, etc.) -- at least efficient in terms of memory (not necessarily in terms of time, where I'd expect at worst a constant factor, barring some rare pathological garbage-collection hiccuping).


"Pythonism"



I dislike the word "pythonic" because I don't find that pythonic is always elegant in my eyes. Nevertheless, map and filter and similar functions (like the very useful itertools module) are probably considered unpythonic in terms of style.



Laziness



In terms of efficiency, like most functional programming constructs, MAP CAN BE LAZY, and in fact is lazy in python. That means you can do this (in python3) and your computer will not run out of memory and lose all your unsaved data:



>>> map(str, range(10**100))
<map object at 0x2201d50>


Try doing that with a list comprehension:



>>> [str(n) for n in range(10**100)]
# DO NOT TRY THIS AT HOME OR YOU WILL BE SAD #


Do note that list comprehensions are also inherently lazy, but python has chosen to implement them as non-lazy. Nevertheless, python does support lazy list comprehensions in the form of generator expressions, as follows:



>>> (str(n) for n in range(10**100))
<generator object <genexpr> at 0xacbdef>


You can basically think of the [...] syntax as passing in a generator expression to the list constructor, like list(x for x in range(5)).



Brief contrived example



from operator import neg
print({x:x**2 for x in map(neg,range(5))})

print({x:x**2 for x in [-y for y in range(5)]})

print({x:x**2 for x in (-y for y in range(5))})


List comprehensions are non-lazy, so may require more memory (unless you use generator comprehensions). The square brackets [...] often make things obvious, especially when in a mess of parentheses. On the other hand, sometimes you end up being verbose like typing [x for x in.... As long as you keep your iterator variables short, list comprehensions are usually clearer if you don't indent your code. But you could always indent your code.



print(
{x:x**2 for x in (-y for y in range(5))}
)


or break things up:



rangeNeg5 = (-y for y in range(5))
print(
{x:x**2 for x in rangeNeg5}
)


Efficiency comparison for python3



map is now lazy:



% python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=map(f,xs)'
1000000 loops, best of 3: 0.336 usec per loop ^^^^^^^^^


Therefore if you will not be using all your data, or do not know ahead of time how much data you need, map in python3 (and generator expressions in python2 or python3) will avoid calculating their values until the last moment necessary. Usually this will usually outweigh any overhead from using map. The downside is that this is very limited in python as opposed to most functional languages: you only get this benefit if you access your data left-to-right "in order", because python generator expressions can only be evaluated the order x[0], x[1], x[2], ....



However let's say that we have a pre-made function f we'd like to map, and we ignore the laziness of map by immediately forcing evaluation with list(...). We get some very interesting results:



% python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=list(map(f,xs))'                                                                                                                                                
10000 loops, best of 3: 165/124/135 usec per loop ^^^^^^^^^^^^^^^
for list(<map object>)

% python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=[f(x) for x in xs]'
10000 loops, best of 3: 181/118/123 usec per loop ^^^^^^^^^^^^^^^^^^
for list(<generator>), probably optimized

% python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=list(f(x) for x in xs)'
1000 loops, best of 3: 215/150/150 usec per loop ^^^^^^^^^^^^^^^^^^^^^^
for list(<generator>)


In results are in the form AAA/BBB/CCC where A was performed with on a circa-2010 Intel workstation with python 3.?.?, and B and C were performed with a circa-2013 AMD workstation with python 3.2.1, with extremely different hardware. The result seems to be that map and list comprehensions are comparable in performance, which is most strongly affected by other random factors. The only thing we can tell seems to be that, oddly, while we expect list comprehensions [...] to perform better than generator expressions (...), map is ALSO more efficient that generator expressions (again assuming that all values are evaluated/used).



It is important to realize that these tests assume a very simple function (the identity function); however this is fine because if the function were complicated, then performance overhead would be negligible compared to other factors in the program. (It may still be interesting to test with other simple things like f=lambda x:x+x)



If you're skilled at reading python assembly, you can use the dis module to see if that's actually what's going on behind the scenes:



>>> listComp = compile('[f(x) for x in xs]', 'listComp', 'eval')
>>> dis.dis(listComp)
1 0 LOAD_CONST 0 (<code object <listcomp> at 0x2511a48, file "listComp", line 1>)
3 MAKE_FUNCTION 0
6 LOAD_NAME 0 (xs)
9 GET_ITER
10 CALL_FUNCTION 1
13 RETURN_VALUE
>>> listComp.co_consts
(<code object <listcomp> at 0x2511a48, file "listComp", line 1>,)
>>> dis.dis(listComp.co_consts[0])
1 0 BUILD_LIST 0
3 LOAD_FAST 0 (.0)
>> 6 FOR_ITER 18 (to 27)
9 STORE_FAST 1 (x)
12 LOAD_GLOBAL 0 (f)
15 LOAD_FAST 1 (x)
18 CALL_FUNCTION 1
21 LIST_APPEND 2
24 JUMP_ABSOLUTE 6
>> 27 RETURN_VALUE


 



>>> listComp2 = compile('list(f(x) for x in xs)', 'listComp2', 'eval')
>>> dis.dis(listComp2)
1 0 LOAD_NAME 0 (list)
3 LOAD_CONST 0 (<code object <genexpr> at 0x255bc68, file "listComp2", line 1>)
6 MAKE_FUNCTION 0
9 LOAD_NAME 1 (xs)
12 GET_ITER
13 CALL_FUNCTION 1
16 CALL_FUNCTION 1
19 RETURN_VALUE
>>> listComp2.co_consts
(<code object <genexpr> at 0x255bc68, file "listComp2", line 1>,)
>>> dis.dis(listComp2.co_consts[0])
1 0 LOAD_FAST 0 (.0)
>> 3 FOR_ITER 17 (to 23)
6 STORE_FAST 1 (x)
9 LOAD_GLOBAL 0 (f)
12 LOAD_FAST 1 (x)
15 CALL_FUNCTION 1
18 YIELD_VALUE
19 POP_TOP
20 JUMP_ABSOLUTE 3
>> 23 LOAD_CONST 0 (None)
26 RETURN_VALUE


 



>>> evalledMap = compile('list(map(f,xs))', 'evalledMap', 'eval')
>>> dis.dis(evalledMap)
1 0 LOAD_NAME 0 (list)
3 LOAD_NAME 1 (map)
6 LOAD_NAME 2 (f)
9 LOAD_NAME 3 (xs)
12 CALL_FUNCTION 2
15 CALL_FUNCTION 1
18 RETURN_VALUE


It seems it is better to use [...] syntax than list(...). Sadly the map class is a bit opaque to disassembly, but we can make due with our speed test.






share|improve this answer





















  • 18





    Believe it or not, some "novice programmers" students learn a functional language first. Not necessarily a bad thing.

    – MarcH
    Aug 1 '12 at 21:57






  • 5





    "the very useful itertools module [is] probably considered unpythonic in terms of style". Hmm. I don't like the term "Pythonic" either, so in some sense I don't care what it means, but I don't think it's fair to those who do use it, to say that according to "Pythonicness" builtins map and filter along with standard library itertools are inherently bad style. Unless GvR actually says they were either a terrible mistake or solely for performance, the only natural conclusion if that's what "Pythonicness" says is to forget about it as stupid ;-)

    – Steve Jessop
    Feb 13 '14 at 17:53








  • 3





    @SteveJessop: Actually, Guido thought dropping map/filter was a great idea for Python 3, and only a rebellion by other Pythonistas kept them in the built-in namespace (while reduce was moved to functools). I personally disagree (map and filter are fine with predefined, particularly built-in, functions, just never use them if a lambda would be needed), but GvR has basically called them not Pythonic for years.

    – ShadowRanger
    Oct 16 '15 at 20:43













  • @ShadowRanger: true, but was GvR ever planning to remove itertools? The part I quote from this answer is the main claim that befuddles me. I don't know whether in his ideal world, map and filter would move to itertools (or functools) or go entirely, but whichever is the case, once one says that itertools is unPythonic in its entirety, then I don't really know what "Pythonic" is supposed to mean but I don't think it can be anything similar to "what GvR recommends people use".

    – Steve Jessop
    Oct 16 '15 at 21:39








  • 2





    @SteveJessop: I was only addressing map/filter, not itertools. Functional programming is perfectly Pythonic (itertools, functools and operator were all designed specifically with functional programming in mind, and I use functional idioms in Python all the time), and itertools provides features that would be a pain to implement yourself, It's specifically map and filter being redundant with generator expressions that made Guido hate them. itertools has always been fine.

    – ShadowRanger
    Oct 16 '15 at 23:05



















84














You should use map and filter instead of list comprehensions.



An objective reason why you should prefer them even though they're not "Pythonic" is this:

They require functions/lambdas as arguments, which introduce a new scope.



I've gotten bitten by this more than once:



for x, y in somePoints:
# (several lines of code here)
squared = [x ** 2 for x in numbers]
# Oops, x was silently overwritten!


but if instead I had said:



for x, y in somePoints:
# (several lines of code here)
squared = map(lambda x: x ** 2, numbers)


then everything would've been fine.



You could say I was being silly for using the same variable name in the same scope.



I wasn't. The code was fine originally -- the two xs weren't in the same scope.

It was only after I moved the inner block to a different section of the code that the problem came up (read: problem during maintenance, not development), and I didn't expect it.



Yes, if you never make this mistake then list comprehensions are more elegant.

But from personal experience (and from seeing others make the same mistake) I've seen it happen enough times that I think it's not worth the pain you have to go through when these bugs creep into your code.



Conclusion:



Use map and filter. They prevent subtle hard-to-diagnose scope-related bugs.



Side note:



Don't forget to consider using imap and ifilter (in itertools) if they are appropriate for your situation!






share|improve this answer





















  • 4





    Thanks for pointing this out. It hadn't explicitly occurred to me that list comprehension was in the same scope and could be an issue. With that said, I think some of the other answers make it clear that list comprehension should be the default approach most of the time but that this is something to remember. This is also a good general reminder to keep functions (and thus scope) small and have thorough unit tests and use assert statements.

    – TimothyAWiseman
    Nov 21 '12 at 19:00






  • 76





    This bug is fixed in Python 3

    – Mirzhan Irkegulov
    Jan 8 '13 at 13:51






  • 11





    @wim: This was only about Python 2, although it applies to Python 3 if you want to stay backwards-compatible. I knew about it and I'd been using Python for a while now (yes, more than just a few months), and yet it happened to me. I've seen others who are smarter than me fall into the same trap. If you're so bright and/or experienced that this isn't a problem for you then I'm happy for you, I don't think most people are like you. If they were, there wouldn't be such an urge to fix it in Python 3.

    – Mehrdad
    Dec 17 '13 at 23:48








  • 8





    I'm sorry but you wrote this in late 2012, well after python 3 is on the scene, and the answer reads like you're recommending an otherwise unpopular style of python coding just because you got bitten by a bug while cutting-and-pasting code. I never claimed to be bright or experienced, I just don't agree that the bold claim is justified by your reasons.

    – wim
    Dec 18 '13 at 0:14








  • 7





    @wim: Huh? Python 2 is still used in a lot of places, the fact that Python 3 exists doesn't change that. And when you say "it's not exactly a subtle bug for anyone that has used Python more than a few months" that sentence literally means "this only concerns inexperienced developers" (clearly not you). And for the record, you clearly didn't read the answer because I said in bold that I was moving, not copying, code. Copy-paste bugs are pretty uniform across languages. This kind of bug is more unique to Python because of its scoping; it's subtler & easier to forget about and miss.

    – Mehrdad
    Dec 18 '13 at 0:38





















38














Actually, map and list comprehensions behave quite differently in the Python 3 language. Take a look at the following Python 3 program:



def square(x):
return x*x
squares = map(square, [1, 2, 3])
print(list(squares))
print(list(squares))


You might expect it to print the line "[1, 4, 9]" twice, but instead it prints "[1, 4, 9]" followed by "". The first time you look at squares it seems to behave as a sequence of three elements, but the second time as an empty one.



In the Python 2 language map returns a plain old list, just like list comprehensions do in both languages. The crux is that the return value of map in Python 3 (and imap in Python 2) is not a list - it's an iterator!



The elements are consumed when you iterate over an iterator unlike when you iterate over a list. This is why squares looks empty in the last print(list(squares)) line.



To summarize:




  • When dealing with iterators you have to remember that they are stateful and that they mutate as you traverse them.

  • Lists are more predictable since they only change when you explicitly mutate them; they are containers.

  • And a bonus: numbers, strings, and tuples are even more predictable since they cannot change at all; they are values.






share|improve this answer


























  • this is probably the best argument for list comprehensions. pythons map is not the functional map but the crippled red-headed stepchild of a functional implementation. Very sad, because I really dislike comprehensions.

    – semiomant
    Jun 29 '17 at 14:26











  • @semiomant I would say lazy map (like in python3) is more 'functional' than eager map (like in python2). For example, map in Haskell is lazy (well, everything in Haskell is lazy...). Anyway, lazy map is better for chaining maps - if you have a map applied to map applied to map, you have a list for each of intermediate map calls in python2, whereas in python3 you have just one resulting list, so its more memory efficient.

    – MnZrK
    Oct 21 '17 at 15:57











  • I guess what I want is for map to produce a data structure, not an iterator. But maybe lazy iterators are easier than lazy data structures. Food for thought. Thanks @MnZrK

    – semiomant
    Oct 24 '17 at 11:01











  • You want to say map returns an iterable, not an iterator.

    – Mehrdad
    Sep 18 '18 at 21:33



















15














I find list comprehensions are generally more expressive of what I'm trying to do than map - they both get it done, but the former saves the mental load of trying to understand what could be a complex lambda expression.



There's also an interview out there somewhere (I can't find it offhand) where Guido lists lambdas and the functional functions as the thing he most regrets about accepting into Python, so you could make the argument that they're un-Pythonic by virtue of that.






share|improve this answer



















  • 9





    Yeah, sigh, but Guido's original intention to remove lambda altogether in Python 3 got a barrage of lobbying against it, so he went back on it despite my stout support -- ah well, guess lambda's just too handy in many SIMPLE cases, the only problem is when it exceeds the bounds of SIMPLE or gets assigned to a name (in which latter case it's a silly hobbled duplicate of def!-).

    – Alex Martelli
    Aug 8 '09 at 3:58






  • 1





    The interview you are thinking about is this one: amk.ca/python/writing/gvr-interview, where Guido says "Sometimes I've been too quick in accepting contributions, and later realized that it was a mistake. One example would be some of the functional programming features, such as lambda functions. lambda is a keyword that lets you create a small anonymous function; built-in functions such as map, filter, and reduce run a function over a sequence type, such as a list."

    – J. Taylor
    Mar 24 '11 at 4:30






  • 3





    @Alex, I don't have your years of experience, but I've seen far more over-complicated list comprehensions than lambdas. Of course, abusing language features is always a difficult temptation to resist. It's interesting that list comprehensions (empirically) seem more prone to abuse than lambdas, though I'm not sure why that should be the case. I'll also point out that "hobbled" isn't always a bad thing. Reducing the scope of "things this line might be doing" can sometimes make it easier on the reader. For example, the const keyword in C++ is a great triumph along these lines.

    – Stuart Berg
    Mar 25 '13 at 15:02











  • > guido . Which is another piece of evidence that Guido is out of his mind. Of course lambda's have been made so lame (no statements..) that they're difficult to use and limited anyways.

    – javadba
    Jan 6 at 23:41





















15














If you plan on writing any asynchronous, parallel, or distributed code, you will probably prefer map over a list comprehension -- as most asynchronous, parallel, or distributed packages provide a map function to overload python's map. Then by passing the appropriate map function to the rest of your code, you may not have to modify your original serial code to have it run in parallel (etc).






share|improve this answer
























  • Could you provide an example?

    – Roman
    Mar 22 '16 at 6:34






  • 1





    How about this github.com/uqfoundation/pathos/blob/master/tests/test_map.py and this github.com/uqfoundation/pathos/blob/master/tests/test_star.py?

    – Mike McKerns
    Mar 22 '16 at 12:51






  • 1





    Python's multiprocessing module does this: docs.python.org/2/library/multiprocessing.html

    – Robert L.
    Jul 31 '17 at 18:07





















14














Here is one possible case:



map(lambda op1,op2: op1*op2, list1, list2)


versus:



[op1*op2 for op1,op2 in zip(list1,list2)]


I am guessing the zip() is an unfortunate and unnecessary overhead you need to indulge in if you insist on using list comprehensions instead of the map. Would be great if someone clarifies this whether affirmatively or negatively.






share|improve this answer


























  • "[op1*op2 from op1,op2 in zip(list1,list2)]" | s/form/for/ And an equivalent list with out zip: (less readable)[list1[i]*list2[i] for i in range(len(list1))]

    – weakish
    Aug 9 '10 at 2:45








  • 2





    Should be "for" not "from" in your second code quote, @andz, and in @weakish's comment too. I thought I had discovered a new syntactical approach to list comprehensions... Darn.

    – physicsmichael
    Oct 12 '10 at 13:12






  • 4





    to add a very late comment, you can make zip lazy by using itertools.izip

    – tacaswell
    Dec 17 '12 at 21:09











  • @tcaswell No longer needed in Python 3000.

    – JeromeJ
    Aug 21 '13 at 21:38






  • 3





    I think I still prefer map(operator.mul, list1, list2). It's on these very simple left side expressions that comprehensions get clumsy.

    – Yann Vernier
    Mar 10 '15 at 23:34



















6














So since Python 3, map() is an iterator, you need to keep in mind what do you need: an iterator or list object.



As @AlexMartelli already mentioned, map() is faster than list comprehension only if you don't use lambda function.



I will present you some time comparisons.




Python 3.5.2 and CPython
I've used Jupiter notebook and especially %timeit built-in magic command

Measurements: s == 1000 ms == 1000 * 1000 µs = 1000 * 1000 * 1000 ns



Setup:



x_list = [(i, i+1, i+2, i*2, i-9) for i in range(1000)]
i_list = list(range(1000))


Built-in function:



%timeit map(sum, x_list)  # creating iterator object
# Output: The slowest run took 9.91 times longer than the fastest.
# This could mean that an intermediate result is being cached.
# 1000000 loops, best of 3: 277 ns per loop

%timeit list(map(sum, x_list)) # creating list with map
# Output: 1000 loops, best of 3: 214 µs per loop

%timeit [sum(x) for x in x_list] # creating list with list comprehension
# Output: 1000 loops, best of 3: 290 µs per loop


lambda function:



%timeit map(lambda i: i+1, i_list)
# Output: The slowest run took 8.64 times longer than the fastest.
# This could mean that an intermediate result is being cached.
# 1000000 loops, best of 3: 325 ns per loop

%timeit list(map(lambda i: i+1, i_list))
# Output: 1000 loops, best of 3: 183 µs per loop

%timeit [i+1 for i in i_list]
# Output: 10000 loops, best of 3: 84.2 µs per loop


There is also such thing as generator expression, see PEP-0289. So i thought it would be useful to add it to comparison



%timeit (sum(i) for i in x_list)
# Output: The slowest run took 6.66 times longer than the fastest.
# This could mean that an intermediate result is being cached.
# 1000000 loops, best of 3: 495 ns per loop

%timeit list((sum(x) for x in x_list))
# Output: 1000 loops, best of 3: 319 µs per loop

%timeit (i+1 for i in i_list)
# Output: The slowest run took 6.83 times longer than the fastest.
# This could mean that an intermediate result is being cached.
# 1000000 loops, best of 3: 506 ns per loop

%timeit list((i+1 for i in i_list))
# Output: 10000 loops, best of 3: 125 µs per loop


You need list object:



Use list comprehension if it's custom function, use list(map()) if there is builtin function



You don't need list object, you just need iterable one:



Always use map()!






share|improve this answer

































    0














    I consider that the most Pythonic way is to use a list comprehension instead of map and filter. The reason is that list comprehensions are clearer than map and filter.



    In [1]: odd_cubes = [x ** 3 for x in range(10) if x % 2 == 1] # using a list comprehension

    In [2]: odd_cubes_alt = list(map(lambda x: x ** 3, filter(lambda x: x % 2 == 1, range(10)))) # using map and filter

    In [3]: odd_cubes == odd_cubes_alt
    Out[3]: True


    As you an see, a comprehension does not require extra lambda expressions as map needs. Furthermore, a comprehension also allows filtering easily, while map requires filter to allow filtering.






    share|improve this answer

























      Your Answer






      StackExchange.ifUsing("editor", function () {
      StackExchange.using("externalEditor", function () {
      StackExchange.using("snippets", function () {
      StackExchange.snippets.init();
      });
      });
      }, "code-snippets");

      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "1"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: true,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: 10,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });














      draft saved

      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f1247486%2flist-comprehension-vs-map%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      9 Answers
      9






      active

      oldest

      votes








      9 Answers
      9






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      564














      map may be microscopically faster in some cases (when you're NOT making a lambda for the purpose, but using the same function in map and a listcomp). List comprehensions may be faster in other cases and most (not all) pythonistas consider them more direct and clearer.



      An example of the tiny speed advantage of map when using exactly the same function:



      $ python -mtimeit -s'xs=range(10)' 'map(hex, xs)'
      100000 loops, best of 3: 4.86 usec per loop
      $ python -mtimeit -s'xs=range(10)' '[hex(x) for x in xs]'
      100000 loops, best of 3: 5.58 usec per loop


      An example of how performance comparison gets completely reversed when map needs a lambda:



      $ python -mtimeit -s'xs=range(10)' 'map(lambda x: x+2, xs)'
      100000 loops, best of 3: 4.24 usec per loop
      $ python -mtimeit -s'xs=range(10)' '[x+2 for x in xs]'
      100000 loops, best of 3: 2.32 usec per loop





      share|improve this answer



















      • 29





        Yep, indeed our internal Python style guide at work explicitly recomments listcomps against map and filter (not even mentioning the tiny but measurable performance improvement map can give in some cases;-).

        – Alex Martelli
        Aug 8 '09 at 3:55






      • 38





        Not to kibash on Alex's infinite style points, but sometimes map seems easier to read to me: data = map(str, some_list_of_objects). Some other ones... operator.attrgetter, operator.itemgetter, etc.

        – Gregg Lind
        Aug 8 '09 at 16:06






      • 48





        map(operator.attrgetter('foo'), objs) easier to read than [o.foo for foo in objs] ?!

        – Alex Martelli
        Aug 8 '09 at 18:42






      • 46





        @Alex: I prefer not to introduce unnecessary names, like o here, and your examples show why.

        – Reid Barton
        Jan 22 '10 at 20:38






      • 25





        I think that @GreggLind has a point, with his str() example, though.

        – Eric O Lebigot
        Oct 5 '11 at 7:55
















      564














      map may be microscopically faster in some cases (when you're NOT making a lambda for the purpose, but using the same function in map and a listcomp). List comprehensions may be faster in other cases and most (not all) pythonistas consider them more direct and clearer.



      An example of the tiny speed advantage of map when using exactly the same function:



      $ python -mtimeit -s'xs=range(10)' 'map(hex, xs)'
      100000 loops, best of 3: 4.86 usec per loop
      $ python -mtimeit -s'xs=range(10)' '[hex(x) for x in xs]'
      100000 loops, best of 3: 5.58 usec per loop


      An example of how performance comparison gets completely reversed when map needs a lambda:



      $ python -mtimeit -s'xs=range(10)' 'map(lambda x: x+2, xs)'
      100000 loops, best of 3: 4.24 usec per loop
      $ python -mtimeit -s'xs=range(10)' '[x+2 for x in xs]'
      100000 loops, best of 3: 2.32 usec per loop





      share|improve this answer



















      • 29





        Yep, indeed our internal Python style guide at work explicitly recomments listcomps against map and filter (not even mentioning the tiny but measurable performance improvement map can give in some cases;-).

        – Alex Martelli
        Aug 8 '09 at 3:55






      • 38





        Not to kibash on Alex's infinite style points, but sometimes map seems easier to read to me: data = map(str, some_list_of_objects). Some other ones... operator.attrgetter, operator.itemgetter, etc.

        – Gregg Lind
        Aug 8 '09 at 16:06






      • 48





        map(operator.attrgetter('foo'), objs) easier to read than [o.foo for foo in objs] ?!

        – Alex Martelli
        Aug 8 '09 at 18:42






      • 46





        @Alex: I prefer not to introduce unnecessary names, like o here, and your examples show why.

        – Reid Barton
        Jan 22 '10 at 20:38






      • 25





        I think that @GreggLind has a point, with his str() example, though.

        – Eric O Lebigot
        Oct 5 '11 at 7:55














      564












      564








      564







      map may be microscopically faster in some cases (when you're NOT making a lambda for the purpose, but using the same function in map and a listcomp). List comprehensions may be faster in other cases and most (not all) pythonistas consider them more direct and clearer.



      An example of the tiny speed advantage of map when using exactly the same function:



      $ python -mtimeit -s'xs=range(10)' 'map(hex, xs)'
      100000 loops, best of 3: 4.86 usec per loop
      $ python -mtimeit -s'xs=range(10)' '[hex(x) for x in xs]'
      100000 loops, best of 3: 5.58 usec per loop


      An example of how performance comparison gets completely reversed when map needs a lambda:



      $ python -mtimeit -s'xs=range(10)' 'map(lambda x: x+2, xs)'
      100000 loops, best of 3: 4.24 usec per loop
      $ python -mtimeit -s'xs=range(10)' '[x+2 for x in xs]'
      100000 loops, best of 3: 2.32 usec per loop





      share|improve this answer













      map may be microscopically faster in some cases (when you're NOT making a lambda for the purpose, but using the same function in map and a listcomp). List comprehensions may be faster in other cases and most (not all) pythonistas consider them more direct and clearer.



      An example of the tiny speed advantage of map when using exactly the same function:



      $ python -mtimeit -s'xs=range(10)' 'map(hex, xs)'
      100000 loops, best of 3: 4.86 usec per loop
      $ python -mtimeit -s'xs=range(10)' '[hex(x) for x in xs]'
      100000 loops, best of 3: 5.58 usec per loop


      An example of how performance comparison gets completely reversed when map needs a lambda:



      $ python -mtimeit -s'xs=range(10)' 'map(lambda x: x+2, xs)'
      100000 loops, best of 3: 4.24 usec per loop
      $ python -mtimeit -s'xs=range(10)' '[x+2 for x in xs]'
      100000 loops, best of 3: 2.32 usec per loop






      share|improve this answer












      share|improve this answer



      share|improve this answer










      answered Aug 7 '09 at 23:45









      Alex MartelliAlex Martelli

      631k12810421284




      631k12810421284








      • 29





        Yep, indeed our internal Python style guide at work explicitly recomments listcomps against map and filter (not even mentioning the tiny but measurable performance improvement map can give in some cases;-).

        – Alex Martelli
        Aug 8 '09 at 3:55






      • 38





        Not to kibash on Alex's infinite style points, but sometimes map seems easier to read to me: data = map(str, some_list_of_objects). Some other ones... operator.attrgetter, operator.itemgetter, etc.

        – Gregg Lind
        Aug 8 '09 at 16:06






      • 48





        map(operator.attrgetter('foo'), objs) easier to read than [o.foo for foo in objs] ?!

        – Alex Martelli
        Aug 8 '09 at 18:42






      • 46





        @Alex: I prefer not to introduce unnecessary names, like o here, and your examples show why.

        – Reid Barton
        Jan 22 '10 at 20:38






      • 25





        I think that @GreggLind has a point, with his str() example, though.

        – Eric O Lebigot
        Oct 5 '11 at 7:55














      • 29





        Yep, indeed our internal Python style guide at work explicitly recomments listcomps against map and filter (not even mentioning the tiny but measurable performance improvement map can give in some cases;-).

        – Alex Martelli
        Aug 8 '09 at 3:55






      • 38





        Not to kibash on Alex's infinite style points, but sometimes map seems easier to read to me: data = map(str, some_list_of_objects). Some other ones... operator.attrgetter, operator.itemgetter, etc.

        – Gregg Lind
        Aug 8 '09 at 16:06






      • 48





        map(operator.attrgetter('foo'), objs) easier to read than [o.foo for foo in objs] ?!

        – Alex Martelli
        Aug 8 '09 at 18:42






      • 46





        @Alex: I prefer not to introduce unnecessary names, like o here, and your examples show why.

        – Reid Barton
        Jan 22 '10 at 20:38






      • 25





        I think that @GreggLind has a point, with his str() example, though.

        – Eric O Lebigot
        Oct 5 '11 at 7:55








      29




      29





      Yep, indeed our internal Python style guide at work explicitly recomments listcomps against map and filter (not even mentioning the tiny but measurable performance improvement map can give in some cases;-).

      – Alex Martelli
      Aug 8 '09 at 3:55





      Yep, indeed our internal Python style guide at work explicitly recomments listcomps against map and filter (not even mentioning the tiny but measurable performance improvement map can give in some cases;-).

      – Alex Martelli
      Aug 8 '09 at 3:55




      38




      38





      Not to kibash on Alex's infinite style points, but sometimes map seems easier to read to me: data = map(str, some_list_of_objects). Some other ones... operator.attrgetter, operator.itemgetter, etc.

      – Gregg Lind
      Aug 8 '09 at 16:06





      Not to kibash on Alex's infinite style points, but sometimes map seems easier to read to me: data = map(str, some_list_of_objects). Some other ones... operator.attrgetter, operator.itemgetter, etc.

      – Gregg Lind
      Aug 8 '09 at 16:06




      48




      48





      map(operator.attrgetter('foo'), objs) easier to read than [o.foo for foo in objs] ?!

      – Alex Martelli
      Aug 8 '09 at 18:42





      map(operator.attrgetter('foo'), objs) easier to read than [o.foo for foo in objs] ?!

      – Alex Martelli
      Aug 8 '09 at 18:42




      46




      46





      @Alex: I prefer not to introduce unnecessary names, like o here, and your examples show why.

      – Reid Barton
      Jan 22 '10 at 20:38





      @Alex: I prefer not to introduce unnecessary names, like o here, and your examples show why.

      – Reid Barton
      Jan 22 '10 at 20:38




      25




      25





      I think that @GreggLind has a point, with his str() example, though.

      – Eric O Lebigot
      Oct 5 '11 at 7:55





      I think that @GreggLind has a point, with his str() example, though.

      – Eric O Lebigot
      Oct 5 '11 at 7:55













      403














      Cases





      • Common case: Almost always, you will want to use a list comprehension in python because it will be more obvious what you're doing to novice programmers reading your code. (This does not apply to other languages, where other idioms may apply.) It will even be more obvious what you're doing to python programmers, since list comprehensions are the de-facto standard in python for iteration; they are expected.


      • Less-common case: However if you already have a function defined, it is often reasonable to use map, though it is considered 'unpythonic'. For example, map(sum, myLists) is more elegant/terse than [sum(x) for x in myLists]. You gain the elegance of not having to make up a dummy variable (e.g. sum(x) for x... or sum(_) for _... or sum(readableName) for readableName...) which you have to type twice, just to iterate. The same argument holds for filter and reduce and anything from the itertools module: if you already have a function handy, you could go ahead and do some functional programming. This gains readability in some situations, and loses it in others (e.g. novice programmers, multiple arguments)... but the readability of your code highly depends on your comments anyway.


      • Almost never: You may want to use the map function as a pure abstract function while doing functional programming, where you're mapping map, or currying map, or otherwise benefit from talking about map as a function. In Haskell for example, a functor interface called fmap generalizes mapping over any data structure. This is very uncommon in python because the python grammar compels you to use generator-style to talk about iteration; you can't generalize it easily. (This is sometimes good and sometimes bad.) You can probably come up with rare python examples where map(f, *lists) is a reasonable thing to do. The closest example I can come up with would be sumEach = partial(map,sum), which is a one-liner that is very roughly equivalent to:




      def sumEach(myLists):
      return [sum(_) for _ in myLists]




      • Just using a for-loop: You can also of course just use a for-loop. While not as elegant from a functional-programming viewpoint, sometimes non-local variables make code clearer in imperative programming languages such as python, because people are very used to reading code that way. For-loops are also, generally, the most efficient when you are merely doing any complex operation that is not building a list like list-comprehensions and map are optimized for (e.g. summing, or making a tree, etc.) -- at least efficient in terms of memory (not necessarily in terms of time, where I'd expect at worst a constant factor, barring some rare pathological garbage-collection hiccuping).


      "Pythonism"



      I dislike the word "pythonic" because I don't find that pythonic is always elegant in my eyes. Nevertheless, map and filter and similar functions (like the very useful itertools module) are probably considered unpythonic in terms of style.



      Laziness



      In terms of efficiency, like most functional programming constructs, MAP CAN BE LAZY, and in fact is lazy in python. That means you can do this (in python3) and your computer will not run out of memory and lose all your unsaved data:



      >>> map(str, range(10**100))
      <map object at 0x2201d50>


      Try doing that with a list comprehension:



      >>> [str(n) for n in range(10**100)]
      # DO NOT TRY THIS AT HOME OR YOU WILL BE SAD #


      Do note that list comprehensions are also inherently lazy, but python has chosen to implement them as non-lazy. Nevertheless, python does support lazy list comprehensions in the form of generator expressions, as follows:



      >>> (str(n) for n in range(10**100))
      <generator object <genexpr> at 0xacbdef>


      You can basically think of the [...] syntax as passing in a generator expression to the list constructor, like list(x for x in range(5)).



      Brief contrived example



      from operator import neg
      print({x:x**2 for x in map(neg,range(5))})

      print({x:x**2 for x in [-y for y in range(5)]})

      print({x:x**2 for x in (-y for y in range(5))})


      List comprehensions are non-lazy, so may require more memory (unless you use generator comprehensions). The square brackets [...] often make things obvious, especially when in a mess of parentheses. On the other hand, sometimes you end up being verbose like typing [x for x in.... As long as you keep your iterator variables short, list comprehensions are usually clearer if you don't indent your code. But you could always indent your code.



      print(
      {x:x**2 for x in (-y for y in range(5))}
      )


      or break things up:



      rangeNeg5 = (-y for y in range(5))
      print(
      {x:x**2 for x in rangeNeg5}
      )


      Efficiency comparison for python3



      map is now lazy:



      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=map(f,xs)'
      1000000 loops, best of 3: 0.336 usec per loop ^^^^^^^^^


      Therefore if you will not be using all your data, or do not know ahead of time how much data you need, map in python3 (and generator expressions in python2 or python3) will avoid calculating their values until the last moment necessary. Usually this will usually outweigh any overhead from using map. The downside is that this is very limited in python as opposed to most functional languages: you only get this benefit if you access your data left-to-right "in order", because python generator expressions can only be evaluated the order x[0], x[1], x[2], ....



      However let's say that we have a pre-made function f we'd like to map, and we ignore the laziness of map by immediately forcing evaluation with list(...). We get some very interesting results:



      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=list(map(f,xs))'                                                                                                                                                
      10000 loops, best of 3: 165/124/135 usec per loop ^^^^^^^^^^^^^^^
      for list(<map object>)

      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=[f(x) for x in xs]'
      10000 loops, best of 3: 181/118/123 usec per loop ^^^^^^^^^^^^^^^^^^
      for list(<generator>), probably optimized

      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=list(f(x) for x in xs)'
      1000 loops, best of 3: 215/150/150 usec per loop ^^^^^^^^^^^^^^^^^^^^^^
      for list(<generator>)


      In results are in the form AAA/BBB/CCC where A was performed with on a circa-2010 Intel workstation with python 3.?.?, and B and C were performed with a circa-2013 AMD workstation with python 3.2.1, with extremely different hardware. The result seems to be that map and list comprehensions are comparable in performance, which is most strongly affected by other random factors. The only thing we can tell seems to be that, oddly, while we expect list comprehensions [...] to perform better than generator expressions (...), map is ALSO more efficient that generator expressions (again assuming that all values are evaluated/used).



      It is important to realize that these tests assume a very simple function (the identity function); however this is fine because if the function were complicated, then performance overhead would be negligible compared to other factors in the program. (It may still be interesting to test with other simple things like f=lambda x:x+x)



      If you're skilled at reading python assembly, you can use the dis module to see if that's actually what's going on behind the scenes:



      >>> listComp = compile('[f(x) for x in xs]', 'listComp', 'eval')
      >>> dis.dis(listComp)
      1 0 LOAD_CONST 0 (<code object <listcomp> at 0x2511a48, file "listComp", line 1>)
      3 MAKE_FUNCTION 0
      6 LOAD_NAME 0 (xs)
      9 GET_ITER
      10 CALL_FUNCTION 1
      13 RETURN_VALUE
      >>> listComp.co_consts
      (<code object <listcomp> at 0x2511a48, file "listComp", line 1>,)
      >>> dis.dis(listComp.co_consts[0])
      1 0 BUILD_LIST 0
      3 LOAD_FAST 0 (.0)
      >> 6 FOR_ITER 18 (to 27)
      9 STORE_FAST 1 (x)
      12 LOAD_GLOBAL 0 (f)
      15 LOAD_FAST 1 (x)
      18 CALL_FUNCTION 1
      21 LIST_APPEND 2
      24 JUMP_ABSOLUTE 6
      >> 27 RETURN_VALUE


       



      >>> listComp2 = compile('list(f(x) for x in xs)', 'listComp2', 'eval')
      >>> dis.dis(listComp2)
      1 0 LOAD_NAME 0 (list)
      3 LOAD_CONST 0 (<code object <genexpr> at 0x255bc68, file "listComp2", line 1>)
      6 MAKE_FUNCTION 0
      9 LOAD_NAME 1 (xs)
      12 GET_ITER
      13 CALL_FUNCTION 1
      16 CALL_FUNCTION 1
      19 RETURN_VALUE
      >>> listComp2.co_consts
      (<code object <genexpr> at 0x255bc68, file "listComp2", line 1>,)
      >>> dis.dis(listComp2.co_consts[0])
      1 0 LOAD_FAST 0 (.0)
      >> 3 FOR_ITER 17 (to 23)
      6 STORE_FAST 1 (x)
      9 LOAD_GLOBAL 0 (f)
      12 LOAD_FAST 1 (x)
      15 CALL_FUNCTION 1
      18 YIELD_VALUE
      19 POP_TOP
      20 JUMP_ABSOLUTE 3
      >> 23 LOAD_CONST 0 (None)
      26 RETURN_VALUE


       



      >>> evalledMap = compile('list(map(f,xs))', 'evalledMap', 'eval')
      >>> dis.dis(evalledMap)
      1 0 LOAD_NAME 0 (list)
      3 LOAD_NAME 1 (map)
      6 LOAD_NAME 2 (f)
      9 LOAD_NAME 3 (xs)
      12 CALL_FUNCTION 2
      15 CALL_FUNCTION 1
      18 RETURN_VALUE


      It seems it is better to use [...] syntax than list(...). Sadly the map class is a bit opaque to disassembly, but we can make due with our speed test.






      share|improve this answer





















      • 18





        Believe it or not, some "novice programmers" students learn a functional language first. Not necessarily a bad thing.

        – MarcH
        Aug 1 '12 at 21:57






      • 5





        "the very useful itertools module [is] probably considered unpythonic in terms of style". Hmm. I don't like the term "Pythonic" either, so in some sense I don't care what it means, but I don't think it's fair to those who do use it, to say that according to "Pythonicness" builtins map and filter along with standard library itertools are inherently bad style. Unless GvR actually says they were either a terrible mistake or solely for performance, the only natural conclusion if that's what "Pythonicness" says is to forget about it as stupid ;-)

        – Steve Jessop
        Feb 13 '14 at 17:53








      • 3





        @SteveJessop: Actually, Guido thought dropping map/filter was a great idea for Python 3, and only a rebellion by other Pythonistas kept them in the built-in namespace (while reduce was moved to functools). I personally disagree (map and filter are fine with predefined, particularly built-in, functions, just never use them if a lambda would be needed), but GvR has basically called them not Pythonic for years.

        – ShadowRanger
        Oct 16 '15 at 20:43













      • @ShadowRanger: true, but was GvR ever planning to remove itertools? The part I quote from this answer is the main claim that befuddles me. I don't know whether in his ideal world, map and filter would move to itertools (or functools) or go entirely, but whichever is the case, once one says that itertools is unPythonic in its entirety, then I don't really know what "Pythonic" is supposed to mean but I don't think it can be anything similar to "what GvR recommends people use".

        – Steve Jessop
        Oct 16 '15 at 21:39








      • 2





        @SteveJessop: I was only addressing map/filter, not itertools. Functional programming is perfectly Pythonic (itertools, functools and operator were all designed specifically with functional programming in mind, and I use functional idioms in Python all the time), and itertools provides features that would be a pain to implement yourself, It's specifically map and filter being redundant with generator expressions that made Guido hate them. itertools has always been fine.

        – ShadowRanger
        Oct 16 '15 at 23:05
















      403














      Cases





      • Common case: Almost always, you will want to use a list comprehension in python because it will be more obvious what you're doing to novice programmers reading your code. (This does not apply to other languages, where other idioms may apply.) It will even be more obvious what you're doing to python programmers, since list comprehensions are the de-facto standard in python for iteration; they are expected.


      • Less-common case: However if you already have a function defined, it is often reasonable to use map, though it is considered 'unpythonic'. For example, map(sum, myLists) is more elegant/terse than [sum(x) for x in myLists]. You gain the elegance of not having to make up a dummy variable (e.g. sum(x) for x... or sum(_) for _... or sum(readableName) for readableName...) which you have to type twice, just to iterate. The same argument holds for filter and reduce and anything from the itertools module: if you already have a function handy, you could go ahead and do some functional programming. This gains readability in some situations, and loses it in others (e.g. novice programmers, multiple arguments)... but the readability of your code highly depends on your comments anyway.


      • Almost never: You may want to use the map function as a pure abstract function while doing functional programming, where you're mapping map, or currying map, or otherwise benefit from talking about map as a function. In Haskell for example, a functor interface called fmap generalizes mapping over any data structure. This is very uncommon in python because the python grammar compels you to use generator-style to talk about iteration; you can't generalize it easily. (This is sometimes good and sometimes bad.) You can probably come up with rare python examples where map(f, *lists) is a reasonable thing to do. The closest example I can come up with would be sumEach = partial(map,sum), which is a one-liner that is very roughly equivalent to:




      def sumEach(myLists):
      return [sum(_) for _ in myLists]




      • Just using a for-loop: You can also of course just use a for-loop. While not as elegant from a functional-programming viewpoint, sometimes non-local variables make code clearer in imperative programming languages such as python, because people are very used to reading code that way. For-loops are also, generally, the most efficient when you are merely doing any complex operation that is not building a list like list-comprehensions and map are optimized for (e.g. summing, or making a tree, etc.) -- at least efficient in terms of memory (not necessarily in terms of time, where I'd expect at worst a constant factor, barring some rare pathological garbage-collection hiccuping).


      "Pythonism"



      I dislike the word "pythonic" because I don't find that pythonic is always elegant in my eyes. Nevertheless, map and filter and similar functions (like the very useful itertools module) are probably considered unpythonic in terms of style.



      Laziness



      In terms of efficiency, like most functional programming constructs, MAP CAN BE LAZY, and in fact is lazy in python. That means you can do this (in python3) and your computer will not run out of memory and lose all your unsaved data:



      >>> map(str, range(10**100))
      <map object at 0x2201d50>


      Try doing that with a list comprehension:



      >>> [str(n) for n in range(10**100)]
      # DO NOT TRY THIS AT HOME OR YOU WILL BE SAD #


      Do note that list comprehensions are also inherently lazy, but python has chosen to implement them as non-lazy. Nevertheless, python does support lazy list comprehensions in the form of generator expressions, as follows:



      >>> (str(n) for n in range(10**100))
      <generator object <genexpr> at 0xacbdef>


      You can basically think of the [...] syntax as passing in a generator expression to the list constructor, like list(x for x in range(5)).



      Brief contrived example



      from operator import neg
      print({x:x**2 for x in map(neg,range(5))})

      print({x:x**2 for x in [-y for y in range(5)]})

      print({x:x**2 for x in (-y for y in range(5))})


      List comprehensions are non-lazy, so may require more memory (unless you use generator comprehensions). The square brackets [...] often make things obvious, especially when in a mess of parentheses. On the other hand, sometimes you end up being verbose like typing [x for x in.... As long as you keep your iterator variables short, list comprehensions are usually clearer if you don't indent your code. But you could always indent your code.



      print(
      {x:x**2 for x in (-y for y in range(5))}
      )


      or break things up:



      rangeNeg5 = (-y for y in range(5))
      print(
      {x:x**2 for x in rangeNeg5}
      )


      Efficiency comparison for python3



      map is now lazy:



      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=map(f,xs)'
      1000000 loops, best of 3: 0.336 usec per loop ^^^^^^^^^


      Therefore if you will not be using all your data, or do not know ahead of time how much data you need, map in python3 (and generator expressions in python2 or python3) will avoid calculating their values until the last moment necessary. Usually this will usually outweigh any overhead from using map. The downside is that this is very limited in python as opposed to most functional languages: you only get this benefit if you access your data left-to-right "in order", because python generator expressions can only be evaluated the order x[0], x[1], x[2], ....



      However let's say that we have a pre-made function f we'd like to map, and we ignore the laziness of map by immediately forcing evaluation with list(...). We get some very interesting results:



      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=list(map(f,xs))'                                                                                                                                                
      10000 loops, best of 3: 165/124/135 usec per loop ^^^^^^^^^^^^^^^
      for list(<map object>)

      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=[f(x) for x in xs]'
      10000 loops, best of 3: 181/118/123 usec per loop ^^^^^^^^^^^^^^^^^^
      for list(<generator>), probably optimized

      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=list(f(x) for x in xs)'
      1000 loops, best of 3: 215/150/150 usec per loop ^^^^^^^^^^^^^^^^^^^^^^
      for list(<generator>)


      In results are in the form AAA/BBB/CCC where A was performed with on a circa-2010 Intel workstation with python 3.?.?, and B and C were performed with a circa-2013 AMD workstation with python 3.2.1, with extremely different hardware. The result seems to be that map and list comprehensions are comparable in performance, which is most strongly affected by other random factors. The only thing we can tell seems to be that, oddly, while we expect list comprehensions [...] to perform better than generator expressions (...), map is ALSO more efficient that generator expressions (again assuming that all values are evaluated/used).



      It is important to realize that these tests assume a very simple function (the identity function); however this is fine because if the function were complicated, then performance overhead would be negligible compared to other factors in the program. (It may still be interesting to test with other simple things like f=lambda x:x+x)



      If you're skilled at reading python assembly, you can use the dis module to see if that's actually what's going on behind the scenes:



      >>> listComp = compile('[f(x) for x in xs]', 'listComp', 'eval')
      >>> dis.dis(listComp)
      1 0 LOAD_CONST 0 (<code object <listcomp> at 0x2511a48, file "listComp", line 1>)
      3 MAKE_FUNCTION 0
      6 LOAD_NAME 0 (xs)
      9 GET_ITER
      10 CALL_FUNCTION 1
      13 RETURN_VALUE
      >>> listComp.co_consts
      (<code object <listcomp> at 0x2511a48, file "listComp", line 1>,)
      >>> dis.dis(listComp.co_consts[0])
      1 0 BUILD_LIST 0
      3 LOAD_FAST 0 (.0)
      >> 6 FOR_ITER 18 (to 27)
      9 STORE_FAST 1 (x)
      12 LOAD_GLOBAL 0 (f)
      15 LOAD_FAST 1 (x)
      18 CALL_FUNCTION 1
      21 LIST_APPEND 2
      24 JUMP_ABSOLUTE 6
      >> 27 RETURN_VALUE


       



      >>> listComp2 = compile('list(f(x) for x in xs)', 'listComp2', 'eval')
      >>> dis.dis(listComp2)
      1 0 LOAD_NAME 0 (list)
      3 LOAD_CONST 0 (<code object <genexpr> at 0x255bc68, file "listComp2", line 1>)
      6 MAKE_FUNCTION 0
      9 LOAD_NAME 1 (xs)
      12 GET_ITER
      13 CALL_FUNCTION 1
      16 CALL_FUNCTION 1
      19 RETURN_VALUE
      >>> listComp2.co_consts
      (<code object <genexpr> at 0x255bc68, file "listComp2", line 1>,)
      >>> dis.dis(listComp2.co_consts[0])
      1 0 LOAD_FAST 0 (.0)
      >> 3 FOR_ITER 17 (to 23)
      6 STORE_FAST 1 (x)
      9 LOAD_GLOBAL 0 (f)
      12 LOAD_FAST 1 (x)
      15 CALL_FUNCTION 1
      18 YIELD_VALUE
      19 POP_TOP
      20 JUMP_ABSOLUTE 3
      >> 23 LOAD_CONST 0 (None)
      26 RETURN_VALUE


       



      >>> evalledMap = compile('list(map(f,xs))', 'evalledMap', 'eval')
      >>> dis.dis(evalledMap)
      1 0 LOAD_NAME 0 (list)
      3 LOAD_NAME 1 (map)
      6 LOAD_NAME 2 (f)
      9 LOAD_NAME 3 (xs)
      12 CALL_FUNCTION 2
      15 CALL_FUNCTION 1
      18 RETURN_VALUE


      It seems it is better to use [...] syntax than list(...). Sadly the map class is a bit opaque to disassembly, but we can make due with our speed test.






      share|improve this answer





















      • 18





        Believe it or not, some "novice programmers" students learn a functional language first. Not necessarily a bad thing.

        – MarcH
        Aug 1 '12 at 21:57






      • 5





        "the very useful itertools module [is] probably considered unpythonic in terms of style". Hmm. I don't like the term "Pythonic" either, so in some sense I don't care what it means, but I don't think it's fair to those who do use it, to say that according to "Pythonicness" builtins map and filter along with standard library itertools are inherently bad style. Unless GvR actually says they were either a terrible mistake or solely for performance, the only natural conclusion if that's what "Pythonicness" says is to forget about it as stupid ;-)

        – Steve Jessop
        Feb 13 '14 at 17:53








      • 3





        @SteveJessop: Actually, Guido thought dropping map/filter was a great idea for Python 3, and only a rebellion by other Pythonistas kept them in the built-in namespace (while reduce was moved to functools). I personally disagree (map and filter are fine with predefined, particularly built-in, functions, just never use them if a lambda would be needed), but GvR has basically called them not Pythonic for years.

        – ShadowRanger
        Oct 16 '15 at 20:43













      • @ShadowRanger: true, but was GvR ever planning to remove itertools? The part I quote from this answer is the main claim that befuddles me. I don't know whether in his ideal world, map and filter would move to itertools (or functools) or go entirely, but whichever is the case, once one says that itertools is unPythonic in its entirety, then I don't really know what "Pythonic" is supposed to mean but I don't think it can be anything similar to "what GvR recommends people use".

        – Steve Jessop
        Oct 16 '15 at 21:39








      • 2





        @SteveJessop: I was only addressing map/filter, not itertools. Functional programming is perfectly Pythonic (itertools, functools and operator were all designed specifically with functional programming in mind, and I use functional idioms in Python all the time), and itertools provides features that would be a pain to implement yourself, It's specifically map and filter being redundant with generator expressions that made Guido hate them. itertools has always been fine.

        – ShadowRanger
        Oct 16 '15 at 23:05














      403












      403








      403







      Cases





      • Common case: Almost always, you will want to use a list comprehension in python because it will be more obvious what you're doing to novice programmers reading your code. (This does not apply to other languages, where other idioms may apply.) It will even be more obvious what you're doing to python programmers, since list comprehensions are the de-facto standard in python for iteration; they are expected.


      • Less-common case: However if you already have a function defined, it is often reasonable to use map, though it is considered 'unpythonic'. For example, map(sum, myLists) is more elegant/terse than [sum(x) for x in myLists]. You gain the elegance of not having to make up a dummy variable (e.g. sum(x) for x... or sum(_) for _... or sum(readableName) for readableName...) which you have to type twice, just to iterate. The same argument holds for filter and reduce and anything from the itertools module: if you already have a function handy, you could go ahead and do some functional programming. This gains readability in some situations, and loses it in others (e.g. novice programmers, multiple arguments)... but the readability of your code highly depends on your comments anyway.


      • Almost never: You may want to use the map function as a pure abstract function while doing functional programming, where you're mapping map, or currying map, or otherwise benefit from talking about map as a function. In Haskell for example, a functor interface called fmap generalizes mapping over any data structure. This is very uncommon in python because the python grammar compels you to use generator-style to talk about iteration; you can't generalize it easily. (This is sometimes good and sometimes bad.) You can probably come up with rare python examples where map(f, *lists) is a reasonable thing to do. The closest example I can come up with would be sumEach = partial(map,sum), which is a one-liner that is very roughly equivalent to:




      def sumEach(myLists):
      return [sum(_) for _ in myLists]




      • Just using a for-loop: You can also of course just use a for-loop. While not as elegant from a functional-programming viewpoint, sometimes non-local variables make code clearer in imperative programming languages such as python, because people are very used to reading code that way. For-loops are also, generally, the most efficient when you are merely doing any complex operation that is not building a list like list-comprehensions and map are optimized for (e.g. summing, or making a tree, etc.) -- at least efficient in terms of memory (not necessarily in terms of time, where I'd expect at worst a constant factor, barring some rare pathological garbage-collection hiccuping).


      "Pythonism"



      I dislike the word "pythonic" because I don't find that pythonic is always elegant in my eyes. Nevertheless, map and filter and similar functions (like the very useful itertools module) are probably considered unpythonic in terms of style.



      Laziness



      In terms of efficiency, like most functional programming constructs, MAP CAN BE LAZY, and in fact is lazy in python. That means you can do this (in python3) and your computer will not run out of memory and lose all your unsaved data:



      >>> map(str, range(10**100))
      <map object at 0x2201d50>


      Try doing that with a list comprehension:



      >>> [str(n) for n in range(10**100)]
      # DO NOT TRY THIS AT HOME OR YOU WILL BE SAD #


      Do note that list comprehensions are also inherently lazy, but python has chosen to implement them as non-lazy. Nevertheless, python does support lazy list comprehensions in the form of generator expressions, as follows:



      >>> (str(n) for n in range(10**100))
      <generator object <genexpr> at 0xacbdef>


      You can basically think of the [...] syntax as passing in a generator expression to the list constructor, like list(x for x in range(5)).



      Brief contrived example



      from operator import neg
      print({x:x**2 for x in map(neg,range(5))})

      print({x:x**2 for x in [-y for y in range(5)]})

      print({x:x**2 for x in (-y for y in range(5))})


      List comprehensions are non-lazy, so may require more memory (unless you use generator comprehensions). The square brackets [...] often make things obvious, especially when in a mess of parentheses. On the other hand, sometimes you end up being verbose like typing [x for x in.... As long as you keep your iterator variables short, list comprehensions are usually clearer if you don't indent your code. But you could always indent your code.



      print(
      {x:x**2 for x in (-y for y in range(5))}
      )


      or break things up:



      rangeNeg5 = (-y for y in range(5))
      print(
      {x:x**2 for x in rangeNeg5}
      )


      Efficiency comparison for python3



      map is now lazy:



      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=map(f,xs)'
      1000000 loops, best of 3: 0.336 usec per loop ^^^^^^^^^


      Therefore if you will not be using all your data, or do not know ahead of time how much data you need, map in python3 (and generator expressions in python2 or python3) will avoid calculating their values until the last moment necessary. Usually this will usually outweigh any overhead from using map. The downside is that this is very limited in python as opposed to most functional languages: you only get this benefit if you access your data left-to-right "in order", because python generator expressions can only be evaluated the order x[0], x[1], x[2], ....



      However let's say that we have a pre-made function f we'd like to map, and we ignore the laziness of map by immediately forcing evaluation with list(...). We get some very interesting results:



      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=list(map(f,xs))'                                                                                                                                                
      10000 loops, best of 3: 165/124/135 usec per loop ^^^^^^^^^^^^^^^
      for list(<map object>)

      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=[f(x) for x in xs]'
      10000 loops, best of 3: 181/118/123 usec per loop ^^^^^^^^^^^^^^^^^^
      for list(<generator>), probably optimized

      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=list(f(x) for x in xs)'
      1000 loops, best of 3: 215/150/150 usec per loop ^^^^^^^^^^^^^^^^^^^^^^
      for list(<generator>)


      In results are in the form AAA/BBB/CCC where A was performed with on a circa-2010 Intel workstation with python 3.?.?, and B and C were performed with a circa-2013 AMD workstation with python 3.2.1, with extremely different hardware. The result seems to be that map and list comprehensions are comparable in performance, which is most strongly affected by other random factors. The only thing we can tell seems to be that, oddly, while we expect list comprehensions [...] to perform better than generator expressions (...), map is ALSO more efficient that generator expressions (again assuming that all values are evaluated/used).



      It is important to realize that these tests assume a very simple function (the identity function); however this is fine because if the function were complicated, then performance overhead would be negligible compared to other factors in the program. (It may still be interesting to test with other simple things like f=lambda x:x+x)



      If you're skilled at reading python assembly, you can use the dis module to see if that's actually what's going on behind the scenes:



      >>> listComp = compile('[f(x) for x in xs]', 'listComp', 'eval')
      >>> dis.dis(listComp)
      1 0 LOAD_CONST 0 (<code object <listcomp> at 0x2511a48, file "listComp", line 1>)
      3 MAKE_FUNCTION 0
      6 LOAD_NAME 0 (xs)
      9 GET_ITER
      10 CALL_FUNCTION 1
      13 RETURN_VALUE
      >>> listComp.co_consts
      (<code object <listcomp> at 0x2511a48, file "listComp", line 1>,)
      >>> dis.dis(listComp.co_consts[0])
      1 0 BUILD_LIST 0
      3 LOAD_FAST 0 (.0)
      >> 6 FOR_ITER 18 (to 27)
      9 STORE_FAST 1 (x)
      12 LOAD_GLOBAL 0 (f)
      15 LOAD_FAST 1 (x)
      18 CALL_FUNCTION 1
      21 LIST_APPEND 2
      24 JUMP_ABSOLUTE 6
      >> 27 RETURN_VALUE


       



      >>> listComp2 = compile('list(f(x) for x in xs)', 'listComp2', 'eval')
      >>> dis.dis(listComp2)
      1 0 LOAD_NAME 0 (list)
      3 LOAD_CONST 0 (<code object <genexpr> at 0x255bc68, file "listComp2", line 1>)
      6 MAKE_FUNCTION 0
      9 LOAD_NAME 1 (xs)
      12 GET_ITER
      13 CALL_FUNCTION 1
      16 CALL_FUNCTION 1
      19 RETURN_VALUE
      >>> listComp2.co_consts
      (<code object <genexpr> at 0x255bc68, file "listComp2", line 1>,)
      >>> dis.dis(listComp2.co_consts[0])
      1 0 LOAD_FAST 0 (.0)
      >> 3 FOR_ITER 17 (to 23)
      6 STORE_FAST 1 (x)
      9 LOAD_GLOBAL 0 (f)
      12 LOAD_FAST 1 (x)
      15 CALL_FUNCTION 1
      18 YIELD_VALUE
      19 POP_TOP
      20 JUMP_ABSOLUTE 3
      >> 23 LOAD_CONST 0 (None)
      26 RETURN_VALUE


       



      >>> evalledMap = compile('list(map(f,xs))', 'evalledMap', 'eval')
      >>> dis.dis(evalledMap)
      1 0 LOAD_NAME 0 (list)
      3 LOAD_NAME 1 (map)
      6 LOAD_NAME 2 (f)
      9 LOAD_NAME 3 (xs)
      12 CALL_FUNCTION 2
      15 CALL_FUNCTION 1
      18 RETURN_VALUE


      It seems it is better to use [...] syntax than list(...). Sadly the map class is a bit opaque to disassembly, but we can make due with our speed test.






      share|improve this answer















      Cases





      • Common case: Almost always, you will want to use a list comprehension in python because it will be more obvious what you're doing to novice programmers reading your code. (This does not apply to other languages, where other idioms may apply.) It will even be more obvious what you're doing to python programmers, since list comprehensions are the de-facto standard in python for iteration; they are expected.


      • Less-common case: However if you already have a function defined, it is often reasonable to use map, though it is considered 'unpythonic'. For example, map(sum, myLists) is more elegant/terse than [sum(x) for x in myLists]. You gain the elegance of not having to make up a dummy variable (e.g. sum(x) for x... or sum(_) for _... or sum(readableName) for readableName...) which you have to type twice, just to iterate. The same argument holds for filter and reduce and anything from the itertools module: if you already have a function handy, you could go ahead and do some functional programming. This gains readability in some situations, and loses it in others (e.g. novice programmers, multiple arguments)... but the readability of your code highly depends on your comments anyway.


      • Almost never: You may want to use the map function as a pure abstract function while doing functional programming, where you're mapping map, or currying map, or otherwise benefit from talking about map as a function. In Haskell for example, a functor interface called fmap generalizes mapping over any data structure. This is very uncommon in python because the python grammar compels you to use generator-style to talk about iteration; you can't generalize it easily. (This is sometimes good and sometimes bad.) You can probably come up with rare python examples where map(f, *lists) is a reasonable thing to do. The closest example I can come up with would be sumEach = partial(map,sum), which is a one-liner that is very roughly equivalent to:




      def sumEach(myLists):
      return [sum(_) for _ in myLists]




      • Just using a for-loop: You can also of course just use a for-loop. While not as elegant from a functional-programming viewpoint, sometimes non-local variables make code clearer in imperative programming languages such as python, because people are very used to reading code that way. For-loops are also, generally, the most efficient when you are merely doing any complex operation that is not building a list like list-comprehensions and map are optimized for (e.g. summing, or making a tree, etc.) -- at least efficient in terms of memory (not necessarily in terms of time, where I'd expect at worst a constant factor, barring some rare pathological garbage-collection hiccuping).


      "Pythonism"



      I dislike the word "pythonic" because I don't find that pythonic is always elegant in my eyes. Nevertheless, map and filter and similar functions (like the very useful itertools module) are probably considered unpythonic in terms of style.



      Laziness



      In terms of efficiency, like most functional programming constructs, MAP CAN BE LAZY, and in fact is lazy in python. That means you can do this (in python3) and your computer will not run out of memory and lose all your unsaved data:



      >>> map(str, range(10**100))
      <map object at 0x2201d50>


      Try doing that with a list comprehension:



      >>> [str(n) for n in range(10**100)]
      # DO NOT TRY THIS AT HOME OR YOU WILL BE SAD #


      Do note that list comprehensions are also inherently lazy, but python has chosen to implement them as non-lazy. Nevertheless, python does support lazy list comprehensions in the form of generator expressions, as follows:



      >>> (str(n) for n in range(10**100))
      <generator object <genexpr> at 0xacbdef>


      You can basically think of the [...] syntax as passing in a generator expression to the list constructor, like list(x for x in range(5)).



      Brief contrived example



      from operator import neg
      print({x:x**2 for x in map(neg,range(5))})

      print({x:x**2 for x in [-y for y in range(5)]})

      print({x:x**2 for x in (-y for y in range(5))})


      List comprehensions are non-lazy, so may require more memory (unless you use generator comprehensions). The square brackets [...] often make things obvious, especially when in a mess of parentheses. On the other hand, sometimes you end up being verbose like typing [x for x in.... As long as you keep your iterator variables short, list comprehensions are usually clearer if you don't indent your code. But you could always indent your code.



      print(
      {x:x**2 for x in (-y for y in range(5))}
      )


      or break things up:



      rangeNeg5 = (-y for y in range(5))
      print(
      {x:x**2 for x in rangeNeg5}
      )


      Efficiency comparison for python3



      map is now lazy:



      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=map(f,xs)'
      1000000 loops, best of 3: 0.336 usec per loop ^^^^^^^^^


      Therefore if you will not be using all your data, or do not know ahead of time how much data you need, map in python3 (and generator expressions in python2 or python3) will avoid calculating their values until the last moment necessary. Usually this will usually outweigh any overhead from using map. The downside is that this is very limited in python as opposed to most functional languages: you only get this benefit if you access your data left-to-right "in order", because python generator expressions can only be evaluated the order x[0], x[1], x[2], ....



      However let's say that we have a pre-made function f we'd like to map, and we ignore the laziness of map by immediately forcing evaluation with list(...). We get some very interesting results:



      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=list(map(f,xs))'                                                                                                                                                
      10000 loops, best of 3: 165/124/135 usec per loop ^^^^^^^^^^^^^^^
      for list(<map object>)

      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=[f(x) for x in xs]'
      10000 loops, best of 3: 181/118/123 usec per loop ^^^^^^^^^^^^^^^^^^
      for list(<generator>), probably optimized

      % python3 -mtimeit -s 'xs=range(1000)' 'f=lambda x:x' 'z=list(f(x) for x in xs)'
      1000 loops, best of 3: 215/150/150 usec per loop ^^^^^^^^^^^^^^^^^^^^^^
      for list(<generator>)


      In results are in the form AAA/BBB/CCC where A was performed with on a circa-2010 Intel workstation with python 3.?.?, and B and C were performed with a circa-2013 AMD workstation with python 3.2.1, with extremely different hardware. The result seems to be that map and list comprehensions are comparable in performance, which is most strongly affected by other random factors. The only thing we can tell seems to be that, oddly, while we expect list comprehensions [...] to perform better than generator expressions (...), map is ALSO more efficient that generator expressions (again assuming that all values are evaluated/used).



      It is important to realize that these tests assume a very simple function (the identity function); however this is fine because if the function were complicated, then performance overhead would be negligible compared to other factors in the program. (It may still be interesting to test with other simple things like f=lambda x:x+x)



      If you're skilled at reading python assembly, you can use the dis module to see if that's actually what's going on behind the scenes:



      >>> listComp = compile('[f(x) for x in xs]', 'listComp', 'eval')
      >>> dis.dis(listComp)
      1 0 LOAD_CONST 0 (<code object <listcomp> at 0x2511a48, file "listComp", line 1>)
      3 MAKE_FUNCTION 0
      6 LOAD_NAME 0 (xs)
      9 GET_ITER
      10 CALL_FUNCTION 1
      13 RETURN_VALUE
      >>> listComp.co_consts
      (<code object <listcomp> at 0x2511a48, file "listComp", line 1>,)
      >>> dis.dis(listComp.co_consts[0])
      1 0 BUILD_LIST 0
      3 LOAD_FAST 0 (.0)
      >> 6 FOR_ITER 18 (to 27)
      9 STORE_FAST 1 (x)
      12 LOAD_GLOBAL 0 (f)
      15 LOAD_FAST 1 (x)
      18 CALL_FUNCTION 1
      21 LIST_APPEND 2
      24 JUMP_ABSOLUTE 6
      >> 27 RETURN_VALUE


       



      >>> listComp2 = compile('list(f(x) for x in xs)', 'listComp2', 'eval')
      >>> dis.dis(listComp2)
      1 0 LOAD_NAME 0 (list)
      3 LOAD_CONST 0 (<code object <genexpr> at 0x255bc68, file "listComp2", line 1>)
      6 MAKE_FUNCTION 0
      9 LOAD_NAME 1 (xs)
      12 GET_ITER
      13 CALL_FUNCTION 1
      16 CALL_FUNCTION 1
      19 RETURN_VALUE
      >>> listComp2.co_consts
      (<code object <genexpr> at 0x255bc68, file "listComp2", line 1>,)
      >>> dis.dis(listComp2.co_consts[0])
      1 0 LOAD_FAST 0 (.0)
      >> 3 FOR_ITER 17 (to 23)
      6 STORE_FAST 1 (x)
      9 LOAD_GLOBAL 0 (f)
      12 LOAD_FAST 1 (x)
      15 CALL_FUNCTION 1
      18 YIELD_VALUE
      19 POP_TOP
      20 JUMP_ABSOLUTE 3
      >> 23 LOAD_CONST 0 (None)
      26 RETURN_VALUE


       



      >>> evalledMap = compile('list(map(f,xs))', 'evalledMap', 'eval')
      >>> dis.dis(evalledMap)
      1 0 LOAD_NAME 0 (list)
      3 LOAD_NAME 1 (map)
      6 LOAD_NAME 2 (f)
      9 LOAD_NAME 3 (xs)
      12 CALL_FUNCTION 2
      15 CALL_FUNCTION 1
      18 RETURN_VALUE


      It seems it is better to use [...] syntax than list(...). Sadly the map class is a bit opaque to disassembly, but we can make due with our speed test.







      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited May 15 '18 at 13:27

























      answered Jun 20 '11 at 5:41









      ninjageckoninjagecko

      61k17113122




      61k17113122








      • 18





        Believe it or not, some "novice programmers" students learn a functional language first. Not necessarily a bad thing.

        – MarcH
        Aug 1 '12 at 21:57






      • 5





        "the very useful itertools module [is] probably considered unpythonic in terms of style". Hmm. I don't like the term "Pythonic" either, so in some sense I don't care what it means, but I don't think it's fair to those who do use it, to say that according to "Pythonicness" builtins map and filter along with standard library itertools are inherently bad style. Unless GvR actually says they were either a terrible mistake or solely for performance, the only natural conclusion if that's what "Pythonicness" says is to forget about it as stupid ;-)

        – Steve Jessop
        Feb 13 '14 at 17:53








      • 3





        @SteveJessop: Actually, Guido thought dropping map/filter was a great idea for Python 3, and only a rebellion by other Pythonistas kept them in the built-in namespace (while reduce was moved to functools). I personally disagree (map and filter are fine with predefined, particularly built-in, functions, just never use them if a lambda would be needed), but GvR has basically called them not Pythonic for years.

        – ShadowRanger
        Oct 16 '15 at 20:43













      • @ShadowRanger: true, but was GvR ever planning to remove itertools? The part I quote from this answer is the main claim that befuddles me. I don't know whether in his ideal world, map and filter would move to itertools (or functools) or go entirely, but whichever is the case, once one says that itertools is unPythonic in its entirety, then I don't really know what "Pythonic" is supposed to mean but I don't think it can be anything similar to "what GvR recommends people use".

        – Steve Jessop
        Oct 16 '15 at 21:39








      • 2





        @SteveJessop: I was only addressing map/filter, not itertools. Functional programming is perfectly Pythonic (itertools, functools and operator were all designed specifically with functional programming in mind, and I use functional idioms in Python all the time), and itertools provides features that would be a pain to implement yourself, It's specifically map and filter being redundant with generator expressions that made Guido hate them. itertools has always been fine.

        – ShadowRanger
        Oct 16 '15 at 23:05














      • 18





        Believe it or not, some "novice programmers" students learn a functional language first. Not necessarily a bad thing.

        – MarcH
        Aug 1 '12 at 21:57






      • 5





        "the very useful itertools module [is] probably considered unpythonic in terms of style". Hmm. I don't like the term "Pythonic" either, so in some sense I don't care what it means, but I don't think it's fair to those who do use it, to say that according to "Pythonicness" builtins map and filter along with standard library itertools are inherently bad style. Unless GvR actually says they were either a terrible mistake or solely for performance, the only natural conclusion if that's what "Pythonicness" says is to forget about it as stupid ;-)

        – Steve Jessop
        Feb 13 '14 at 17:53








      • 3





        @SteveJessop: Actually, Guido thought dropping map/filter was a great idea for Python 3, and only a rebellion by other Pythonistas kept them in the built-in namespace (while reduce was moved to functools). I personally disagree (map and filter are fine with predefined, particularly built-in, functions, just never use them if a lambda would be needed), but GvR has basically called them not Pythonic for years.

        – ShadowRanger
        Oct 16 '15 at 20:43













      • @ShadowRanger: true, but was GvR ever planning to remove itertools? The part I quote from this answer is the main claim that befuddles me. I don't know whether in his ideal world, map and filter would move to itertools (or functools) or go entirely, but whichever is the case, once one says that itertools is unPythonic in its entirety, then I don't really know what "Pythonic" is supposed to mean but I don't think it can be anything similar to "what GvR recommends people use".

        – Steve Jessop
        Oct 16 '15 at 21:39








      • 2





        @SteveJessop: I was only addressing map/filter, not itertools. Functional programming is perfectly Pythonic (itertools, functools and operator were all designed specifically with functional programming in mind, and I use functional idioms in Python all the time), and itertools provides features that would be a pain to implement yourself, It's specifically map and filter being redundant with generator expressions that made Guido hate them. itertools has always been fine.

        – ShadowRanger
        Oct 16 '15 at 23:05








      18




      18





      Believe it or not, some "novice programmers" students learn a functional language first. Not necessarily a bad thing.

      – MarcH
      Aug 1 '12 at 21:57





      Believe it or not, some "novice programmers" students learn a functional language first. Not necessarily a bad thing.

      – MarcH
      Aug 1 '12 at 21:57




      5




      5





      "the very useful itertools module [is] probably considered unpythonic in terms of style". Hmm. I don't like the term "Pythonic" either, so in some sense I don't care what it means, but I don't think it's fair to those who do use it, to say that according to "Pythonicness" builtins map and filter along with standard library itertools are inherently bad style. Unless GvR actually says they were either a terrible mistake or solely for performance, the only natural conclusion if that's what "Pythonicness" says is to forget about it as stupid ;-)

      – Steve Jessop
      Feb 13 '14 at 17:53







      "the very useful itertools module [is] probably considered unpythonic in terms of style". Hmm. I don't like the term "Pythonic" either, so in some sense I don't care what it means, but I don't think it's fair to those who do use it, to say that according to "Pythonicness" builtins map and filter along with standard library itertools are inherently bad style. Unless GvR actually says they were either a terrible mistake or solely for performance, the only natural conclusion if that's what "Pythonicness" says is to forget about it as stupid ;-)

      – Steve Jessop
      Feb 13 '14 at 17:53






      3




      3





      @SteveJessop: Actually, Guido thought dropping map/filter was a great idea for Python 3, and only a rebellion by other Pythonistas kept them in the built-in namespace (while reduce was moved to functools). I personally disagree (map and filter are fine with predefined, particularly built-in, functions, just never use them if a lambda would be needed), but GvR has basically called them not Pythonic for years.

      – ShadowRanger
      Oct 16 '15 at 20:43







      @SteveJessop: Actually, Guido thought dropping map/filter was a great idea for Python 3, and only a rebellion by other Pythonistas kept them in the built-in namespace (while reduce was moved to functools). I personally disagree (map and filter are fine with predefined, particularly built-in, functions, just never use them if a lambda would be needed), but GvR has basically called them not Pythonic for years.

      – ShadowRanger
      Oct 16 '15 at 20:43















      @ShadowRanger: true, but was GvR ever planning to remove itertools? The part I quote from this answer is the main claim that befuddles me. I don't know whether in his ideal world, map and filter would move to itertools (or functools) or go entirely, but whichever is the case, once one says that itertools is unPythonic in its entirety, then I don't really know what "Pythonic" is supposed to mean but I don't think it can be anything similar to "what GvR recommends people use".

      – Steve Jessop
      Oct 16 '15 at 21:39







      @ShadowRanger: true, but was GvR ever planning to remove itertools? The part I quote from this answer is the main claim that befuddles me. I don't know whether in his ideal world, map and filter would move to itertools (or functools) or go entirely, but whichever is the case, once one says that itertools is unPythonic in its entirety, then I don't really know what "Pythonic" is supposed to mean but I don't think it can be anything similar to "what GvR recommends people use".

      – Steve Jessop
      Oct 16 '15 at 21:39






      2




      2





      @SteveJessop: I was only addressing map/filter, not itertools. Functional programming is perfectly Pythonic (itertools, functools and operator were all designed specifically with functional programming in mind, and I use functional idioms in Python all the time), and itertools provides features that would be a pain to implement yourself, It's specifically map and filter being redundant with generator expressions that made Guido hate them. itertools has always been fine.

      – ShadowRanger
      Oct 16 '15 at 23:05





      @SteveJessop: I was only addressing map/filter, not itertools. Functional programming is perfectly Pythonic (itertools, functools and operator were all designed specifically with functional programming in mind, and I use functional idioms in Python all the time), and itertools provides features that would be a pain to implement yourself, It's specifically map and filter being redundant with generator expressions that made Guido hate them. itertools has always been fine.

      – ShadowRanger
      Oct 16 '15 at 23:05











      84














      You should use map and filter instead of list comprehensions.



      An objective reason why you should prefer them even though they're not "Pythonic" is this:

      They require functions/lambdas as arguments, which introduce a new scope.



      I've gotten bitten by this more than once:



      for x, y in somePoints:
      # (several lines of code here)
      squared = [x ** 2 for x in numbers]
      # Oops, x was silently overwritten!


      but if instead I had said:



      for x, y in somePoints:
      # (several lines of code here)
      squared = map(lambda x: x ** 2, numbers)


      then everything would've been fine.



      You could say I was being silly for using the same variable name in the same scope.



      I wasn't. The code was fine originally -- the two xs weren't in the same scope.

      It was only after I moved the inner block to a different section of the code that the problem came up (read: problem during maintenance, not development), and I didn't expect it.



      Yes, if you never make this mistake then list comprehensions are more elegant.

      But from personal experience (and from seeing others make the same mistake) I've seen it happen enough times that I think it's not worth the pain you have to go through when these bugs creep into your code.



      Conclusion:



      Use map and filter. They prevent subtle hard-to-diagnose scope-related bugs.



      Side note:



      Don't forget to consider using imap and ifilter (in itertools) if they are appropriate for your situation!






      share|improve this answer





















      • 4





        Thanks for pointing this out. It hadn't explicitly occurred to me that list comprehension was in the same scope and could be an issue. With that said, I think some of the other answers make it clear that list comprehension should be the default approach most of the time but that this is something to remember. This is also a good general reminder to keep functions (and thus scope) small and have thorough unit tests and use assert statements.

        – TimothyAWiseman
        Nov 21 '12 at 19:00






      • 76





        This bug is fixed in Python 3

        – Mirzhan Irkegulov
        Jan 8 '13 at 13:51






      • 11





        @wim: This was only about Python 2, although it applies to Python 3 if you want to stay backwards-compatible. I knew about it and I'd been using Python for a while now (yes, more than just a few months), and yet it happened to me. I've seen others who are smarter than me fall into the same trap. If you're so bright and/or experienced that this isn't a problem for you then I'm happy for you, I don't think most people are like you. If they were, there wouldn't be such an urge to fix it in Python 3.

        – Mehrdad
        Dec 17 '13 at 23:48








      • 8





        I'm sorry but you wrote this in late 2012, well after python 3 is on the scene, and the answer reads like you're recommending an otherwise unpopular style of python coding just because you got bitten by a bug while cutting-and-pasting code. I never claimed to be bright or experienced, I just don't agree that the bold claim is justified by your reasons.

        – wim
        Dec 18 '13 at 0:14








      • 7





        @wim: Huh? Python 2 is still used in a lot of places, the fact that Python 3 exists doesn't change that. And when you say "it's not exactly a subtle bug for anyone that has used Python more than a few months" that sentence literally means "this only concerns inexperienced developers" (clearly not you). And for the record, you clearly didn't read the answer because I said in bold that I was moving, not copying, code. Copy-paste bugs are pretty uniform across languages. This kind of bug is more unique to Python because of its scoping; it's subtler & easier to forget about and miss.

        – Mehrdad
        Dec 18 '13 at 0:38


















      84














      You should use map and filter instead of list comprehensions.



      An objective reason why you should prefer them even though they're not "Pythonic" is this:

      They require functions/lambdas as arguments, which introduce a new scope.



      I've gotten bitten by this more than once:



      for x, y in somePoints:
      # (several lines of code here)
      squared = [x ** 2 for x in numbers]
      # Oops, x was silently overwritten!


      but if instead I had said:



      for x, y in somePoints:
      # (several lines of code here)
      squared = map(lambda x: x ** 2, numbers)


      then everything would've been fine.



      You could say I was being silly for using the same variable name in the same scope.



      I wasn't. The code was fine originally -- the two xs weren't in the same scope.

      It was only after I moved the inner block to a different section of the code that the problem came up (read: problem during maintenance, not development), and I didn't expect it.



      Yes, if you never make this mistake then list comprehensions are more elegant.

      But from personal experience (and from seeing others make the same mistake) I've seen it happen enough times that I think it's not worth the pain you have to go through when these bugs creep into your code.



      Conclusion:



      Use map and filter. They prevent subtle hard-to-diagnose scope-related bugs.



      Side note:



      Don't forget to consider using imap and ifilter (in itertools) if they are appropriate for your situation!






      share|improve this answer





















      • 4





        Thanks for pointing this out. It hadn't explicitly occurred to me that list comprehension was in the same scope and could be an issue. With that said, I think some of the other answers make it clear that list comprehension should be the default approach most of the time but that this is something to remember. This is also a good general reminder to keep functions (and thus scope) small and have thorough unit tests and use assert statements.

        – TimothyAWiseman
        Nov 21 '12 at 19:00






      • 76





        This bug is fixed in Python 3

        – Mirzhan Irkegulov
        Jan 8 '13 at 13:51






      • 11





        @wim: This was only about Python 2, although it applies to Python 3 if you want to stay backwards-compatible. I knew about it and I'd been using Python for a while now (yes, more than just a few months), and yet it happened to me. I've seen others who are smarter than me fall into the same trap. If you're so bright and/or experienced that this isn't a problem for you then I'm happy for you, I don't think most people are like you. If they were, there wouldn't be such an urge to fix it in Python 3.

        – Mehrdad
        Dec 17 '13 at 23:48








      • 8





        I'm sorry but you wrote this in late 2012, well after python 3 is on the scene, and the answer reads like you're recommending an otherwise unpopular style of python coding just because you got bitten by a bug while cutting-and-pasting code. I never claimed to be bright or experienced, I just don't agree that the bold claim is justified by your reasons.

        – wim
        Dec 18 '13 at 0:14








      • 7





        @wim: Huh? Python 2 is still used in a lot of places, the fact that Python 3 exists doesn't change that. And when you say "it's not exactly a subtle bug for anyone that has used Python more than a few months" that sentence literally means "this only concerns inexperienced developers" (clearly not you). And for the record, you clearly didn't read the answer because I said in bold that I was moving, not copying, code. Copy-paste bugs are pretty uniform across languages. This kind of bug is more unique to Python because of its scoping; it's subtler & easier to forget about and miss.

        – Mehrdad
        Dec 18 '13 at 0:38
















      84












      84








      84







      You should use map and filter instead of list comprehensions.



      An objective reason why you should prefer them even though they're not "Pythonic" is this:

      They require functions/lambdas as arguments, which introduce a new scope.



      I've gotten bitten by this more than once:



      for x, y in somePoints:
      # (several lines of code here)
      squared = [x ** 2 for x in numbers]
      # Oops, x was silently overwritten!


      but if instead I had said:



      for x, y in somePoints:
      # (several lines of code here)
      squared = map(lambda x: x ** 2, numbers)


      then everything would've been fine.



      You could say I was being silly for using the same variable name in the same scope.



      I wasn't. The code was fine originally -- the two xs weren't in the same scope.

      It was only after I moved the inner block to a different section of the code that the problem came up (read: problem during maintenance, not development), and I didn't expect it.



      Yes, if you never make this mistake then list comprehensions are more elegant.

      But from personal experience (and from seeing others make the same mistake) I've seen it happen enough times that I think it's not worth the pain you have to go through when these bugs creep into your code.



      Conclusion:



      Use map and filter. They prevent subtle hard-to-diagnose scope-related bugs.



      Side note:



      Don't forget to consider using imap and ifilter (in itertools) if they are appropriate for your situation!






      share|improve this answer















      You should use map and filter instead of list comprehensions.



      An objective reason why you should prefer them even though they're not "Pythonic" is this:

      They require functions/lambdas as arguments, which introduce a new scope.



      I've gotten bitten by this more than once:



      for x, y in somePoints:
      # (several lines of code here)
      squared = [x ** 2 for x in numbers]
      # Oops, x was silently overwritten!


      but if instead I had said:



      for x, y in somePoints:
      # (several lines of code here)
      squared = map(lambda x: x ** 2, numbers)


      then everything would've been fine.



      You could say I was being silly for using the same variable name in the same scope.



      I wasn't. The code was fine originally -- the two xs weren't in the same scope.

      It was only after I moved the inner block to a different section of the code that the problem came up (read: problem during maintenance, not development), and I didn't expect it.



      Yes, if you never make this mistake then list comprehensions are more elegant.

      But from personal experience (and from seeing others make the same mistake) I've seen it happen enough times that I think it's not worth the pain you have to go through when these bugs creep into your code.



      Conclusion:



      Use map and filter. They prevent subtle hard-to-diagnose scope-related bugs.



      Side note:



      Don't forget to consider using imap and ifilter (in itertools) if they are appropriate for your situation!







      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited Jul 9 '14 at 13:43









      Paolo Moretti

      37.4k188985




      37.4k188985










      answered Nov 20 '12 at 22:28









      MehrdadMehrdad

      129k90415758




      129k90415758








      • 4





        Thanks for pointing this out. It hadn't explicitly occurred to me that list comprehension was in the same scope and could be an issue. With that said, I think some of the other answers make it clear that list comprehension should be the default approach most of the time but that this is something to remember. This is also a good general reminder to keep functions (and thus scope) small and have thorough unit tests and use assert statements.

        – TimothyAWiseman
        Nov 21 '12 at 19:00






      • 76





        This bug is fixed in Python 3

        – Mirzhan Irkegulov
        Jan 8 '13 at 13:51






      • 11





        @wim: This was only about Python 2, although it applies to Python 3 if you want to stay backwards-compatible. I knew about it and I'd been using Python for a while now (yes, more than just a few months), and yet it happened to me. I've seen others who are smarter than me fall into the same trap. If you're so bright and/or experienced that this isn't a problem for you then I'm happy for you, I don't think most people are like you. If they were, there wouldn't be such an urge to fix it in Python 3.

        – Mehrdad
        Dec 17 '13 at 23:48








      • 8





        I'm sorry but you wrote this in late 2012, well after python 3 is on the scene, and the answer reads like you're recommending an otherwise unpopular style of python coding just because you got bitten by a bug while cutting-and-pasting code. I never claimed to be bright or experienced, I just don't agree that the bold claim is justified by your reasons.

        – wim
        Dec 18 '13 at 0:14








      • 7





        @wim: Huh? Python 2 is still used in a lot of places, the fact that Python 3 exists doesn't change that. And when you say "it's not exactly a subtle bug for anyone that has used Python more than a few months" that sentence literally means "this only concerns inexperienced developers" (clearly not you). And for the record, you clearly didn't read the answer because I said in bold that I was moving, not copying, code. Copy-paste bugs are pretty uniform across languages. This kind of bug is more unique to Python because of its scoping; it's subtler & easier to forget about and miss.

        – Mehrdad
        Dec 18 '13 at 0:38
















      • 4





        Thanks for pointing this out. It hadn't explicitly occurred to me that list comprehension was in the same scope and could be an issue. With that said, I think some of the other answers make it clear that list comprehension should be the default approach most of the time but that this is something to remember. This is also a good general reminder to keep functions (and thus scope) small and have thorough unit tests and use assert statements.

        – TimothyAWiseman
        Nov 21 '12 at 19:00






      • 76





        This bug is fixed in Python 3

        – Mirzhan Irkegulov
        Jan 8 '13 at 13:51






      • 11





        @wim: This was only about Python 2, although it applies to Python 3 if you want to stay backwards-compatible. I knew about it and I'd been using Python for a while now (yes, more than just a few months), and yet it happened to me. I've seen others who are smarter than me fall into the same trap. If you're so bright and/or experienced that this isn't a problem for you then I'm happy for you, I don't think most people are like you. If they were, there wouldn't be such an urge to fix it in Python 3.

        – Mehrdad
        Dec 17 '13 at 23:48








      • 8





        I'm sorry but you wrote this in late 2012, well after python 3 is on the scene, and the answer reads like you're recommending an otherwise unpopular style of python coding just because you got bitten by a bug while cutting-and-pasting code. I never claimed to be bright or experienced, I just don't agree that the bold claim is justified by your reasons.

        – wim
        Dec 18 '13 at 0:14








      • 7





        @wim: Huh? Python 2 is still used in a lot of places, the fact that Python 3 exists doesn't change that. And when you say "it's not exactly a subtle bug for anyone that has used Python more than a few months" that sentence literally means "this only concerns inexperienced developers" (clearly not you). And for the record, you clearly didn't read the answer because I said in bold that I was moving, not copying, code. Copy-paste bugs are pretty uniform across languages. This kind of bug is more unique to Python because of its scoping; it's subtler & easier to forget about and miss.

        – Mehrdad
        Dec 18 '13 at 0:38










      4




      4





      Thanks for pointing this out. It hadn't explicitly occurred to me that list comprehension was in the same scope and could be an issue. With that said, I think some of the other answers make it clear that list comprehension should be the default approach most of the time but that this is something to remember. This is also a good general reminder to keep functions (and thus scope) small and have thorough unit tests and use assert statements.

      – TimothyAWiseman
      Nov 21 '12 at 19:00





      Thanks for pointing this out. It hadn't explicitly occurred to me that list comprehension was in the same scope and could be an issue. With that said, I think some of the other answers make it clear that list comprehension should be the default approach most of the time but that this is something to remember. This is also a good general reminder to keep functions (and thus scope) small and have thorough unit tests and use assert statements.

      – TimothyAWiseman
      Nov 21 '12 at 19:00




      76




      76





      This bug is fixed in Python 3

      – Mirzhan Irkegulov
      Jan 8 '13 at 13:51





      This bug is fixed in Python 3

      – Mirzhan Irkegulov
      Jan 8 '13 at 13:51




      11




      11





      @wim: This was only about Python 2, although it applies to Python 3 if you want to stay backwards-compatible. I knew about it and I'd been using Python for a while now (yes, more than just a few months), and yet it happened to me. I've seen others who are smarter than me fall into the same trap. If you're so bright and/or experienced that this isn't a problem for you then I'm happy for you, I don't think most people are like you. If they were, there wouldn't be such an urge to fix it in Python 3.

      – Mehrdad
      Dec 17 '13 at 23:48







      @wim: This was only about Python 2, although it applies to Python 3 if you want to stay backwards-compatible. I knew about it and I'd been using Python for a while now (yes, more than just a few months), and yet it happened to me. I've seen others who are smarter than me fall into the same trap. If you're so bright and/or experienced that this isn't a problem for you then I'm happy for you, I don't think most people are like you. If they were, there wouldn't be such an urge to fix it in Python 3.

      – Mehrdad
      Dec 17 '13 at 23:48






      8




      8





      I'm sorry but you wrote this in late 2012, well after python 3 is on the scene, and the answer reads like you're recommending an otherwise unpopular style of python coding just because you got bitten by a bug while cutting-and-pasting code. I never claimed to be bright or experienced, I just don't agree that the bold claim is justified by your reasons.

      – wim
      Dec 18 '13 at 0:14







      I'm sorry but you wrote this in late 2012, well after python 3 is on the scene, and the answer reads like you're recommending an otherwise unpopular style of python coding just because you got bitten by a bug while cutting-and-pasting code. I never claimed to be bright or experienced, I just don't agree that the bold claim is justified by your reasons.

      – wim
      Dec 18 '13 at 0:14






      7




      7





      @wim: Huh? Python 2 is still used in a lot of places, the fact that Python 3 exists doesn't change that. And when you say "it's not exactly a subtle bug for anyone that has used Python more than a few months" that sentence literally means "this only concerns inexperienced developers" (clearly not you). And for the record, you clearly didn't read the answer because I said in bold that I was moving, not copying, code. Copy-paste bugs are pretty uniform across languages. This kind of bug is more unique to Python because of its scoping; it's subtler & easier to forget about and miss.

      – Mehrdad
      Dec 18 '13 at 0:38







      @wim: Huh? Python 2 is still used in a lot of places, the fact that Python 3 exists doesn't change that. And when you say "it's not exactly a subtle bug for anyone that has used Python more than a few months" that sentence literally means "this only concerns inexperienced developers" (clearly not you). And for the record, you clearly didn't read the answer because I said in bold that I was moving, not copying, code. Copy-paste bugs are pretty uniform across languages. This kind of bug is more unique to Python because of its scoping; it's subtler & easier to forget about and miss.

      – Mehrdad
      Dec 18 '13 at 0:38













      38














      Actually, map and list comprehensions behave quite differently in the Python 3 language. Take a look at the following Python 3 program:



      def square(x):
      return x*x
      squares = map(square, [1, 2, 3])
      print(list(squares))
      print(list(squares))


      You might expect it to print the line "[1, 4, 9]" twice, but instead it prints "[1, 4, 9]" followed by "". The first time you look at squares it seems to behave as a sequence of three elements, but the second time as an empty one.



      In the Python 2 language map returns a plain old list, just like list comprehensions do in both languages. The crux is that the return value of map in Python 3 (and imap in Python 2) is not a list - it's an iterator!



      The elements are consumed when you iterate over an iterator unlike when you iterate over a list. This is why squares looks empty in the last print(list(squares)) line.



      To summarize:




      • When dealing with iterators you have to remember that they are stateful and that they mutate as you traverse them.

      • Lists are more predictable since they only change when you explicitly mutate them; they are containers.

      • And a bonus: numbers, strings, and tuples are even more predictable since they cannot change at all; they are values.






      share|improve this answer


























      • this is probably the best argument for list comprehensions. pythons map is not the functional map but the crippled red-headed stepchild of a functional implementation. Very sad, because I really dislike comprehensions.

        – semiomant
        Jun 29 '17 at 14:26











      • @semiomant I would say lazy map (like in python3) is more 'functional' than eager map (like in python2). For example, map in Haskell is lazy (well, everything in Haskell is lazy...). Anyway, lazy map is better for chaining maps - if you have a map applied to map applied to map, you have a list for each of intermediate map calls in python2, whereas in python3 you have just one resulting list, so its more memory efficient.

        – MnZrK
        Oct 21 '17 at 15:57











      • I guess what I want is for map to produce a data structure, not an iterator. But maybe lazy iterators are easier than lazy data structures. Food for thought. Thanks @MnZrK

        – semiomant
        Oct 24 '17 at 11:01











      • You want to say map returns an iterable, not an iterator.

        – Mehrdad
        Sep 18 '18 at 21:33
















      38














      Actually, map and list comprehensions behave quite differently in the Python 3 language. Take a look at the following Python 3 program:



      def square(x):
      return x*x
      squares = map(square, [1, 2, 3])
      print(list(squares))
      print(list(squares))


      You might expect it to print the line "[1, 4, 9]" twice, but instead it prints "[1, 4, 9]" followed by "". The first time you look at squares it seems to behave as a sequence of three elements, but the second time as an empty one.



      In the Python 2 language map returns a plain old list, just like list comprehensions do in both languages. The crux is that the return value of map in Python 3 (and imap in Python 2) is not a list - it's an iterator!



      The elements are consumed when you iterate over an iterator unlike when you iterate over a list. This is why squares looks empty in the last print(list(squares)) line.



      To summarize:




      • When dealing with iterators you have to remember that they are stateful and that they mutate as you traverse them.

      • Lists are more predictable since they only change when you explicitly mutate them; they are containers.

      • And a bonus: numbers, strings, and tuples are even more predictable since they cannot change at all; they are values.






      share|improve this answer


























      • this is probably the best argument for list comprehensions. pythons map is not the functional map but the crippled red-headed stepchild of a functional implementation. Very sad, because I really dislike comprehensions.

        – semiomant
        Jun 29 '17 at 14:26











      • @semiomant I would say lazy map (like in python3) is more 'functional' than eager map (like in python2). For example, map in Haskell is lazy (well, everything in Haskell is lazy...). Anyway, lazy map is better for chaining maps - if you have a map applied to map applied to map, you have a list for each of intermediate map calls in python2, whereas in python3 you have just one resulting list, so its more memory efficient.

        – MnZrK
        Oct 21 '17 at 15:57











      • I guess what I want is for map to produce a data structure, not an iterator. But maybe lazy iterators are easier than lazy data structures. Food for thought. Thanks @MnZrK

        – semiomant
        Oct 24 '17 at 11:01











      • You want to say map returns an iterable, not an iterator.

        – Mehrdad
        Sep 18 '18 at 21:33














      38












      38








      38







      Actually, map and list comprehensions behave quite differently in the Python 3 language. Take a look at the following Python 3 program:



      def square(x):
      return x*x
      squares = map(square, [1, 2, 3])
      print(list(squares))
      print(list(squares))


      You might expect it to print the line "[1, 4, 9]" twice, but instead it prints "[1, 4, 9]" followed by "". The first time you look at squares it seems to behave as a sequence of three elements, but the second time as an empty one.



      In the Python 2 language map returns a plain old list, just like list comprehensions do in both languages. The crux is that the return value of map in Python 3 (and imap in Python 2) is not a list - it's an iterator!



      The elements are consumed when you iterate over an iterator unlike when you iterate over a list. This is why squares looks empty in the last print(list(squares)) line.



      To summarize:




      • When dealing with iterators you have to remember that they are stateful and that they mutate as you traverse them.

      • Lists are more predictable since they only change when you explicitly mutate them; they are containers.

      • And a bonus: numbers, strings, and tuples are even more predictable since they cannot change at all; they are values.






      share|improve this answer















      Actually, map and list comprehensions behave quite differently in the Python 3 language. Take a look at the following Python 3 program:



      def square(x):
      return x*x
      squares = map(square, [1, 2, 3])
      print(list(squares))
      print(list(squares))


      You might expect it to print the line "[1, 4, 9]" twice, but instead it prints "[1, 4, 9]" followed by "". The first time you look at squares it seems to behave as a sequence of three elements, but the second time as an empty one.



      In the Python 2 language map returns a plain old list, just like list comprehensions do in both languages. The crux is that the return value of map in Python 3 (and imap in Python 2) is not a list - it's an iterator!



      The elements are consumed when you iterate over an iterator unlike when you iterate over a list. This is why squares looks empty in the last print(list(squares)) line.



      To summarize:




      • When dealing with iterators you have to remember that they are stateful and that they mutate as you traverse them.

      • Lists are more predictable since they only change when you explicitly mutate them; they are containers.

      • And a bonus: numbers, strings, and tuples are even more predictable since they cannot change at all; they are values.







      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited Dec 2 '13 at 18:52

























      answered Oct 1 '13 at 13:09









      raekraek

      1,7481416




      1,7481416













      • this is probably the best argument for list comprehensions. pythons map is not the functional map but the crippled red-headed stepchild of a functional implementation. Very sad, because I really dislike comprehensions.

        – semiomant
        Jun 29 '17 at 14:26











      • @semiomant I would say lazy map (like in python3) is more 'functional' than eager map (like in python2). For example, map in Haskell is lazy (well, everything in Haskell is lazy...). Anyway, lazy map is better for chaining maps - if you have a map applied to map applied to map, you have a list for each of intermediate map calls in python2, whereas in python3 you have just one resulting list, so its more memory efficient.

        – MnZrK
        Oct 21 '17 at 15:57











      • I guess what I want is for map to produce a data structure, not an iterator. But maybe lazy iterators are easier than lazy data structures. Food for thought. Thanks @MnZrK

        – semiomant
        Oct 24 '17 at 11:01











      • You want to say map returns an iterable, not an iterator.

        – Mehrdad
        Sep 18 '18 at 21:33



















      • this is probably the best argument for list comprehensions. pythons map is not the functional map but the crippled red-headed stepchild of a functional implementation. Very sad, because I really dislike comprehensions.

        – semiomant
        Jun 29 '17 at 14:26











      • @semiomant I would say lazy map (like in python3) is more 'functional' than eager map (like in python2). For example, map in Haskell is lazy (well, everything in Haskell is lazy...). Anyway, lazy map is better for chaining maps - if you have a map applied to map applied to map, you have a list for each of intermediate map calls in python2, whereas in python3 you have just one resulting list, so its more memory efficient.

        – MnZrK
        Oct 21 '17 at 15:57











      • I guess what I want is for map to produce a data structure, not an iterator. But maybe lazy iterators are easier than lazy data structures. Food for thought. Thanks @MnZrK

        – semiomant
        Oct 24 '17 at 11:01











      • You want to say map returns an iterable, not an iterator.

        – Mehrdad
        Sep 18 '18 at 21:33

















      this is probably the best argument for list comprehensions. pythons map is not the functional map but the crippled red-headed stepchild of a functional implementation. Very sad, because I really dislike comprehensions.

      – semiomant
      Jun 29 '17 at 14:26





      this is probably the best argument for list comprehensions. pythons map is not the functional map but the crippled red-headed stepchild of a functional implementation. Very sad, because I really dislike comprehensions.

      – semiomant
      Jun 29 '17 at 14:26













      @semiomant I would say lazy map (like in python3) is more 'functional' than eager map (like in python2). For example, map in Haskell is lazy (well, everything in Haskell is lazy...). Anyway, lazy map is better for chaining maps - if you have a map applied to map applied to map, you have a list for each of intermediate map calls in python2, whereas in python3 you have just one resulting list, so its more memory efficient.

      – MnZrK
      Oct 21 '17 at 15:57





      @semiomant I would say lazy map (like in python3) is more 'functional' than eager map (like in python2). For example, map in Haskell is lazy (well, everything in Haskell is lazy...). Anyway, lazy map is better for chaining maps - if you have a map applied to map applied to map, you have a list for each of intermediate map calls in python2, whereas in python3 you have just one resulting list, so its more memory efficient.

      – MnZrK
      Oct 21 '17 at 15:57













      I guess what I want is for map to produce a data structure, not an iterator. But maybe lazy iterators are easier than lazy data structures. Food for thought. Thanks @MnZrK

      – semiomant
      Oct 24 '17 at 11:01





      I guess what I want is for map to produce a data structure, not an iterator. But maybe lazy iterators are easier than lazy data structures. Food for thought. Thanks @MnZrK

      – semiomant
      Oct 24 '17 at 11:01













      You want to say map returns an iterable, not an iterator.

      – Mehrdad
      Sep 18 '18 at 21:33





      You want to say map returns an iterable, not an iterator.

      – Mehrdad
      Sep 18 '18 at 21:33











      15














      I find list comprehensions are generally more expressive of what I'm trying to do than map - they both get it done, but the former saves the mental load of trying to understand what could be a complex lambda expression.



      There's also an interview out there somewhere (I can't find it offhand) where Guido lists lambdas and the functional functions as the thing he most regrets about accepting into Python, so you could make the argument that they're un-Pythonic by virtue of that.






      share|improve this answer



















      • 9





        Yeah, sigh, but Guido's original intention to remove lambda altogether in Python 3 got a barrage of lobbying against it, so he went back on it despite my stout support -- ah well, guess lambda's just too handy in many SIMPLE cases, the only problem is when it exceeds the bounds of SIMPLE or gets assigned to a name (in which latter case it's a silly hobbled duplicate of def!-).

        – Alex Martelli
        Aug 8 '09 at 3:58






      • 1





        The interview you are thinking about is this one: amk.ca/python/writing/gvr-interview, where Guido says "Sometimes I've been too quick in accepting contributions, and later realized that it was a mistake. One example would be some of the functional programming features, such as lambda functions. lambda is a keyword that lets you create a small anonymous function; built-in functions such as map, filter, and reduce run a function over a sequence type, such as a list."

        – J. Taylor
        Mar 24 '11 at 4:30






      • 3





        @Alex, I don't have your years of experience, but I've seen far more over-complicated list comprehensions than lambdas. Of course, abusing language features is always a difficult temptation to resist. It's interesting that list comprehensions (empirically) seem more prone to abuse than lambdas, though I'm not sure why that should be the case. I'll also point out that "hobbled" isn't always a bad thing. Reducing the scope of "things this line might be doing" can sometimes make it easier on the reader. For example, the const keyword in C++ is a great triumph along these lines.

        – Stuart Berg
        Mar 25 '13 at 15:02











      • > guido . Which is another piece of evidence that Guido is out of his mind. Of course lambda's have been made so lame (no statements..) that they're difficult to use and limited anyways.

        – javadba
        Jan 6 at 23:41


















      15














      I find list comprehensions are generally more expressive of what I'm trying to do than map - they both get it done, but the former saves the mental load of trying to understand what could be a complex lambda expression.



      There's also an interview out there somewhere (I can't find it offhand) where Guido lists lambdas and the functional functions as the thing he most regrets about accepting into Python, so you could make the argument that they're un-Pythonic by virtue of that.






      share|improve this answer



















      • 9





        Yeah, sigh, but Guido's original intention to remove lambda altogether in Python 3 got a barrage of lobbying against it, so he went back on it despite my stout support -- ah well, guess lambda's just too handy in many SIMPLE cases, the only problem is when it exceeds the bounds of SIMPLE or gets assigned to a name (in which latter case it's a silly hobbled duplicate of def!-).

        – Alex Martelli
        Aug 8 '09 at 3:58






      • 1





        The interview you are thinking about is this one: amk.ca/python/writing/gvr-interview, where Guido says "Sometimes I've been too quick in accepting contributions, and later realized that it was a mistake. One example would be some of the functional programming features, such as lambda functions. lambda is a keyword that lets you create a small anonymous function; built-in functions such as map, filter, and reduce run a function over a sequence type, such as a list."

        – J. Taylor
        Mar 24 '11 at 4:30






      • 3





        @Alex, I don't have your years of experience, but I've seen far more over-complicated list comprehensions than lambdas. Of course, abusing language features is always a difficult temptation to resist. It's interesting that list comprehensions (empirically) seem more prone to abuse than lambdas, though I'm not sure why that should be the case. I'll also point out that "hobbled" isn't always a bad thing. Reducing the scope of "things this line might be doing" can sometimes make it easier on the reader. For example, the const keyword in C++ is a great triumph along these lines.

        – Stuart Berg
        Mar 25 '13 at 15:02











      • > guido . Which is another piece of evidence that Guido is out of his mind. Of course lambda's have been made so lame (no statements..) that they're difficult to use and limited anyways.

        – javadba
        Jan 6 at 23:41
















      15












      15








      15







      I find list comprehensions are generally more expressive of what I'm trying to do than map - they both get it done, but the former saves the mental load of trying to understand what could be a complex lambda expression.



      There's also an interview out there somewhere (I can't find it offhand) where Guido lists lambdas and the functional functions as the thing he most regrets about accepting into Python, so you could make the argument that they're un-Pythonic by virtue of that.






      share|improve this answer













      I find list comprehensions are generally more expressive of what I'm trying to do than map - they both get it done, but the former saves the mental load of trying to understand what could be a complex lambda expression.



      There's also an interview out there somewhere (I can't find it offhand) where Guido lists lambdas and the functional functions as the thing he most regrets about accepting into Python, so you could make the argument that they're un-Pythonic by virtue of that.







      share|improve this answer












      share|improve this answer



      share|improve this answer










      answered Aug 7 '09 at 23:59









      DanDan

      879716




      879716








      • 9





        Yeah, sigh, but Guido's original intention to remove lambda altogether in Python 3 got a barrage of lobbying against it, so he went back on it despite my stout support -- ah well, guess lambda's just too handy in many SIMPLE cases, the only problem is when it exceeds the bounds of SIMPLE or gets assigned to a name (in which latter case it's a silly hobbled duplicate of def!-).

        – Alex Martelli
        Aug 8 '09 at 3:58






      • 1





        The interview you are thinking about is this one: amk.ca/python/writing/gvr-interview, where Guido says "Sometimes I've been too quick in accepting contributions, and later realized that it was a mistake. One example would be some of the functional programming features, such as lambda functions. lambda is a keyword that lets you create a small anonymous function; built-in functions such as map, filter, and reduce run a function over a sequence type, such as a list."

        – J. Taylor
        Mar 24 '11 at 4:30






      • 3





        @Alex, I don't have your years of experience, but I've seen far more over-complicated list comprehensions than lambdas. Of course, abusing language features is always a difficult temptation to resist. It's interesting that list comprehensions (empirically) seem more prone to abuse than lambdas, though I'm not sure why that should be the case. I'll also point out that "hobbled" isn't always a bad thing. Reducing the scope of "things this line might be doing" can sometimes make it easier on the reader. For example, the const keyword in C++ is a great triumph along these lines.

        – Stuart Berg
        Mar 25 '13 at 15:02











      • > guido . Which is another piece of evidence that Guido is out of his mind. Of course lambda's have been made so lame (no statements..) that they're difficult to use and limited anyways.

        – javadba
        Jan 6 at 23:41
















      • 9





        Yeah, sigh, but Guido's original intention to remove lambda altogether in Python 3 got a barrage of lobbying against it, so he went back on it despite my stout support -- ah well, guess lambda's just too handy in many SIMPLE cases, the only problem is when it exceeds the bounds of SIMPLE or gets assigned to a name (in which latter case it's a silly hobbled duplicate of def!-).

        – Alex Martelli
        Aug 8 '09 at 3:58






      • 1





        The interview you are thinking about is this one: amk.ca/python/writing/gvr-interview, where Guido says "Sometimes I've been too quick in accepting contributions, and later realized that it was a mistake. One example would be some of the functional programming features, such as lambda functions. lambda is a keyword that lets you create a small anonymous function; built-in functions such as map, filter, and reduce run a function over a sequence type, such as a list."

        – J. Taylor
        Mar 24 '11 at 4:30






      • 3





        @Alex, I don't have your years of experience, but I've seen far more over-complicated list comprehensions than lambdas. Of course, abusing language features is always a difficult temptation to resist. It's interesting that list comprehensions (empirically) seem more prone to abuse than lambdas, though I'm not sure why that should be the case. I'll also point out that "hobbled" isn't always a bad thing. Reducing the scope of "things this line might be doing" can sometimes make it easier on the reader. For example, the const keyword in C++ is a great triumph along these lines.

        – Stuart Berg
        Mar 25 '13 at 15:02











      • > guido . Which is another piece of evidence that Guido is out of his mind. Of course lambda's have been made so lame (no statements..) that they're difficult to use and limited anyways.

        – javadba
        Jan 6 at 23:41










      9




      9





      Yeah, sigh, but Guido's original intention to remove lambda altogether in Python 3 got a barrage of lobbying against it, so he went back on it despite my stout support -- ah well, guess lambda's just too handy in many SIMPLE cases, the only problem is when it exceeds the bounds of SIMPLE or gets assigned to a name (in which latter case it's a silly hobbled duplicate of def!-).

      – Alex Martelli
      Aug 8 '09 at 3:58





      Yeah, sigh, but Guido's original intention to remove lambda altogether in Python 3 got a barrage of lobbying against it, so he went back on it despite my stout support -- ah well, guess lambda's just too handy in many SIMPLE cases, the only problem is when it exceeds the bounds of SIMPLE or gets assigned to a name (in which latter case it's a silly hobbled duplicate of def!-).

      – Alex Martelli
      Aug 8 '09 at 3:58




      1




      1





      The interview you are thinking about is this one: amk.ca/python/writing/gvr-interview, where Guido says "Sometimes I've been too quick in accepting contributions, and later realized that it was a mistake. One example would be some of the functional programming features, such as lambda functions. lambda is a keyword that lets you create a small anonymous function; built-in functions such as map, filter, and reduce run a function over a sequence type, such as a list."

      – J. Taylor
      Mar 24 '11 at 4:30





      The interview you are thinking about is this one: amk.ca/python/writing/gvr-interview, where Guido says "Sometimes I've been too quick in accepting contributions, and later realized that it was a mistake. One example would be some of the functional programming features, such as lambda functions. lambda is a keyword that lets you create a small anonymous function; built-in functions such as map, filter, and reduce run a function over a sequence type, such as a list."

      – J. Taylor
      Mar 24 '11 at 4:30




      3




      3





      @Alex, I don't have your years of experience, but I've seen far more over-complicated list comprehensions than lambdas. Of course, abusing language features is always a difficult temptation to resist. It's interesting that list comprehensions (empirically) seem more prone to abuse than lambdas, though I'm not sure why that should be the case. I'll also point out that "hobbled" isn't always a bad thing. Reducing the scope of "things this line might be doing" can sometimes make it easier on the reader. For example, the const keyword in C++ is a great triumph along these lines.

      – Stuart Berg
      Mar 25 '13 at 15:02





      @Alex, I don't have your years of experience, but I've seen far more over-complicated list comprehensions than lambdas. Of course, abusing language features is always a difficult temptation to resist. It's interesting that list comprehensions (empirically) seem more prone to abuse than lambdas, though I'm not sure why that should be the case. I'll also point out that "hobbled" isn't always a bad thing. Reducing the scope of "things this line might be doing" can sometimes make it easier on the reader. For example, the const keyword in C++ is a great triumph along these lines.

      – Stuart Berg
      Mar 25 '13 at 15:02













      > guido . Which is another piece of evidence that Guido is out of his mind. Of course lambda's have been made so lame (no statements..) that they're difficult to use and limited anyways.

      – javadba
      Jan 6 at 23:41







      > guido . Which is another piece of evidence that Guido is out of his mind. Of course lambda's have been made so lame (no statements..) that they're difficult to use and limited anyways.

      – javadba
      Jan 6 at 23:41













      15














      If you plan on writing any asynchronous, parallel, or distributed code, you will probably prefer map over a list comprehension -- as most asynchronous, parallel, or distributed packages provide a map function to overload python's map. Then by passing the appropriate map function to the rest of your code, you may not have to modify your original serial code to have it run in parallel (etc).






      share|improve this answer
























      • Could you provide an example?

        – Roman
        Mar 22 '16 at 6:34






      • 1





        How about this github.com/uqfoundation/pathos/blob/master/tests/test_map.py and this github.com/uqfoundation/pathos/blob/master/tests/test_star.py?

        – Mike McKerns
        Mar 22 '16 at 12:51






      • 1





        Python's multiprocessing module does this: docs.python.org/2/library/multiprocessing.html

        – Robert L.
        Jul 31 '17 at 18:07


















      15














      If you plan on writing any asynchronous, parallel, or distributed code, you will probably prefer map over a list comprehension -- as most asynchronous, parallel, or distributed packages provide a map function to overload python's map. Then by passing the appropriate map function to the rest of your code, you may not have to modify your original serial code to have it run in parallel (etc).






      share|improve this answer
























      • Could you provide an example?

        – Roman
        Mar 22 '16 at 6:34






      • 1





        How about this github.com/uqfoundation/pathos/blob/master/tests/test_map.py and this github.com/uqfoundation/pathos/blob/master/tests/test_star.py?

        – Mike McKerns
        Mar 22 '16 at 12:51






      • 1





        Python's multiprocessing module does this: docs.python.org/2/library/multiprocessing.html

        – Robert L.
        Jul 31 '17 at 18:07
















      15












      15








      15







      If you plan on writing any asynchronous, parallel, or distributed code, you will probably prefer map over a list comprehension -- as most asynchronous, parallel, or distributed packages provide a map function to overload python's map. Then by passing the appropriate map function to the rest of your code, you may not have to modify your original serial code to have it run in parallel (etc).






      share|improve this answer













      If you plan on writing any asynchronous, parallel, or distributed code, you will probably prefer map over a list comprehension -- as most asynchronous, parallel, or distributed packages provide a map function to overload python's map. Then by passing the appropriate map function to the rest of your code, you may not have to modify your original serial code to have it run in parallel (etc).







      share|improve this answer












      share|improve this answer



      share|improve this answer










      answered Jun 8 '14 at 17:03









      Mike McKernsMike McKerns

      18.5k47290




      18.5k47290













      • Could you provide an example?

        – Roman
        Mar 22 '16 at 6:34






      • 1





        How about this github.com/uqfoundation/pathos/blob/master/tests/test_map.py and this github.com/uqfoundation/pathos/blob/master/tests/test_star.py?

        – Mike McKerns
        Mar 22 '16 at 12:51






      • 1





        Python's multiprocessing module does this: docs.python.org/2/library/multiprocessing.html

        – Robert L.
        Jul 31 '17 at 18:07





















      • Could you provide an example?

        – Roman
        Mar 22 '16 at 6:34






      • 1





        How about this github.com/uqfoundation/pathos/blob/master/tests/test_map.py and this github.com/uqfoundation/pathos/blob/master/tests/test_star.py?

        – Mike McKerns
        Mar 22 '16 at 12:51






      • 1





        Python's multiprocessing module does this: docs.python.org/2/library/multiprocessing.html

        – Robert L.
        Jul 31 '17 at 18:07



















      Could you provide an example?

      – Roman
      Mar 22 '16 at 6:34





      Could you provide an example?

      – Roman
      Mar 22 '16 at 6:34




      1




      1





      How about this github.com/uqfoundation/pathos/blob/master/tests/test_map.py and this github.com/uqfoundation/pathos/blob/master/tests/test_star.py?

      – Mike McKerns
      Mar 22 '16 at 12:51





      How about this github.com/uqfoundation/pathos/blob/master/tests/test_map.py and this github.com/uqfoundation/pathos/blob/master/tests/test_star.py?

      – Mike McKerns
      Mar 22 '16 at 12:51




      1




      1





      Python's multiprocessing module does this: docs.python.org/2/library/multiprocessing.html

      – Robert L.
      Jul 31 '17 at 18:07







      Python's multiprocessing module does this: docs.python.org/2/library/multiprocessing.html

      – Robert L.
      Jul 31 '17 at 18:07













      14














      Here is one possible case:



      map(lambda op1,op2: op1*op2, list1, list2)


      versus:



      [op1*op2 for op1,op2 in zip(list1,list2)]


      I am guessing the zip() is an unfortunate and unnecessary overhead you need to indulge in if you insist on using list comprehensions instead of the map. Would be great if someone clarifies this whether affirmatively or negatively.






      share|improve this answer


























      • "[op1*op2 from op1,op2 in zip(list1,list2)]" | s/form/for/ And an equivalent list with out zip: (less readable)[list1[i]*list2[i] for i in range(len(list1))]

        – weakish
        Aug 9 '10 at 2:45








      • 2





        Should be "for" not "from" in your second code quote, @andz, and in @weakish's comment too. I thought I had discovered a new syntactical approach to list comprehensions... Darn.

        – physicsmichael
        Oct 12 '10 at 13:12






      • 4





        to add a very late comment, you can make zip lazy by using itertools.izip

        – tacaswell
        Dec 17 '12 at 21:09











      • @tcaswell No longer needed in Python 3000.

        – JeromeJ
        Aug 21 '13 at 21:38






      • 3





        I think I still prefer map(operator.mul, list1, list2). It's on these very simple left side expressions that comprehensions get clumsy.

        – Yann Vernier
        Mar 10 '15 at 23:34
















      14














      Here is one possible case:



      map(lambda op1,op2: op1*op2, list1, list2)


      versus:



      [op1*op2 for op1,op2 in zip(list1,list2)]


      I am guessing the zip() is an unfortunate and unnecessary overhead you need to indulge in if you insist on using list comprehensions instead of the map. Would be great if someone clarifies this whether affirmatively or negatively.






      share|improve this answer


























      • "[op1*op2 from op1,op2 in zip(list1,list2)]" | s/form/for/ And an equivalent list with out zip: (less readable)[list1[i]*list2[i] for i in range(len(list1))]

        – weakish
        Aug 9 '10 at 2:45








      • 2





        Should be "for" not "from" in your second code quote, @andz, and in @weakish's comment too. I thought I had discovered a new syntactical approach to list comprehensions... Darn.

        – physicsmichael
        Oct 12 '10 at 13:12






      • 4





        to add a very late comment, you can make zip lazy by using itertools.izip

        – tacaswell
        Dec 17 '12 at 21:09











      • @tcaswell No longer needed in Python 3000.

        – JeromeJ
        Aug 21 '13 at 21:38






      • 3





        I think I still prefer map(operator.mul, list1, list2). It's on these very simple left side expressions that comprehensions get clumsy.

        – Yann Vernier
        Mar 10 '15 at 23:34














      14












      14








      14







      Here is one possible case:



      map(lambda op1,op2: op1*op2, list1, list2)


      versus:



      [op1*op2 for op1,op2 in zip(list1,list2)]


      I am guessing the zip() is an unfortunate and unnecessary overhead you need to indulge in if you insist on using list comprehensions instead of the map. Would be great if someone clarifies this whether affirmatively or negatively.






      share|improve this answer















      Here is one possible case:



      map(lambda op1,op2: op1*op2, list1, list2)


      versus:



      [op1*op2 for op1,op2 in zip(list1,list2)]


      I am guessing the zip() is an unfortunate and unnecessary overhead you need to indulge in if you insist on using list comprehensions instead of the map. Would be great if someone clarifies this whether affirmatively or negatively.







      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited Aug 3 '11 at 21:33









      ninjagecko

      61k17113122




      61k17113122










      answered Nov 2 '09 at 8:42









      AndzAndz

      1,04911114




      1,04911114













      • "[op1*op2 from op1,op2 in zip(list1,list2)]" | s/form/for/ And an equivalent list with out zip: (less readable)[list1[i]*list2[i] for i in range(len(list1))]

        – weakish
        Aug 9 '10 at 2:45








      • 2





        Should be "for" not "from" in your second code quote, @andz, and in @weakish's comment too. I thought I had discovered a new syntactical approach to list comprehensions... Darn.

        – physicsmichael
        Oct 12 '10 at 13:12






      • 4





        to add a very late comment, you can make zip lazy by using itertools.izip

        – tacaswell
        Dec 17 '12 at 21:09











      • @tcaswell No longer needed in Python 3000.

        – JeromeJ
        Aug 21 '13 at 21:38






      • 3





        I think I still prefer map(operator.mul, list1, list2). It's on these very simple left side expressions that comprehensions get clumsy.

        – Yann Vernier
        Mar 10 '15 at 23:34



















      • "[op1*op2 from op1,op2 in zip(list1,list2)]" | s/form/for/ And an equivalent list with out zip: (less readable)[list1[i]*list2[i] for i in range(len(list1))]

        – weakish
        Aug 9 '10 at 2:45








      • 2





        Should be "for" not "from" in your second code quote, @andz, and in @weakish's comment too. I thought I had discovered a new syntactical approach to list comprehensions... Darn.

        – physicsmichael
        Oct 12 '10 at 13:12






      • 4





        to add a very late comment, you can make zip lazy by using itertools.izip

        – tacaswell
        Dec 17 '12 at 21:09











      • @tcaswell No longer needed in Python 3000.

        – JeromeJ
        Aug 21 '13 at 21:38






      • 3





        I think I still prefer map(operator.mul, list1, list2). It's on these very simple left side expressions that comprehensions get clumsy.

        – Yann Vernier
        Mar 10 '15 at 23:34

















      "[op1*op2 from op1,op2 in zip(list1,list2)]" | s/form/for/ And an equivalent list with out zip: (less readable)[list1[i]*list2[i] for i in range(len(list1))]

      – weakish
      Aug 9 '10 at 2:45







      "[op1*op2 from op1,op2 in zip(list1,list2)]" | s/form/for/ And an equivalent list with out zip: (less readable)[list1[i]*list2[i] for i in range(len(list1))]

      – weakish
      Aug 9 '10 at 2:45






      2




      2





      Should be "for" not "from" in your second code quote, @andz, and in @weakish's comment too. I thought I had discovered a new syntactical approach to list comprehensions... Darn.

      – physicsmichael
      Oct 12 '10 at 13:12





      Should be "for" not "from" in your second code quote, @andz, and in @weakish's comment too. I thought I had discovered a new syntactical approach to list comprehensions... Darn.

      – physicsmichael
      Oct 12 '10 at 13:12




      4




      4





      to add a very late comment, you can make zip lazy by using itertools.izip

      – tacaswell
      Dec 17 '12 at 21:09





      to add a very late comment, you can make zip lazy by using itertools.izip

      – tacaswell
      Dec 17 '12 at 21:09













      @tcaswell No longer needed in Python 3000.

      – JeromeJ
      Aug 21 '13 at 21:38





      @tcaswell No longer needed in Python 3000.

      – JeromeJ
      Aug 21 '13 at 21:38




      3




      3





      I think I still prefer map(operator.mul, list1, list2). It's on these very simple left side expressions that comprehensions get clumsy.

      – Yann Vernier
      Mar 10 '15 at 23:34





      I think I still prefer map(operator.mul, list1, list2). It's on these very simple left side expressions that comprehensions get clumsy.

      – Yann Vernier
      Mar 10 '15 at 23:34











      6














      So since Python 3, map() is an iterator, you need to keep in mind what do you need: an iterator or list object.



      As @AlexMartelli already mentioned, map() is faster than list comprehension only if you don't use lambda function.



      I will present you some time comparisons.




      Python 3.5.2 and CPython
      I've used Jupiter notebook and especially %timeit built-in magic command

      Measurements: s == 1000 ms == 1000 * 1000 µs = 1000 * 1000 * 1000 ns



      Setup:



      x_list = [(i, i+1, i+2, i*2, i-9) for i in range(1000)]
      i_list = list(range(1000))


      Built-in function:



      %timeit map(sum, x_list)  # creating iterator object
      # Output: The slowest run took 9.91 times longer than the fastest.
      # This could mean that an intermediate result is being cached.
      # 1000000 loops, best of 3: 277 ns per loop

      %timeit list(map(sum, x_list)) # creating list with map
      # Output: 1000 loops, best of 3: 214 µs per loop

      %timeit [sum(x) for x in x_list] # creating list with list comprehension
      # Output: 1000 loops, best of 3: 290 µs per loop


      lambda function:



      %timeit map(lambda i: i+1, i_list)
      # Output: The slowest run took 8.64 times longer than the fastest.
      # This could mean that an intermediate result is being cached.
      # 1000000 loops, best of 3: 325 ns per loop

      %timeit list(map(lambda i: i+1, i_list))
      # Output: 1000 loops, best of 3: 183 µs per loop

      %timeit [i+1 for i in i_list]
      # Output: 10000 loops, best of 3: 84.2 µs per loop


      There is also such thing as generator expression, see PEP-0289. So i thought it would be useful to add it to comparison



      %timeit (sum(i) for i in x_list)
      # Output: The slowest run took 6.66 times longer than the fastest.
      # This could mean that an intermediate result is being cached.
      # 1000000 loops, best of 3: 495 ns per loop

      %timeit list((sum(x) for x in x_list))
      # Output: 1000 loops, best of 3: 319 µs per loop

      %timeit (i+1 for i in i_list)
      # Output: The slowest run took 6.83 times longer than the fastest.
      # This could mean that an intermediate result is being cached.
      # 1000000 loops, best of 3: 506 ns per loop

      %timeit list((i+1 for i in i_list))
      # Output: 10000 loops, best of 3: 125 µs per loop


      You need list object:



      Use list comprehension if it's custom function, use list(map()) if there is builtin function



      You don't need list object, you just need iterable one:



      Always use map()!






      share|improve this answer






























        6














        So since Python 3, map() is an iterator, you need to keep in mind what do you need: an iterator or list object.



        As @AlexMartelli already mentioned, map() is faster than list comprehension only if you don't use lambda function.



        I will present you some time comparisons.




        Python 3.5.2 and CPython
        I've used Jupiter notebook and especially %timeit built-in magic command

        Measurements: s == 1000 ms == 1000 * 1000 µs = 1000 * 1000 * 1000 ns



        Setup:



        x_list = [(i, i+1, i+2, i*2, i-9) for i in range(1000)]
        i_list = list(range(1000))


        Built-in function:



        %timeit map(sum, x_list)  # creating iterator object
        # Output: The slowest run took 9.91 times longer than the fastest.
        # This could mean that an intermediate result is being cached.
        # 1000000 loops, best of 3: 277 ns per loop

        %timeit list(map(sum, x_list)) # creating list with map
        # Output: 1000 loops, best of 3: 214 µs per loop

        %timeit [sum(x) for x in x_list] # creating list with list comprehension
        # Output: 1000 loops, best of 3: 290 µs per loop


        lambda function:



        %timeit map(lambda i: i+1, i_list)
        # Output: The slowest run took 8.64 times longer than the fastest.
        # This could mean that an intermediate result is being cached.
        # 1000000 loops, best of 3: 325 ns per loop

        %timeit list(map(lambda i: i+1, i_list))
        # Output: 1000 loops, best of 3: 183 µs per loop

        %timeit [i+1 for i in i_list]
        # Output: 10000 loops, best of 3: 84.2 µs per loop


        There is also such thing as generator expression, see PEP-0289. So i thought it would be useful to add it to comparison



        %timeit (sum(i) for i in x_list)
        # Output: The slowest run took 6.66 times longer than the fastest.
        # This could mean that an intermediate result is being cached.
        # 1000000 loops, best of 3: 495 ns per loop

        %timeit list((sum(x) for x in x_list))
        # Output: 1000 loops, best of 3: 319 µs per loop

        %timeit (i+1 for i in i_list)
        # Output: The slowest run took 6.83 times longer than the fastest.
        # This could mean that an intermediate result is being cached.
        # 1000000 loops, best of 3: 506 ns per loop

        %timeit list((i+1 for i in i_list))
        # Output: 10000 loops, best of 3: 125 µs per loop


        You need list object:



        Use list comprehension if it's custom function, use list(map()) if there is builtin function



        You don't need list object, you just need iterable one:



        Always use map()!






        share|improve this answer




























          6












          6








          6







          So since Python 3, map() is an iterator, you need to keep in mind what do you need: an iterator or list object.



          As @AlexMartelli already mentioned, map() is faster than list comprehension only if you don't use lambda function.



          I will present you some time comparisons.




          Python 3.5.2 and CPython
          I've used Jupiter notebook and especially %timeit built-in magic command

          Measurements: s == 1000 ms == 1000 * 1000 µs = 1000 * 1000 * 1000 ns



          Setup:



          x_list = [(i, i+1, i+2, i*2, i-9) for i in range(1000)]
          i_list = list(range(1000))


          Built-in function:



          %timeit map(sum, x_list)  # creating iterator object
          # Output: The slowest run took 9.91 times longer than the fastest.
          # This could mean that an intermediate result is being cached.
          # 1000000 loops, best of 3: 277 ns per loop

          %timeit list(map(sum, x_list)) # creating list with map
          # Output: 1000 loops, best of 3: 214 µs per loop

          %timeit [sum(x) for x in x_list] # creating list with list comprehension
          # Output: 1000 loops, best of 3: 290 µs per loop


          lambda function:



          %timeit map(lambda i: i+1, i_list)
          # Output: The slowest run took 8.64 times longer than the fastest.
          # This could mean that an intermediate result is being cached.
          # 1000000 loops, best of 3: 325 ns per loop

          %timeit list(map(lambda i: i+1, i_list))
          # Output: 1000 loops, best of 3: 183 µs per loop

          %timeit [i+1 for i in i_list]
          # Output: 10000 loops, best of 3: 84.2 µs per loop


          There is also such thing as generator expression, see PEP-0289. So i thought it would be useful to add it to comparison



          %timeit (sum(i) for i in x_list)
          # Output: The slowest run took 6.66 times longer than the fastest.
          # This could mean that an intermediate result is being cached.
          # 1000000 loops, best of 3: 495 ns per loop

          %timeit list((sum(x) for x in x_list))
          # Output: 1000 loops, best of 3: 319 µs per loop

          %timeit (i+1 for i in i_list)
          # Output: The slowest run took 6.83 times longer than the fastest.
          # This could mean that an intermediate result is being cached.
          # 1000000 loops, best of 3: 506 ns per loop

          %timeit list((i+1 for i in i_list))
          # Output: 10000 loops, best of 3: 125 µs per loop


          You need list object:



          Use list comprehension if it's custom function, use list(map()) if there is builtin function



          You don't need list object, you just need iterable one:



          Always use map()!






          share|improve this answer















          So since Python 3, map() is an iterator, you need to keep in mind what do you need: an iterator or list object.



          As @AlexMartelli already mentioned, map() is faster than list comprehension only if you don't use lambda function.



          I will present you some time comparisons.




          Python 3.5.2 and CPython
          I've used Jupiter notebook and especially %timeit built-in magic command

          Measurements: s == 1000 ms == 1000 * 1000 µs = 1000 * 1000 * 1000 ns



          Setup:



          x_list = [(i, i+1, i+2, i*2, i-9) for i in range(1000)]
          i_list = list(range(1000))


          Built-in function:



          %timeit map(sum, x_list)  # creating iterator object
          # Output: The slowest run took 9.91 times longer than the fastest.
          # This could mean that an intermediate result is being cached.
          # 1000000 loops, best of 3: 277 ns per loop

          %timeit list(map(sum, x_list)) # creating list with map
          # Output: 1000 loops, best of 3: 214 µs per loop

          %timeit [sum(x) for x in x_list] # creating list with list comprehension
          # Output: 1000 loops, best of 3: 290 µs per loop


          lambda function:



          %timeit map(lambda i: i+1, i_list)
          # Output: The slowest run took 8.64 times longer than the fastest.
          # This could mean that an intermediate result is being cached.
          # 1000000 loops, best of 3: 325 ns per loop

          %timeit list(map(lambda i: i+1, i_list))
          # Output: 1000 loops, best of 3: 183 µs per loop

          %timeit [i+1 for i in i_list]
          # Output: 10000 loops, best of 3: 84.2 µs per loop


          There is also such thing as generator expression, see PEP-0289. So i thought it would be useful to add it to comparison



          %timeit (sum(i) for i in x_list)
          # Output: The slowest run took 6.66 times longer than the fastest.
          # This could mean that an intermediate result is being cached.
          # 1000000 loops, best of 3: 495 ns per loop

          %timeit list((sum(x) for x in x_list))
          # Output: 1000 loops, best of 3: 319 µs per loop

          %timeit (i+1 for i in i_list)
          # Output: The slowest run took 6.83 times longer than the fastest.
          # This could mean that an intermediate result is being cached.
          # 1000000 loops, best of 3: 506 ns per loop

          %timeit list((i+1 for i in i_list))
          # Output: 10000 loops, best of 3: 125 µs per loop


          You need list object:



          Use list comprehension if it's custom function, use list(map()) if there is builtin function



          You don't need list object, you just need iterable one:



          Always use map()!







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited May 23 '17 at 11:47









          Community

          11




          11










          answered Dec 3 '16 at 14:18









          vishes_shellvishes_shell

          10.7k34046




          10.7k34046























              0














              I consider that the most Pythonic way is to use a list comprehension instead of map and filter. The reason is that list comprehensions are clearer than map and filter.



              In [1]: odd_cubes = [x ** 3 for x in range(10) if x % 2 == 1] # using a list comprehension

              In [2]: odd_cubes_alt = list(map(lambda x: x ** 3, filter(lambda x: x % 2 == 1, range(10)))) # using map and filter

              In [3]: odd_cubes == odd_cubes_alt
              Out[3]: True


              As you an see, a comprehension does not require extra lambda expressions as map needs. Furthermore, a comprehension also allows filtering easily, while map requires filter to allow filtering.






              share|improve this answer






























                0














                I consider that the most Pythonic way is to use a list comprehension instead of map and filter. The reason is that list comprehensions are clearer than map and filter.



                In [1]: odd_cubes = [x ** 3 for x in range(10) if x % 2 == 1] # using a list comprehension

                In [2]: odd_cubes_alt = list(map(lambda x: x ** 3, filter(lambda x: x % 2 == 1, range(10)))) # using map and filter

                In [3]: odd_cubes == odd_cubes_alt
                Out[3]: True


                As you an see, a comprehension does not require extra lambda expressions as map needs. Furthermore, a comprehension also allows filtering easily, while map requires filter to allow filtering.






                share|improve this answer




























                  0












                  0








                  0







                  I consider that the most Pythonic way is to use a list comprehension instead of map and filter. The reason is that list comprehensions are clearer than map and filter.



                  In [1]: odd_cubes = [x ** 3 for x in range(10) if x % 2 == 1] # using a list comprehension

                  In [2]: odd_cubes_alt = list(map(lambda x: x ** 3, filter(lambda x: x % 2 == 1, range(10)))) # using map and filter

                  In [3]: odd_cubes == odd_cubes_alt
                  Out[3]: True


                  As you an see, a comprehension does not require extra lambda expressions as map needs. Furthermore, a comprehension also allows filtering easily, while map requires filter to allow filtering.






                  share|improve this answer















                  I consider that the most Pythonic way is to use a list comprehension instead of map and filter. The reason is that list comprehensions are clearer than map and filter.



                  In [1]: odd_cubes = [x ** 3 for x in range(10) if x % 2 == 1] # using a list comprehension

                  In [2]: odd_cubes_alt = list(map(lambda x: x ** 3, filter(lambda x: x % 2 == 1, range(10)))) # using map and filter

                  In [3]: odd_cubes == odd_cubes_alt
                  Out[3]: True


                  As you an see, a comprehension does not require extra lambda expressions as map needs. Furthermore, a comprehension also allows filtering easily, while map requires filter to allow filtering.







                  share|improve this answer














                  share|improve this answer



                  share|improve this answer








                  edited Sep 11 '17 at 19:26

























                  answered Sep 4 '17 at 17:20









                  lmiguelvargasflmiguelvargasf

                  12.6k1187108




                  12.6k1187108






























                      draft saved

                      draft discarded




















































                      Thanks for contributing an answer to Stack Overflow!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f1247486%2flist-comprehension-vs-map%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      Florida Star v. B. J. F.

                      Danny Elfman

                      Lugert, Oklahoma