[ Why are generators faster? ]
I understand that generators are faster than iterators. I also understand that generators can be implemented using for
loop syntax. For example:
import time
startT = time.time()
def myGen(n):
for i in range(n):
yield x
def myIter(n):
for i in range(n):
pass
def main():
n=100
startT=time.time()
myIter(n)
print 'myIter took ', time.time() - startT
startT=time.time()
myGen(n)
print 'myGen(n) took ', time.time() - startT
This is just one example of the results:
myIter took 0.09234782
myGen(n) took 0.017847266
Since this uses the for
loop syntax, then I don't understand how it is faster than an iterator. This generator uses an iterator, because the "for" loop is implemented using an iterator. If you time these, the generator is consistently faster. Why is this, when the generator uses an iterator?
Thanks.
Answer 1
In your code, myIter(n)
actually does work -- it loops 100 times.
myGen(n)
, on the other hand, simply builds the generator -- and that's it. It doesn't count to 100. All you're doing is timing how long it takes to build the object, and you're timing it in an unreliable way. If we use timeit
(here using IPython to make things simpler):
>>> %timeit myIter(100)
1000000 loops, best of 3: 1 µs per loop
>>> %timeit myGen(100)
10000000 loops, best of 3: 163 ns per loop
>>> %timeit myGen(10**1000)
10000000 loops, best of 3: 163 ns per loop
And we see that the myGen(n)
time is independent of n
, because it's not doing anything. In fact, we can see your code was never executed another way:
>>> list(myGen(100))
Traceback (most recent call last):
File "<ipython-input-11-dd43d937402a>", line 1, in <module>
list(myGen(100))
File "<ipython-input-1-ba968e48e9fd>", line 3, in myGen
yield x
NameError: name 'x' is not defined
If we fix this typo, and then try a fast way to consume the generator, we get something like
>>> %timeit myIter(100)
1000000 loops, best of 3: 1 µs per loop
>>> %timeit consume(myGen(100), 100)
100000 loops, best of 3: 3.44 µs per loop
and the generator version is slower, as is often the case.