[ Why are generators faster? ]

I understand that generators are faster than iterators. I also understand that generators can be implemented using for loop syntax. For example:

    import time 


startT = time.time()


def myGen(n):
    for i in range(n):
        yield x         


def myIter(n):
    for i in range(n):
        pass

def main():
    n=100
    startT=time.time()
    myIter(n)
    print 'myIter took ', time.time() - startT

    startT=time.time()
    myGen(n)
    print 'myGen(n) took ', time.time() - startT

This is just one example of the results:

myIter took 0.09234782
myGen(n) took 0.017847266

Since this uses the for loop syntax, then I don't understand how it is faster than an iterator. This generator uses an iterator, because the "for" loop is implemented using an iterator. If you time these, the generator is consistently faster. Why is this, when the generator uses an iterator?

Thanks.

Answer 1

In your code, myIter(n) actually does work -- it loops 100 times.

myGen(n), on the other hand, simply builds the generator -- and that's it. It doesn't count to 100. All you're doing is timing how long it takes to build the object, and you're timing it in an unreliable way. If we use timeit (here using IPython to make things simpler):

>>> %timeit myIter(100)
1000000 loops, best of 3: 1 µs per loop
>>> %timeit myGen(100)
10000000 loops, best of 3: 163 ns per loop
>>> %timeit myGen(10**1000)
10000000 loops, best of 3: 163 ns per loop

And we see that the myGen(n) time is independent of n, because it's not doing anything. In fact, we can see your code was never executed another way:

>>> list(myGen(100))
Traceback (most recent call last):
  File "<ipython-input-11-dd43d937402a>", line 1, in <module>
    list(myGen(100))
  File "<ipython-input-1-ba968e48e9fd>", line 3, in myGen
    yield x
NameError: name 'x' is not defined

If we fix this typo, and then try a fast way to consume the generator, we get something like

>>> %timeit myIter(100)
1000000 loops, best of 3: 1 µs per loop
>>> %timeit consume(myGen(100), 100)
100000 loops, best of 3: 3.44 µs per loop

and the generator version is slower, as is often the case.