Help! My python code is slow

Posted: 2023-10-22

TLDR: List comprehension is faster then append

Ok, so this isn’t a picture of a python snake (this is a boa constrictor), but it is the same story with append vs yield: they both kind of work in the same manner.

Most popular patterns in Python to create a list of widgets, can be summarised into these two categories: - for loop and then append on the list:

    l = []
    for x in sequence:
        y = do_something( x )
        l.append( y )

• list comprehension with generator expresion

  def gen( sequence ):
      for x in sequence:
          yield do_something( x )
  
  l = [ x for x in gen( sequence ) ]
  

What is interesting however is that list comprehension is much faster. Here’s the chart to show to ilustrate the performance of these two solutions

– insert chart of both – tested with python 3.6 and cpu

def do_something( x ):
    return x

def m1(sequence):
    l = []
    for x in sequence:
        y = do_something( x )
        l.append( y )

def gen( sequence ):
    for x in sequence:
        yield do_something( x )

def m2(sequence):
    l = [ x for x in gen( sequence ) ]


sizes = [ 1000, 10000, 100000, 1000000, 10000000, 100000000 ]


def iter(arr):
    for s in sizes:
        seq = range(s)
        r1 = %timeit -o m1(seq)
        r2 = %timeit -o m2(seq)
    
        yield (s, r1, r2)

results = [ i for i in iter(sizes) ]

import matplotlib.pyplot as plt                                                                                                                      

x = sizes
y1 = [ i[1].average for i in results]
y2 = [ i[2].average for i in results]
plt.loglog( x, y1, x, y2 )

Photo by David Clode on Unsplash