What are Generators in Python and How to use them?

Generators have been an important part of python ever since they were introduced with PEP 255.

Generator in python are special routine that can be used to control the iteration behaviour of a loop. A generator is similar to a function returning an array. A generator has parameter, which we can call and it generates a sequence of numbers. But unlike functions, which return a whole array, a generator yields one value at a time which requires less memory.

Any python function with a keyword “yield” may be called as generator. A normal python function starts execution from first line and continues until we get a return statement or an exception or end of the function. However, any of the local variables created during the function scope are destroyed and not accessible further. In case of generator, when it encounters a yield keyword the state of the function is frozen and all the variables are stored in memory until the generator is called again.

“Ready to take your python skills to the next level? Sign up for a free demo today!”

We can use generator in accordance with an iterator or can be explicitly called using the “next” keyword.

Generally generators in Python are:

• Defined with the def keyword
• Use the yield keyword
• May contain several yield keywords.
• Returns an iterator.

Generators with Iterators

def generator_thr_iter():
   yield 'xyz'
   yield 246
   yield 40.50
for i in generator_thr_iter():
   print(i)

Output

xyz
246
40.5

Generator using next

def generator_thr_iter():
   yield 'xyz'
   yield 246
   yield 40.50
>>> g = generator_thr_iter()
>>> g.__next__()
'xyz'
>>> g.__next__()
246
>>> g.__next__()
40.5
>>> g.__next__()
Traceback (most recent call last):
File "<pyshell#39>", line 1, in <module>
g.__next__()
StopIteration

We can think of generators as the one returning multiple items one by one instead of all at once and the generator function is paused until the next item is requested.

Program to print square of numbers from 1 to n

Consider we want to calculate the square of number from 1 to n, where n is really big number, such that creating a list of numbers up to ‘n’ would occupy the entire system memory space.

Without generator, our approach will be something like –

>>> n= 200000000000
>>> number_list = range(1, n+1)
>>> for i in number_list:
   print(i*i)

Above approach will consume lot of system memory. Better approach is to iterate over the numbers without ever creating the list of numbers so that the system memory is not occupied. Here comes the use of generators.

Our generator program for the same would be –

def num_generator(n):
num =1
while True:
   yield num
   if num == n:
      return
   else:
      num += 1
for i in num_generator(200000000000):
   print (i*i)

So in above approach, when the for loop is first initialised the num_generator is called and the value of n = 200000000000 is stored in memory and num=1 is initialised and is entered into while loop which loops forever. Then the yield num is encountered, at this time the while loop is frozen and all the local variables are stored in memory. Since num=1, yield num is returned to the for loop and is assigned to I, where 1(i*i) is printed and the next call to num_generator is made.

Now the execution starts from the point where it has frozen previously, so it executes the line num == n (1 == 200000000000), which is false so num +=1 is executed which comes to num = 2 and the while loop is executed once again and the process continues.

Finally while loop is executed till n=200000000000, when 200000000000 is yielded then the next line ‘num == n’(200000000000 == 200000000000) is executed, since it is true the return statement is executed.

So when generator executes a return statement or encounters exception or reach the end of the generator the “StopIteration” exception is raised and the for loop iteration stops at the moment. So we are able to print square of number upto 200000000000 without ever creating a big list of numbers which would be have occupied large system memory.

Python Generators – A complete guide to create and use generators