Table of Contents
Generators have been an important part of python ever since they were introduced with PEP 255.
Generator in python are special routine that can be used to control the iteration behaviour of a loop. A generator is similar to a function returning an array. A generator has parameter, which we can call and it generates a sequence of numbers. But unlike functions, which return a whole array, a generator yields one value at a time which requires less memory.
Any python function with a keyword “yield” may be called as generator. A normal python function starts execution from first line and continues until we get a return statement or an exception or end of the function. However, any of the local variables created during the function scope are destroyed and not accessible further. In case of generator, when it encounters a yield keyword the state of the function is frozen and all the variables are stored in memory until the generator is called again.
“Ready to take your python skills to the next level? Sign up for a free demo today!”
We can use generator in accordance with an iterator or can be explicitly called using the “next” keyword.
Generally generators in Python are:
- Defined with the def keyword
- Use the yield keyword
- May contain several yield keywords.
- Returns an iterator.
Generators with Iterators
def generator_thr_iter(): yield 'xyz' yield 246 yield 40.50 for i in generator_thr_iter(): print(i)
Output
xyz 246 40.5
Generator using next
def generator_thr_iter(): yield 'xyz' yield 246 yield 40.50 >>> g = generator_thr_iter() >>> g.__next__() 'xyz' >>> g.__next__() 246 >>> g.__next__() 40.5 >>> g.__next__() Traceback (most recent call last): File "<pyshell#39>", line 1, in <module> g.__next__() StopIteration
We can think of generators as the one returning multiple items one by one instead of all at once and the generator function is paused until the next item is requested.
Program to print square of numbers from 1 to n
Consider we want to calculate the square of number from 1 to n, where n is really big number, such that creating a list of numbers up to ‘n’ would occupy the entire system memory space.
Without generator, our approach will be something like –
>>> n= 200000000000 >>> number_list = range(1, n+1) >>> for i in number_list: print(i*i)
Above approach will consume lot of system memory. Better approach is to iterate over the numbers without ever creating the list of numbers so that the system memory is not occupied. Here comes the use of generators.
Our generator program for the same would be –
def num_generator(n): num =1 while True: yield num if num == n: return else: num += 1 for i in num_generator(200000000000): print (i*i)
So in above approach, when the for loop is first initialised the num_generator is called and the value of n = 200000000000 is stored in memory and num=1 is initialised and is entered into while loop which loops forever. Then the yield num is encountered, at this time the while loop is frozen and all the local variables are stored in memory. Since num=1, yield num is returned to the for loop and is assigned to I, where 1(i*i) is printed and the next call to num_generator is made.
Now the execution starts from the point where it has frozen previously, so it executes the line num == n (1 == 200000000000), which is false so num +=1 is executed which comes to num = 2 and the while loop is executed once again and the process continues.
Finally while loop is executed till n=200000000000, when 200000000000 is yielded then the next line ‘num == n’(200000000000 == 200000000000) is executed, since it is true the return statement is executed.
So when generator executes a return statement or encounters exception or reach the end of the generator the “StopIteration” exception is raised and the for loop iteration stops at the moment. So we are able to print square of number upto 200000000000 without ever creating a big list of numbers which would be have occupied large system memory.
Python Generators – A complete guide to create and use generators
Generators are also used to create functions that behave like iterators. It is a different approach to create iterators.
This article will teach you about Python generators, how you can create and use generators effectively in your code.
We will also see learn about its applications.
“Experience the power of our web development course with a free demo – enroll now!”
What are Python Generator Functions?
Generator functions are special kind of functions that returns an iterator and we can loop it through just like a list, to access the objects one at a time.
Python generator functions are a simple way to create iterators. They solve the common problem of creating iterable objects. The traditional way was to create a class and then we have to implement __iter__() and __next__() methods.
We also have to manage the internal state and raise the StopIteration exception when the generator ends.
How to Create a Python Generator?
A Python generator is created like a normal function but we do not use the return statement in generators.
For generating a value, we use the yield keyword. When we use the yield keyword inside a function, it automatically becomes a generator function.
Let us understand the working of a generator with a simple generator.
Output:
20
40
Traceback (most recent call last):
File “C:/Users/Techvidvan/AppData/Local/Programs/Python/Python38-32/test.py”, line 11, in <module>
print(next(gen))
StopIteration
Here, the mygen() function is a generator function in which we used 3 yield statements.
The next() function used on the generator object returns the value of the first yield statement. Again calling the next() will then return the value from the second function and so on. When it gets exhausted the generator function raises the StopIteration exception.
We can also use the for loop for easy traversing the elements.
Output:
3
6
End
One thing to notice here is that when the generator yields the values, it remembers the value of the variable in between each call.
Python Generators with Loops
The above examples were simple, only for understanding the working of the generators. Now we will see generators with a loop that is more practically applicable for creating customized iterable objects. We can use for-loop to yield values.
Let us generate a sequence of Fibonacci numbers:
Output:
1
1
2
3
5
8
13
21
34
Using Generators with List() Function
Output:
Python Generator Expressions
If you are familiar with list comprehensions then this would be very easy. We have even a more shorthand technique to create python generators.
In list comprehensions we use [] brackets while in generator expressions we use () parenthesis. They are used at places where we quickly want to use a generator right away.
Output:
1
4
9
16
25
36
49
64
81
Here, we created a generator that is used to generate a square of 0-9 numbers. If we print the gen object then we see that it is a generator object.
Output:
Why Generators are Used?
- Generators are used to create iterable objects.
- The generators yield one value at a time from a set of items.
- So they are memory efficient as they require less space and we don’t have to store everything at once.
- It is also easier to implement an infinite generation of series.
- We cannot store infinite series of a sequence in memory so with generators we can continuously generate values or infinite values(theoretically)
- Generators are also helpful in reading and performing operations on large files.
- We can use generators to effectively write code for our programs.
“Ready to take your python skills to the next level? Sign up for a free demo today!”
Summary
Python Generators are often misinterpreted as a hard topic but in fact, they are very useful and easy, once you start understanding how it works. We saw what Python generators are, how to create them and iterate over generators with for loops.
Moreover, we saw an even more easy way to create generators i.e. generators expression.
