in Python, yield function with List is confusing

Question

Here I try to print in two different ways, one is with list, and another one is with for loop.
However, the result of using for loop make sense, but using list is different from what I'd have thought.

Can you help me explain the reason? And in what situation you might use List function instead of for loop? Thank you!

def my_gen():
n = 1
print('This is printed first')
yield n

n += 1
print('This is printed second')
yield n

n += 1
print('This is printed at last')
yield n

print (list(my_gen()))

# Using for loop below, and this make sense
for item in my_gen():
print(item)

Answer 1

All the list() function does is it converts whatever iterable it gets to a list, and returns it.

>>> list(range(10))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> list((3, 5, 8, 13))
[3, 5, 8, 13]

>>> list('abcd')
['a', 'b', 'c', 'd']

-----------------------

If you use a simpler generator or a generator expression it might be clearer

>>> def cube_gen(x):
for i in range(x):
yield i*i*i

>>> list(cube_gen(5))
[0, 1, 8, 27, 64]

>>> cube_gen = (i*i*i for i in range(5))

>>> cube_gen
<generator object <genexpr> at 0x00ADB060>

>>> list(cube_gen)
[0, 1, 8, 27, 64]

Answer 2

Oh okay, in that case your doubt might be about how generator functions or iterators in general work. You might want to give these answers a read:
https://stackoverflow.com/questions/231767/what-does-the-yield-keyword-do/231855#231855

-------------------

Long story short, a generator function returns an iterator (not a regular iterable like a list or a string). What this means is that you can call the next() function on it, and it will run the code till the next 'yield' and pause there.

You can test this yourself:

>>> x = my_gen()
>>> x
<generator object my_gen at 0x03BC0870>

>>> next(x)
This is printed first
1

>>> next(x)
This is printed second
2

>>> next(x)
This is printed at last
3

>>> next(x)
Traceback (most recent call last):
File "<pyshell#1337>", line 1, in <module>
next(x)
StopIteration

-------------------

This is what a for loop does. It takes the iterator it gets, and keeps calling the next() function on it till a StopIteration exception is raised. If that went over your head, try this article:

http://treyhunner.com/2016/12/JUMP_LINK__&&__python__&&__JUMP_LINK-iterator-protocol-how-for-loops-work/

-------------------

This process is called the iterator protocol, and also happens to be what the list() function does. It goes through the entire iterable (in our case the genertor object) and makes a list out of all the items.

But, while going through the generator, before it gets to the 'yield' statements, the print() statements have to be executed. This is why the three lines are printed, before the list object is returned.

-------------------

If you want to get a better understanding of how iterators and iterables work, check this site:
https://stackoverflow.com/questions/9884132/what-exactly-are-iterator-iterable-and-iteration

Let me know whether any of this made sense

Answer 3

Thank you for the feedback! I guess my question is the format of the first result though.

def my_gen():
n = 1
print('This is printed first')
yield n

n += 1
print('This is printed second')
yield n

n += 1
print('This is printed at last')
yield n

print (list(my_gen()))

interesting thing is that it prints out three lines and one line of list for numbers, and that I have no clue :(

Answer 4

@ Just A Rather Ridiculously Long Username, thank you so much! Your explanation really help me clear things up! :)

Answer 5

No problem, cheers :)