About the inner workings of class attributes

Question

Please consider this code:

class C:
x = 5
def f(self):
self.x += 1

c = C()
c.f()
print(C.x, c.x)

Alright, so to my understanding, when you refer to an attribute by self and it doesn't exist in the instance, Python will look in the class if there's a class attribute with that name.

This is what should be happening here as well, right?

So since the instance doesn't have x, I expected that either I get an error (referenced before assignment) or the class attribute is changed.

Instead, although += is used, the instance suddenly gets the incremented value as a new *own* attribute while the class variable isn't touched.

So C.x still is 5, and c now suddenly has an x (== 6) of its own.

Can someone explain what exactly is going on here?

Answer 1

You need to think of it like a dictionary amongst which there is a key to a sub dictionary for class_variables and another key for instance_variables.
When you run the code your py file is parsed and the original structure of the class dict is created. When you create an instance of the class a copy of that dict is made. At this time there is only the class level x within the class_variables sub dict. When you run the f() self.x or use c.x etc a new element in the instance_variables sub dict is added with the key x. This now gives you 2 variables at different levels with the name x for this instance.

Answer 2

See this
https://code.sololearn.com/c14ThHy4U2sD/?ref=app

The instance of the class doesn't have an attribute named `x` when it is formed. But when try to access `self.x`, Python first checks if there is an attribute named `x` for the instance, and if not, checks if the class has an attribute `x`. But when you assign to `self.x`, it creates a new attribute.

So when you call
self.x += 1
which can be written as
self.x = self.x + 1

As there isn't an attribute `x` for the instance, it instead takes the class attribute `x`
self.x = C.x + 1
self.x = 5 + 1

Not sure if that's correct but seems logical. Also, I did not understand Théophile completely, but I think he wanted to say the same thing. Correct me if I'm wrong.

Answer 3

You also need to the lookup order of variables via scope etc. Just like when you have a var inside a function with the same name as a var outside a function. Python follows LEGB local, enclosing, global, built-in. For an instance this also holds true. If it's not found in the instance level then it looks at the class level. You can access the class var through a class level method.

Answer 4

ChaoticDawg, exactly, mutable vs. immutable. Quoting myself:

'It happens what was to be expected: When you 'add' something to the list, it's still that list, while if you add to an int, you get another object.'

The interesting thing, though, is that the reference is still written into the instance instead of the class.

I'm interested in the underlying mechanism, which is probably something basic and was purposefully (I hope) designed like that.

Answer 5

Hm, Théophile, I'm not yet sure what to make of it.

z is not in t, but it is in T, which was to be expected since it's a class attribute. *scratches head*

Maybe it's the time (00:15 here). I will look at this again tomorrow...

Answer 6

Thank you, Théophile.

Reading your posts again now shows me you've been basically saying the same (correct) things over and over.

Maybe it's finally time for me to dig deeper into that data model...

Answer 7

Théophile, I knew about the difference between method object and the class's function that is internally called.

I've explained it for practical purposes in here:

https://code.sololearn.com/ce664AekBveN/?ref=app

I didn't know about the whole descriptor business and the background mechanics though.

We're casting a bright light onto that 'data model' in the documentation here - it becomes more and more attractive to get a deeper understanding of the inner workings!

Answer 8

x is not a static variable nor does it behave like a static variable in Java. If this were the case then both calls in
print(C.x, c.x) would return the same value. static class variables in Java are the same across all instances of that class, and only 1 var is held in memory.

https://code.sololearn.com/ceaGDT4gyUlP/?ref=app

variables (they aren't actually variables, but names to a PyObject) in python are held in dictionary like structure (not actually a dictionary, but visualize it as such) for each level of scope, global, class, instance, local, block, etc. When the py file is parsed the class variable x and its value are stored as is. When you create an instance of this class, a copy of that info is made and used when constructing the instance of the class. The variable x for the class and the variable x for the instance point to two different locations in memory. So, when you call c.f() the value of x is changed only for that instance.

Answer 9

Here, maybe this will help you understand my explanation better.

https://code.sololearn.com/ckAvNw2d2Kv3/?ref=app

Answer 10

IDK, never tried, I'd have to look into it. They aren't really dicts though. It doesn't actually happen exactly how I described, but that is an abstract way of looking at it to help with the understanding of what's happening.

Answer 11

Okay... that's really not so easy to understand. 😅

Keep me updated!

I'll also try to investigate more a bit later.

Answer 12

~ swim ~, wow, that's one more thing to think closely about.

Is this behavior controlled by duck typing?

So may it be different for other build-in types as well?

(Man, you really shouldn't take a look under that hood - there's an abyss staring back at you. 🤣)

Answer 13

You can access and assign them separate from each other. It does however, look like if the instance attribute doesn't exist when called with +=, -=, *=, /=, //=, **=, %=, etc if the key, value isn't found then it will go up the "scope ladder" and check if it exists, get its value, set its value and add it to the instance attributes.

https://code.sololearn.com/cj5IgNYy8Cwi/?ref=app

Answer 14

IDK, if it has anything to do with duck typing. I was envisioning that it may be some catch block cause-effect thing going on.

It seems like I watched a PyCon video on this some time ago, but apparently I didn't retain that info. Lol

~ swim ~ that looks like some good reading that I'll have to get around to at some point. These eyes are burning too much from lack of sleep and staring at my screens too long. Lol

Answer 15

I don't think it's that complicated, ChaoticDawg.

By using instance methods or classmethods, you have the referred-to object, either instance or class, available as the implicitly passed first argument, so either self or cls.

That's just how Python's OOP is set up.

And this is what you're accessing from the methods.

The method object passes the call on to the class, c.f(x) becoming C.f(c, x).

So writing self.x is the very same thing as writing c.x, and writing cls.x is the same as writing C.x.

You are just writing dynamically into either the class or the instance, and the other doesn't necessarily know about it.

If you had tried to access c.z, after you had stored z in C, c would find it.

First it would search in its own place, and when it didn't find anything, it would look on in C where it's stored.

Answer 16

HonFu when doing :
self.x += 1
Three things are happening :
- 1st : python performs a getattribute on the class instance. It finds nothing, so it checks the class variables. It founds 'x' and return its value
- 2nd : integers are immutable, so '+ 1' creates a new integer
- 3rd : python performs a setattr on the class instance, with 'x' as key and the new integer as value (remember : __setattr__(self, key, value)). Python adds this new variable to the instance dictionary.

Answer 17

https://code.sololearn.com/cYj45ayaHntk/?ref=app

You can see better the steps... And what's happening inside.

Answer 18

Hmmm, that's probably due to mutable vs immutable objects, similar to how functions react with default values. Will to do some more testing and research, but I got to get a bit of rest.

Answer 19

https://code.sololearn.com/clGoyqWOc9Ic/?ref=app

OK, I try to explain everything here, through multiple examples... I hope you will understand.

Answer 20

HonFu Yes, I'm aware of everything you stated. I think I covered all of that already in my previous posts. I was just hypothesizing as to the underlying reason as to the behavior. I just provided the code bit so it could be seen in the class and instance dicts.