Python3: Mutable, Immutable…
Introduction:
For a lot of us, Python is a beautiful programing language. It’s easy to pick up, very readable, and easy to maintain. But the main reason why it’s so easy is that there’s so much under the hood. While many people are satisfied with being aware of the complexity of a language without needing to be well acquainted with those complexities, understanding the in’s and out’s of Python will help prevent you from creating major bugs in your programs. The aim of this article is to familiarize you with these details.
Objects
Let’s start off with the glue that holds everything together: objects. Literally everything is an object. Your module is an object, a function is an object, an integer is an object. Objects are simply instances of classes. We can think of objects as individuals while classes are the groups to which they belong. We are all humans (class human), but I am a unique individual (an instance of class human).
In Python, everything is an object. Every object has its own data type and internal state (data). Let us first try to understand how data is stored in the memory with below two examples.
Example 1: When Python executes a = 1000, this creates an integer object at some memory location say 0x1111 & stores value 1000. Here a doesn’t contain value 1000 but just holds the reference (memory address) to this object. When the next statement a=1500 is executed, another integer object is created at memory location 0x2222 with value 1500 and a now references this new object.
a = 100
a = 1500
Example 2: When Python executes a = 2000, as explained in the above example, it creates an integer object at memory location 0x3333 and a will now point to this object. After the execution of b = a, now b also points to the same memory location a is referring to. Note that the value of a is not copied to b but the reference (address) is copied to b.
a = 2000
b = a
Identity and Type:
Before we dive into the intricacies of objects in Python, let’s start off with some background on the two built-in functions: id() and type(). These two functions can give us crucial information about a variable in question. The type function will return the data type of the object, which is most often the same as the class that the object belongs to. For example, if I have a list l, performing type(l) will return the class list:
The id function will return the variable identifier of the variable of your choosing. All variables in Python refer to an object and the variable identifier is an integer number that refers to a specific object. In the CPython implementation, this number is the memory address of the object. The id of an object is used to help differentiate between two variables being identical and them being linked to the same object. We can use “==” to determine if they are identical, while “is” can be used to determine if both variables are pointing to the same object. If we have two lists x and y, we can see their ids are unique:
Therefore:
But if we do this, things start to look weird:
“Wait wait wait !!! What??” you might say, “Two seconds ago I thought those two lists were considered to be different. Why are these the same?” To answer that question, we must go deeper into the subject of mutability.
Mutable Objects:
So, what is mutability? Simply said it is the ability to mutate — or alter — an object. Some built-in mutable types in Python include lists, sets, dicts, and bytearrays. Let’s look at an example with our good friend the list. We can append items to our list:
We can remove items from our list:
Like I said before, everything in Python is an object. So every element in the list is also an object. If we start looking a little closer, things start to get weird again:
Why did l[1]’s id change? The list’s id didn’t change when we altered its values, so why would its element? It’s because integers and strings are not mutable.
Immutable Objects:
Immutability is exactly what it sounds like: the opposite of mutability. This means that you cannot change the contents of the object. Some built-in immutable types include ints, floats, complexes, strings, tuples, frozensets, and bytes. So in the above example, the list object initially contained references to the objects “1”, “2”, and “3”. Since we cannot change the object itself, the list changed which reference it held, such that it now contains a reference to the object “Hello”. In other words, a reassignment occurred. While we cannot append an immutable object, we can still do:
While it seems like we appended “Hello” to “ World”, what actually happened was something more along the lines of:
s = s + “ World”
The value of s, “Hello”, plus “ World” was calculated and s was given the reference to this new object. A similar process happens when adding numbers:
If you try to change an element in an immutable set, however, the interpreter will raise an exception:
Similarly, while you cannot actually remove an element in an immutable object, you can slice the object, which will return a new object containing the desired values:
Since Python must create a separate object for each unique immutable value (which takes up a lot of memory) the interpreter cleverly optimizes its creation of objects. It does so by using the same reference number for objects that cannot change, such as strings:
In the CPython interpretation, it goes one step further and stores all common small int objects (-5(inclusive) to 257(exclusive)) in an array for easy reference. Unlike other objects, which are created as needed and destroyed once there are no more variables referencing that object, these small int objects remain throughout the program.
Preallocation in Python
Now, some homework for you:
1. Create two variables with values between -5 and 256 and then check if they reference the same object.
2. Do the same as above but use values for the variables out of the range above.
What happened?
In Python, upon startup, Python3 keeps an array of integer objects, from -5 to 256. For example, for the int object, macros called NSMALLPOSINTS and NSMALLNEGINTS are used. Let’s go check the source code:
#ifndef NSMALLPOSINTS
#define NSMALLPOSINTS 257
#endif
#ifndef NSMALLNEGINTS
#define NSMALLNEGINTS 5
#endif
#if NSMALLNEGINTS + NSMALLPOSINTS > 0
/* References to small integers are saved in this array so that they
can be shared.
The integers that are saved are those in the range
-NSMALLNEGINTS (inclusive) to NSMALLPOSINTS (not inclusive).
*/
static PyIntObject *small_ints[NSMALLNEGINTS + NSMALLPOSINTS];
#endif
#ifdef COUNT_ALLOCS
Py_ssize_t quick_int_allocs;
Py_ssize_t quick_neg_int_allocs;
#endifWhat does this mean? This means that when you create an int from the range of -5 and 256, you are actually referencing the existing object.
This is done so as to avoid to creating objects that are commonly used and because — in this way — you can represent any ASCII character.
Why Does It Matter?
When assigning variables, understanding the differences between mutable and immutable objects is key. Mutable objects have quite some tricks up their sleeves. In an earlier example we showed:
However, if we assign y like so:
How is y the same object as x? I thought they were different objects before? Well, the way in which assignments occurs in Python is crucial. In this case, we have not assigned a new object to y, but instead aliased it to x; meaning that we gave it the same reference as x. Since both now point to the same object, a change in one will affect a change in another:
But what if you want to make a copy of your object without worrying about altering your original? Just like how slicing of immutable objects returns a new object, so does the slicing of lists:
Appending to lists can be a bit tricky as well. In Python, whatever is evaluated on the right side of the assignment expression is then referenced by whatever is on the left side of the expression. Therefore:
A new list is created since the right side of the assignment was evaluated as [1, 2, 3, 4, 5] and that new object was made referenced by l. However, if we do:
We can see that the object remains the same. This is called assignment in-place, and is equivalent to appending to the list. However, since immutable objects cannot be changed, the expressions a+= b and a = a + b would function the same, similar to our “Hello World” example above.
Passing Arguments to Functions:
In other programming languages, arguments are often said to be passed one of two ways: by value (wherein a new variable is given the same value as the argument passed) or by reference (wherein a variable holds a reference to the data such that it could be changed). Python, however, is unique. Python passes the reference to the object. Similar to how we aliased x = y, the function’s argument is aliased to the object reference by the variable passed to the function. This means that if a reassignment occurs inside of the function, it will not alter the original variable passed to the function. Since immutable types cannot be altered any change to the variable inside of the function will not persist:
However, since lists are mutable, you can alter their contents:
Notice how the reference to the list remained the same throughout the program. If we were to reassign l inside the function, however, it would not affect the original list:
Exceptions in immutability
Not all of the immutable objects are actually immutable. Confused? Let me explain.
As discussed earlier, Python containers liked tuples are immutable. That means value of a tuple can't be changed after it is created. But the "value" of a tuple is infact a sequence of names with unchangeable bindings to objects. The key thing to note is that the bindings are unchangeable, not the objects they are bound to.
Let us consider a tuple t = (‘holberton’, [1, 2, 3])
The above tuple t contains elements of different data types, the first one is an immutable string and the second one is a mutable list.The tuple itself isn’t mutable (i.e. it doesn’t have any methods for changing its contents). Likewise, the string is immutable because strings don’t have any mutating methods. But the list object does have mutating methods, so it can be changed. This is a subtle point, but nonetheless important: the “value” of an immutable object can’t change, but it’s constituent objects can.
Conclusion
Having a grasp of the ins and outs of a language can be immensely helpful. While many programmers won’t take the time to become familiar with the languages they use on a daily basis, taking the time to become acquainted with Python is a great way to avoid bugs and get a leg up on the competition. In addition to this, having the discipline to sit down and learn about what goes on under the hood of a language is a great skill that will stand you in good stead as you progress on your coding journey. I hope this article has been helpful, please leave any feedback you have in the comments section!
