Python can be a tricky language due to the fact that it’s dynamic. Not everyone likes this and it’s true that while it solve one group of problems, it also creates another (there are no solutions in real life, just trade-offs). I’m going to show you what can be a problem and how to solve them using Python’s slots mechanism.
Dynamic traps
First, let’s begin with something obvious. You create a class and want to assign a variable, but you make a typo:
class C:
my_var: int
c = C()
c.myvar = 123
print(c.myvar)
Notice the lack of an underscore. While it can be argued that this can be solved with linters, or explicit setter calling (not the property setters) as in this case we will get an AttributeError:
class C:
_my_var: int
def set_my_var(self, value):
self._my_var = value
def get_my_var(self) -> int:
return self._my_var
c = C()
c.set_myvar(123)
# Result:
# AttributeError: 'C' object has no attribute 'set_myvar'
Another option would be to use a static typed language, but let’s assume we want to avoid it 😉
While encapsulation is a very powerful tool, adding setters and getters for each and every variable is in my opinion just an illusion of proper encapsulation and in such simple case as discussed above there is no reason for using it.
The above problems stem from the fact that Python’s classes use dictionaries under the hood which may be a bit expensive in memory allocation. This leads to the next issue, namely the efficiency of typical python classes.
Python slots to solve memory and attributes
Python has a built-in mechanism called slots that may help solve both of those issues (or at least to make it less problematic). Let’s begin with the unintended attribute assignment:
class C:
__slots__ = ("my_var",)
my_var: int
c = C()
c.myvar = 123
print(c.myvar)
# Results in:
# AttributeError: 'C' object has no attribute 'myvar'
The second issue is the memory efficiency. Let’s profile the usage with the memory profiler library:
from memory_profiler import profile
class C:
my_var: int
@profile
def main():
objects = []
for _ in range(100_000):
c = C()
c.my_var = 123
objects.append(c)
if __name__ == "__main__":
main()
Result:
(venv) ➜ tmp python -m memory_profiler main.py
Filename: main.py
Line # Mem usage Increment Occurrences Line Contents
=============================================================
9 18.9 MiB 18.9 MiB 1 @profile
10 def main():
11 18.9 MiB 0.0 MiB 1 objects = []
12 35.3 MiB 0.0 MiB 100001 for _ in range(100_000):
13 35.3 MiB 5.4 MiB 100000 c = C()
14 35.3 MiB 9.0 MiB 100000 c.my_var = 123
15 35.3 MiB 1.9 MiB 100000 objects.append(c)
And now with a minor upgrade:
from memory_profiler import profile
class C:
__slots__ = ("my_var",)
my_var: int
@profile
def main():
objects = []
for _ in range(100_000):
c = C()
c.my_var = 123
objects.append(c)
if __name__ == "__main__":
main()
The result:
(venv) ➜ tmp python -m memory_profiler main.py
Filename: main.py
Line # Mem usage Increment Occurrences Line Contents
=============================================================
10 19.0 MiB 19.0 MiB 1 @profile
11 def main():
12 19.0 MiB 0.0 MiB 1 objects = []
13 24.5 MiB 0.0 MiB 100001 for _ in range(100_000):
14 24.5 MiB 3.6 MiB 100000 c = C()
15 24.5 MiB 0.0 MiB 100000 c.my_var = 123
16 24.5 MiB 2.0 MiB 100000 objects.append(c)
We got 24.5 MB instead of 35.3 MB – over 10 MB saved.
Final thoughts
While slots are interesting mechanism, its strengths will show off with large data sets (notice that the memory savings were made for 100 000 objects). Other problems can be solved by using libraries like pydantic if you also want to have data serialization, validation etc. If you want to implement own solution, slots may be a valuable supplement for dataclasses.
You may also be interested in: