17 Nov 2013 by Ben. We’ve mentioned before how Oyster.com’s Python-based web servers cache huge amounts of static content in huge Python dicts (hash tables). Well, we recently saved over 2 GB in each of four 6 GB server processes with a single line of code — using slots on our Image class. Here’s a screenshot of RAM usage before and after deploying this change on one of our servers. Our first test will look at how they allocate memory. Our second test will look at their runtimes. This benchmarking for memory and runtime is done on Python 3.8.5 using the modules tracemalloc for memory allocation tracing and timeit for the runtime evaluation. Results may vary on your personal computer.

Originally posted on towardsdatascience.

Python Slots it. For instance, you may get a $25 no deposit bonus with a 30x wagering requirement. This means you will have to wager a total of Python Slots $750 – 30 times $25 – to cashout the maximum cap winning amount. But slots in the python exists from version 2.2. It's main purpose is memory optimisation because it allow to get rid of dict object, which created by interpreter for each instance of class. If you create many small objects, with predefined structure, and meet a memory limit, than slots can help you to overcome that limitation. An OS-specific virtual memory manager carves out a chunk of memory for the Python process. The darker gray boxes in the image below are now owned by the Python process. Python uses a portion of the memory for internal use and non-object memory. The other portion is dedicated to object storage (your int, dict, and the like). Note that this was.

A simple way to improve your Python code

When we create an object from a class, the attributes of the object will be stored in a dictionary called __dict__. We use this dictionary to get and set attributes. It allows us to dynamically create new attributes after the creation of the object.

Let’s create a simple class Article that initially has 2 attributes date and writer. If we print out __dict__ of the object, we will get the key and value of each attribute. Meanwhile, we also print __dict__ of the class which will be needed later. After that, a new attribute reviewer is added to the object, and we can see it in the updated __dict__.

Good enough?

Well, we can’t say this is bad until we find a better solution. Dictionary is very powerful in Python, but when it comes to creating thousands or millions of objects, we might face some issues:

1. Dictionary needs memory. Millions of objects will definitely eat up the RAM usage.
2. Dictionary is in fact a hash map. The worst case of the time complexity of get/set in a hash mapis O(n).

__slots__ solution

From Python documentation: __slots__ allows us to explicitly declare data members (like properties) and deny the creation of __dict__ and __weakref__ (unless explicitly declared in __slots__ or available in a parent.)

So, how does it relate to the issues I’ve mentioned?

Let’s create a class ArticleWithSlots. The only difference between 2 classes is the extra field __slots__.

__slots__ is created on the class level, which means if we print ArticleWithSlots.__dict__, we should be able to see it. Besides, we also see 2 extra attributes on the class level, date: <member 'date' ..> and writer: <member 'writer' ..>, which belong to class member_descriptor.

What is a descriptor in Python?

Before we talk about descriptor, we should understand the default behaviour of accessing attributes in Python. When you do article.writer, Python will call the method __getattribute__(), where it does a look up in __dict__, self.__dict__['writer'] and return the value.

If the look up key is an object with one of the descriptor methods, then the default behaviour will be overwritten by the descriptor method.

Descriptor methods include __get__(), __set__() and __delete__(). And a descriptor is simply a Python object that implements at least one descriptor methods.

__slots__ automatically creates a descriptor for each attribute with the implementation of descriptor methods. You can find them in the screenshot. It means that the object will use __get__(), __set__() and __delete__() to interact with attributes instead of the default behavior.

According to Guido van Rossum, the implementation of __get__(), __set__() uses an array instead of the dictionary and it’s entirely implemented in C which is highly efficient.

__slots__ has faster attribute access

In the following code, I compare the object creation time and attribute access time of Article and ArticleWithSlots. __slots__ is around 10% faster.

__slots__ has slightly better performance is because thetime complexity of get/set operationin a list is faster than a dictionary in the worst case. As O(n) only happens in the worst case, we will not notice the difference most of the time, especially when you have a small volume of data.

__slots__ reduces RAM usage

Since attributes can be accessed as data members (like properties), there is no need to store them in the dictionary __dict__. Actually, __slots__ will deny the creation of __dict__ at all. So if you print article_slots.__dict__, you will get the AttributeError exception.

And this behavior reduces the RAM usage of an object. I will compare the size of article and article_slots using pympler. The reason for not using sys.getsizeof() is that getsizeof() doesn’t include the size of referenced objects. However, __dict__ is a referenced object which will be ignored in getsizeof().

It turns out that article_slotssaves more than 50% of the memory. WoW, such an amazing improvement!

Such a good performance is because article_slots doesn’t have __dict__ attribute which actually saves a lot of memory.

When to use and not use __slots__?

So far, it looks like __slots__ is such a nice feature. Can we add it to every class?

The answer is NO! Apparently, there are some trade-offs.

Fixed attributes

One of the reasons to use __dict__ is its flexibility after creating the object where you can add new attributes. However, __slots__ will fix the attributes when you create the class. So, it’s not possible to add new attributes later.

But …

In some cases where you want to take advantage of __slots__, and also have the flexibility of adding new attributes in the runtime. You can achieve this by adding __dict__ inside __slots__ as an attribute. Only the newly added attributes will appear in __dict__. This can be useful when your class has 10+ fixed attributes and you want to have 1 or 2 dynamic attributes later.

Inheritance

Slot Machine In Python

If you want to inherit a class that includes __slots__, you don’t have to repeat those attributes again in the subclass. Otherwise, the subclass will take up more space. Besides, the repeated attributes will be inaccessible in the parent class.

It works the same when you inherit a NamedTuple. You don’t need to repeat attributes in the subclass. If you want to understand more about NamedTuple, you can read my article dedicated to this topic.

You can also add __dict__ attribute in the subclass. Or, you don’t put __slots__ in the subclass, it will by default have __dict__.

If you inherit a class without __slots__, then the subclass will contain __dict__.

Conclusion

Monty Python Slots

I hope you’ve understood what __slots__ is and some details of the implementation. At the end of the article, I want to share the pros and cons that coming from my own experience and the internet (linked in the Reference).

pros

__slots__ can be definitely useful when you have a pressure on memory usage. It’s extremely easy to add or remove with just one-line of code. The possibility of having __dict__ as an attribute in __slots__ gives developers more flexibility to manage attributes while taking care of the performance.

cons

You need to be clear about what you are doing and what you want to achieve with __slots__, especially when inheriting a class with it. The order of inheritance, the attribute names can make a huge difference in the performance.

You can’t inherit a built-in type such as int, bytes, tuple with non-empty __slots__. Besides, you can’t assign a default value to attributes in __slots__. This is because these attributes are supposed to be descriptors. Instead, you can assign the default value in __init__().

Source: towardsdatascience

Introduction

In Python, every object instance comes pre-built with standard functions and attributes. For example, Python uses a dictionary to store an object's instance attributes. This has many benefits, like allowing us to add new attributes at runtime. However, this convenience comes at a cost.

Dictionaries can consume a fair chunk of memory, especially if we have many instance objects with a large number of attributes. If the performance and memory efficiency of code are critical, we can trade the convenience of dictionaries for __slots__.

In this tutorial, we will look at how what __slots__ are and how to use them in Python. We'll also discuss the tradeoffs for using __slots__, and look at their performance when compared to typical classes that store their instance attributes with dictionaries.

What Are _slots_ and How to Use Them?

Slots are class variables that can be assigned a string, an iterable, or a sequence of strings of the instance variable names. When using slots, you name an object's instance variables up front, losing the ability to add them dynamically.

An object instance using slots does not have a built-in dictionary. As a result, more space is saved and accessing attributes is faster.

Let's see it in action. Consider this regular class:

In the above snippet:

• organization is a class variable
• name and location are instance variables (note the keyword self in front of them)

While every object instance of the class is created, a dynamic dictionary is allocated under the attribute name as __dict__ which includes all of an object's writable attributes. The output of the above code snippet is:

This can be pictorially represented as:

Now, let's see how we can implement this class using slots:

In the above snippet:

• organization is a class variable
• name and location are instance variables
• The keyword __slots__ is a class variable holding the list of the instance variables (name and location)

Running that code will give us this error:

That's right! Object instances of classes with slots do not have a __dict__ attribute. Behind the scenes, instead of storing the instance variables in a dictionary, the values are mapped with the index locations as shown in the figure below:

While there's no __dict__ attribute, you still access the object properties as you would typically do:

Slots were created purely for performance improvements as stated by Guido in his authoritative blog post.

Let's see if they outperform standard classes.

Efficiency and Velocity of Slots

We are going to compare objects instantiated with slots to objects instantiated with dictionaries with two tests. Our first test will look at how they allocate memory. Our second test will look at their runtimes.

This benchmarking for memory and runtime is done on Python 3.8.5 using the modules tracemalloc for memory allocation tracing and timeit for the runtime evaluation.

Results may vary on your personal computer:

In the above snippet, the calculate_memory() function determines the allocated memory, and the calculate_runtime() function determines the runtime evaluation of the class with slots vs the class without slots.

The results will look something along these lines:

It's evident that using __slots__ gives an edge over using dictionaries in size and speed. While the speed difference is not particularly noticeable, the size difference is significant.

Python Memory Release

Gotchas Using Slots

Before you jump into using slots in all of your classes, there are a few caveats to be mindful of:

1. It can only store attributes defined in the __slots__ class variable. For example, in the following snippet when we try to set an attribute for an instance that is not present in the __slots__ variable, we get an AttributeError:

Output:

With slots, you need to know all the attributes present in the class and define them in the __slots__ variable.

2. Sub-classes will not follow the __slots__ assignment in the superclass. Let's say your base class has the __slots__ attribute assigned and this is inherited to a subclass, the subclass will have a __dict__ attribute by default.

Consider the following snippet where the object of the subclass is checked if its directory contains the __dict__ attribute and the output turns out to be True:

Output:

This can be averted by declaring the __slots__ variable one more time for the subclass for all the instance variables present in the subclass. Although this seems redundant, the effort can be weighed against the amount of memory saved:

Output:

Conclusion

In this article, we have learned the basics about the __slots__ attribute, and how classes with slots differ from classes with dictionaries. We also benchmarked those two classes with slots being significantly more memory efficient. Finally, we discussed some known caveats with using slots in classes.

If used in the right places, __slots__ can boost performance and optimize the code in making it more memory efficient.