Evgenii Legotckoi
Evgenii LegotckoiApril 19, 2022, 3:39 a.m.

Django - Tutorial 058. Database Growth Due to django_session Table

Lately, I have noticed that the fatal moment is approaching, when the disk space for the site on the hosting will catastrophically cease to be enough. And the database dump becomes incredibly huge, although there are no obvious prerequisites for this. The content size of the site isn't growing that fast, and the number of registered users isn't growing that fast either.

After examining the database, it was found that the size of the django_session table is just a gigantic almost 7 GB, and the size of the index also reaches almost 6.5 GB, despite the fact that the size of the database itself is 14 GB.

At the same time, the size of the second largest table is only 11 MB and this is a third-party application with a list of cities. And the size of the third table, which contains messages on the forum, is only 8 MB.

Accordingly, it was decided to figure out why this is happening and how to fix it.

Right now, I probably won’t reveal anything new to those who actively administer PostgreSQL databases, but for beginners and those who mainly deal with Django as a PET project, without professional use, the information may be useful.


How to check table size

https://evileg.com/ru/knowledge/article/add/#
To do this, simply execute the following query in the PostreSQL administration interface. And we get a sorted output of information on the database tables.

  1. select table_name, pg_relation_size(quote_ident(table_name)), pg_size_pretty(pg_relation_size(quote_ident(table_name))) from information_schema.tables where table_schema = 'public' order by 2;
  2.  
  3. forum_forumpost | 8290304 | 8096 kB
  4. cities_light_city | 11108352 | 11 MB
  5. django_session | 7225204736 | 6890 MB
  6. (110 rows)

As you can see, in my case, the djang_session table has grown very much over the 6 years of the site's existence on the Django engine.
Thanks to DDOS visitors, the mechanism for creating session keys for all anonymous users, and the fact that by default PostgreSQL does not reduce the size of the database file even when deleting records.

And the size of the database can be seen like this

  1. SELECT pg_size_pretty( pg_database_size('databasename') );
  2. pg_size_pretty
  3. ----------------
  4. 14 GB
  5. (1 row)
  6.  

Here is such an unpleasant size came out - 14 GB.

Deleting expired sessions

When a site is DDOSed or simply flooded with users, a huge number of sessions are created that are usually not deleted in Django, and the table index grows additionally.

Therefore, the first thing to do is to remove obsolete sessions. Django has the clearsessions command for this.

Therefore, in the console we activate the python environment of your project, go to the folder with your project and execute the following command.

  1. python manage.py clearsessions

This will delete all old sessions. You can also schedule this command to run via cron.
For example, using the django-session-cleanup battery, it requires the use of celery .

Run the garbage collector

After you have completed the removal of old sessions, you need to free up the space occupied by the database.
This is necessary because the database's priority is performance over disk space savings. Thus, the database file grows due to the growth of the index, and the data has not been there for a long time. And also, by default, the garbage collector does not start by itself, for this you need to configure it to start on a schedule, for example, using a daemon.

But personally, I've done it manually so far. The garbage collector in PostgreSQL is started with the vacuum command.

  1. vacuum FULL ANALYZE django_session;

After performing this operation, I check the size of the database again and see

  1. SELECT pg_size_pretty( pg_database_size('databasename') );
  2. pg_size_pretty
  3. ----------------
  4. 494 MB
  5. (1 row)
  6.  

Now the database size is only 494 MB, which is good news.

It will be necessary over time to configure the garbage collector to run at least once a week, but more on that in the next article.

We recommend hosting TIMEWEB
We recommend hosting TIMEWEB
Stable hosting, on which the social network EVILEG is located. For projects on Django we recommend VDS hosting.
u
  • May 15, 2022, 5:58 p.m.
  • (edited)

А если хранить сессии в SESSION_ENGINE = 'django.contrib.sessions.backends.signed_cookies' ?
Интересно стало насколько безопасно хранить сессии в печеньках... стоит оно того или нет?)

Evgenii Legotckoi
  • May 15, 2022, 7:27 p.m.

Думаю, что скорее всего это будет менее безопасно, но на практике я не проверял.

Comments

Only authorized users can post comments.
Please, Log in or Sign up
Ua

Qt - Test 001. Signals and slots

  • Result:84points,
  • Rating points4
Ua

Qt - Test 001. Signals and slots

  • Result:42points,
  • Rating points-8
ОК

Qt - Test 001. Signals and slots

  • Result:47points,
  • Rating points-6
Last comments
ИМ
Игорь МаксимовNov. 22, 2024, 9:51 p.m.
Django - Tutorial 017. Customize the login page to Django Добрый вечер Евгений! Я сделал себе авторизацию аналогичную вашей, все работает, кроме возврата к предидущей странице. Редеректит всегда на главную, хотя в логах сервера вижу запросы на правильн…
Evgenii Legotckoi
Evgenii LegotckoiOct. 31, 2024, 11:37 p.m.
Django - Lesson 064. How to write a Python Markdown extension Добрый день. Да, можно. Либо через такие же плагины, либо с постобработкой через python библиотеку Beautiful Soup
A
ALO1ZEOct. 19, 2024, 5:19 p.m.
Fb3 file reader on Qt Creator Подскажите как это запустить? Я не шарю в программировании и кодинге. Скачал и установаил Qt, но куча ошибок выдается и не запустить. А очень надо fb3 переконвертировать в html
ИМ
Игорь МаксимовOct. 5, 2024, 4:51 p.m.
Django - Lesson 064. How to write a Python Markdown extension Приветствую Евгений! У меня вопрос. Можно ли вставлять свои классы в разметку редактора markdown? Допустим имея стандартную разметку: <ul> <li></li> <li></l…
d
dblas5July 5, 2024, 8:02 p.m.
QML - Lesson 016. SQLite database and the working with it in QML Qt Здравствуйте, возникает такая проблема (я новичок): ApplicationWindow неизвестный элемент. (М300) для TextField и Button аналогично. Могу предположить, что из-за более новой верси…
Now discuss on the forum
f
firstlunoxodFeb. 15, 2025, 1:46 p.m.
Рисование на QGraphicsScene при зажатой кнопке мыши Подскажите, пожалуйста! Как данный класс можно дополнить, чтобы созданные объекты можно было перемещать мышкой по сцене?
Дмитрий
ДмитрийFeb. 3, 2025, 4:24 p.m.
Создание deb-пакета. Как создать ярлык на рабочем столе после установки собственного deb-пакета? Всем привет. Сделал свой deb-пакет с программой. Всё устанавливается и работает. Ставлю по пути /usr/bin/my_application. Как для пользователя при установке пакета сразу создать ярлык на раб…
NW
Nayo WaiJan. 30, 2025, 7:22 p.m.
не запускается компьютер!!! Не запускается компьютер (точнее работает блок , но сам монитор вообще жесть)В общем я ничего с интернета не скачивала в последнее время. На компе никаких левых пр…
n
nklyJan. 3, 2025, 12:52 p.m.
Нужно запретить перемещение только некоторых итемов, остальные перемещать можно. Вопрос решен. Узнать QModelIndex элемента на который мы перетаскиваем другой элемент, можно с помощью функции indexAt(event->position().toPoint()) представления QTreeViev вызываемой в переопр…
M
MarselAug. 17, 2023, 12:26 a.m.
OAuth2.0 через VK, получение email Спасибо большое за помощь и простите за то что отнял время своей невнимательностью.

Follow us in social networks