Out of memory on a VPS: what the OOM killer is and what to do

When memory and swap run out, the Linux kernel picks a process and kills it to save the system. It is called the OOM killer (out of memory). It usually picks one of the biggest, and on a web server that is often the database. The symptom you see is a service that vanishes without warning and “comes back” once restarted. To confirm, look for the mark in the kernel log; to fix it, find out who is using the memory and shrink it, or give the machine more.

Confirming it was this

1 Look for the mark in the kernel log: sudo dmesg -T | grep -i -E "out of memory|killed process" or sudo journalctl -k | grep -i "killed process". It shows the name of the killed process and the time.
2 See the memory state: free -h. The column that matters is available, not “free”: Linux uses free memory for cache, which is normal.
3 See who spends it: ps aux --sort=-%mem | head lists the ten biggest. Watch for many identical processes (PHP-FPM, Apache) and for programs that grow over time.
4 Match it with the time. If the kill coincides with a traffic peak, a backup or a cron job, you have the cause. Seeing what uses CPU, memory and disk explains how to follow this through the day.

The remedies, cheapest first

What to do When it fits
Reduce what runs Turn off services you do not use; look at what starts with the server (systemctl list-unit-files --state=enabled).
Tune the web server and PHP workers Each PHP-FPM or Apache process takes memory; if the allowed maximum (in PHP-FPM, pm.max_children) times the size of each exceeds the memory, the server dies in a peak. Lower the maximum.
Tune the database MariaDB and MySQL reserve memory for cache (in InnoDB, innodb_buffer_pool_size). A value meant for a bigger machine smothers a small one.
Cap one service In systemd, MemoryMax= in a unit (create an override with sudo systemctl edit name). A runaway service is stopped before it drags the others down. For containers, Docker has the --memory option.
Fix the cause in the application A memory leak, a query that loads everything, a plugin that got heavy. The OOM is just the warning.
Have swap Gives slack for short peaks, but it is disk, and far slower. A cushion, not a cure. How to add it: adding swap to a VPS.
More memory When normal use no longer fits. See the plans at VPS servers and what your service allows in Meu Interweb.
If the database was killed mid-write, check it after starting it. In the database log look for recovery messages or “crashed” tables, and check the site. See tables marked as crashed. A recent copy is what saves it: backing up with mysqldump and pg_dump.
It is not the same as PHP’s “Allowed memory size exhausted”. That is PHP’s own limit for one script, and the server stays up: PHP’s memory limit. The OOM is the whole system’s.
Before buying more memory, measure. Many cases are solved by lowering one configuration number. And if you plan to run heavy programs (Docker, several sites, browser automation), add up what each needs, according to their documentation, before choosing the plan. See also common VPS and Cloud errors.

A service keeps dying and you cannot see why? Paste us the kernel log lines and the plan name, and we will tell you what we see from our side.

Open a support ticket

SEE ALSO

Seeing what uses CPU, memory and disk

A service will not start: reading systemctl status

Common VPS and Cloud errors

VPS, Cloud or Dedicated: how to choose

RECOMMENDED PRODUCT

VPS server with root access

Resources of your own, the OS you choose, reinstall whenever you like. from 7.560,00 Kz/mo (3-year plan, with coupon)

See plans
  • 0 Users Found This Useful
Was this answer helpful?