How to troubleshoot high CPU and memory usage on a Linux server

To find what's using CPU or memory on a Linux server, run top and press P to sort by CPU or M to sort by memory; the process at the top is your answer most of the time. For a snapshot you can paste into a ticket, ps -eo pid,user,%cpu,%mem,rss,etime,comm --sort=-%cpu | head does the same without the full screen.

The harder part is reading the numbers right: a high load average isn't always CPU, and "used" memory isn't the number that matters. This tutorial goes from the quick look to the details, in the order you'd use them. We ran every command on Ubuntu 24.04 with a CPU load generator and a deliberately leaking process; the same tools exist on Debian, AlmaLinux and Rocky Linux.

The one-minute check#

uptime
free -h

 14:32:10 up 12 days,  3:04,  1 user,  load average: 2.39, 2.12, 1.18
               total        used        free      shared  buff/cache   available
Mem:           3.8Gi       1.2Gi       0.4Gi        20Mi       2.2Gi       2.4Gi
Swap:          2.0Gi          0B       2.0Gi

Load average is the number of processes running or waiting, averaged over 1, 5 and 15 minutes. Compare it with the number of vCPUs (nproc): on a 2-vCPU server, a load of 2 means fully busy, and a load staying above it means work is queueing. The three numbers show the trend; here it's rising.

available is the memory programs can still get, including cache the kernel will give back. Ignore free: Linux uses spare memory as disk cache on purpose, so free is always low on a healthy server. Worry when available is small and swap is in use.

Find the CPU hog#

top -o %CPU

Or install htop (sudo apt install htop or sudo dnf install htop), which is easier to read: F6 to sort, F4 to filter by name, F9 to send a signal, q to quit.

In top, the line that starts %Cpu(s) breaks the time down:

%Cpu(s): 10.6 us,  5.9 sy,  0.0 ni, 80.3 id,  0.0 wa,  0.0 hi,  3.1 si,  0.0 st

A snapshot sorted by CPU, with how long each process has been running:

ps -eo pid,user,%cpu,%mem,rss,etime,comm --sort=-%cpu | head -n 10

%cpu in ps is the average over the process's whole life, which hides a short spike. For the last few seconds, use pidstat from the sysstat package:

sudo apt install -y sysstat     # or: sudo dnf install -y sysstat
pidstat 1 5

It prints per-process CPU every second for five seconds, then an average.

Is it CPU, disk or memory?#

vmstat answers that in one table:

vmstat 1 5

procs -----------memory---------- ---swap-- -----io---- -system-- -------cpu-------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st gu
 5  0      0 19727940 453520 8053068    0    0    18   186  696    0  0  0 100  0  0  0
 6  0      0 19642132 453620 8094520    0    0     0 60780 49394 81056 16  9 74  0  0  0

Ignore the first line (averages since boot). Then:

• r above the vCPU count: CPU-bound.
• b above zero, with high wa: processes blocked on the disk. Find the writer with sudo iotop or pidstat -d 1.
• si and so above zero for more than a moment: the server is swapping, so it's short of memory.

The kernel's own pressure counters say the same thing directly. some avg10= is the share of the last 10 seconds that at least one task was stalled:

cat /proc/pressure/cpu /proc/pressure/memory /proc/pressure/io

Find the memory hog#

ps -eo pid,user,%mem,rss,comm --sort=-rss | head -n 10

rss is resident memory in KB. For services, systemd can total it per unit, including all child processes, which is often clearer:

systemd-cgtop --order=memory

A process whose memory only ever grows over hours or days is probably leaking. Restarting it buys time; the fix is in the application. In the meantime, cap it so it can't take the whole server down with it, in its systemd unit (see write a systemd service):

[Service]
MemoryMax=1G

Did the OOM killer strike?#

When memory runs out completely, the kernel's out-of-memory killer picks a process and kills it. It's logged in the kernel log:

sudo journalctl -k | grep -iE 'out of memory|killed process'

kernel: Memory cgroup out of memory: Killed process 9171 (python3) total-vm:83492kB, anon-rss:65280kB, ...

Out of memory: Killed process means the whole server ran out; Memory cgroup out of memory means a unit hit its own MemoryMax. systemd also records it for the service:

systemctl status myapp

Active: failed (Result: oom-kill)

If a database or your app keeps getting killed, either give the server more memory, lower what the app may use (worker counts, cache sizes, shared_buffers), or add swap as a buffer for short peaks.

Deal with the process#

Once you know the culprit:

sudo systemctl restart nginx          # a service: restart it the proper way
sudo kill 12345                       # a stray process: ask it to stop (SIGTERM)
sudo kill -9 12345                    # only if it ignores that
sudo renice +10 -p 12345              # keep it running, at lower priority

Then find out why. The process's logs (journalctl -u name --since "1 hour ago"), a cron job or timer that runs at that time (systemctl list-timers, crontab -l), or plain traffic: the web server's access log shows if a crawler or an attack is behind it.

An unknown process using 100% CPU under a strange name, often from /tmp or /dev/shm, is a classic sign of a cryptominer on a compromised server. Don't just kill it: read what to do if your server is compromised.

Troubleshooting#

Load is high but CPU is mostly idle. Look at wa and the b column in vmstat: processes are waiting for the disk (or for a slow network file system), and they count towards load too.

top shows high st (steal). The hypervisor is giving CPU time to other virtual machines. A few percent now and then is normal on any shared host; sustained high steal isn't something you can fix inside the server, so tell your provider with the times you saw it.

Memory used is high but nothing in ps is big. Check buff/cache and available in free -h first. If available really is low, look at Shmem and Slab in /proc/meminfo (tmpfs files and kernel caches), and at containers: docker stats.

pidstat: command not found. Install sysstat.

Next steps#

• Keep a history so the next spike can be explained: monitor with Netdata.
• Read what the process logged at the time: journalctl.
• Slow because the disk is full? Find and free disk space.

Doing it on VPSNine#

The VPSNine panel graphs each server's CPU and memory and can email you when either stays high, with resource alerts, so you hear about a spike before your users do. If you've tuned everything and the server is still short, the larger plans have more vCPUs and memory. Runs the same on Ubuntu 24.04 and every other image.