[tutorial] How to use psutil to monitor CPU and memory usage in python

What is psutil?

psutil is an open source package for python, which can help us to get the information of the system (CPU, memory, disks, network etc..).

The installation of psutil

pip install psutil

Usefull functions in psutil

CPU

psutil.cpu_count() # get the number of CPU
psutil.cpu_percent() # get the usage of all CPU
psutil.cpu_percent(percpu=True) # get the usage of per CPU
psutil.cpu_percent(interval=1) # get the usage of all CPU in 1 second, using interval will get more robust result

Memory

info = psutil.virtual_memory()

# total memory
info.total / 1024 / 1024 / 1024 # bytes -> GB

# the usage of memory
info.percent

MacOS

# svmem(total=25769803776, available=7726628864, percent=70.0, used=9045114880, free=791134208, active=6949453824, inactive=6915768320, wired=2095661056)

Ubuntu

# will get more information like buffers, cached etc..
# svmem(total=33541988352, available=25899790336, percent=22.8, used=7087771648, free=2703233024, active=15287881728, inactive=14264188928, buffers=965103616, cached=22785880064, shared=8265728, slab=904495104)

How to integrate psutil into our code?

We can use psutil to monitor each function cpu and memory usage by using decorator.

import psutil

def monitor(func):
    def wrapper(*args, **kwargs):
        process = psutil.Process()
        cur_func_pid = process.pid
        process = psutil.Process(cur_func_pid)

        func(*args, **kwargs)

        cpu_usage = process.cpu_percent(interval=0.01)
        memory_usage = process.memory_info().rss / 1024 / 1024 / 1024 # bytes -> GB

        print(f"CPU usage: {cpu_usage}%")
        print(f"Memory usage: {memory_usage}GB")

    return wrapper

Example

@monitor
def example_code():
    import numpy as np

    for i in range(100):
        np.random.rand(100, 100, 100)

example_code()

Reference

• Python常用库之psutil使用指南
• 简单介绍psutil库（virtual_memory()、cpu_percent()
• Python System Monitoring and Profiling with the psutil Module
• Psutil module in Python
• Administrative Tasks for Oracle Machine Learning for Python