C
OS list
Previous3 / 5Next
OS/Memory/Lesson 03

Memory Management — Virtual Memory, Paging, and Allocation

45 min·theory

Memory Management — Virtual Memory, Paging, and Allocation

🎯 After Reading This Lesson

After finishing this lesson, you will be able to confidently handle the following three topics.

  • ▸✅ Stack / Heap / Code / Data segments
  • ▸✅ Paging vs Segmentation + Virtual Memory
  • ▸✅ Page Fault + LRU page replacement

Keep the learning goals as a checklist, and close the lesson once you can answer all of them.

Virtual Memory — *The Illusion of Infinite Memory*

One-liner: The illusion that each process owns all of memory. In reality, the OS maps pages to physical RAM.

Why it is needed:

ReasonMeaning
IsolationProcess A cannot access Process B's memory (security and stability)
AbstractionA unified address space starting at 0x00000000 (simplifies development)
OvercommitUse 64 GB of virtual memory on a 16 GB RAM machine (pages loaded on demand)
SwapUnused pages are moved to disk (more efficient RAM use)

6 steps to access a variable (e.g., int x = 5; printf("%d", x);):
1. Virtual address — &x = 0x7fff5fbff8ac (stack segment)
2. MMU translation — the Memory Management Unit looks up the page table
3. TLB cache — frequently used mappings are cached in the CPU (1-cycle hit)
4. Physical address — 0x7fff5fbff8ac → physical RAM 0x12345678
5. Cache lookup — L1 → L2 → L3 → RAM (4 → 12 → 40 → 200 cycles)
6. Value returned — 5 is loaded into a register

> 💡 Page Fault = no mapping in the page table → OS loads from disk → very slow (several ms).

Paging — Memory Management in 4 KB Units

Page = 4 KB (Linux standard). Both virtual and physical memory are managed in page-sized units.

Page table structure:

  • ▸A separate page table for each process
  • ▸Virtual page number → physical page number + permissions (r/w/x)
  • ▸64-bit systems = 4-level page table (PML4 → PDPT → PD → PT)

TLB (Translation Lookaside Buffer):

  • ▸A cache of page mappings inside the CPU (typically 64–1024 entries)
  • ▸1 cycle on a hit; page table walk on a miss (10–100 cycles)

Paging pitfalls:

ProblemCauseSolution
Page FaultNo mapping → load from diskKeep frequently used data in LRU cache
ThrashingToo many page faults → CPU stalls waiting on diskAdd RAM or reduce the working set
Memory fragmentationSmall free blocks scattered aroundBuddy/Slab allocator (handled by the kernel)
OOM KillerRAM + swap full → OS forcibly kills a processLower oom_score and fix memory leaks

Huge Pages (2 MB / 1 GB):

  • ▸Used by databases and JVMs that handle large amounts of memory
  • ▸Fewer TLB misses → better performance
  • ▸Linux: echo 1024 > /proc/sys/vm/nr_hugepages

Memory Leaks + OOM Debugging

Memory Leak: allocated memory that is never freed → accumulates over time → eventually causes OOM.

Risk by language:

LanguageLeak RiskReason
C/C++Very highForgetting to call free after malloc causes an immediate leak
Java/Python/GoLow (GC)But GC cannot collect objects if references are still held (listeners, caches, global variables)
RustAlmost noneThe ownership system guarantees freedom from leaks at compile time

Leak diagnosis tools:

  • ▸Linux: valgrind --leak-check=full ./app (C/C++)
  • ▸Java: jmap -heap <PID> → jhat or VisualVM
  • ▸Python: tracemalloc module or objgraph
  • ▸Node.js: --inspect + Chrome DevTools Heap Snapshot
  • ▸All languages: If RSS in top or htop keeps rising over time, suspect a leak

OOM Killer behavior:
1. RAM + swap are nearly exhausted
2. The kernel computes an oom_score (based on memory usage and priority)
3. The process with the highest score receives SIGKILL
4. Logged in /var/log/syslog or dmesg (Out of memory: Kill process ...)

> 💡 On a production server, set vm.swappiness=10 (minimize swapping) and configure memory monitoring alerts.

💻 📌 Memory Analysis Through Scenarios 
# ============================================================
# Scenario 1: "An alarm came in saying server memory is low"
# ============================================================
# Start with system-wide status — human-readable
free -h
#               total   used   free  shared  buff/cache   available
# Mem:          15Gi    11Gi   1.2Gi  200Mi   3.0Gi       3.5Gi
# Swap:         2.0Gi   1.5Gi  500Mi
#
# Key point:
#   available  = *actually usable new* memory (look at this, not used)
#   buff/cache = disk cache (automatically reclaimed if needed)
#   swap used > 0 = RAM shortage signal ⚠️

# More details (50+ items)
cat /proc/meminfo

# View trends at 1-second intervals
vmstat 1 5                    # Print 5 times
# Pay attention to free·si(swap in)·so(swap out) columns

# ============================================================
# Scenario 2: "Which process is using a lot of memory?"
# ============================================================
ps aux --sort=-%mem | head -10
# RSS column = actual memory used (KB)
# VSZ column = virtual memory (for reference)

# Color interactive
top -o %MEM                   # Can also sort with M key

# Detailed *memory map* of a specific process
pmap -x 28391                 # Usage by library, heap, and stack
cat /proc/28391/status | grep -E 'VmRSS|VmSize|VmSwap'
#   VmRSS = actual physical memory (RAM)
#   VmSize = virtual memory (reserved address space)
#   VmSwap = swapped amount

# ============================================================
# Scenario 3: "OOM Killer killed my process"
# ============================================================
# 1) Check OOM events
dmesg | grep -i 'out of memory'
sudo journalctl -k | grep -i oom
# Example output: Out of memory: Kill process 28391 (java) score 854 or sacrifice child

# 2) Check the process's OOM score (higher means killed first)
cat /proc/28391/oom_score
# 0~1000. Based on memory usage and priority

# 3) Protect important processes
echo -1000 > /proc/28391/oom_score_adj  # Never kill (root only)
# Or in systemd unit file:
#   OOMScoreAdjust=-500

# 4) Adjust swappiness (usually lowered on servers)
cat /proc/sys/vm/swappiness          # Default 60
sudo sysctl vm.swappiness=10         # Recommended for production (minimize swap)

# ============================================================
# Scenario 4: "Java app is suspected of memory leak"
# ============================================================
# 1) One-line heap statistics
jmap -heap 28391
# Heap Configuration:
#   MaxHeapSize = 4096 MB
# Heap Usage:
#   PS Young Generation: Eden 256/256MB (100% used)  ← Dangerous
#   PS Old Generation:   2800/3072 MB (91% used)     ← Very dangerous

# 2) GC statistics every 1 second (view long-term trends)
jstat -gcutil 28391 1s
# S0  S1  E    O    M    YGC  YGCT  FGC  FGCT  GCT
# 0   85  92   91   95   1820 25.5  234  120   145
# FGC (Full GC) frequent or long duration = suspected leak

# 3) Heap dump (for analysis)
jmap -dump:live,format=b,file=/tmp/heap.bin 28391

# 4) Open and analyze with Eclipse MAT or VisualVM
# - Leak Suspects report
# - Which classes are taking up memory

# ============================================================
# Scenario 5: "Python app RSS increases over time"
# ============================================================
# tracemalloc — track memory allocations (insert into code)
# import tracemalloc
# tracemalloc.start()
# ... run code ...
# snapshot = tracemalloc.take_snapshot()
# for stat in snapshot.statistics('lineno')[:10]:
#     print(stat)
# Output: /app/main.py:42: size=125 MB ...

# memory-profiler — line-by-line memory measurement
# pip install memory-profiler
# python -m memory_profiler app.py

# ============================================================
# Scenario 6: "Improve performance by enabling Huge Pages"
# ============================================================
# Reduces TLB misses in DBs and JVMs handling large memory
cat /proc/sys/vm/nr_hugepages         # Current (usually 0)
echo 1024 | sudo tee /proc/sys/vm/nr_hugepages   # 1024 × 2MB = 2GB
# Permanent application: vm.nr_hugepages=1024 in /etc/sysctl.conf

🤖 Try Asking AI Like This

Knowing the concepts in this lesson lets you give AI specific, precise instructions. Instead of a vague "fix this," you can make vocabulary-driven requests — and that is where token savings start.

  • ▸"Analyze the memory usage pattern (heap/stack) of this code"
  • ▸"Tell me the command to measure RSS / Virtual Memory of this process"

Why This Reduces Tokens

Without the right concepts, you have to ask "What does that mean?" after every AI response. Those follow-up questions eat up tokens. Learn the concept once, and the conversation wraps up in one go.

Read this first: OS Fundamentals + Processes & Threads
Up next: Concurrency — Lock·Deadlock·Synchronous/Asynchronous
PreviousListNext
Memory Management — Virtual Memory, Paging, and Allocation - OS