Memory Hierarchy Stack vs Heap Allocation Garbage Collection Memory Leaks Profiling Best Practices

Memory Management - Deep Dive

Interview Tip: Memory management is critical for system performance and stability. Understanding stack vs heap, garbage collection algorithms, and how to detect memory leaks demonstrates systems-level thinking.

1. Memory Hierarchy

Memory is organized in a hierarchy based on speed and cost:

Fastest/Most Expensive ┌─────────────────┐ │ CPU Registers │ ~1 cycle, < 1 KB ├─────────────────┤ │ L1 Cache │ ~4 cycles, 32-64 KB per core ├─────────────────┤ │ L2 Cache │ ~12 cycles, 256-512 KB per core ├─────────────────┤ │ L3 Cache │ ~40 cycles, 8-32 MB shared ├─────────────────┤ │ Main Memory │ ~100-200 cycles, GB │ (RAM) │ ├─────────────────┤ │ SSD │ ~10,000 cycles, TB ├─────────────────┤ │ Hard Disk │ ~1,000,000 cycles, TB └─────────────────┘ Slowest/Least Expensive
Cache Locality: Programs that access memory sequentially (good spatial locality) or repeatedly access the same memory (good temporal locality) run faster due to cache hits.

Virtual Memory

The OS provides an abstraction layer between physical RAM and what processes see:

2. Stack vs Heap Memory

Aspect Stack Heap
Allocation Automatic (function calls) Manual (new/malloc) or GC
Deallocation Automatic (function returns) Manual (delete/free) or GC
Speed Very fast (pointer increment) Slower (search free blocks)
Size Limited (1-8 MB typical) Large (limited by RAM)
Fragmentation None Can fragment over time
Access Pattern LIFO (Last In, First Out) Random access
Thread Safety Each thread has its own Shared between threads
Use Case Local variables, function calls Dynamic data, large objects

Stack Memory Example

void function_a() { int x = 10; // Allocated on stack char buffer[100]; // Allocated on stack function_b(); // New stack frame pushed } // Stack frame popped, memory freed Stack Growth: High Address ┌──────────────┐ │ function_a() │ x, buffer │ frame │ ├──────────────┤ │ function_b() │ New frame │ frame │ ├──────────────┤ ← Stack Pointer │ │ │ (unused) │ │ │ └──────────────┘ Low Address

Heap Memory Example

// C++ int* ptr = new int[1000]; // Allocated on heap delete[] ptr; // Must manually free // Java List<String> list = new ArrayList<>(); // Allocated on heap // Garbage collector handles deallocation // Python data = [1, 2, 3, 4, 5] // Allocated on heap # Reference counting + GC handles deallocation
Stack Overflow: Occurs when stack size limit is exceeded (deep recursion, large local arrays). Heap overflow is different - it's when you run out of heap memory.

3. Memory Allocation Strategies

Fixed-Size Allocation

Memory divided into fixed-size blocks. Fast but can waste space.

Block Size: 64 bytes ┌────┬────┬────┬────┬────┐ │ 64 │ 64 │ 64 │ 64 │ 64 │ └────┴────┴────┴────┴────┘ Allocating 100 bytes wastes 28 bytes (uses 2 blocks)

Variable-Size Allocation

Several algorithms for managing variable-sized blocks:

Free List Example: ┌──────┬────────┬──────┬──────────┐ │ Used │ Free │ Used │ Free │ │ 100B │ 200B │ 150B │ 300B │ └──────┴────────┴──────┴──────────┘ Request 180B: - First Fit: Uses 200B block - Best Fit: Uses 200B block - Worst Fit: Uses 300B block

Slab Allocation

Used for kernel objects. Pre-allocates common object sizes:

Slab for "task_struct" objects (fixed size): ┌─────┬─────┬─────┬─────┬─────┐ │ Free│ Free│ Used│ Free│ Used│ └─────┴─────┴─────┴─────┴─────┘ No fragmentation, very fast allocation

4. Garbage Collection

Automatic memory management that reclaims memory no longer in use.

Reference Counting

Track number of references to each object. When count reaches 0, free it.

// Python uses reference counting a = [1, 2, 3] # refcount = 1 b = a # refcount = 2 c = a # refcount = 3 del a # refcount = 2 del b # refcount = 1 del c # refcount = 0 → freed
Circular Reference Problem:
a = [] b = [] a.append(b) # a references b b.append(a) # b references a del a, b # Both have refcount > 0, but unreachable!
Python uses cycle detection to handle this.

Mark and Sweep

Two-phase algorithm used by many GC systems:

Phase 1 - Mark: 1. Start from "roots" (global variables, stack, registers) 2. Traverse all reachable objects 3. Mark each reachable object ┌─────┐ ┌─────┐ ┌─────┐ │ A │────→│ B │────→│ C │ (All marked) └─────┘ └─────┘ └─────┘ │ ↓ ┌─────┐ │ D │ (Marked) └─────┘ ┌─────┐ │ E │ (Unreachable - not marked) └─────┘ Phase 2 - Sweep: Free all unmarked objects (E)
Stop-the-World: Traditional mark-and-sweep pauses application threads during GC. Modern collectors minimize pause times.

Generational Garbage Collection

Based on observation: most objects die young.

┌─────────────────────────────────────────┐ │ Young Generation │ │ ┌─────────┬──────────┬──────────┐ │ │ │ Eden │ Survivor │ Survivor │ │ │ │ Space │ 0 │ 1 │ │ │ └─────────┴──────────┴──────────┘ │ │ │ │ New objects allocated here │ │ Collected frequently (minor GC) │ └─────────────────────────────────────────┘ ↓ (survivors promoted) ┌─────────────────────────────────────────┐ │ Old Generation │ │ │ │ Long-lived objects │ │ Collected infrequently (major GC) │ └─────────────────────────────────────────┘

Java Generational GC Process:

  1. Objects allocated in Eden space
  2. Minor GC: Eden survivors → Survivor 0
  3. Next GC: Eden + Survivor 0 survivors → Survivor 1
  4. After N cycles, promote to Old Generation
  5. Major GC: Collect Old Generation (expensive)

Concurrent and Incremental GC

Collector Type Pause Time Use Case
Serial GC Stop-the-world Long Single-threaded apps
Parallel GC Stop-the-world, multi-threaded Medium Throughput-focused
CMS (Concurrent Mark Sweep) Mostly concurrent Short Low-latency apps
G1 GC Generational, concurrent Predictable Large heaps, balanced
ZGC / Shenandoah Concurrent, region-based Sub-millisecond Very low latency

Tri-Color Marking (Concurrent GC)

Allows GC to run concurrently with application threads:

White: Not yet visited Gray: Visited, but children not scanned Black: Visited and children scanned Initial: All objects white, roots gray During: Scan gray objects → mark children gray → mark self black Final: White objects are garbage (unreachable) ┌───────┐ │ Root │ (Gray initially) └───┬───┘ │ ↓ ┌───────┐ ┌───────┐ │ A │────→│ B │ (White → Gray → Black) └───────┘ └───────┘ │ ↓ ┌───────┐ │ C │ (White → Gray → Black) └───────┘ ┌───────┐ │ D │ (Stays white → unreachable → free) └───────┘

5. Memory Leaks

Memory that's allocated but never freed, eventually exhausting available memory.

Common Causes

1. Forgotten References (Java/JavaScript)

// Memory leak - static collection grows unbounded class Cache { private static Map<String, Object> cache = new HashMap<>(); public void addToCache(String key, Object value) { cache.put(key, value); // Never removed! } } // Fix: Use WeakHashMap or implement eviction policy private static Map<String, Object> cache = new WeakHashMap<>();

2. Event Listeners Not Removed (JavaScript)

// Memory leak function setupButton() { const button = document.getElementById('myButton'); const largeData = new Array(1000000); button.addEventListener('click', () => { console.log(largeData.length); // Closure keeps largeData alive }); // Listener never removed, largeData never freed } // Fix: Remove listener when done function setupButton() { const button = document.getElementById('myButton'); const handler = () => console.log('clicked'); button.addEventListener('click', handler); // Later: button.removeEventListener('click', handler); }

3. Missing delete/free (C/C++)

// Memory leak void processData() { int* data = new int[1000]; // ... process data ... // Missing: delete[] data; } // Memory leaked // Fix: Use RAII void processData() { std::unique_ptr<int[]> data(new int[1000]); // ... process data ... } // Automatically deleted

4. Circular References (without GC)

// C++: Smart pointer cycle class Node { std::shared_ptr<Node> next; // Strong reference std::shared_ptr<Node> prev; // Strong reference }; auto node1 = std::make_shared<Node>(); auto node2 = std::make_shared<Node>(); node1->next = node2; node2->prev = node1; // Cycle! Neither can be freed // Fix: Use weak_ptr for back-references class Node { std::shared_ptr<Node> next; std::weak_ptr<Node> prev; // Weak reference breaks cycle };

5. Unclosed Resources

// Memory leak (file descriptors, connections) void readFile() { FileInputStream fis = new FileInputStream("data.txt"); // ... read data ... // Missing: fis.close(); } // Fix: Try-with-resources (Java) or defer (Go) void readFile() { try (FileInputStream fis = new FileInputStream("data.txt")) { // ... read data ... } // Automatically closed }

Detecting Memory Leaks

Language Tools What to Look For
C/C++ Valgrind, AddressSanitizer, Dr. Memory Unfreed allocations, invalid accesses
Java VisualVM, JProfiler, Eclipse MAT Heap dumps, growing collections
JavaScript Chrome DevTools, heap snapshots Detached DOM nodes, retained objects
Python tracemalloc, objgraph, memory_profiler Growing object counts, ref cycles
Go pprof, runtime/trace Goroutine leaks, heap growth

6. Memory Profiling

Heap Profiling (Go example)

import _ "net/http/pprof" import "runtime/pprof" // Enable profiling endpoint go func() { http.ListenAndServe("localhost:6060", nil) }() // Take heap snapshot f, _ := os.Create("heap.prof") pprof.WriteHeapProfile(f) f.Close() // Analyze: // go tool pprof heap.prof // (pprof) top10 // (pprof) list functionName

Valgrind (C/C++ example)

# Compile with debug symbols g++ -g myprogram.cpp -o myprogram # Run with Valgrind valgrind --leak-check=full \ --show-leak-kinds=all \ --track-origins=yes \ ./myprogram # Output: # ==12345== HEAP SUMMARY: # ==12345== in use at exit: 1,000 bytes in 1 blocks # ==12345== total heap usage: 2 allocs, 1 frees, 2,000 bytes allocated # ==12345== # ==12345== 1,000 bytes in 1 blocks are definitely lost # ==12345== at malloc (vg_replace_malloc.c:299) # ==12345== by main (myprogram.cpp:10)

Chrome DevTools (JavaScript)

1. Open DevTools → Memory tab 2. Take heap snapshot 3. Perform action that might leak 4. Take another snapshot 5. Compare snapshots Look for: - Objects that should have been freed but weren't - Detached DOM nodes (nodes removed from DOM but still referenced) - Growing arrays/objects between snapshots - Event listeners that weren't removed

Java Heap Dump Analysis

# Generate heap dump jmap -dump:live,format=b,file=heap.bin <pid> # Or automatically on OutOfMemoryError java -XX:+HeapDumpOnOutOfMemoryError \ -XX:HeapDumpPath=/tmp/heap.hprof \ MyApplication # Analyze with Eclipse MAT # Look for: # - Dominator tree (what's holding most memory) # - Leak suspects (automated analysis) # - Duplicate strings # - Collection fill ratios

7. Best Practices

General Principles

1. Prefer Stack Over Heap
// Good: Stack allocation void process() { int data[100]; // Stack // ... use data ... } // Automatically cleaned up // Avoid if possible: Heap allocation void process() { int* data = new int[100]; // Heap // ... use data ... delete[] data; // Manual cleanup required }
2. Use RAII (Resource Acquisition Is Initialization)

Tie resource lifetime to object lifetime:

// C++: Use smart pointers std::unique_ptr<Widget> widget = std::make_unique<Widget>(); // Automatically deleted when out of scope // Java: Try-with-resources try (Connection conn = getConnection()) { // Use connection } // Automatically closed // Python: Context managers with open('file.txt') as f: data = f.read() # Automatically closed // Go: Defer f, _ := os.Open("file.txt") defer f.Close() // Executed when function returns
3. Minimize Object Creation
// Bad: Creates many temporary objects String result = ""; for (int i = 0; i < 10000; i++) { result += i; // New string each iteration! } // Good: Reuse buffer StringBuilder sb = new StringBuilder(); for (int i = 0; i < 10000; i++) { sb.append(i); } String result = sb.toString(); // JavaScript: Similar issue let result = ""; for (let i = 0; i < 10000; i++) { result += i; // Bad } // Better const parts = []; for (let i = 0; i < 10000; i++) { parts.push(i); } const result = parts.join('');
4. Object Pooling

For frequently created/destroyed objects:

// Connection pooling class ConnectionPool { private Queue<Connection> available; private int maxSize = 10; public Connection acquire() { if (!available.isEmpty()) { return available.poll(); // Reuse existing } if (totalConnections < maxSize) { return createNewConnection(); } // Wait or throw exception } public void release(Connection conn) { available.offer(conn); // Return to pool } } // Used in: // - Database connections // - Thread pools // - Buffer pools // - Object pools for game entities
5. Clear References
// Java: Clear references in collections class Stack { private Object[] elements; private int size = 0; public Object pop() { if (size == 0) throw new EmptyStackException(); Object result = elements[--size]; elements[size] = null; // Clear reference! return result; } } // JavaScript: Clear event listeners class Component { constructor() { this.handler = this.onClick.bind(this); element.addEventListener('click', this.handler); } destroy() { element.removeEventListener('click', this.handler); this.handler = null; // Clear reference } }
6. Use Weak References

For caches and callbacks that shouldn't prevent GC:

// Java: WeakHashMap for caches Map<Key, Value> cache = new WeakHashMap<>(); // Entries automatically removed when key is GC'd // JavaScript: WeakMap const cache = new WeakMap(); const obj = {data: 'important'}; cache.set(obj, 'cached value'); // When obj is no longer referenced, entry can be GC'd // Python: weakref import weakref cache = weakref.WeakValueDictionary() obj = SomeObject() cache['key'] = obj # When obj is deleted, entry is automatically removed
7. Profile Before Optimizing
8. Set Memory Limits
# Java: Set heap size java -Xms512m -Xmx2g MyApplication # Initial Max # Node.js: Set max old space node --max-old-space-size=4096 app.js # Go: Set GOGC (GC trigger percentage) export GOGC=100 # Default: GC when heap doubles # Docker: Limit container memory docker run -m 512m myapp

Language-Specific Tips

C/C++

Java

JavaScript

Python

Go

Common Interview Questions

Q: Explain the difference between stack and heap memory.
A: Stack is for automatic, short-lived local variables with LIFO allocation. It's fast but limited in size. Heap is for dynamic, long-lived objects with manual or GC'd allocation. It's larger but slower and can fragment. Each thread has its own stack, but the heap is shared.
Q: How does mark-and-sweep garbage collection work?
A: Two phases: (1) Mark - starting from roots (globals, stack), traverse all reachable objects and mark them. (2) Sweep - iterate through heap and free all unmarked objects. Modern variants include generational GC (young vs old), concurrent marking (tri-color), and incremental collection to reduce pause times.
Q: What causes memory leaks in garbage-collected languages?
A: Even with GC, leaks occur when objects are unintentionally retained: (1) Forgotten event listeners, (2) Static collections that grow unbounded, (3) Closures capturing large objects, (4) Unclosed resources (files, connections), (5) Caches without eviction policies. The GC can't free objects that are still reachable.
Q: How would you investigate a memory leak in production?
A: (1) Monitor metrics (heap usage, GC frequency) to confirm leak, (2) Take heap dumps/snapshots at intervals, (3) Compare snapshots to find growing objects, (4) Use profiling tools (Valgrind, Chrome DevTools, jmap), (5) Review recent code changes for common patterns (event listeners, static collections, unclosed resources), (6) Add instrumentation/logging, (7) Reproduce in staging if possible.
Q: What is memory fragmentation and how do you prevent it?
A: Fragmentation occurs when free memory is scattered in small blocks, preventing large allocations despite sufficient total free space. Internal fragmentation wastes space within allocated blocks. External fragmentation creates unusable gaps between blocks. Prevention: (1) Use memory pools for fixed-size objects, (2) Buddy system allocation, (3) Compacting GC that moves objects, (4) Generational GC that segregates by lifetime, (5) Avoid frequent allocation/deallocation of variable-sized objects.
Q: Explain virtual memory and why it's important.
A: Virtual memory provides each process with its own isolated address space mapped to physical RAM via page tables. Benefits: (1) Process isolation - processes can't access each other's memory, (2) Simplified memory management - contiguous virtual memory can map to fragmented physical memory, (3) Demand paging - only load needed pages from disk, (4) Larger address space than physical RAM via swapping, (5) Memory protection and security.
Key Takeaway: Effective memory management requires understanding the trade-offs between stack vs heap, manual vs automatic management, and throughput vs latency. Profile first, optimize based on data, and use appropriate tools and patterns for your language and use case.