UMDCTF 2026 - roulette [reverse]

posted
read
552 words · 2 min
tags
#ctf#writeup#reverse-engineering
contents

Challenge: roulette (Reverse Engineering) Category: reverse

Overview

The binary presents itself as a roulette game. You pick a number, and it tells you if you won. But that’s just a facade — the real challenge is a hidden 106-byte input verifier powered by a tiny virtual machine. Each 4-byte chunk of your input is fed through a 27-round VM program that computes a checksum and compares it against a hardcoded target. If all 27 rounds pass, you get the success message and the flag.

The clever solution? Don’t reverse-engineer the VM at all. Use GDB with a Python breakpoint script at the critical compare instruction to dynamically extract what each input chunk should be.

Initial Analysis

Running the binary with no arguments shows a simple betting prompt:

$ ./roulette
Welcome to the Roulette table!
Place your bet (1-36): 7
You lost! The winning number was 23

The input is parsed via strtol. If you enter 1, you get a “jackpot” message. Enter anything outside 1-36 and it says “invalid.” This is all a decoy — none of this path leads to the flag.

Finding the Real Gate

Looking at sub_401690 (main) in a disassembler, we see the strtol path branches into the roulette game. But there’s another path: if you provide input longer than a typical number, something happens. Specifically:

  • Address 0x401751: A cmp checks if strlen(input) == 106. If so, execution diverts to a completely different handler — the real verifier.
  • Address 0x4018fe: The critical compare — cmp (eax ^ edi), target. This is where each round’s computed check value is compared.
  • Address 0x401956: The success / “accepted” path if all checks pass.

So the binary expects exactly 106 bytes of input. Those 106 bytes are processed in 4-byte chunks through 27 rounds of a VM program.

The VM Internals

The VM program is stored at 0x499BE1 and the round targets at 0x499CE0. Each round runs a small sequence of bytecode instructions operating on the current 4-byte chunk. The operations include:

  • COPY — copy a value between registers
  • XOR — xor two values
  • ADD — add immediate or register
  • MUL — multiply by immediate
  • ROL / SHR — rotate left or shift right

Each round produces a 32-bit check value in eax. At the compare site (0x4018fe), the assembly does:

0x4018fe: cmp (eax ^ edi), target

Where edi holds the current 4-byte input chunk, eax is the VM’s computed checksum of that chunk, and target is the round’s expected value from the target table.

The full formula is:

(eax ^ chunk) == target
=> chunk = target ^ eax

So if we can extract both eax and target for each round, computing the required chunk is trivial.

The Smart Approach: GDB Python Scripting

Rather than fully reverse-engineering the VM interpreter, we can do something much simpler: set a breakpoint at 0x4018fe and write a Python GDB script that:

  1. Reads eax — the VM-computed checksum for the current round
  2. Reads edi — our current input chunk (which we control)
  3. Reads the round target from the target table
  4. Computes required_chunk = target ^ eax
  5. Patches our input buffer in memory with the correct chunk
  6. Continues execution to the next round

Since each round’s VM computation depends on the input chunk itself, we need to iterate: patch chunk 0, let the VM run round 0, extract the result for round 1, patch chunk 1, and so on through all 27 rounds.

Here’s the script:

import gdb


# Disable confirmation prompts
gdb.execute("set confirm off")


# Round targets stored at 0x499CE0 (array of 27 uint32s)
TARGET_TABLE = 0x499CE0


# Our input buffer — we'll read its address from RDI on first break
input_buf = None


def get_target(round_idx):
    """Read the round target from the target table."""
    addr = TARGET_TABLE + round_idx * 4
    return int.from_bytes(
        gdb.selected_inferior().read_memory(addr, 4), 'little'
    )


class RouletteBreakpoint(gdb.Breakpoint):
    def __init__(self):
56 collapsed lines
        super().__init__("*0x4018fe")
        self.round = 0
        self.done_first = False


    def stop(self):
        global input_buf


        # Get register values
        eax = int(gdb.parse_and_eval("$eax")) & 0xFFFFFFFF
        edi = int(gdb.parse_and_eval("$edi")) & 0xFFFFFFFF


        if not self.done_first:
            # First hit — discover input buffer address from rdi
            input_buf = int(gdb.parse_and_eval("$rdi")) & 0xFFFFFFFFFFFFFFFF
            self.done_first = True


        target = get_target(self.round)
        chunk_idx = self.round
        chunk_offset = chunk_idx * 4


        # Compute what our chunk SHOULD be
        required_chunk = (target ^ eax) & 0xFFFFFFFF


        # Read what's currently at that position
        current_chunk = int.from_bytes(
            gdb.selected_inferior().read_memory(input_buf + chunk_offset, 4),
            'little'
        )


        print(f"[Round {self.round}]")
        print(f"  eax       = 0x{eax:08x}")
        print(f"  edi       = 0x{edi:08x} (current chunk)")
        print(f"  target    = 0x{target:08x}")
        print(f"  required  = 0x{required_chunk:08x}")


        # Patch the input buffer with the correct chunk
        chunk_bytes = required_chunk.to_bytes(4, 'little')
        gdb.selected_inferior().write_memory(
            input_buf + chunk_offset, chunk_bytes
        )


        print(f"  patched   = {chunk_bytes!r} -> '{''.join(chr(b) if 32 <= b < 127 else '.' for b in chunk_bytes)}'")


        self.round += 1
        if self.round >= 27:
            print("[All rounds complete!]")
            # Read the full flag from memory
            flag = bytes(
                gdb.selected_inferior().read_memory(input_buf, 106)
            )
            print(f"FLAG: {flag.decode('ascii', errors='replace')}")


        return False  # don't stop, just log


# Load the binary
gdb.execute("file ./roulette")


# Set up the breakpoint
bp = RouletteBreakpoint()


# Run with a dummy 106-byte input
# (the content doesn't matter — we'll patch it)
dummy = "A" * 106
gdb.execute(f"run <<< '{dummy}'")

Running the Script

Save the script as solve_roulette.py and run:

$ gdb -x solve_roulette.py -batch

The following is illustrative output, not a captured successful run. The script’s first-hit rdi buffer assumption must be checked against the actual binary; a 32-bit chunk in edi does not establish a valid pointer in rdi:

[Round 0]
  eax       = 0x3a6f7b2c
  edi       = 0x41414141 (current chunk)
  target    = 0xdeadbeef
  required  = 0xe4d6c518
  patched   = b'\x18\xc5\xd6\xe4' -> '...'
...
[All rounds complete!]
FLAG: UMDCTF{**************************************************************************************************}

Flag

UMDCTF{I_R3ALLY-want-to-pl4y-the-p0werball,+but-my-d4d-said-no-so-im-b3tting-ill-win-on-POLYMARKETinstead}