247CTF - Flag Errata

posted
read
923 words · 3 min
contents

We stored the flag within this binary, but made a few errors when trying to print it. Can you make use of these errors to recover the flag?

Overview

Terminal window
❯ file flag_errata.exe
flag_errata.exe: PE32+ executable for MS Windows 5.02 (console), x86-64 (stripped to external PDB), 9 sections

A PE64 binary (GCC 7.3 / MinGW-w64) with a monstrous 53KB main function. It prompts for a 40-character password, validates each character through error-code accumulation, then returns -247 on any mismatch.

The core pattern repeats 40 times:

sub_401560(); // intentionally failing WinAPI call
LastError = GetLastError(); // harvest the error code
sub_40159B();
v5 = GetLastError() + LastError; // accumulate
sub_4015D6();
v6 = GetLastError() + v5;
// ... N calls total ...
if ( Buffer[i] != (int)(accumulated) % 126 )
return -247; // failure — 247CTF signature

The password is never stored anywhere. It’s implicitly encoded by the count and sequence of WinAPI calls whose error codes reconstruct each character.

The Error Code Cycle

13 functions form a fixed cycle, each designed to fail deterministically:

#FunctionAddressAPI CallExpected ErrorCode
0errgen_open_247ctf_dll_system320x401560CreateFileA("C:\Windows\System32\247CTF.dll", ...)ERROR_FILE_NOT_FOUND2
1errgen_open_247ctf_dll_nested0x40159BCreateFileA("C:\247CTF\247CTF\247CTF.dll", ...)ERROR_PATH_NOT_FOUND3
2errgen_open_user32_dll_readwrite0x4015D6CreateFileA("user32.dll", GENERIC_RW, exclusive)ERROR_SHARING_VIOLATION32
3errgen_getthreadtimes_null0x401611GetThreadTimes(0, 0, 0, 0, 0)ERROR_INVALID_HANDLE6
4errgen_findfiles_exhaust0x401634FindFirstFile("C:\*") + loop FindNextFileERROR_NO_MORE_FILES18
5errgen_virtualqueryex_null0x401674VirtualQueryEx(proc, NULL, NULL, 8)ERROR_BAD_LENGTH24
6errgen_open_cmdexe_twice0x401697CreateFileA("cmd.exe") x2 (exclusive)ERROR_SHARING_VIOLATION32
7errgen_open_user32_invalid_create0x401704CreateFileA("user32.dll", ..., dwCreation=0)ERROR_INVALID_PARAMETER87
8errgen_setconsoledisplaymode0x40173FSetConsoleDisplayMode(FULLSCREEN)Varies by OS~87
9errgen_open_247ctf_colon0x401764CreateFileA("247CTF:", ...)ERROR_INVALID_NAME123
10errgen_loadlibrary_fake_dll0x4017A2LoadLibraryA("C:\247CTF.DLL")ERROR_MOD_NOT_FOUND126
11errgen_createmutex_twice0x4017B0CreateMutexA("247CTF") x2ERROR_ALREADY_EXISTS183
12errgen_createprocess_user320x4017DFCreateProcessA("user32.dll" as exe)ERROR_BAD_EXE_FORMAT193

Each character block calls NN consecutive functions from this cycle, accumulates the error codes, then checks sum % 126 against the input.

Algebraic Solution

The key insight: since the error codes cycle deterministically, we can bypass the Windows environment entirely and solve this as pure modular arithmetic.

I use Arch BTW.

Let:

  • S=∑k=012ekS = \sum_{k=0}^{12} e_k, the sum of one full cycle (all 13 error codes)
  • P(r)=∑k=0r−1ekP(r) = \sum_{k=0}^{r-1} e_k, the partial sum of the first rr codes

For character ii with nin_i total API calls:

qi=⌊ni13⌋,ri=ni mod 13q_i = \left\lfloor \frac{n_i}{13} \right\rfloor, \qquad r_i = n_i \bmod 13 password[i]≡qi⋅S+P(ri)(mod126)\text{password}[i] \equiv q_i \cdot S + P(r_i) \pmod{126}

The flag format 247CTF{<flag-redacted>} gives us 8 known characters for free. Three of them — blocks 2, 3, 6 — share the same remainder r=9r = 9: same position in the error-code cycle, different lap counts. Shared P(9)P(9) cancels on subtraction.

Blocks 2, 3, 6 correspond to 7 (ASCII 5555, q=16q = 16), C (6767, q=10q = 10), { (123123, q=24q = 24):

16S+P(9)≡55(mod126)(1)10S+P(9)≡67(mod126)(2)24S+P(9)≡123(mod126)(3)\begin{aligned} 16S + P(9) &\equiv 55 \pmod{126} && \quad (1) \\ 10S + P(9) &\equiv 67 \pmod{126} && \quad (2) \\ 24S + P(9) &\equiv 123 \pmod{126} && \quad (3) \end{aligned}

Subtracting (2)(2) from (1)(1) eliminates P(9)P(9):

(16−10) S≡55−67(mod126)6S≡−12≡114(mod126)\begin{aligned} (16 - 10)\,S &\equiv 55 - 67 \pmod{126} \\ 6S &\equiv -12 \equiv 114 \pmod{126} \end{aligned}

Since gcd⁡(6,126)=6\gcd(6, 126) = 6 divides 114114, we may divide the entire congruence by 66 (reducing the modulus to 126/6=21126/6 = 21):

S≡19(mod21)⟹S∈{19,  40,  61,  82,  103,  124}S \equiv 19 \pmod{21} \qquad \Longrightarrow \qquad S \in \{19,\; 40,\; 61,\; 82,\; 103,\; 124\}

Similarly, (3)−(1)(3) - (1) yields a second constraint:

8S≡68(mod126)4S≡34(mod63)(dividing by gcd⁡(2,126)=2)\begin{aligned} 8S &\equiv 68 \pmod{126} \\ 4S &\equiv 34 \pmod{63} && \quad (\text{dividing by } \gcd(2, 126) = 2) \end{aligned}

To isolate SS, multiply both sides by the modular inverse 4−1≡16(mod63)4^{-1} \equiv 16 \pmod{63}, since 4×16=64≡14 \times 16 = 64 \equiv 1:

S≡16×34=544≡40(mod63)S \equiv 16 \times 34 = 544 \equiv 40 \pmod{63}

Intersecting via CRT: 40 mod 21=19  ✓40 \bmod 21 = 19\;\checkmark, so both congruences agree.

 S=40 \boxed{\,S = 40\,}

Back-substituting into (1)(1):

16×40+P(9)≡55(mod126)640+P(9)≡55(mod126)10+P(9)=55(640 mod 126=10)P(9)=45\begin{aligned} 16 \times 40 + P(9) &\equiv 55 \pmod{126} \\ 640 + P(9) &\equiv 55 \pmod{126} \\ 10 + P(9) &= 55 && \quad (640 \bmod 126 = 10) \\ P(9) &= 45 \end{aligned}

Same process with the remaining known characters fills all partial sums:

P(r)P(r)ValueSource
P(3)P(3)10'2' (ASCII 50)
P(4)P(4)16e3=6e_3 = 6
P(5)P(5)34e4=18e_4 = 18
P(6)P(6)58'4' (ASCII 52)
P(7)P(7)90'T' (ASCII 84)
P(9)P(9)45'7' (ASCII 55)
P(10)P(10)42'F' (ASCII 70)
P(12)P(12)99e10,e11e_{10}, e_{11}

With SS and all P(r)P(r) determined, every character computes directly:

BlockCallsqqrrCalculationChar
01613(40+10) mod 126(40+10) \bmod 1262
7191149(560+45) mod 126(560+45) \bmod 126e
910985(320+34) mod 126(320+34) \bmod 126f
152852112(840+99) mod 126(840+99) \bmod 1269
202982212(880+99) mod 126(880+99) \bmod 126a
393529(80+45) mod 126(80+45) \bmod 126}

All 40 characters land in the valid charset — 247CTF{<flag-redacted>} suffix — confirming the solution without ever running the binary.

Character-by-Character Breakdown

BlockCallsqrFormulaChar
01613(1×40+10) mod 126=50(1 \times 40 + 10) \bmod 126 = 502
14536(3×40+58) mod 126=52(3 \times 40 + 58) \bmod 126 = 524
2217169(16×40+45) mod 126=55(16 \times 40 + 45) \bmod 126 = 557
3139109(10×40+45) mod 126=67(10 \times 40 + 45) \bmod 126 = 67C
44637(3×40+90) mod 126=84(3 \times 40 + 90) \bmod 126 = 84T
5101710(7×40+42) mod 126=70(7 \times 40 + 42) \bmod 126 = 70F
6321249(24×40+45) mod 126=123(24 \times 40 + 45) \bmod 126 = 123{
7191149(14×40+45) mod 126=101(14 \times 40 + 45) \bmod 126 = 101e
8280217(21×40+90) mod 126=48(21 \times 40 + 90) \bmod 126 = 480
910985(8×40+34) mod 126=102(8 \times 40 + 34) \bmod 126 = 102f
101916(1×40+58) mod 126=98(1 \times 40 + 58) \bmod 126 = 98b
11256199(19×40+45) mod 126=49(19 \times 40 + 45) \bmod 126 = 491
1210985(8×40+34) mod 126=102(8 \times 40 + 34) \bmod 126 = 102f
13241187(18×40+90) mod 126=54(18 \times 40 + 90) \bmod 126 = 546
14215167(16×40+90) mod 126=100(16 \times 40 + 90) \bmod 126 = 100d
152852112(21×40+99) mod 126=57(21 \times 40 + 99) \bmod 126 = 579
161714(1×40+16) mod 126=56(1 \times 40 + 16) \bmod 126 = 568
17215167(16×40+90) mod 126=100(16 \times 40 + 90) \bmod 126 = 100d
18217169(16×40+45) mod 126=55(16 \times 40 + 45) \bmod 126 = 557
19215167(16×40+90) mod 126=100(16 \times 40 + 90) \bmod 126 = 100d
202982212(22×40+99) mod 126=97(22 \times 40 + 99) \bmod 126 = 97a
212852112(21×40+99) mod 126=57(21 \times 40 + 99) \bmod 126 = 579
22280217(21×40+90) mod 126=48(21 \times 40 + 90) \bmod 126 = 480
23215167(16×40+90) mod 126=100(16 \times 40 + 90) \bmod 126 = 100d
24217169(16×40+45) mod 126=55(16 \times 40 + 45) \bmod 126 = 557
2512012(0×40+99) mod 126=99(0 \times 40 + 99) \bmod 126 = 99c
2638212(2×40+99) mod 126=53(2 \times 40 + 99) \bmod 126 = 535
2712012(0×40+99) mod 126=99(0 \times 40 + 99) \bmod 126 = 99c
28191149(14×40+45) mod 126=101(14 \times 40 + 45) \bmod 126 = 101e
2910985(8×40+34) mod 126=102(8 \times 40 + 34) \bmod 126 = 102f
301613(1×40+10) mod 126=50(1 \times 40 + 10) \bmod 126 = 502
31217169(16×40+45) mod 126=55(16 \times 40 + 45) \bmod 126 = 557
3212012(0×40+99) mod 126=99(0 \times 40 + 99) \bmod 126 = 99c
331613(1×40+10) mod 126=50(1 \times 40 + 10) \bmod 126 = 502
3410985(8×40+34) mod 126=102(8 \times 40 + 34) \bmod 126 = 102f
354536(3×40+58) mod 126=52(3 \times 40 + 58) \bmod 126 = 524
36217169(16×40+45) mod 126=55(16 \times 40 + 45) \bmod 126 = 557
372852112(21×40+99) mod 126=57(21 \times 40 + 99) \bmod 126 = 579
381714(1×40+16) mod 126=56(1 \times 40 + 16) \bmod 126 = 568
393529(2×40+45) mod 126=125(2 \times 40 + 45) \bmod 126 = 125}

Anti-Analysis

  • TLS Callbacks: Two callbacks (0x401C50, 0x401C20) execute before main — a common anti-debug vector.
  • 53KB of bloat: Up to 321 API calls per character, making manual static analysis impractical without scripting.
  • Environment sensitivity: Functions like CreateProcessA and SetConsoleDisplayMode return different error codes under a debugger, making dynamic analysis a trap. The algebraic approach sidesteps this entirely by reasoning at the mathematical level, independent of the Windows runtime.

247CTF{[flag redacted]}