What does `rep ret` mean?
assembly, branch-prediction, micro-optimization, x86
Solution
There's a whole blog named after this instruction. And the first post describes the reason behind it: http://repzret.org/p/repzret/
Basically, there was an issue in the AMD's branch predictor when a single-byte `ret` immediately followed a conditional jump as in the code you quoted (and a few other situations), and the workaround was to add the `rep` prefix, which is ignored by CPU but fixes the predictor penalty.
Problem
I was testing some code on Visual Studio 2008 and noticed `security_cookie`. I can understand the point of it, but I don't understand what the purpose of this instruction is. ``` rep ret /* REP to avoid AMD branch prediction penalty */ ``` Of course I can understand the comment :) but what is this prefix exaclty doing in context with the `ret` and what happens if `ecx` is != 0? Apparently the loop count from `ecx` is ignored when I debug it, which is to be expected. The code where I found this was here (injected by the compiler for security): ``` void __declspec(naked) __fastcall __security_check_cookie(UINT_PTR cookie) { /* x86 version written in asm to preserve all regs */ __asm { cmp ecx, __security_cookie jne failure rep ret /* REP to avoid AMD branch prediction penalty */ failure: jmp __report_gsfailure } } ```