al-khaser | Public malware techniques used in the wild : Virtual Machine | Reverse Engineering library

Both of those techniques are relatively old. I think they were in use in the early 2000s, if not the late 90s.

The first uses the fact that the Thread-Information Block (TIB) resides at fs:[0] for windows processes (at least as late as WinXP...I haven't looked since then).

Offset 0x30 into the TIB is a pointer to the Process Environment Block (PEB). From that link, we see that offset 0x2 into the PEB is the 'being debugged' flag. This is the value that is read by the API call IsDebuggerPresent.

The second example demonstrates two detection mechanisms:

This answer gives good detail on the function of int 0x2d on windows. For the moment, we'll just note that it's part of Windows' built-in debugging support.

The simpler mechanism is this: Structured exception handling in Windows would catch things that the language-based exceptions couldn't, because it was hooked into the OS's exception handling framework, not the language-runtime's.

So the int 0x2d would generate a fault; any debugger present would handle the fault, and because this interrupt is for debugging, the debugger would return to control-flow as usual. Thus the structured-exception handling mechanism wouldn't be invoked...so the 'catch' would never be reached, and the return value of the function would thereby change. This method was pretty trivially defeated by later SEH-aware debuggers, by telling the debugger to pass the exception to the program, in which case the SEH would fire.

The other mechanism is based in the detail of the of way int 0x2d is processed by windows, which was apparently not emulated perfectly by OllyDbg (a once phenomenal debugger that went defunct and came back full of malware). I am not personally familiar with this method, so I would direct you to the linked answer.

Just for context, the early days of anti-debugger techniques were studies of particularly popular debug tools. In whatever document you've dug these techniques from, you will no doubt also see ones dedicated to detecting NuMega Softice --- probably the first kernel debugger for Windows (circa NT 4.0). The instruction sequence from whence my handle is derived was used as a method of detecting this debugger.

Thanks for the trip down memory lane ;)