aosabook | The Architecture of Open Source Applications | Architecture library

Your best guide is the pypy architecture documentation, and the actual JIT documentation.

What jumped out the most for me is this:

we have a tracing JIT that traces the interpreter written in RPython, rather than the user program that it interprets.

This is covered in more detail in the JIT overview.

It seems to be that the "core" is this (from here):

Once the meta-interpreter has verified that it has traced a loop, it decides how to compile what it has. There is an optional optimization phase between these actions which is covered future down this page. The backend converts the trace operations into assembly for the particular machine. It then hands the compiled loop back to the frontend. The next time the loop is seen in application code, the optimized assembly can be run instead of the normal interpreter.

This paper (PDF) might also be helpful.

Edit: Looking at the x86 backend rpython/jit/backend/x86/rx86.py, the backend doesn't so much as compile but spit out machine code directly. Look at the X86_64_CodeBuilder and AbstractX86CodeBuilder classes. One level higher is the Assembler386 class in rpython/jit/backend/x86/assembler.py. This assembler uses the MachineCodeBlockWrapper from rpython/jit/backend/x86/codebuf.py which is based on the X86_64_CodeBuilder for x86-64.