Someone knowledgeable should answer this, but now I tend to think the programmer just pulls the reset-pin low, then dumps the hex-file on memory address 0x000. Then pulls reset back up.
It's somewhat more complicated than that, there is some command verification, a magic number enable sequence, and it's command/response based. Still, it's pretty simple and it's probably possible to implement it just using the SPI and a single GPIO for reset. I'm surprised there's not already a driver in avrdude for the Linux SPI interface, so perhaps it's more complicated than I think as well
The relevant app note is AVR910, which describes the low-voltage protocol in its entirety.
avrdude does most of the work, and drives the hardware directly (as well as reading and parsing the input files, handling user commands and so on) with its drivers. An implementation on RPi would send all the commands via Linux GPIO or SPI interfaces. That said, the line does move around depending on how 'low level' the interface to the hardware is. Driving something like a USB AVRISP mkII is going to be a very different thing than the bit-banged serial port driver.