Mocking up an ISP on a breadboard as described below is handy as a once-off but gets old very quickly. The pictures show a little "shield" for the homebrew Arduino, which costs about a euro for parts and a couple of hours of time.
Although this shield takes a little longer to construct than it does to wire up the breadboard, it also boasts status LEDs, for Heartbeat, Error and Programming, which ArduinoISP can drive but which were always too much trouble to wire up. (The Heartbeat LED is particularly pretty.)
Since they're so cheap, there's no point not to make separate shields for the 8-pin and 14-pin Tiny AVRs. We'll do these as needed.