Have you tried booting into the Hardware Tests - on your System Install DVD, just follow the instructions on the disc. Run the Extended Tests and it will test the logic board and video RAM, amongst other stuff. It does sound like a hardware problem, a mate of mine had a similar effect on his 1st generation aluminium iMac (and that was a known issue with early models of those too). He got a new logic board on his I think.

Mine does it very occasionally too, seemingly most often when it's been driving a big screen. Re-closing the lid and starting again seems to do the trick most of the time.