You might be ok, I have seen shift registers used in a massive display before, I think you would need to look a t a different power supply to power both LEDs and Pi.
When we had a display at work that consisted of an LED matrix (8x5 modules but lots of them to display txt) I stripped the units down and they had fairly sizeable 5v PSU's on them but it worked out hundreds of LEDs they were powering plus a module that held the data (these were being scrapped so I kept the bits.)
theres some info here: http://arduino.cc/en/Tutorial/ShiftOut
about connecting shift registers to drive sets of LEDs. I have seen 512 LEDs driven by shift registers and an Arduino before, but theres no way the arduino has the oompth to power that many on its own.
So basically I think you'd be ok with your plans, but you would have to design it in such a way as to have a PSU and a common 5v rail to power everything off of.
I'm not 100% but hopefully that is a pointer in the right direction.