Does the type of movie file effect playback?
MKV vs MOV or MP4?
Is the GPU being taken advantage of or is all the processing being done by the CPU?
What format the video is encoded in makes the difference when it comes to performance. The only two licensed codecs for the GPU are MPEG4 and h.264:
As long as the video is encoded with one of those two formats the GPU will do the decoding otherwise it will be the CPU that does the decoding. I have played both formats on my Pi in HD and assuming I meet IO requirements (see earlier post on network throughput) they are smooth as butter.