Pixel and wandering mouse cursor.

[madtom1999]

Probably pushing the envelope here: Having had much success on the Zero with PIXEL I thought I'd try in on an old 256MB A (?) and upgraded from raspbian lite with sudo apt-get install raspberrypi-ui-mods and it nearly works.
You have to leave it for a while after it comes up and then if you take your time you can run a few things (even synaptic) though the pointer wanders off to the bottom right of the screen when its busy. However sometimes the pointer just wanders off and the system is unresponsive to the mouse. I know this is a lot to ask of a little machine but has anyone experienced this and found a solution to it?