Despite the issues with Raspbian not being a real-time deterministic operating system I'm sure that can be worked around by preventing Linux from scheduling on a core with suitable "isolcpus" parameters on it's command line and running ones control code on the isolated CPU with "taskset".
But the other issues raised were about physical size and power requirements.
Once again I have to ask: What is it you want to control exactly? And how?
It's impossible to suggest an optimal controller hardware solution without knowing what it is expected to do.