That is what I was worried about.
I played around with adding authorized keys into the user home folder that exists on the PiServer host.
(accessing the folder using root)
and changing settings through CHROOT.
Was curious if anyone else tried and had success.