You could start with Raspbian Lite and only install the Raspberry Pi desktop and applications you need. You could even skip the full desktop and just install a window manager, file manager and panel for a basic GUI environment.
How restricted do you want "everything else" to be?
Is not having anything else installed enough?
Or do you need it so nothing else can be installed?
And are you going to be the only user of the system?
If you are the admin user on the system it would be quite difficult truly restrict anything. If someone else could manage the system they could create a user for you that has no admin privileges, so you could use the applications, but could not alter the OS or applications in any way.
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups...and where is that annoying music coming from?