I found the premade binaries, even the aarch64 version 3.2-3.3, this saves compiling fpc.
They come with the text mode IDE FP so you can start doing stuff.
I have even used Geany to compile free pascal code
The Laz compile scripts need to be modified, but that is a learning curve I have yet to master.
There is another IDE , MSEide and MSEgui that could be of use, sadly Martin has passed but many hope his code goes on.
I use Gentoo64 so as well as the native aarch64 version an armhf cross compiler is needed for 32bit code.
I also hope to cross compile for the RISC-V 32 and 64 one day.
It is all looking possible now with the latest versions.
If I remember right you can force Raspbian to install a newer version than the default.
Recently Debian split the install for fpc and lazarus to make this easier.
As Raspbian Debian is moving to 4.19 and 64bit, two versions will be needed.
It is much better now than 6 months ago, it will be easy in another 6 months.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges