A simple project should be OK, provided you have a 512 MB model. Even quite -understatement- complex projects like Lazarus itself should compile ok..
Do you have the latest RaspBian? Do you maybe use a fixed memory split like 256/256? 256 is not enough for Lazarus (but it is for fpc..)
Lazarus works best with a memory split 64-128 for GPU or automatic, afaik this is the default in newer RaspBians.
A simple example would help

And check your config.txt from RaspBian.