However, something really odd:
After letting it rest overnight without any card inserted, it worked. I shut it down and then, the problem reappeared.
Could it be capacitor related somehow, taking some time to cooldown or something ?
More likely a polyfuse** issue (partial tripping for some reason). What's the voltage on the "5V" supply rail and what are you powering it with? (One possible cause of "partial trips" could be a "spikey" PSU).
** How many polyfuses does it have? - (early) B1's had three (two extra, 140mA ones on the USB ports), B2's onewards, just the one.
Still running Raspbian Jessie or Stretch on some older Pi's (an A, B1, 2xB2, B+, P2B, 3xP0, P0W, 2xP3A+, P3B+, P3B, B+, and a A+) but Buster on the P4B's. See: https://www.cpmspectrepi.uk/raspberry_pi/raspiidx.htm