An old laptop running Linux would certainly give you access to an environment suitable for learning programming. In this respect it will deliver everything that the Pi will offer with perhaps less of the hassle in terms of establishing a work system. Perhaps because your kleage will depend very much on the hardware in you laptop, hiwever you will be able to run a 'Live CD' distro to check its viabilty prior to making any commitment in terms of installing the OS. It is even possible to maintain your existing Windoze system and run Linux in parallel on a dual-boot system.
If you like the idea of playing with some hardware and are attracted by the Pis GPIO etc then you need look no farther than the Arduino microcontroller
http://www.arduino.cc/. Progamming is perhaps more restrictive inasmuch as you will have to use C/C++ and the available memory can be a lot less than the Pi. A free IDE (Integrated Development Enviornment) is available, which runs under both Windows and Linux, in which to develop programs, compile and upload to the microcontroller via a standard USB lead. The Arduino has both digital pins (as per the Pis GPIO) and also analogue pins. All can be configured as input or output, again all vey Pi-ish.
Cost-wise an Arduino can be as expensive as the Pi. But smaller ones may be half (or less) the price. The Arduino has no disk, Ethernet, Wi-Fi and so on but these can be added with 'shields' which simply stack on the basic board if they are required. A variety of libraries are freely available to aid the development of larger porjects but coding to drive a few LEDs can be accomplished very readily without the need for learning a number of disparate skills.
Arduino starter kits are widely avalable offering the controller, breadboard, electronic components (such as LEDs, resistors, switches), wires and some useful physical hardware layout and coding. This gets you up and runnign very qucikly and you will not dissipate your enthusiam in peripheral issues.
HTH