You are looking at finding an Analog Telephone Adapter (ATA) that works via USB. I see the acronyms FXS and FXO thrown around too...
Caller ID information is sent over the line at 300 baud between the first and second ring. Assuming you subscribe to the service.
Quick look on the internet for such turns up ...
http://www.cigear.com/sangoma-usb-2-por ... -1520.html