It depends on the device used, and whether or not the kernel support or drivers have been added. Some things may work out of the box, others won't (which might be why they're supplying the OS for you to use).
Unless you're incredibly picky about sound, I'm going to suggest a cheap USB sound card, which sounds significantly better than what the Pi dishes out through the analog audio port, and works fine with things like guitarix (which works with real time audio processing (as long as it's plugged directly into the Pi (some USB hubs can be weird))).
There's one like this, which is a bit chunky, and can get in the way of other USB ports (and ideally needs a male to female extension cable):
http://www.ebay.com/itm/291620588938
Or there's this one, which comes with a cable:
http://www.ebay.com/itm/221899723705
For $1 or $1.54 Australian dollars, it's not a huge loss, and makes life a whole lot easier.