Have you selected analog audio as the output device (left / right mouse click the speaker icon in Raspbian)?
Also, what program are you using to play audio? If you're using omxplayer, you can use command line switches to use analog audio output. VLC has the list of audio devices in one of the menus.