yamanoorsai
Posts: 9
Joined: Wed Oct 31, 2012 2:38 am

Reading Audio stream for FFT

Sun Mar 03, 2013 6:18 pm

I am trying to run an FFT on the audio stream so that I could control RGB LEDs. I am trying to use this example from here:

http://julip.co/2012/05/arduino-python- ... -spectrum/

I am getting the following error when I execute the program:

Code: Select all

input_device_index = device)
File "/usr/share/pyshared/pyaudio.py", line 714, in open
stream = Stream(self, *args, **kwargs)
File "/usr/share/pyshared/pyaudio.py", line 396, in __init__
self._stream = pa.open(**arguments)
IOError: [Errno Invalid number of channels] -9998
I have set the device correctly (The device index is 3).

Has anyone tried this before? What am I missing here?

-rst-
Posts: 1316
Joined: Thu Nov 01, 2012 12:12 pm
Location: Dublin, Ireland

Re: Reading Audio stream for FFT

Mon Mar 04, 2013 5:12 pm

Hmm, cannot access that site:
High Risk Website Blocked
Location: julip.co/2012/05/arduino-python-soundlight-spectrum
Access has been blocked as the threat Mal/HTMLGen-A has been found on this website.
Return to the page you were previously viewing.
http://raspberrycompote.blogspot.com/ - Low-level graphics and 'Coding Gold Dust'

yamanoorsai
Posts: 9
Joined: Wed Oct 31, 2012 2:38 am

Re: Reading Audio stream for FFT

Mon Mar 04, 2013 5:16 pm

I have posted the code on pastebin now:

http://pastebin.com/Dmx2bMAb

-rst-
Posts: 1316
Joined: Thu Nov 01, 2012 12:12 pm
Location: Dublin, Ireland

Re: Reading Audio stream for FFT

Mon Mar 04, 2013 5:28 pm

Is that the full error message in the original post?

Maybe you could try this (perhaps even in the interactive python interpreter):

Code: Select all

import pyaudio
p = pyaudio.PyAudio()
stream = p.open(format = pyaudio.paInt16,
                    channels = 1,
                    rate = 44100,
                    input = True,
                    frames_per_buffer = 2**11,
                    input_device_index = 3)
and see what the output is (just to isolate it from the other stuff in that code)?
http://raspberrycompote.blogspot.com/ - Low-level graphics and 'Coding Gold Dust'

yamanoorsai
Posts: 9
Joined: Wed Oct 31, 2012 2:38 am

Re: Reading Audio stream for FFT

Mon Mar 04, 2013 7:24 pm

I tried that:

This is the complete error:

Code: Select all

Expression 'parameters->channelCount <= maxChans' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1438
Expression 'ValidateParameters( inputParameters, hostApi, StreamDirection_In )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2742
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
  File "/usr/lib/pymodules/python2.7/pyaudio.py", line 714, in open
    stream = Stream(self, *args, **kwargs)
  File "/usr/lib/pymodules/python2.7/pyaudio.py", line 396, in __init__
    self._stream = pa.open(**arguments)
IOError: [Errno Invalid number of channels] -9998
This confirms that the code seems to fail at this point. I am not sure what's wrong here

For the command:

Code: Select all

p = pyaudio.PyAudio()
I got the following warning:

Code: Select all

jack server is not running or cannot be started
Is this relevant?

-rst-
Posts: 1316
Joined: Thu Nov 01, 2012 12:12 pm
Location: Dublin, Ireland

Re: Reading Audio stream for FFT

Tue Mar 05, 2013 11:58 am

That message about 'jack' does not seem to be relevant - pyaudio just reports there is no jack (http://jackaudio.org/) running (using it is optional).

Can you play and record audio using some ready-made ALSA tools?
http://raspberrycompote.blogspot.com/ - Low-level graphics and 'Coding Gold Dust'

texy
Forum Moderator
Forum Moderator
Posts: 5155
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England

Re: Reading Audio stream for FFT

Tue Mar 05, 2013 1:04 pm

Interesting. How does it get access to the music stream? I,m guessing its to do with the serial read. Just trying to get my head around the overall sequence of events :?

Texy
Various male/female 40- and 26-way GPIO header for sale here ( IDEAL FOR YOUR PiZero ):
https://www.raspberrypi.org/forums/viewtopic.php?f=93&t=147682#p971555

yamanoorsai
Posts: 9
Joined: Wed Oct 31, 2012 2:38 am

Re: Reading Audio stream for FFT

Tue Mar 05, 2013 2:33 pm

Yeah I am able to play wave files using aplay and mp3 using mpg321.

@texy: The code is supposed to perform an FFT while an audio is being played. In my understanding, it involves tapping into the stream.

-rst-
Posts: 1316
Joined: Thu Nov 01, 2012 12:12 pm
Location: Dublin, Ireland

Re: Reading Audio stream for FFT

Tue Mar 05, 2013 3:26 pm

How about recording? As it is what you are trying to do with the code...

One random thought: is your audio device input stereo - would channels=2 work?
http://raspberrycompote.blogspot.com/ - Low-level graphics and 'Coding Gold Dust'

yamanoorsai
Posts: 9
Joined: Wed Oct 31, 2012 2:38 am

Re: Reading Audio stream for FFT

Tue Mar 05, 2013 7:09 pm

I tried number of channels as 2. The error seems to be the same.

My objective is to run an FFT on the audio output so that I could control RGB lights based on the frequency that is detected in the file that is being played.

-rst-
Posts: 1316
Joined: Thu Nov 01, 2012 12:12 pm
Location: Dublin, Ireland

Re: Reading Audio stream for FFT

Wed Mar 06, 2013 10:38 am

Uh, now I managed to read that code in enough detail... :oops:

So:
- you are playing an audio file on RPi
- you (try to) open the sound card output as an input for pyaudio
- do FFT on that input
- control some leds based on the FFT result (in the example Arduino code the result is written to the serial output)

You do have a sound card that supports this "# Enable stereo mixing in your sound card # to make you sound output an input"?

What does the list_devices() output look like?
http://raspberrycompote.blogspot.com/ - Low-level graphics and 'Coding Gold Dust'

texy
Forum Moderator
Forum Moderator
Posts: 5155
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England

Re: Reading Audio stream for FFT

Wed Mar 06, 2013 11:00 am

Yeah, that is the bit I don't understand - when you are playing audio on the pi, does it effectively open an internal serial stream, that pyaudio has 'read access' too? Otherwise I still do not get how the stream is tapped into.
Texy
Various male/female 40- and 26-way GPIO header for sale here ( IDEAL FOR YOUR PiZero ):
https://www.raspberrypi.org/forums/viewtopic.php?f=93&t=147682#p971555

-rst-
Posts: 1316
Joined: Thu Nov 01, 2012 12:12 pm
Location: Dublin, Ireland

Re: Reading Audio stream for FFT

Wed Mar 06, 2013 12:29 pm

I suppose the original example is for PCs, where some (full-duplex?) sound cards have this thing for 'listening to the output' - so for example you could play streaming stuff from the web and record the audio directly from the same soundcard...
http://raspberrycompote.blogspot.com/ - Low-level graphics and 'Coding Gold Dust'

SpaceGerbil
Posts: 27
Joined: Sat Feb 02, 2013 10:03 am

Re: Reading Audio stream for FFT

Wed Mar 06, 2013 10:31 pm

What a coincidence - I've been looking at exactly this since yesterday.

OK - this is what I have so far...
The code posted here works fine on my Windows PC running python 2.7 (and my laptop running Linux Mint 14):
http://julip.co/2012/05/arduino-python- ... -spectrum/

I modified the program to print the levels before I move on to outputting these to LEDs.

You need to check the input devices present. This is important because the RPi returns nothing for pyaudio to work with if you are only using the onboard audio. rst is spot on in his reply - you need a USB sound card that can open and read the stream. Forget about any code referring to 'serial' for the time being - this is how she sent the outputs to the LEDs with the Arduino (I'm assuming you will be using the GPIO pins on the Rpi). Once I had chosen the correct device (Line In), I could connect any audio source to the soundcard's Line In. After removing all references to the serial output I checked that the program was reading the stream correctly and outputting something reasonable. Bingo - no problems so far.

Now the Rpi. I gave up on my cheapo USB audio device (way too much noise and crackling). The next thing to try was to stream audio from a file. This seems to be working great, but my code needs tidying.

I'm a bit busy at the moment, but will try to post my code and (hopefully) a working LED equalizer at the weekend.

texy
Forum Moderator
Forum Moderator
Posts: 5155
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England

Re: Reading Audio stream for FFT

Thu Mar 07, 2013 7:46 am

Good stuff - it'll be interesting to see the code ;)
So, apart from drving the LED's, you code currently reads an audio file (mp3, wav, etc), and can analyse the frequencies and get values for you chosen bands 'real time' as it is playing out audio? ie, only 1 process is running. Or does it still need aplay, mpg321 to play the file?

Texy
Various male/female 40- and 26-way GPIO header for sale here ( IDEAL FOR YOUR PiZero ):
https://www.raspberrypi.org/forums/viewtopic.php?f=93&t=147682#p971555

-rst-
Posts: 1316
Joined: Thu Nov 01, 2012 12:12 pm
Location: Dublin, Ireland

Re: Reading Audio stream for FFT

Thu Mar 07, 2013 10:30 am

Well, doesn't the 'opening the audio output as an input' (instead of just reading the audio file directly) suggest that some process has to be playing the audio to the output in the first place?

Hmm, like I remember someone mentioning hooking up to some Python player to get (some) levels while playing the file...
http://raspberrycompote.blogspot.com/ - Low-level graphics and 'Coding Gold Dust'

yamanoorsai
Posts: 9
Joined: Wed Oct 31, 2012 2:38 am

Re: Reading Audio stream for FFT

Sat Mar 09, 2013 2:27 pm

Sincere apologies for the late reply. I did not have a USB sound card. I was trying it out with the onboard sound drivers. I ordered a USB sound card and the code worked like a charm. Thanks so much for the help.

yamanoorsai
Posts: 9
Joined: Wed Oct 31, 2012 2:38 am

Re: Reading Audio stream for FFT

Sat Mar 09, 2013 4:48 pm

I ran into another problem today. I am able to run a program that simply calculates the volume levels of the audio. If I try to perform an FFT, I am getting buffer overflow errors. I tried increasing the sample rate to 48000 and 88200. I also increased the buffer size but with no luck.

The error is as follows:

Code: Select all

   data = stream.read(chunk)
  File "/usr/lib64/python2.7/site-packages/pyaudio.py", line 564, in read
    return pa.read_stream(self._stream, num_frames)
IOError: [Errno Input overflowed] -9981
The code is here:
http://pastebin.com/pjcbPPLa

Can this be considered as a limitation of pyaudio

-rst-
Posts: 1316
Joined: Thu Nov 01, 2012 12:12 pm
Location: Dublin, Ireland

Re: Reading Audio stream for FFT

Mon Mar 11, 2013 3:16 pm

Might be a performance issue - RPi CPU too slow for that amount of data... could be something else too of course.
http://raspberrycompote.blogspot.com/ - Low-level graphics and 'Coding Gold Dust'

SpaceGerbil
Posts: 27
Joined: Sat Feb 02, 2013 10:03 am

Re: Reading Audio stream for FFT

Tue Mar 12, 2013 11:29 pm

:evil: Arrrrrrrrrggggggghhhhhhh.
This is driving me nuts. It shouldn't be that hard. Everything works like a dream on my laptop, but the RPi is not playing ball. It is quite probable that I am too stupid.
Might be a performance issue - RPi CPU too slow for that amount of data.
This occurred to me, so I paused the stream before processing the FFT. Bizarrely it worked briefly at times, but not others. I seem to be successful in recording the chunks, but the code locks at different stages when calculating the levels (it cannot get past the struct.unpack command). I think that this is where it converts shorts to floats.
Is that the issue?

I am looking at other ways to handle this but we might need an RPi guru :ugeek: to help us out.

yamanoorsai
Posts: 9
Joined: Wed Oct 31, 2012 2:38 am

Re: Reading Audio stream for FFT

Wed Mar 13, 2013 12:41 am

I would appreciate it if you could post an update when you find a solution. It's killing me :cry:

texy
Forum Moderator
Forum Moderator
Posts: 5155
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England

Re: Reading Audio stream for FFT

Wed Mar 13, 2013 7:21 am

SpaceGerbil wrote::evil: Arrrrrrrrrggggggghhhhhhh.
This is driving me nuts. It shouldn't be that hard. Everything works like a dream on my laptop, but the RPi is not playing ball. It is quite probable that I am too stupid.
Might be a performance issue - RPi CPU too slow for that amount of data.
This occurred to me, so I paused the stream before processing the FFT. Bizarrely it worked briefly at times, but not others. I seem to be successful in recording the chunks, but the code locks at different stages when calculating the levels (it cannot get past the struct.unpack command). I think that this is where it converts shorts to floats.
Is that the issue?

I am looking at other ways to handle this but we might need an RPi guru :ugeek: to help us out.
The first thing anyone would ask is for you to post your code ;)

T.
Various male/female 40- and 26-way GPIO header for sale here ( IDEAL FOR YOUR PiZero ):
https://www.raspberrypi.org/forums/viewtopic.php?f=93&t=147682#p971555

-rst-
Posts: 1316
Joined: Thu Nov 01, 2012 12:12 pm
Location: Dublin, Ireland

Re: Reading Audio stream for FFT

Sat Mar 16, 2013 5:39 pm

Looks to me it is the same code yamanoorsai used (links posted above).

This may be too obvious, but have you checked what length of data causes the issue? I'd start by adding some debug output around the code at line 98 (behind the paste.bin link):

Code: Select all

    # Convert raw sound data to Numpy array
    datalen = len(data)
    print "datalen = %d" % datalen
    fmt = "%dH"%(datalen/2)
    data2 = struct.unpack(fmt, data)
    data2 = numpy.array(data2, dtype='h')
..maybe there is a zero length data passed at times and the unpack does not like it, or something...
http://raspberrycompote.blogspot.com/ - Low-level graphics and 'Coding Gold Dust'

SpaceGerbil
Posts: 27
Joined: Sat Feb 02, 2013 10:03 am

Re: Reading Audio stream for FFT

Sat Mar 16, 2013 7:05 pm

I'm pretty sure that you are correct about this. I eventually abandoned pyaudio and, on a recommendation went for alsaaudio. This also threw up errors, but it was much easier to spot where they were.

Anyway, I've got it all working!

YouTube video of the RPi in action below:

http://www.youtube.com/watch?v=YPf-5CJ1f3g

I'm off to watch the French get beaten at rugby. Will post the code later tonight.

SpaceGerbil
Posts: 27
Joined: Sat Feb 02, 2013 10:03 am

Re: Reading Audio stream for FFT

Sun Mar 17, 2013 12:27 am

OK - France won, but came last.

I was starting to get very annoyed with this little RPi beast. It would seem that a bad workman always blames his tools....

Here is the code:

Code: Select all

#!/usr/bin/env python
 
import alsaaudio as aa
import audioop
from time import sleep

import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BOARD)
GPIO.setwarnings(False)

#Define physical header pin numbers for 6 LEDs
RPiPins=[11,12,13,15,16,18]
#Set all pins as output
for pin in RPiPins:
		GPIO.setup(pin, GPIO.OUT)

# Set up audio
data_in = aa.PCM(aa.PCM_CAPTURE, aa.PCM_NONBLOCK)
data_in.setchannels(2)
data_in.setrate(44100)
data_in.setformat(aa.PCM_FORMAT_S16_LE)

data_in.setperiodsize(256)

while True:
	# Read data from device
	l,data = data_in.read()
	if l:
		# catch frame error
		try:
			max_vol=audioop.max(data,2)
			scaled_vol = max_vol//4680		
			if scaled_vol==0:
				for pin in range(0,6):
					GPIO.output(RPiPins[pin], False)
			else:
				for pin in range(0,scaled_vol):
					GPIO.output(RPiPins[pin], True)
				for pin in range(scaled_vol,6):
					GPIO.output(RPiPins[pin], False)
				
		except audioop.error, e:
			if e.message !="not a whole number of frames":
				raise e
On and off, I spent two weeks trying to get this working in ALSA. It seemed my cheapo USB audio was not so bad after all, but needed modding.
Gave up on pyaudio. Too much baggage.
Alsaaudio on the other hand is low maintenance.

I will post all the pfaffing I did on my website in the next couple of days.

I have not tackled FFT just yet. This code gets over most buffer problems, but everything I do with floating point numbers locks the RPi. Working on it..........

Please query if you have any problems.

PS I have this running with my music as I type. It is soooooooo cool.

PPS Working on a guitar tuner for my daughter...

PPPS Working on a Stephen Hawking speech emulator for my son.

PPPPS Working on a Darth Vader voice for me.

Return to “Python”