User avatar
flatmax
Posts: 336
Joined: Thu May 26, 2016 10:36 pm

Tutorial 2 - Audio programming with gtkIOStream

Wed Jul 26, 2017 12:42 pm

gtkIOStream is a versatile software for signal processing and audio processing, amongst other features (such as GUI programming). It interfaces directly with jackd for low latency audio processing and routing. gtkIOStream also provides a nice port monitor interface which shows which jackd ports are in use and how (similar to the qjackctl connect GUI but different).

In this tutorial we will focus on full duplex streaming using jackd. Full duplex is a term given to both audio output and audio input (half duplex means either output or input). This tutorial will also introduce how to start jackd, the JackPortMonitorGui application and show you the playback and capture mixers settings required for the Audio Injector stereo cards to get input and output audio working so that you can hear something on the input at the output of your hardware.

Before we start, you have to have compiled and installed gtkIOStream. For certain components of gtkIOStream its compilation and installation isn't necessary, but for certain parts of audio processing it is necessary. Tutorial 0 and contents guides you through how to install and test the installation of gtkIOStream. Before you start this tutorial, you must have it installed first.

If you haven't already done so, install jackd2 like so :

Code: Select all

sudo apt-get install jackd2
Our first step is to open the standard lightweight Raspberry Pi IDE with a blank file for us to code up in ... like so :

Code: Select all

geany JackClient.C
Now if you have used ssh to get onto your Pi, then you may have to logout and log in with the "-X" flag like so :
Last edited by flatmax on Wed Jul 26, 2017 1:28 pm, edited 2 times in total.
Check the Ultra 2 sound card - use our shop instead of Amazon Europe (Amazon USA is live).
Sound card for the Raspberry Pi with inbuilt microphone : www.audioinjector.net
Audio Inector Octo multitrack GPIO sound card

User avatar
flatmax
Posts: 336
Joined: Thu May 26, 2016 10:36 pm

Re: Tutorial 2 - Audio programming with gtkIOStream

Wed Jul 26, 2017 12:57 pm

In geany, cut and past the following code :

Code: Select all

#include <JackClient.H>
#include <iostream>
using namespace std;

#include <unistd.h> // for sleep

/** This jack client will play back what it captures
*/
class JackFullDuplex : public JackClient {

	/// The Jack client callback - copy input to output
    int processAudio(jack_nframes_t nframes) {
		if (outputPorts.size()!=inputPorts.size()){
			cout<<"Different input and output port count, don't know how to copy. In port cnt="<<inputPorts.size()<<" output port count="<<outputPorts.size()<<endl;
			return -1;
		}

        for (uint i=0; i<outputPorts.size(); i++) { // for each channel of audio, copy input to output
            jack_default_audio_sample_t *out = ( jack_default_audio_sample_t* ) jack_port_get_buffer ( outputPorts[i], nframes ); // the output sample buffer
            jack_default_audio_sample_t *in = ( jack_default_audio_sample_t* ) jack_port_get_buffer ( inputPorts[i], nframes ); // the inputs sample buffer
            for (uint j=0; j<nframes; j++) // rather then copying, you could do some signal processing here!
                out[j]=in[j];
        }
        return 0;
    }
};

int main(int argc, char *argv[]) {
    JackFullDuplex jackClient; // init the full duplex jack client

    // connect to the jack server
    int res=jackClient.connect("jack full duplex client");
    if (res!=0)
        return JackDebug().evaluateError(res);

    cout<<"Jack : sample rate set to : "<<jackClient.getSampleRate()<<" Hz"<<endl;
    cout<<"Jack : block size set to : "<<jackClient.getBlockSize()<<" samples"<<endl;

	// create the ports
    res=jackClient.createPorts("in ", 2, "out ", 2);
    if (res!=0)
        return JackDebug().evaluateError(res);

    // start the client connecting ports to system ports
    res=jackClient.startClient(2, 2, true);
    if (res!=0)
        return JackDebug().evaluateError(res);

	while (1) sleep(10); // sleep forever
    return 0;
}
Lets step through the code in order of execution.
We start with good old main, which has the standard (unused) input variables argc and argv. The first thing main does is instantiate our jackd client JackFullDuplex :

Code: Select all

int main(int argc, char *argv[]) {
    JackFullDuplex jackClient;
The JackFullDuplex is our C++ class which inherits from the gtkIOStream JackClient, so we include this in order to define it :

Code: Select all

#include <JackClient.H>
#include <iostream>
using namespace std;

#include <unistd.h>
Here you will see the JackClient.H file which defines our JackClient. You will also notice that we include iostream and unistd for console output (cout, endl, etc.) and the call to the function sleep respectively.

Next we define our JackFullDuplex class which inherits from the JackClient class :

Code: Select all

class JackFullDuplex : public JackClient {
Now inside this class, we have to overload the "processAudio" method, because that method will be called by jackd every time it has nframes of audio for us to process. It looks like so :

Code: Select all

    int processAudio(jack_nframes_t nframes) {
		if (outputPorts.size()!=inputPorts.size()){
			cout<<"Different input and output port count, don't know how to copy. In port cnt="<<inputPorts.size()<<" output port count="<<outputPorts.size()<<endl;
			return -1;
		}
This class method inputs the number of frames (audio samples for each channel) require processing and returns a number which indicates 0 for success and keep going, and any other number for error and stop calling this jackd client. The first thing we do is confirm that we have the same number of input and output channels (channels are ports in jackd) and if they differ, we print an error to console and return non zero to tell jackd to stop calling us.

Next, as we are a full duplex "copy" client, we take the input audio and copy it directly to the output. We loop over all of our ports (channels) :

Code: Select all

        for (uint i=0; i<outputPorts.size(); i++) { // for each channel of audio, copy input to output
            jack_default_audio_sample_t *out = ( jack_default_audio_sample_t* ) jack_port_get_buffer ( outputPorts[i], nframes ); // the output sample buffer
            jack_default_audio_sample_t *in = ( jack_default_audio_sample_t* ) jack_port_get_buffer ( inputPorts[i], nframes ); // the inputs sample buffer
            for (uint j=0; j<nframes; j++) // rather then copying, you could do some signal processing here!
                out[j]=in[j];
        }
and we acquire the output and input audio buffers for channel i using the jack_port_get_buffer function. We then loop over all the frames using the variable j and copy the input sample to the output sample.
Finally, we return 0 indicating that everything is good and we want to keep processing next time there are nframes of audio samples available :

Code: Select all

        return 0;
    }
Great ! So now you understand how to program a jackd audio client - it is that simple ! Oh ... if you don't want to use the input channels, then simply don't deal with them in the processAudio JackClient method. If you don't want to use the output channels, then simply don't deal with them in the processAudio JackClient method.

OK - what about the rest of main ? In the rest of the main function we tell our JackFullDuplex client (jackClient in the code) to connect to the jackd server :

Code: Select all

    int res=jackClient.connect("jack full duplex client");
    if (res!=0)
        return JackDebug().evaluateError(res);
we return here on error, where the JackDebug::evaluateError method prints the error to console and returns the error number for us to return to the main caller (the shell).

We print information about the sample rate in use and the number of samples per channel per frame (nframes) :

Code: Select all

    cout<<"Jack : sample rate set to : "<<jackClient.getSampleRate()<<" Hz"<<endl;
    cout<<"Jack : block size set to : "<<jackClient.getBlockSize()<<" samples"<<endl;
We want to create ports (channels) for our client, and in our case we will simply work with a stereo full duplex client, so we define 2 input and output ports :

Code: Select all

    res=jackClient.createPorts("in ", 2, "out ", 2);
    if (res!=0)
        return JackDebug().evaluateError(res);
again, returning on error.

We want to start our client and auto-connect our ports to the system input and output ports :

Code: Select all

    res=jackClient.startClient(2, 2, true);
    if (res!=0)
        return JackDebug().evaluateError(res);
and we are at the races ... at this point, jackd starts calling our JackFullDuplex::processAudio client.

Finally we sleep forever and if for whatever reason we stop sleeping, we return 0 :

Code: Select all

	while (1) sleep(10); // sleep forever
    return 0;
That is it ! You have now completed the flow of this jackd client program. We now have to compile and run this program.
Check the Ultra 2 sound card - use our shop instead of Amazon Europe (Amazon USA is live).
Sound card for the Raspberry Pi with inbuilt microphone : www.audioinjector.net
Audio Inector Octo multitrack GPIO sound card

User avatar
flatmax
Posts: 336
Joined: Thu May 26, 2016 10:36 pm

Re: Tutorial 2 - Audio programming with gtkIOStream

Wed Jul 26, 2017 1:32 pm

We compile it with the following command :

Code: Select all

g++ `pkg-config --cflags --libs gtkIOStream` -o JackClient JackClient.C 
this command should be reasonably familiar to you now, we are compiling the code (JackClient.C) and calling it JackClient.

We have to start jackd before we start our client, if you are using a Pi, you will have to have a sound card installed with audio input. You start jackd with two input and output ports (channels) like so :

Code: Select all

jackd -d alsa -r 48000 -i 2 -o 2
Here we tell jackd to use the sample rate of 48 kHz.
If you don't have a DISPLAY env. variable, jackd will complain and not stare, so either log back in with "ssh -X" (assuming you used ssh to access your term) or declare one first :

Code: Select all

export DISPLAY=:0.0
jackd -d alsa -r 48000 -i 2 -o 2
One of the nice things about gtkIOStream is that it includes a port monitor which is available in both gui and non gui form. Make sure you "ssh -X" into your remote computer (if you are working over a network) so that you can see and execute the following GUI command :

Code: Select all

JackPortMonitorGui
which brings up a gui which looks like so :
JackPortMonitorGui.noClients.png
JackPortMonitorGui without any jackd clients
JackPortMonitorGui.noClients.png (15.81 KiB) Viewed 1608 times
Note that you can drag and drop ALL the buttons to connect/disconnect ports.

Finally we start our jackd client :

Code: Select all

$ ./JackClient 
Jack : sample rate set to : 48000 Hz
Jack : block size set to : 1024 samples
input port 0 latency = 0
input port 1 latency = 0
output port 0 latency = 0
output port 1 latency = 0
JackClient::bufferSizeChangeStatic : New buffer size = 1024
It tells us the sample rate and block size and some information about latency - which isn't relevant here.

You will now observe that the JackPortMonitorGui app shows that the system and our client are fully connected and looks like so :
JackPortMonitorGui.withClient.png
JackPortMonitorGui with our JackFullDuplex client connected.
JackPortMonitorGui.withClient.png (25.8 KiB) Viewed 1608 times
You will be able to drag and drop the buttons from the left side to the right side dropping on any buttons on the right to connect ports together. You can disconnect ports by dragging buttons from the right side to the left side dropping them on the ports you want to disconnect.


At this point you can verify that your client is working by plugging in an audio source to the input lines and playing the output lines through an amplifier or pair of headphones.

If you don't hear any audio then either your player isn't playing or your mixer isn't set up correctly. In the case of the audio injector stereo cards, your playback alsamixer should look like so :
AudioInjector.playback.mixerSettings.lineInput.png
Audio Injector playback settings for line input.
AudioInjector.playback.mixerSettings.lineInput.png (29.82 KiB) Viewed 1608 times
Your capture alsamixer setting should look Image.
Check the Ultra 2 sound card - use our shop instead of Amazon Europe (Amazon USA is live).
Sound card for the Raspberry Pi with inbuilt microphone : www.audioinjector.net
Audio Inector Octo multitrack GPIO sound card

Return to “C/C++”