SPI DMA Driver for Raspberry Pi 2

[jmtkelly]

I am currently working on a project in which I would like to perform an SPI Transfer
of approx 500 bytes each time a GPIO_GEN_0 interrupt occurs. I have successfully
authored code in userspace using ioctl to perform a SPI transfer (as a test of the hardware)
and have also authored a kernel module which installs an Interrupt Service Routine that is called
on the positive edge of the GPIO_GEN_0 input. I now need to write some SPI transfer code in
the kernel module that handles the GPIO interrupt so that this transfer is done in an interrupt
service routine each time the GPIO_GEN_0 line is asserted. This is where the confusion comes in.

I was thinking that using DMA for SPI would be the best way to do this which lead me to the msperl
spi-bcm2835 driver. If I go to the msperl git repository and view the spi-bcm2835.c file, I see that it
is authored by Chris Boot (which appears to be the same module that is loaded if I load the spi driver
using raspi-config). I do not see any reference to DMA functions in this c source file. It would appear that
the original DMA source was writted by notro and then forked by msperl. Is this code the code that is currently
distributed as part of raspbian distro? Where are the DMA functions? Is there any sample code available to
illustrated how the SPI DMA code should be used inside a kernel module? I found this example but do not see the
functions that it uses in the msperls git repository:

viewtopic.php?&t=50893

Any help would be appreciated

[joan]

It may be best to PM msperl direct or to reply on one of his recent threads.

[jmtkelly]

It may be best to PM msperl direct or to reply on one of his recent threads.

Thanks Joan. I will try sending msperl a message then.

[msperl]

From userspace just use SPIDEV, which shall use DMA transparently with any of the new foundation-provided kernels (e.g. 4.1 or higher) - this works for all versions of the rpi.

[msperl]

Example code:

Code: Select all

struct mydevice_priv {
  struct spi_device *spi;
  struct spi_message spi_msg;
  struct spi_transfer spi_xfer;
  char data_tx[500];
  char data_rx[500];
};

static irqreturn_t mydevice_irq_handler(int irq, void *dev_id)
{
  struct mydevice_priv *priv = dev_id;
  struct spi_device *spi = priv->spi;

  /* execute the prepared spi-message asynchronously */
  spi_async(spi, spi_msg);
  /* note that you need to make sure that only one interrupt occurs,
   * as the same message can not be in flight multiple times...*/
}

static void mydevice_spi_msg_finished(void *context)
{
  struct mydevice_priv *priv = context;
  /* do whatever you want here */
}

static int mydevice_probe(struct spi_device *spi)
{
  int err;
  struct mydevice_priv *priv;

  /* allocate private structure */
  priv = devm_zalloc(sizeof(*priv),GPF_KERNEL);
  if (!priv)
    return -ENOMEM;

  /* set up things */
  priv->spi = dev;

  /* initialize spi message */
  spi_message_init(&priv->spi_msg);
  spi_message_add_tail(&priv->spi_xfer, &priv->spi_msg);

  /* only transfer data to device */
  priv->spi_xfer.len = sizeof(data);
  priv->spi_xfer.tx = data_tx;
  priv->spi_xfer.rx = data_rx;

  /* what to call asynchronously when the transfer is done */
  priv->spi_msg.complete = mydevice_spi_msg_finished;
  priv->spi_msg.context = priv;

  /* register irq handler */  
  err = devm_request_irq(spi->irq, NULL, mydevice_irq_handler, 0, DEVICE_NAME, priv);
  if (!ret)
    return err;

  /* other preparation stuff */
  ...

  /* return ok */
  return 0;
}

static const struct of_device_id mydevice_of_match[] = {
        {
                /* compatiblity string - typically <vendor>,<chip> */
                .compatible     = "mydevice-name-in-devicetree",
        },
        { }
};
MODULE_DEVICE_TABLE(of, mydevice_of_match);

static struct spi_driver mydevice_driver = {
        .driver = {
                .name = DEVICE_NAME,
                .owner = THIS_MODULE,
                .of_match_table = mydevice_of_match,
        },
        .id_table = mydevice_id_table,
        .probe = mydevice_probe,
};
module_spi_driver(mydevice_driver);

MODULE_AUTHOR("whatever");
MODULE_DESCRIPTION("whatever");
MODULE_LICENSE("GPL");

This is the asynchronous approach, but you could also do a synchronous approach using:
devm_request_threaded_irq and then you can use spi_sync() to run your transfers inside the interrupt handler - no need for complete or other stuff...

Note that you will require to configure your device tree like this (maybe via an overlay):

Code: Select all

/* set up GPIO used for IRQ as input */
&gpio {
  mydevice_gpio_in: mydevice_gpio_in {
    bcrm,function = <0>; /* input */
    bcrm,pins = <&gpio PINNUM>;
  };
};

/* register the device on chip-select X*/
&spi {
  /* disable spidev on chip-select X first */
  [email protected] {
    status = "disabled";
  };

  /* and register your device instead for chip-select X */
  mydevice {
    /* chip-select */
    reg = <X>; 
    /* compatiblity string - typically <vendor>,<chip> */
    /* this maps to the driver via one of the above tables in the module */
    compatible = "mydevice-name-in-devicetree";
    /* the spi frequency to use */
    spi-max-frequency = <1000000>;
    /* set up gpios for real - when the device gets activated */
    pinctrl-names = "default";
    pinctrl-0 = <&mydevice_gpio_in>;
    /* define the interrupt to use */
    interrupt-parent = <&gpio>;
    interrupts = <PINNUM 0x2>;
    /* activate the device */
    status = "okay";
  };
};

[tuck1s]

Hi, what throughput can you realistically get out of an SPI -> Raspi transfer?
The project I'm working on would need approx 1.6Mbytes/second minimum (one way i.e. into the Raspi).

[joan]

Hi, what throughput can you realistically get out of an SPI -> Raspi transfer?
The project I'm working on would need approx 1.6Mbytes/second minimum (one way i.e. into the Raspi).

Is that 1 transfer per second of 1.6M, 2 per second of 0.8M, 16 per second of 100K, a thousand per second of 1.6K?

[tuck1s]

I can adjust the transfer size, as the other end is a high speed XMOS device (with limited RAM).
So it makes sense to consider transfer sizes that will be a good fit for the final destination (disk) without running out of RAM. So let's consider (using the actual numbers)

16 channels of 16-bit, 48kHz audio = 1,536,000 bytes per second

16384 byte transfer chunks = 93.75 transfers per second
32768 byte transfer chunks = 46.875 transfers per second

The Raspi code would transfer this to a hard drive. I know the USB interface on the Raspi is not ideal, but a simple experiment shows that it could easily achieve this rate with C code writing to a laptop USB hard drive, with only 11% CPU utilisation. So it's the SPI interface that's the concern.

Does the Raspi have to be the clock master? It would be better if the transfers were synchronous with the data source.
Otherwise I suppose I'll need to TLV (type-length-value) encode the reply packets to allow them to be non-full.

[joan]

Those sort of transfer rates will be fine. My concern was if you needed 20k or so transfers per second.

Why not just try and see? Strap MISO low or high and try the sustained transfer, you don't need anything at the other end of the link to respond as SPI can't tell.

[mjubes]

>Why not just try and see?

With a test userspace program using spidev (ioctl calls), jessie 4.1.10, and a RPI B+, I can manage 2k in/out byte transfers in about 700uS each (give or take the odd kernel interruption)

However, there seems to be a rather constant 600uS gap between consecutive xfer calls, which also seem to be independent of packet size, i.e. a 1K transfer shows the same gap.

Does anybody know where this gap comes from?

[joan]

What bits per second are you using? I am getting some surprisingly slow results as well.

[mjubes]

>What bits per second are you using? I am getting some surprisingly slow results as well.

I am using 32000 for the IOC_WR_MAX_SPEED ioctl() call, the highest speed it will go, higher values run at the same speed.

The average transfer rate I am getting with spidev is about 2K bytes per 1mS.(including kernel hiccups and inter xfer gaps)
With the spi_bcm2708 driver the CPU itself is running at 80-90% usage (B+)
With the spi_bcm2835 driver the CPU is running at 15% usage (B+)

So it sounds like the 2835 driver may be using DMA, whereas the 2708 relies on too many interrupts (can anybody confirm this?)

To get the spi-bcm2835 driver running (instead of the default 2708) I disabled SPI in raspi-config, and then added the line
dtoverlay=spi-bcm2835 to "/boot/config.txt"

Running similar tests with I2S transfers, in and out, (48000 * 2 X 24 bits per sec) gets the CPU running at 3-4% usage.

[Kroltez]

Example code:

Code: Select all

struct mydevice_priv {
  struct spi_device *spi;
  struct spi_message spi_msg;
  struct spi_transfer spi_xfer;
  char data_tx[500];
  char data_rx[500];
};

static irqreturn_t mydevice_irq_handler(int irq, void *dev_id)
{
  struct mydevice_priv *priv = dev_id;
  struct spi_device *spi = priv->spi;

  /* execute the prepared spi-message asynchronously */
  spi_async(spi, spi_msg);
  /* note that you need to make sure that only one interrupt occurs,
   * as the same message can not be in flight multiple times...*/
}

static void mydevice_spi_msg_finished(void *context)
{
  struct mydevice_priv *priv = context;
  /* do whatever you want here */
}

static int mydevice_probe(struct spi_device *spi)
{
  int err;
  struct mydevice_priv *priv;

  /* allocate private structure */
  priv = devm_zalloc(sizeof(*priv),GPF_KERNEL);
  if (!priv)
    return -ENOMEM;

  /* set up things */
  priv->spi = dev;

  /* initialize spi message */
  spi_message_init(&priv->spi_msg);
  spi_message_add_tail(&priv->spi_xfer, &priv->spi_msg);

  /* only transfer data to device */
  priv->spi_xfer.len = sizeof(data);
  priv->spi_xfer.tx = data_tx;
  priv->spi_xfer.rx = data_rx;

  /* what to call asynchronously when the transfer is done */
  priv->spi_msg.complete = mydevice_spi_msg_finished;
  priv->spi_msg.context = priv;

  /* register irq handler */  
  err = devm_request_irq(spi->irq, NULL, mydevice_irq_handler, 0, DEVICE_NAME, priv);
  if (!ret)
    return err;

  /* other preparation stuff */
  ...

  /* return ok */
  return 0;
}

static const struct of_device_id mydevice_of_match[] = {
        {
                /* compatiblity string - typically <vendor>,<chip> */
                .compatible     = "mydevice-name-in-devicetree",
        },
        { }
};
MODULE_DEVICE_TABLE(of, mydevice_of_match);

static struct spi_driver mydevice_driver = {
        .driver = {
                .name = DEVICE_NAME,
                .owner = THIS_MODULE,
                .of_match_table = mydevice_of_match,
        },
        .id_table = mydevice_id_table,
        .probe = mydevice_probe,
};
module_spi_driver(mydevice_driver);

MODULE_AUTHOR("whatever");
MODULE_DESCRIPTION("whatever");
MODULE_LICENSE("GPL");

This is the asynchronous approach, but you could also do a synchronous approach using:
devm_request_threaded_irq and then you can use spi_sync() to run your transfers inside the interrupt handler - no need for complete or other stuff...

Note that you will require to configure your device tree like this (maybe via an overlay):

Code: Select all

/* set up GPIO used for IRQ as input */
&gpio {
  mydevice_gpio_in: mydevice_gpio_in {
    bcrm,function = <0>; /* input */
    bcrm,pins = <&gpio PINNUM>;
  };
};

/* register the device on chip-select X*/
&spi {
  /* disable spidev on chip-select X first */
  [email protected] {
    status = "disabled";
  };

  /* and register your device instead for chip-select X */
  mydevice {
    /* chip-select */
    reg = <X>; 
    /* compatiblity string - typically <vendor>,<chip> */
    /* this maps to the driver via one of the above tables in the module */
    compatible = "mydevice-name-in-devicetree";
    /* the spi frequency to use */
    spi-max-frequency = <1000000>;
    /* set up gpios for real - when the device gets activated */
    pinctrl-names = "default";
    pinctrl-0 = <&mydevice_gpio_in>;
    /* define the interrupt to use */
    interrupt-parent = <&gpio>;
    interrupts = <PINNUM 0x2>;
    /* activate the device */
    status = "okay";
  };
};

Do you have a more complete version of this example Ijust get errors if I try and compile ?