kernelcode
Posts: 16
Joined: Tue Nov 22, 2011 3:47 pm

DMA SRC_INC not working as (I) expected

Sun Feb 28, 2016 8:30 pm

I'm playing around with the DMA engine, but it's not quite working like I expected.

I'm trying to "repeat" one word from one memory buffer into another - think memset() but with a 32-bit value. I figured I could do this by setting DEST_INC and clearing SRC_INC in the DMA control block.

I've set up the control block with

Code: Select all

TI = (1 << 4)
(DEST_INC = 1), but it seems like the source is also incrementing in a "strange" way - I get a repeating pattern of the first 16 bytes of the source in my destination buffer (whereas I expected the first 4 bytes repeated)

I'm using DMA channel 5 on a mainline 4.4 kernel if it matters, but I don't think it does. My code is in a kernel module, and I'm using the "streaming" DMA API to sync my buffers between the CPU and the DMA.

Here's some annotated printouts from my code:

Code: Select all

/* Allocate a buffer, and dma_map it */
noddy_open: dma_mapped vaddr da932000 -> paddr 0x1a932000 -> dma 0x1a932000

/* My control block in RAM */
TI:  00000010 /* DEST_INC */
SRC: 1a932020
DST: 1a933000
LEN: 00000020 /* 32 byte transfer */
STR: 00000000
NXT: 00000000

CB: /* Dumped from RAM */
da932000: 10 00 00 00 20 20 93 1a 00 30 93 1a 20 00 00 00
da932010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Source Buffer: /* Initialised to 1-31 */
da932020: 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
da932030: 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f

Dest Buffer: /* Zeroed */
da933000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
da933010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

/* Transfer started and completed (Too fast to time) */
32 byte transfer took 1000 ns

/* Registers read from peripheral */
DMA Registers:
CS:  000f000a /* High prority, DREQ and END */
CB:  00000000
TI:  00000010 /* DEST_INC */
SRC: 1a932020 /* Source address as expected: not changed */
DST: 1a933020 /* Dest address as expected: (start + 32) */
LEN: 00000000
STR: 00000000
NXT: 00000000

CB: /* As before */
da932000: 10 00 00 00 20 20 93 1a 00 30 93 1a 20 00 00 00
da932010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Source Buffer: /* As before */
da932020: 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
da932030: 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f

Dest Buffer: /* Expected repeating "00 01 02 03" */
da933000: 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
da933010: 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
I think the above shows my issue clearly enough but I can post the code if someone thinks it will help - can anyone explain the behaviour? Am I assuming the wrong thing about SRC_INC and DEST_INC?

You'll notice my addresses are in the '0x00000000-prefixed' bus-address region - this is what is returned from the DMA APIs. I also tried manually setting it to the '0xc0000000-prefixed' bus addresses, but it made no difference. I don't think it should matter anyway as the DMA engine isn't behind either of the caches, but I'm happy to be told I'm wrong.

I've tried fiddling with forcing no write bursts and setting burst length = 1, but it seems to make no difference.

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2396
Joined: Thu Jul 11, 2013 2:37 pm

Re: DMA SRC_INC not working as (I) expected

Mon Feb 29, 2016 8:32 am

DMA reads in 128-bit stripes by default. You need to set the source width parameter to 32-bit.
Rockets are loud.
https://astro-pi.org

kernelcode
Posts: 16
Joined: Tue Nov 22, 2011 3:47 pm

Re: DMA SRC_INC not working as (I) expected

Mon Feb 29, 2016 8:47 am

That's not what the documentation says, is that a documentation error?
Source Transfer Width
1 = Use 128-bit source read width.
0 = Use 32-bit source read width.
Image

I'll give it a try tonight anyway.

Thanks

User avatar
joan
Posts: 14948
Joined: Thu Jul 05, 2012 5:09 pm
Location: UK

Re: DMA SRC_INC not working as (I) expected

Mon Feb 29, 2016 9:27 am

I'd put my money on it being a caching problem. Are you sure the memory you are using is cache coherent?

kernelcode
Posts: 16
Joined: Tue Nov 22, 2011 3:47 pm

Re: DMA SRC_INC not working as (I) expected

Mon Feb 29, 2016 10:11 am

Hi Joan,

I don't think this can be a caching problem. I have a couple of reasons for this:

1) I'm using the streaming DMA API to avoid caching problems (no the memory is not cache coherent, but I use the DMA API to perform cache maintenance before and after CPU/device accesses)
- Granted, this could be broken

2) Because the CPU view of the destination buffer shows it zeroed before the DMA transaction, and filled with "something" after the DMA transaction.
I can't see any way for "caching" to cause the data viewed *after* the transaction to be both non-zero and also not-what-is-in-RAM.
If I had caching problems, I would expect that after the transaction, the CPU would see cached data (i.e. zeroes). The fact that it's not seeing zeroes means that it's reading data from somewhere (I guess main RAM). So unless the cache itself is broken, and is reading the wrong part of RAM (yeah... not likely) I don't think "caching" can explain what I'm seeing.

FWIW, if I use 2D mode, with DEST_INC, and strides of 0 I see the behaviour I expected in my test above (i.e. destination buffer is filled with "00 01 02 03")

kernelcode
Posts: 16
Joined: Tue Nov 22, 2011 3:47 pm

Re: DMA SRC_INC not working as (I) expected

Mon Feb 29, 2016 6:50 pm

@jdb, I tried setting the width bits (contrary to the docs) and it made no difference. So I guess that's not it

Code: Select all

[  117.593186] dma_mapped vaddr dd824000 -> paddr 0x1d824000 -> dma 0x1d824000
[  117.608090] TI:  00000230
[  117.613976] SRC: 1d824020
[  117.619739] DST: 1d825000
[  117.625379] LEN: 00000020
[  117.630848] STR: 00000000
[  117.636231] NXT: 00000000
[  117.641464] CB:
[  117.645859] dd824000: 30 02 00 00 20 40 82 1d 00 50 82 1d 20 00 00 00
[  117.655080] dd824010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  117.664149] Source Buffer:
[  117.669496] dd824020: 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
[  117.678659] dd824030: 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f
[  117.687713] Dest Buffer
[  117.692646] dd825000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  117.701722] dd825010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  117.710861] noddy_open: 32 byte transfer took 1000 ns
[  117.718515] DMA Registers:
[  117.723774] CS:  000f000a
[  117.728982] CB:  00000000
[  117.734068] TI:  00000230
[  117.739096] SRC: 1d824020
[  117.744050] DST: 1d825020
[  117.748997] LEN: 00000000
[  117.753930] STR: 00000000
[  117.758764] NXT: 00000000
[  117.763395] CB:
[  117.767155] dd824000: 30 02 00 00 20 40 82 1d 00 50 82 1d 20 00 00 00
[  117.775879] dd824010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  117.784568] Source Buffer:
[  117.789568] dd824020: 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
[  117.798504] dd824030: 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f
[  117.807507] Dest Buffer
[  117.812462] dd825000: 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
[  117.821553] dd825010: 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f

chregu82
Posts: 2
Joined: Mon Aug 10, 2020 1:08 pm

Re: DMA SRC_INC not working as (I) expected

Mon Aug 10, 2020 1:12 pm

I am trying to do the same thing, writing a repeating 4 byte source to a incementing destination. I am experencing the same strange behaviour where at the destination is some data of the source but also random data. I'm using the same options, incrementing dest, but not source.

Has anyone ever solved this problem? I'm going to try it with the 2D option.

jayben
Posts: 59
Joined: Mon Aug 19, 2019 9:56 pm

Re: DMA SRC_INC not working as (I) expected

Tue Aug 11, 2020 10:21 am

In my DMA experimentation ( https://iosoft.blog/raspberry-pi-dma-programming/ ) I did occasionally have problems where non-incrementing memory transfers didn't behave as expected, but peripheral transfers always worked OK. I was able to work around the issue, and assumed it was due to bugs in my code, but you have got me wondering - maybe there is a problem when running un-paced transfers to a single location.

Subsequently, I did some SMI DMA experimentation, and discovered seriously weird symptoms when running flat out on the PI 3 and 4, reinforcing the idea that maybe the cache on a multi-core device does start interfering when a location is pushed hard by DMA.

Do keep us posted.

chregu82
Posts: 2
Joined: Mon Aug 10, 2020 1:08 pm

Re: DMA SRC_INC not working as (I) expected

Wed Aug 12, 2020 5:43 am

Here are more details about what I'm doing. I'd like to fill de screen with pixel data, one byte is one pixel. I'm running this bare metal, without MMU and cache. It acts the same ond all Pi generations.
If i copy pixel data with the 2D DMA transfer, all works fine. But not with the standard transfer if the source is a 32 bit integer with 4 pixels, incrementing destination, but not source. The first 2 bytes seem to be copied, then 2 bytes random data, then the pattern repeats. Normal transfers seem to work, if I transfer one section of the screen to another, so incrementing source and destination.

Return to “Interfacing (DSI, CSI, I2C, etc.)”