markus3141
Posts: 39
Joined: Thu May 31, 2012 2:15 pm
Location: near Hamburg, Germany

CM4 MSI-X support (Coral TPU)

Thu Dec 03, 2020 10:04 am

Hi everyone,

from the datasheet I understand the CM4 currently does not support PCIe MSI-X. Is this a firmware/driver or actual hardware limitation, i.e. is there a chance it could get implemented at some point?

I'm planning a CM4 product, but the PCIe device I'm planning on using apparently needs MSI-X. I haven't received my hardware yet, so I can't test if it works anyway, but I doubt it.


Thank you
Last edited by markus3141 on Thu Jan 14, 2021 7:01 pm, edited 1 time in total.

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 3834
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: CM4 MSI-X support

Thu Dec 03, 2020 11:12 am

It's a bit of both. I'm just playing with this at the moment and I found that if you modify the pcie-brcmstb driver so that the msi_domain_info structure includes MSI_FLAG_PCI_MSIX in the .flags field then it will work, but you are limited to a maximum of 32 MSI-Xs. I've also got a hack that allows the same MSI value to be issued to two MSI-Xs, providing support for 64 MSI-Xs by effectively sharing interrupts, but it's not been tested with any kind of contention yet.

markus3141
Posts: 39
Joined: Thu May 31, 2012 2:15 pm
Location: near Hamburg, Germany

Re: CM4 MSI-X support

Thu Dec 03, 2020 11:34 am

Thank you, that sounds promising. The driver I need (Google Apex for Coral TPUs) seems to use 12 interrupts (https://coral.googlesource.com/linux-im ... iver.c#191), so it could be fine.

If anyone has tested Coral PCIe accelerators with the CM4, I'd love to hear. Mine won't be here for at least a week or two.

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 3834
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: CM4 MSI-X support

Thu Dec 03, 2020 11:38 am

Since my reply, one has appeared on my desk...

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 3834
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: CM4 MSI-X support

Thu Dec 03, 2020 2:26 pm

Merely setting the magic MSIX flag in the PCIe driver is enough to allow the Coral driver start up. I'm in the middle of something else at the moment and reluctant to mess up my current CM4 image (I should try a Lite) in order to test the Tensor development software, but I've pushed that change to the rpi-5.10.y branch - it will be set in future kernel releases.

markus3141
Posts: 39
Joined: Thu May 31, 2012 2:15 pm
Location: near Hamburg, Germany

Re: CM4 MSI-X support

Thu Dec 03, 2020 2:29 pm

Thank you very much, that is very good to hear.

acsmith
Posts: 8
Joined: Thu Mar 19, 2020 5:41 pm

Re: CM4 MSI-X support

Fri Dec 04, 2020 2:00 am

Hi,

This is awesome news. Using a Gumstix board - https://www.gumstix.com/cm4-development-board.html and an M.2 Accelerator B+M key module I got the following.
  1. Clone the rpi-5.10.y branch given above.
  2. Add the newest gasket driver from Google - https://coral.googlesource.com/linux-im ... ng/gasket/
  3. Compile and flash with the newest Raspberry Pi OS 64bit - https://downloads.raspberrypi.org/raspi ... -arm64.zip
  4. Confirm /dev/apex_0 is present
  5. Follow all steps 1-6 https://coral.ai/docs/m2/get-started/#2a-on-linux except remove gasket-dkms from step 2
  6. pip3 install https://github.com/google-coral/pycoral ... arch64.whl
  7. Run the model https://coral.ai/docs/m2/get-started/#4 ... w-lite-api
When I run python3 classify_image.py ... I get the following output

Code: Select all

Traceback (most recent call last):
  File "classify_image.py", line 122, in <module>
    main()
  File "classify_image.py", line 99, in main
    interpreter = make_interpreter(args.model)
  File "classify_image.py", line 73, in make_interpreter
    {'device': device[0]} if device else {})
  File "/home/pi/.local/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 155, in load_delegate
    library, str(e)))
ValueError: Failed to load delegate from libedgetpu.so.1
If I remove the PCIe device, plug in a USB accelerator, restart and then run the same code it outputs the expected inference times. Below is a few more details about the modules installed and the PCIe info. Looks like MSI-X is enabled but the device isn't loading. I'm talking with the folks at Google to see if they've got any suggestions. Any suggestions for things to try or look into would be greatly appreciated. Thanks.

Code: Select all

pi@raspberrypi:~ $ modinfo gasket
filename:       /lib/modules/5.10.0-rc6-v8+/kernel/drivers/staging/gasket/gasket.ko
author:         Rob Springer <rspringer@google.com>
license:        GPL v2
version:        1.1.3
description:    Google Gasket driver framework
srcversion:     E193514267853028E5C035A
depends:        
staging:        Y
intree:         Y
name:           gasket
vermagic:       5.10.0-rc6-v8+ SMP preempt mod_unload modversions aarch64
parm:           dma_bit_mask:int

Code: Select all

pi@raspberrypi:~ $ modinfo apex
filename:       /lib/modules/5.10.0-rc6-v8+/kernel/drivers/staging/gasket/apex.ko
author:         John Joseph <jnjoseph@google.com>
license:        GPL v2
version:        1.1
description:    Google Apex driver
srcversion:     D861FD5D8C12609DD8551D5
alias:          pci:v00001AC1d0000089Asv*sd*bc*sc*i*
depends:        gasket
staging:        Y
intree:         Y
name:           apex
vermagic:       5.10.0-rc6-v8+ SMP preempt mod_unload modversions aarch64
parm:           allow_power_save:int
parm:           allow_sw_clock_gating:int
parm:           allow_hw_clock_gating:int
parm:           bypass_top_level:int
parm:           trip_point0_temp:int
parm:           trip_point1_temp:int
parm:           trip_point2_temp:int
parm:           hw_temp_warn1:int
parm:           hw_temp_warn2:int
parm:           hw_temp_warn1_en:bool
parm:           hw_temp_warn2_en:bool
parm:           temp_poll_interval:int

Code: Select all

pi@raspberrypi:~ $ sudo lspci -vvv -s 01:
01:00.0 System peripheral: Device 1ac1:089a (prog-if ff)
	Subsystem: Device 1ac1:089a
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 50
	Region 0: Memory at 600100000 (64-bit, prefetchable) [size=16K]
	Region 2: Memory at 600000000 (64-bit, prefetchable) [size=1M]
	Capabilities: [80] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
			ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [d0] MSI-X: Enable+ Count=128 Masked-
		Vector table: BAR=2 offset=00046800
		PBA: BAR=2 offset=00046068
	Capabilities: [e0] MSI: Enable- Count=1/32 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [f8] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Vendor Specific Information: ID=1556 Rev=1 Len=008 <?>
	Capabilities: [108 v1] Latency Tolerance Reporting
		Max snoop latency: 0ns
		Max no snoop latency: 0ns
	Capabilities: [110 v1] L1 PM Substates
		L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
			  PortCommonModeRestoreTime=10us PortTPowerOnTime=10us
		L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
			   T_CommonMode=0us LTR1.2_Threshold=0ns
		L1SubCtl2: T_PwrOn=10us
	Capabilities: [200 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Kernel driver in use: apex
	Kernel modules: apex

Valdior
Posts: 1
Joined: Thu Dec 31, 2020 4:58 am

Re: CM4 MSI-X support

Thu Dec 31, 2020 5:03 am

acsmith wrote:
Fri Dec 04, 2020 2:00 am
Hi,

This is awesome news. Using a Gumstix board - https://www.gumstix.com/cm4-development-board.html and an M.2 Accelerator B+M key module I got the following.
  1. Clone the rpi-5.10.y branch given above.
  2. Add the newest gasket driver from Google - https://coral.googlesource.com/linux-im ... ng/gasket/
  3. Compile and flash with the newest Raspberry Pi OS 64bit - https://downloads.raspberrypi.org/raspi ... -arm64.zip
  4. Confirm /dev/apex_0 is present
  5. Follow all steps 1-6 https://coral.ai/docs/m2/get-started/#2a-on-linux except remove gasket-dkms from step 2
  6. pip3 install https://github.com/google-coral/pycoral ... arch64.whl
  7. Run the model https://coral.ai/docs/m2/get-started/#4 ... w-lite-api
When I run python3 classify_image.py ... I get the following output

Code: Select all

Traceback (most recent call last):
  File "classify_image.py", line 122, in <module>
    main()
  File "classify_image.py", line 99, in main
    interpreter = make_interpreter(args.model)
  File "classify_image.py", line 73, in make_interpreter
    {'device': device[0]} if device else {})
  File "/home/pi/.local/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 155, in load_delegate
    library, str(e)))
ValueError: Failed to load delegate from libedgetpu.so.1
If I remove the PCIe device, plug in a USB accelerator, restart and then run the same code it outputs the expected inference times. Below is a few more details about the modules installed and the PCIe info. Looks like MSI-X is enabled but the device isn't loading. I'm talking with the folks at Google to see if they've got any suggestions. Any suggestions for things to try or look into would be greatly appreciated. Thanks.

[/code]

Hi! I tried the same with my CM4 + IO Board and the latest 5.10 branch from Github - results are the same. It sees the pcie module, apex driver, but no connection with coral.
Any progress yet?

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1693
Joined: Sat Sep 10, 2011 11:43 am

Re: CM4 MSI-X support

Thu Dec 31, 2020 2:01 pm

Google is looking at the driver to work around an issue with writing to 64bit registers over the PCIe bus
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

markus3141
Posts: 39
Joined: Thu May 31, 2012 2:15 pm
Location: near Hamburg, Germany

Re: CM4 MSI-X support

Thu Dec 31, 2020 2:45 pm

I tried to patch up the module myself by replacing the 64bit writes and reads with two 32bit writes and reads where necessary to get a 32bit kernel compiled. I don't know if thats a valid way to fix this since writes aren't atomic anymore. The module compiles, but on loading I get the following messages:

Code: Select all

[    7.331426] apex 0000:01:00.0: enabling device (0140 -> 0142)
[    7.331957] apex 0000:01:00.0: Cannot get BAR2 base address
[    7.332768] apex 0000:01:00.0: error adding gasket device
[    7.332815] apex: probe of 0000:01:00.0 failed with error -12
Since I'm not really a kernel development expert, I have no idea if that's related to my changes or a separate problem. Maybe somebody else hat a go at this?

markus3141
Posts: 39
Joined: Thu May 31, 2012 2:15 pm
Location: near Hamburg, Germany

Re: CM4 MSI-X support

Thu Jan 14, 2021 4:27 pm

Turns out there were a few ulong casts which cut the upper 64bit off. I fixed those, now the driver loads and I can read the temperature etc.

Sadly libedgetpu still won't cooperate as there still seem to be some issues with reading 64bit registers from the TPU. In this case the lower and upper 32bit seems to be the same for some reason.

This is what libedgetpu reports:

Code: Select all

I :273] Starting in normal mode
I :83] Opening /dev/apex_0. read_only=0
I :97] mmap_offset=0x0000000000040000, mmap_size=4096
I :108] Got map addr at 0x0xb6f49000
I :97] mmap_offset=0x0000000000044000, mmap_size=4096
I :108] Got map addr at 0x0xb6f48000
I :97] mmap_offset=0x0000000000048000, mmap_size=4096
I :108] Got map addr at 0x0xb6f47000
I :211] Read: offset = 0x00000000000486f0, value: = 0x0000000000000000
I :190] Write: offset = 0x00000000000487a8, value = 0x0000000000000000
I :211] Read: offset = 0x0000000000048578, value: = 0x0000001000000010
I :296] descriptor_result=68719476752, sizeof=16 (I added this debug statement, 68719476752=0x1000000010)
I :122] Closing /dev/apex_0. mmap_offset=0x0000000000040000, mmap_size=4096, read_only=0
I :122] Closing /dev/apex_0. mmap_offset=0x0000000000044000, mmap_size=4096, read_only=0
I :122] Closing /dev/apex_0. mmap_offset=0x0000000000048000, mmap_size=4096, read_only=0
I :384] Failed to open device [Apex (PCIe)] at [/dev/apex_0]: Internal: Size of |Element| does not match with the hardware.
I've got the "fixed" and updated driver here in case anyone feels like hacking around: https://github.com/markus-k/rpi-linux/t ... 0.y-gasket

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 3834
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: CM4 MSI-X support

Thu Jan 14, 2021 4:54 pm

That's a known "limitation" of the PCIe root complex - it doesn't support 64-bit wide accesses (not to be confused with 64-bit addresses).

markus3141
Posts: 39
Joined: Thu May 31, 2012 2:15 pm
Location: near Hamburg, Germany

Re: CM4 MSI-X support

Thu Jan 14, 2021 4:57 pm

I'm doing two 32bit accesses:

Code: Select all

static inline u64 gasket_readq_relaxed(const volatile __iomem void *addr)
{
	const volatile __iomem u32 *p = (const volatile __iomem u32 *)addr;
	u32 low, high;

	low = readl_relaxed(p);
	high = readl_relaxed(p + 1);

	return low + ((u64)high << 32);
}
I wonder if thats a limitation of the TPU then, not allowing 32bit accesses to 64bit registers.

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2582
Joined: Thu Jul 11, 2013 2:37 pm

Re: CM4 MSI-X support (Coral TPU)

Fri Jan 15, 2021 9:37 am

What happens if you do 16-bit reads to each of the 16-bit fields in the 64-bit register? Are the numbers you get back different than the 32-bit reads?
Rockets are loud.
https://astro-pi.org

markus3141
Posts: 39
Joined: Thu May 31, 2012 2:15 pm
Location: near Hamburg, Germany

Re: CM4 MSI-X support (Coral TPU)

Fri Jan 15, 2021 10:57 am

Okay I was (obviously) looking in the wrong location. Register writes are done by the user-space driver (libedgetpu), which attempts 64bit reads. Replacing those with two 32bit reads seems to help, although I'm not sure why the compiler isn't doing this correctly on a 32bit binary. 16bit reads seem to work as well (good hint!). Now there are probably 64bit accesses in the user-space driver everywhere that have to be fixed in order to work on the Pi...

User avatar
geerlingguy
Posts: 208
Joined: Sun Feb 15, 2015 3:43 am
Location: St. Louis, MO, USA
Contact: Website Twitter YouTube

Re: CM4 MSI-X support (Coral TPU)

Sat Jan 16, 2021 2:45 am

markus3141 wrote:
Fri Jan 15, 2021 10:57 am
Now there are probably 64bit accesses in the user-space driver everywhere that have to be fixed in order to work on the Pi...
To you, who are about to debug, we salute you.
The question is not whether something should be done on a Raspberry Pi, it is whether it can be done on a Raspberry Pi.

Justin.Hiltz
Posts: 1
Joined: Fri Jan 22, 2021 3:39 pm

Re: CM4 MSI-X support (Coral TPU)

Tue Jan 26, 2021 7:12 pm

Hi everyone,

I'll just leave the following in case anyone doesn't have access to a board that the TPU is working on. It's the output of libedgetpu with verbosity at 10 on another SBC. Might be useful to compare against. Note that I have libedgetpu doing 16 bit read/writes but I don't think that changes anything other than the 'read' logging.

Code: Select all

I :273] Starting in normal mode
I :83] Opening /dev/apex_0. read_only=0
I :97] mmap_offset=0x0000000000040000, mmap_size=4096
I :108] Got map addr at 0x0xffff8f83e000
I :97] mmap_offset=0x0000000000044000, mmap_size=4096
I :108] Got map addr at 0x0xffff8f83d000
I :97] mmap_offset=0x0000000000048000, mmap_size=4096
I :108] Got map addr at 0x0xffff8f83c000
I :218] Read: offset = 0x00000000000486f0, value: = 0x0000000000000000, w0=0x0000, w1=0x0000, w2=0x0000, w3=0x0000
I :193] Write: offset = 0x00000000000487a8, value = 0x0000000000000000
I :218] Read: offset = 0x0000000000048578, value: = 0x0000000000000010, w0=0x0010, w1=0x0000, w2=0x0000, w3=0x0000
I :136] MmuMapper#Map() : 0000ffff8f3aa000 -> 0000000001000000 (1 pages) flags=00000000.
I :55] MapMemory() page-aligned : device_address = 0x0000000001000000
I :169] Queue base : 0xffff8f3aa000 -> 0x0000000001000000 [4096 bytes]
I :136] MmuMapper#Map() : 0000ffff8f3ab000 -> 0000000001001000 (1 pages) flags=00000000.
I :55] MapMemory() page-aligned : device_address = 0x0000000001001000
I :179] Queue status block : 0xffff8f3ab000 -> 0x0000000001001000 [16 bytes]
I :193] Write: offset = 0x0000000000048590, value = 0x0000000001000000
I :193] Write: offset = 0x0000000000048598, value = 0x0000000001001000
I :193] Write: offset = 0x00000000000485a0, value = 0x0000000000000100
I :193] Write: offset = 0x0000000000048568, value = 0x0000000000000005
I :218] Read: offset = 0x0000000000048570, value: = 0x0000000000000001, w0=0x0001, w1=0x0000, w2=0x0000, w3=0x0000
I :218] Read: offset = 0x00000000000486d0, value: = 0x0000000000000000, w0=0x0000, w1=0x0000, w2=0x0000, w3=0x0000
I :193] Write: offset = 0x0000000000044018, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000044158, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000044198, value = 0x0000000000000001
I :193] Write: offset = 0x00000000000441d8, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000044218, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000048788, value = 0x000000000000007f
I :218] Read: offset = 0x0000000000048788, value: = 0x000000000000007f, w0=0x007f, w1=0x0000, w2=0x0000, w3=0x0000
I :193] Write: offset = 0x00000000000400c0, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000040150, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000040110, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000040250, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000040298, value = 0x0000000000000001
I :193] Write: offset = 0x00000000000402e0, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000040328, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000040190, value = 0x0000000000000001
I :193] Write: offset = 0x00000000000401d0, value = 0x0000000000000001
I :193] Write: offset = 0x0000000000040210, value = 0x0000000000000001
I :193] Write: offset = 0x00000000000486e8, value = 0x0000000000000000
I :45] Set event fd : event_id:0 -> event_fd:7, 
I :45] Set event fd : event_id:4 -> event_fd:11, 
I :62] event_fd=7. Monitor thread begin.
I :45] Set event fd : event_id:5 -> event_fd:12, 
I :62] event_fd=11. Monitor thread begin.
I :45] Set event fd : event_id:6 -> event_fd:13, 
I :62] event_fd=12. Monitor thread begin.
I :45] Set event fd : event_id:7 -> event_fd:14, 
I :62] event_fd=13. Monitor thread begin.
I :45] Set event fd : event_id:8 -> event_fd:15, 
I :62] event_fd=14. Monitor thread begin.
I :45] Set event fd : event_id:9 -> event_fd:16, 
I :45] Set event fd : event_id:10 -> event_fd:17, 
I :62] event_fd=16. Monitor thread begin.
I :45] Set event fd : event_id:11 -> event_fd:18, 
I :62] event_fd=15. Monitor thread begin.
I :62] event_fd=17. Monitor thread begin.
I :45] Set event fd : event_id:12 -> event_fd:19, 
I :62] event_fd=18. Monitor thread begin.
I :193] Write: offset = 0x00000000000486a0, value = 0x000000000000000f
I :193] Write: offset = 0x00000000000485c0, value = 0x0000000000000001
I :193] Write: offset = 0x00000000000486c0, value = 0x0000000000000001
I :172] Opening device at /dev/apex_0
I :62] event_fd=19. Monitor thread begin.
I :47] Adding input "map/TensorArrayStack/TensorArrayGatherV3" with 150528 bytes.
I :58] Adding output "prediction" with 965 bytes.
I :167] Request prepared, total batch size: 1, total TPU requests required: 1.
I :310] Request [0]: Submitting P0 request immediately.
I :373] Request [0]: Need to map parameters.
I :136] MmuMapper#Map() : 0000ffff82360000 -> 8000000000000000 (953 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000000000
I :252] Mapped params : Buffer(ptr=0xffff82360000) -> 0x8000000000000000, 3900864 bytes.
I :252] Mapped params : Buffer(ptr=(nil)) -> 0x0000000000000000, 0 bytes.
I :387] Request [0]: Need to do parameter-caching.
I :80] [0] Request constructed.
I :46] InstructionBuffers created.
I :653] Created new instruction buffers.
I :75] Mapped scratch : Buffer(ptr=(nil)) -> 0x0000000000000000, 0 bytes.
I :368] MapDataBuffers() done.
I :187] Linking Parameter: 0x8000000000000000
I :136] MmuMapper#Map() : 0000000019641000 -> 8000000000400000 (3 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000400000
I :223] Mapped "instructions" : Buffer(ptr=0x19641000) -> 0x8000000000400000, 9680 bytes. Direction=1
I :384] MapInstructionBuffers() done.
I :481] [0] SetState old=0, new=1.
I :393] [0] NotifyRequestSubmitted()
I :481] [0] SetState old=1, new=2.
I :83] Request[0]: Submitted
I :401] [0] NotifyRequestActive()
I :481] [0] SetState old=2, new=3.
I :133] Request[0]: Scheduling DMA[0]
I :392] Adding an element to the host queue.
I :193] Write: offset = 0x00000000000485a8, value = 0x0000000000000001
I :80] [1] Request constructed.
I :113] Adding input "map/TensorArrayStack/TensorArrayGatherV3" with 150528 bytes.
I :75] event_fd=7. Monitor thread got num_events=1.
I :188] Adding output "prediction" with 965 bytes.
I :425] Completed 1 elements.
I :193] Write: offset = 0x00000000000485c8, value = 0x0000000000000000
I :155] Completing DMA[0]
I :46] InstructionBuffers created.
I :653] Created new instruction buffers.
I :75] Mapped scratch : Buffer(ptr=(nil)) -> 0x0000000000000000, 0 bytes.
I :136] MmuMapper#Map() : 00000000195f4000 -> 8000000000440000 (38 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000440000
I :223] Mapped "map/TensorArrayStack/TensorArrayGatherV3" : Buffer(ptr=0x195f4a40) -> 0x8000000000440a40, 150528 bytes. Direction=1
I :136] MmuMapper#Map() : 0000000019647000 -> 8000000000404000 (1 pages) flags=00000004.
I :55] MapMemory() page-aligned : device_address = 0x8000000000404000
I :223] Mapped "prediction" : Buffer(ptr=0x19647000) -> 0x8000000000404000, 968 bytes. Direction=2
I :368] MapDataBuffers() done.
I :93] Linking map/TensorArrayStack/TensorArrayGatherV3[0]: 0x8000000000440a40
I :93] Linking prediction[0]: 0x8000000000404000
I :136] MmuMapper#Map() : 0000000019649000 -> 8000000000420000 (32 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000420000
I :223] Mapped "instructions" : Buffer(ptr=0x19649000) -> 0x8000000000420000, 128896 bytes. Direction=1
I :384] MapInstructionBuffers() done.
I :481] [1] SetState old=0, new=1.
I :393] [1] NotifyRequestSubmitted()
I :481] [1] SetState old=1, new=2.
I :83] Request[1]: Submitted
I :401] [1] NotifyRequestActive()
I :481] [1] SetState old=2, new=3.
I :133] Request[1]: Scheduling DMA[0]
I :392] Adding an element to the host queue.
I :193] Write: offset = 0x00000000000485a8, value = 0x0000000000000002
I :75] event_fd=11. Monitor thread got num_events=1.
I :193] Write: offset = 0x00000000000486a8, value = 0x000000000000000e
I :218] Read: offset = 0x00000000000486d0, value: = 0x0000000000000001, w0=0x0001, w1=0x0000, w2=0x0000, w3=0x0000
I :75] event_fd=7. Monitor thread got num_events=1.
I :413] [0] NotifyCompletion()
I :425] Completed 1 elements.
I :193] Write: offset = 0x00000000000485c8, value = 0x0000000000000000
I :172] MmuMaper#Unmap() : 0000000019641000 -> 8000000000400000 (3 pages).
I :155] Completing DMA[0]
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000400000, num_pages = 3
I :664] Returned instruction buffers back to executable reference
I :481] [0] SetState old=3, new=4.
I :235] Request[0]: Completed
I :96] [0] Request destroyed.
I :75] event_fd=11. Monitor thread got num_events=1.
I :193] Write: offset = 0x00000000000486a8, value = 0x000000000000000e
I :218] Read: offset = 0x00000000000486d0, value: = 0x0000000000000002, w0=0x0002, w1=0x0000, w2=0x0000, w3=0x0000
I :413] [1] NotifyCompletion()
I :172] MmuMaper#Unmap() : 0000000019649000 -> 8000000000420000 (32 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000420000, num_pages = 32
I :172] MmuMaper#Unmap() : 00000000195f4000 -> 8000000000440000 (38 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000440000, num_pages = 38
I :172] MmuMaper#Unmap() : 0000000019647000 -> 8000000000404000 (1 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000404000, num_pages = 1
I :664] Returned instruction buffers back to executable reference
I :481] [1] SetState old=3, new=4.
I :235] Request[1]: Completed
I :96] [1] Request destroyed.
I :47] Adding input "map/TensorArrayStack/TensorArrayGatherV3" with 150528 bytes.
I :58] Adding output "prediction" with 965 bytes.
I :167] Request prepared, total batch size: 1, total TPU requests required: 1.
I :310] Request [1]: Submitting P0 request immediately.
I :80] [2] Request constructed.
I :113] Adding input "map/TensorArrayStack/TensorArrayGatherV3" with 150528 bytes.
I :188] Adding output "prediction" with 965 bytes.
I :645] Reusing old instruction buffers.
I :75] Mapped scratch : Buffer(ptr=(nil)) -> 0x0000000000000000, 0 bytes.
I :136] MmuMapper#Map() : 00000000195f4000 -> 8000000000400000 (38 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000400000
I :223] Mapped "map/TensorArrayStack/TensorArrayGatherV3" : Buffer(ptr=0x195f4a40) -> 0x8000000000400a40, 150528 bytes. Direction=1
I :136] MmuMapper#Map() : 0000000019647000 -> 8000000000440000 (1 pages) flags=00000004.
I :55] MapMemory() page-aligned : device_address = 0x8000000000440000
I :223] Mapped "prediction" : Buffer(ptr=0x19647000) -> 0x8000000000440000, 968 bytes. Direction=2
I :368] MapDataBuffers() done.
I :93] Linking map/TensorArrayStack/TensorArrayGatherV3[0]: 0x8000000000400a40
I :93] Linking prediction[0]: 0x8000000000440000
I :136] MmuMapper#Map() : 0000000019649000 -> 8000000000460000 (32 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000460000
I :223] Mapped "instructions" : Buffer(ptr=0x19649000) -> 0x8000000000460000, 128896 bytes. Direction=1
I :384] MapInstructionBuffers() done.
I :481] [2] SetState old=0, new=1.
I :393] [2] NotifyRequestSubmitted()
I :481] [2] SetState old=1, new=2.
I :83] Request[2]: Submitted
I :401] [2] NotifyRequestActive()
I :481] [2] SetState old=2, new=3.
I :133] Request[2]: Scheduling DMA[0]
I :392] Adding an element to the host queue.
I :193] Write: offset = 0x00000000000485a8, value = 0x0000000000000003
I :75] event_fd=7. Monitor thread got num_events=1.
I :425] Completed 1 elements.
I :193] Write: offset = 0x00000000000485c8, value = 0x0000000000000000
I :155] Completing DMA[0]
I :75] event_fd=11. Monitor thread got num_events=1.
I :193] Write: offset = 0x00000000000486a8, value = 0x000000000000000e
I :218] Read: offset = 0x00000000000486d0, value: = 0x0000000000000003, w0=0x0003, w1=0x0000, w2=0x0000, w3=0x0000
I :413] [2] NotifyCompletion()
I :172] MmuMaper#Unmap() : 0000000019649000 -> 8000000000460000 (32 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000460000, num_pages = 32
I :172] MmuMaper#Unmap() : 00000000195f4000 -> 8000000000400000 (38 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000400000, num_pages = 38
I :172] MmuMaper#Unmap() : 0000000019647000 -> 8000000000440000 (1 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000440000, num_pages = 1
I :664] Returned instruction buffers back to executable reference
I :481] [2] SetState old=3, new=4.
I :235] Request[2]: Completed
I :96] [2] Request destroyed.
I :47] Adding input "map/TensorArrayStack/TensorArrayGatherV3" with 150528 bytes.
I :58] Adding output "prediction" with 965 bytes.
I :167] Request prepared, total batch size: 1, total TPU requests required: 1.
I :310] Request [2]: Submitting P0 request immediately.
I :80] [3] Request constructed.
I :113] Adding input "map/TensorArrayStack/TensorArrayGatherV3" with 150528 bytes.
I :188] Adding output "prediction" with 965 bytes.
I :645] Reusing old instruction buffers.
I :75] Mapped scratch : Buffer(ptr=(nil)) -> 0x0000000000000000, 0 bytes.
I :136] MmuMapper#Map() : 00000000195f4000 -> 8000000000400000 (38 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000400000
I :223] Mapped "map/TensorArrayStack/TensorArrayGatherV3" : Buffer(ptr=0x195f4a40) -> 0x8000000000400a40, 150528 bytes. Direction=1
I :136] MmuMapper#Map() : 0000000019647000 -> 8000000000440000 (1 pages) flags=00000004.
I :55] MapMemory() page-aligned : device_address = 0x8000000000440000
I :223] Mapped "prediction" : Buffer(ptr=0x19647000) -> 0x8000000000440000, 968 bytes. Direction=2
I :368] MapDataBuffers() done.
I :93] Linking map/TensorArrayStack/TensorArrayGatherV3[0]: 0x8000000000400a40
I :93] Linking prediction[0]: 0x8000000000440000
I :136] MmuMapper#Map() : 0000000019649000 -> 8000000000460000 (32 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000460000
I :223] Mapped "instructions" : Buffer(ptr=0x19649000) -> 0x8000000000460000, 128896 bytes. Direction=1
I :384] MapInstructionBuffers() done.
I :481] [3] SetState old=0, new=1.
I :393] [3] NotifyRequestSubmitted()
I :481] [3] SetState old=1, new=2.
I :83] Request[3]: Submitted
I :401] [3] NotifyRequestActive()
I :481] [3] SetState old=2, new=3.
I :133] Request[3]: Scheduling DMA[0]
I :392] Adding an element to the host queue.
I :193] Write: offset = 0x00000000000485a8, value = 0x0000000000000004
I :75] event_fd=7. Monitor thread got num_events=1.
I :425] Completed 1 elements.
I :193] Write: offset = 0x00000000000485c8, value = 0x0000000000000000
I :155] Completing DMA[0]
I :75] event_fd=11. Monitor thread got num_events=1.
I :193] Write: offset = 0x00000000000486a8, value = 0x000000000000000e
I :218] Read: offset = 0x00000000000486d0, value: = 0x0000000000000004, w0=0x0004, w1=0x0000, w2=0x0000, w3=0x0000
I :413] [3] NotifyCompletion()
I :172] MmuMaper#Unmap() : 0000000019649000 -> 8000000000460000 (32 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000460000, num_pages = 32
I :172] MmuMaper#Unmap() : 00000000195f4000 -> 8000000000400000 (38 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000400000, num_pages = 38
I :172] MmuMaper#Unmap() : 0000000019647000 -> 8000000000440000 (1 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000440000, num_pages = 1
I :664] Returned instruction buffers back to executable reference
I :481] [3] SetState old=3, new=4.
I :235] Request[3]: Completed
I :96] [3] Request destroyed.
I :47] Adding input "map/TensorArrayStack/TensorArrayGatherV3" with 150528 bytes.
I :58] Adding output "prediction" with 965 bytes.
I :167] Request prepared, total batch size: 1, total TPU requests required: 1.
I :310] Request [3]: Submitting P0 request immediately.
I :80] [4] Request constructed.
I :113] Adding input "map/TensorArrayStack/TensorArrayGatherV3" with 150528 bytes.
I :188] Adding output "prediction" with 965 bytes.
I :645] Reusing old instruction buffers.
I :75] Mapped scratch : Buffer(ptr=(nil)) -> 0x0000000000000000, 0 bytes.
I :136] MmuMapper#Map() : 00000000195f4000 -> 8000000000400000 (38 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000400000
I :223] Mapped "map/TensorArrayStack/TensorArrayGatherV3" : Buffer(ptr=0x195f4a40) -> 0x8000000000400a40, 150528 bytes. Direction=1
I :136] MmuMapper#Map() : 0000000019647000 -> 8000000000440000 (1 pages) flags=00000004.
I :55] MapMemory() page-aligned : device_address = 0x8000000000440000
I :223] Mapped "prediction" : Buffer(ptr=0x19647000) -> 0x8000000000440000, 968 bytes. Direction=2
I :368] MapDataBuffers() done.
I :93] Linking map/TensorArrayStack/TensorArrayGatherV3[0]: 0x8000000000400a40
I :93] Linking prediction[0]: 0x8000000000440000
I :136] MmuMapper#Map() : 0000000019649000 -> 8000000000460000 (32 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000460000
I :223] Mapped "instructions" : Buffer(ptr=0x19649000) -> 0x8000000000460000, 128896 bytes. Direction=1
I :384] MapInstructionBuffers() done.
I :481] [4] SetState old=0, new=1.
I :393] [4] NotifyRequestSubmitted()
I :481] [4] SetState old=1, new=2.
I :83] Request[4]: Submitted
I :401] [4] NotifyRequestActive()
I :481] [4] SetState old=2, new=3.
I :133] Request[4]: Scheduling DMA[0]
I :392] Adding an element to the host queue.
I :193] Write: offset = 0x00000000000485a8, value = 0x0000000000000005
I :75] event_fd=7. Monitor thread got num_events=1.
I :425] Completed 1 elements.
I :193] Write: offset = 0x00000000000485c8, value = 0x0000000000000000
I :155] Completing DMA[0]
I :75] event_fd=11. Monitor thread got num_events=1.
I :193] Write: offset = 0x00000000000486a8, value = 0x000000000000000e
I :218] Read: offset = 0x00000000000486d0, value: = 0x0000000000000005, w0=0x0005, w1=0x0000, w2=0x0000, w3=0x0000
I :413] [4] NotifyCompletion()
I :172] MmuMaper#Unmap() : 0000000019649000 -> 8000000000460000 (32 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000460000, num_pages = 32
I :172] MmuMaper#Unmap() : 00000000195f4000 -> 8000000000400000 (38 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000400000, num_pages = 38
I :172] MmuMaper#Unmap() : 0000000019647000 -> 8000000000440000 (1 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000440000, num_pages = 1
I :664] Returned instruction buffers back to executable reference
I :481] [4] SetState old=3, new=4.
I :235] Request[4]: Completed
I :96] [4] Request destroyed.
I :47] Adding input "map/TensorArrayStack/TensorArrayGatherV3" with 150528 bytes.
I :58] Adding output "prediction" with 965 bytes.
I :167] Request prepared, total batch size: 1, total TPU requests required: 1.
I :310] Request [4]: Submitting P0 request immediately.
I :80] [5] Request constructed.
I :113] Adding input "map/TensorArrayStack/TensorArrayGatherV3" with 150528 bytes.
I :188] Adding output "prediction" with 965 bytes.
I :645] Reusing old instruction buffers.
I :75] Mapped scratch : Buffer(ptr=(nil)) -> 0x0000000000000000, 0 bytes.
I :136] MmuMapper#Map() : 00000000195f4000 -> 8000000000400000 (38 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000400000
I :223] Mapped "map/TensorArrayStack/TensorArrayGatherV3" : Buffer(ptr=0x195f4a40) -> 0x8000000000400a40, 150528 bytes. Direction=1
I :136] MmuMapper#Map() : 0000000019647000 -> 8000000000440000 (1 pages) flags=00000004.
I :55] MapMemory() page-aligned : device_address = 0x8000000000440000
I :223] Mapped "prediction" : Buffer(ptr=0x19647000) -> 0x8000000000440000, 968 bytes. Direction=2
I :368] MapDataBuffers() done.
I :93] Linking map/TensorArrayStack/TensorArrayGatherV3[0]: 0x8000000000400a40
I :93] Linking prediction[0]: 0x8000000000440000
I :136] MmuMapper#Map() : 0000000019649000 -> 8000000000460000 (32 pages) flags=00000002.
I :55] MapMemory() page-aligned : device_address = 0x8000000000460000
I :223] Mapped "instructions" : Buffer(ptr=0x19649000) -> 0x8000000000460000, 128896 bytes. Direction=1
I :384] MapInstructionBuffers() done.
I :481] [5] SetState old=0, new=1.
I :393] [5] NotifyRequestSubmitted()
I :481] [5] SetState old=1, new=2.
I :83] Request[5]: Submitted
I :401] [5] NotifyRequestActive()
I :481] [5] SetState old=2, new=3.
I :133] Request[5]: Scheduling DMA[0]
I :392] Adding an element to the host queue.
I :193] Write: offset = 0x00000000000485a8, value = 0x0000000000000006
I :75] event_fd=7. Monitor thread got num_events=1.
I :425] Completed 1 elements.
I :193] Write: offset = 0x00000000000485c8, value = 0x0000000000000000
I :155] Completing DMA[0]
I :75] event_fd=11. Monitor thread got num_events=1.
I :193] Write: offset = 0x00000000000486a8, value = 0x000000000000000e
I :218] Read: offset = 0x00000000000486d0, value: = 0x0000000000000006, w0=0x0006, w1=0x0000, w2=0x0000, w3=0x0000
I :413] [5] NotifyCompletion()
I :172] MmuMaper#Unmap() : 0000000019649000 -> 8000000000460000 (32 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000460000, num_pages = 32
I :172] MmuMaper#Unmap() : 00000000195f4000 -> 8000000000400000 (38 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000400000, num_pages = 38
I :172] MmuMaper#Unmap() : 0000000019647000 -> 8000000000440000 (1 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000440000, num_pages = 1
I :664] Returned instruction buffers back to executable reference
I :481] [5] SetState old=3, new=4.
I :235] Request[5]: Completed
I :96] [5] Request destroyed.
I :172] MmuMaper#Unmap() : 0000ffff82360000 -> 8000000000000000 (953 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x8000000000000000, num_pages = 953
I :51] InstructionBuffers destroyed.
I :51] InstructionBuffers destroyed.
I :226] Releasing Edge TPU device at /dev/apex_0
I :178] Closing Edge TPU device at /dev/apex_0
I :193] Write: offset = 0x00000000000486d8, value = 0x0000000000000001
I :218] Read: offset = 0x00000000000486e0, value: = 0x0000000000000001, w0=0x0001, w1=0x0000, w2=0x0000, w3=0x0000
I :193] Write: offset = 0x0000000000044018, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000044158, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000044198, value = 0x0000000000000002
I :193] Write: offset = 0x00000000000441d8, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000044218, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000048788, value = 0x000000000000007f
I :218] Read: offset = 0x0000000000048788, value: = 0x000000000000007f, w0=0x007f, w1=0x0000, w2=0x0000, w3=0x0000
I :193] Write: offset = 0x00000000000400c0, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000040150, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000040110, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000040250, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000040298, value = 0x0000000000000002
I :193] Write: offset = 0x00000000000402e0, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000040328, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000040190, value = 0x0000000000000002
I :193] Write: offset = 0x00000000000401d0, value = 0x0000000000000002
I :193] Write: offset = 0x0000000000040210, value = 0x0000000000000002
I :193] Write: offset = 0x00000000000486c0, value = 0x0000000000000000
I :193] Write: offset = 0x00000000000485c0, value = 0x0000000000000000
I :193] Write: offset = 0x00000000000486a0, value = 0x0000000000000000
I :75] event_fd=7. Monitor thread got num_events=1.
I :85] event_fd=7. Monitor thread exit.
I :75] event_fd=11. Monitor thread got num_events=1.
I :85] event_fd=11. Monitor thread exit.
I :75] event_fd=12. Monitor thread got num_events=1.
I :85] event_fd=12. Monitor thread exit.
I :75] event_fd=13. Monitor thread got num_events=1.
I :85] event_fd=13. Monitor thread exit.
I :75] event_fd=14. Monitor thread got num_events=1.
I :85] event_fd=14. Monitor thread exit.
I :75] event_fd=15. Monitor thread got num_events=1.
I :85] event_fd=15. Monitor thread exit.
I :75] event_fd=16. Monitor thread got num_events=1.
I :85] event_fd=16. Monitor thread exit.
I :75] event_fd=17. Monitor thread got num_events=1.
I :85] event_fd=17. Monitor thread exit.
I :75] event_fd=18. Monitor thread got num_events=1.
I :85] event_fd=18. Monitor thread exit.
I :75] event_fd=19. Monitor thread got num_events=1.
I :85] event_fd=19. Monitor thread exit.
I :193] Write: offset = 0x0000000000048568, value = 0x0000000000000000
I :218] Read: offset = 0x0000000000048570, value: = 0x0000000000000000, w0=0x0000, w1=0x0000, w2=0x0000, w3=0x0000
I :193] Write: offset = 0x00000000000485a8, value = 0x0000000000000000
I :193] Write: offset = 0x0000000000048590, value = 0x0000000000000000
I :193] Write: offset = 0x0000000000048598, value = 0x0000000000000000
I :172] MmuMaper#Unmap() : 0000ffff8f3aa000 -> 0000000001000000 (1 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x0000000001000000, num_pages = 1
I :172] MmuMaper#Unmap() : 0000ffff8f3ab000 -> 0000000001001000 (1 pages).
I :83] UnmapMemory() page-aligned : device_address = 0x0000000001001000, num_pages = 1
I :122] Closing /dev/apex_0. mmap_offset=0x0000000000040000, mmap_size=4096, read_only=0
I :122] Closing /dev/apex_0. mmap_offset=0x0000000000044000, mmap_size=4096, read_only=0
I :122] Closing /dev/apex_0. mmap_offset=0x0000000000048000, mmap_size=4096, read_only=0

markus3141
Posts: 39
Joined: Thu May 31, 2012 2:15 pm
Location: near Hamburg, Germany

Re: CM4 MSI-X support (Coral TPU)

Wed Jan 27, 2021 11:16 am

Thank you, that will certainly help. That was next on my todo list, but I'm kinda busy getting my thesis finished at the moment.

I tested my patches on a 64bit Raspbian yesterday, but to no surprise it's failing in a very similar way.

It seems something is wrong with the DMA. It never completes, but instead the fatal error interrupt fires, raising an HIB Error.

Return to “Compute Module”