grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Sat Feb 03, 2018 6:44 pm

Getting closer I feel, come to find out, I think the FPGA example I first downloaded does not work, or at least, the configuration was wrong. Scoping it didn't look ANYTHING like what I see coming out of the camera MIPI ports. So, found a different example and pulled that instead.

I think where I am hung up now is the timing. The FPGA RTL has a set of timing parameters for the MIPI lanes as such:

Code: Select all

    parameter LP01CLK_dly           = 6,
    parameter LP00CLK_dly           = 6,
    parameter HS00CLK_dly           = 6,
    parameter HSXXCLK_dly           = 6,
    parameter CLK2DATA_dly          = 6,
    parameter LP01DATA_dly          = 6,
    parameter LP00DATA_dly          = 6,
    parameter HS00DATA_dly          = 6,
    parameter HSXXDATA_dly          = 6
Which I am trying to relate to the numbers in the raspiraw configurations. I've looked through the documents and the camera chip data sheet. I feel like my only option here is to figure out what the timing numbers that are being send to the camera, then map those settings to what the data sheet says, then try and figure out the appropriate similar settings here. I've looked through the register calls but there is A LOT to go through.

I do see the termination being applied when I scope the port while the CSI port is being attempted to be read, so at least I see SOMETHING happening, hahaha.

Still getting 0 data loaded, which as you indicated means I am not detecting the proper packets I would expect to get.

HermannSW
Posts: 792
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Feb 04, 2018 2:04 am

@6by9,

with v2 camera "raspivid" option "-fps" allows to set values up to 120 that are working with v2 camera (setting higher framerates results in 120fps taken) for "-md 7" (640x480):
viewtopic.php?f=43&t=201728&p=1256653#p1256653

raspivid "--fps" option works happily for 240fps in "--mode 7" (640x480):
viewtopic.php?f=43&t=109137&start=375#p1263896

I searched in whole userland sources
https://github.com/6by9/userland

where the maximal value of 120(90) for "-fps" gets enforced for "raspivid" v2(v1) camera and did not find it.

Where is "raspivid" maximal framerate capped? In userland? Or in closed source GPU code?
bookmark list: https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/fork-raspiraw      https://github.com/Hermann-SW/userland
https://github.com/Hermann-SW/wireless-control-Eachine-E52-drone      https://twitter.com/HermannSW

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Feb 04, 2018 9:30 pm

Status update, I can get some sort of data now. I slowed down each of the values above to '50' each and reduced the timing numbers to '1' in my driver. Clearly this all needs tuning, and might be why the data I am getting is noisy (or, maybe I have to much capacitance on the wires with my board).

Given how many people would like to hook a Lattice Machx02 to a raspberry pi, this is why I'm detailing my progress here. This thread has been A LOT of help.

What I have discovered also is that the Machx02 reference design pattern generator does NOT produce proper RAW10 sequencing. Given the format is 40 bits, they should be breaking that out into 5 transmissions per pixel. Instead they are just taking an RGB 24-bit value (888) and using the 10 lowest bits and feeding that out as RAW10. This of course generates a BW set of bars and then just whatever was in memory I imaging (on the pi).

Next step is to try and make a better test bar generator that is proper RAW10. (Ignore the coloring in the image I captured, that was just be playing with the dcraw settings)
test7.png
test7.png (210.89 KiB) Viewed 2854 times

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 05, 2018 7:44 am

One things for sure, I don't think a lot of physical testing went into the Lattice test design. I switched back to working with the RGB888 version because the RAW10 doesn't make sense. What CSI-2 spec says that means and what the Lattice is sending don't match. I will ask them about it.

The second problem is whomever wrote the pattern generator has mixed up vertical and horizontal settings for their CSI-2 pattern generator instance, basically making a 480x640.

In addition, their documentation shows using LVDS25E ports, but those are limited to 150Mhz. The demo code appears to be writing out at 250Mhz. That's not going to work. I slowed down the system to 125Mhz and that's working (with the glitching) but any slower and the Pi wasn't detecting it.

I've got some emails into them to ask why not use the LVDS25 (or LVDS33) as an output? And if I use that, what does that change the recommended series resistor off of the port on the lattice side.

You were correct, the termination set to 1 was needed.

A few questions:

1) I see mention of managing the stride length, but I don't see how that's set. In the individual board driver files, the modes reference min_vts and line_time_ns. Both of those seem relative to exposure time settings so I assume those have no meaning for me. How do I set a stride length?

2) Since I am sending RGB888, I don't need it to be unpacked, just received and put into a buffer. I see in the code we create vc.ril.isp and vc.ril.render. Are these required just to get the data into memory? In the mode I have the encoding type set to MMAL_ENCODING_RGB888 and bayer_order set to 0.

3) native bit depth, is that per pixel? So 8 would be correct?

4) What is 'cpi_timing'?

I feel like I am getting a lot closer, but I realize I will need to respin this board soon. I didn't realize how important the matching was on the lines. I matched their length, but looking at the requirements, I did not match the 100R impedance required, nor did I handle the serpentine routing properly. Given I now have the proper equations I am going to calculate what I had just out of curiosity and fix my design.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5816
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 05, 2018 10:05 am

HermannSW wrote:
Sun Feb 04, 2018 2:04 am
Where is "raspivid" maximal framerate capped? In userland? Or in closed source GPU code?
GPU firmware.
I'll see what I get if I up the limit to 240fps. The numbers were provided by Sony and implemented as provided. If it works at higher rates then that's a win for free.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5816
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 05, 2018 10:27 am

grimepoch wrote:
Mon Feb 05, 2018 7:44 am
You were correct, the termination set to 1 was needed.
Great. Switching to low speed mode has significant power saving implications which are necessary in mobile applications.
Having seen a fair number of the V4L2 sensor drivers in the kernel now they are tending to using continuous clock. It's not a big deal, but you do have to set it correctly.
grimepoch wrote:A few questions:

1) I see mention of managing the stride length, but I don't see how that's set. In the individual board driver files, the modes reference min_vts and line_time_ns. Both of those seem relative to exposure time settings so I assume those have no meaning for me. How do I set a stride length?
Stride is set via the port format.

Code: Select all

	output->format->es->video.crop.width = sensor_mode->width;
	output->format->es->video.crop.height = sensor_mode->height;
	output->format->es->video.width = VCOS_ALIGN_UP(sensor_mode->width, 16);
	output->format->es->video.height = VCOS_ALIGN_UP(sensor_mode->height, 16);
So the ACTIVE pixels in the buffer are sensor_mode->width per line, and sensor_mode->height lines.
The stride will be the value given by

Code: Select all

mmal_encoding_width_to_stride(output->format->encoding, output->format->es->video.width)
(mmal_encoding_width_to_stride is in the userland repo under interface/mmal/util/mmal_util.c)
The overall buffer size is

Code: Select all

mmal_encoding_width_to_stride(output->format->encoding, output->format->es->video.width) * output->format->es->video.height
grimepoch wrote:2) Since I am sending RGB888, I don't need it to be unpacked, just received and put into a buffer. I see in the code we create vc.ril.isp and vc.ril.render. Are these required just to get the data into memory? In the mode I have the encoding type set to MMAL_ENCODING_RGB888 and bayer_order set to 0.
That's fine.
If you look at the variant of raspiraw that supports the TC358743 HDMI to CSI2 bridge, that is set up to receive BGR888 from the chip using rawcam.
The isp and render components are there to put the images on the screen, which requires them to be in a sensible format (not Bayer or a YUYV variant). The ISP is doing that conversion, and video_render renders the image. Neither are required if you use the code path that is saving the data to file.
grimepoch wrote:3) native bit depth, is that per pixel? So 8 would be correct?
Ignore it if you're not producing Bayer. The code at line 1076:

Code: Select all

	if (sensor_mode->encoding || cfg.bit_depth == sensor_mode->native_bit_depth)
	{
		rx_cfg.unpack = MMAL_CAMERA_RX_CONFIG_UNPACK_NONE;
		rx_cfg.pack = MMAL_CAMERA_RX_CONFIG_PACK_NONE;
	}
	else
	{
		switch(sensor_mode->native_bit_depth)
		{
			//set up the packing/unpacking
So set encoding to MMAL_ENCODING_RGB24 and native_bit_depth to 0.
grimepoch wrote:4) What is 'cpi_timing'?
Again ignore it. CPI = Camera Parallel Interface. Present in the hardware, but not accessible on the Pi.
grimepoch wrote:I feel like I am getting a lot closer, but I realize I will need to respin this board soon. I didn't realize how important the matching was on the lines. I matched their length, but looking at the requirements, I did not match the 100R impedance required, nor did I handle the serpentine routing properly. Given I now have the proper equations I am going to calculate what I had just out of curiosity and fix my design.
For prototyping you'll probably get away with it. I have an analog video to CSI2 eval board on my desk which has SMA outputs for each side of each lane. I bought 2 30cm SMA to SMA leads, chopped them in half, and soldered them onto a dead Pi camera board. It works fine.
Likewise I'd previously wired a sensor in using loosely twisted pairs of standard wire without significant issues.
In a product then you do want to follow design rules to get the best signal integrity.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 05, 2018 3:38 pm

Excellent information, thanks!

Pertaining to the respin, I am going to dig deeper of course and try to understand why the Pi is having such problems decoding what I am sending it. I have no other noise sources near the wires, If you look at the screenshot I sent, you can see that for some reason, it's getting corrupted. I've also locked up the Pi by sending it counting up colors as well. (just taking the RGB888 value and incrementing by 1 as it walks across the screen).

All I can think is that I am getting too large of reflections on on the wire, and by slowing everything down as much as I can, it's let's that reflection settle a bit. I'm thinking the larger the swing, the longer the ring, and that's why I am getting away with setting much longer timings in my CSI-2 driver.

The timing settings I am setting in my raspiraw driver, if I get those wrong, what happens? I notice when I decode I go from flags 0004 to 0084 all the time, all I could think is that the ID is getting corrupted because I am not sending any other data. (Also given sometimes it doesn't see anything at all, which means the whole start frame packet is probably corrupted there).

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5816
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 05, 2018 4:51 pm

grimepoch wrote:
Mon Feb 05, 2018 3:38 pm
The timing settings I am setting in my raspiraw driver, if I get those wrong, what happens? I notice when I decode I go from flags 0004 to 0084 all the time, all I could think is that the ID is getting corrupted because I am not sending any other data. (Also given sometimes it doesn't see anything at all, which means the whole start frame packet is probably corrupted there).
0x4 = MMAL_BUFFER_HEADER_FLAG_FRAME_END
0x80 = MMAL_BUFFER_HEADER_FLAG_CODECSIDEINFO
As in the first post:
The handling of the sensor non-image data path is not tested. You will find that you always get a pair of buffers back with the same timestamp. One has the MMAL_BUFFER_HEADER_FLAG_CODECSIDEINFO flag set and should be the non-image data. I have not tested this at all as yet, and the length will always come through as the full buffer size at the moment.
Unless you are sending back embedded data on an alternate data type, ignore the buffer with MMAL_BUFFER_HEADER_FLAG_CODECSIDEINFO set. You get both buffers back at frame end as there wasn't a simple mechanism to package up the length of the data frame.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 05, 2018 10:38 pm

I am not sending any non-image data (at least, I don't see that in the RTL code at all), are you saying whether I send it or not, I am going to get some buffers trigger 0x0084 with zero length?

(Notice the 4 in there as well)

They definitely are coming though with size set, that's why I thought the ID was getting corrupted on occasion.

I talked with someone else, they find it hard to believe I'd have issues with the wiring for how short it is I am running and the lack of other noise sources.

The only other thing I can imagine is the timing# settings maybe. But I've tried some pretty will ranges of values and haven't seen much of a difference.

HermannSW
Posts: 792
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Feb 07, 2018 9:54 am

6by9 wrote:
Mon Feb 05, 2018 10:05 am
...GPU firmware.
I'll see what I get if I up the limit to 240fps. The numbers were provided by Sony and implemented as provided. If it works at higher rates then that's a win for free.
I could not resist and tried myself, although I don't have access to closed source GPU code as you do.
See new thread "High framerate raspivid (initially 640x480 at 180fps)":
viewtopic.php?f=43&t=204775

Yes, an initial cruel hack of raspivid, but 640x480 at 180fps works without frameskips (above that framerate frameskips start to happen).
This is mostly a hack because I found no better way than to "usleep(200000)" to get past GPU emitted I2C commands.
  1. is ther a better wait to wait for end of GPU emitted I2C commands to camera?
  2. if not, could you add a callback function into GPU firmware allowing to better do further porting of raspiraw high framerate features to raspivid?
Image
bookmark list: https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/fork-raspiraw      https://github.com/Hermann-SW/userland
https://github.com/Hermann-SW/wireless-control-Eachine-E52-drone      https://twitter.com/HermannSW

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5816
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Feb 07, 2018 10:34 am

grimepoch wrote:
Mon Feb 05, 2018 10:38 pm
I am not sending any non-image data (at least, I don't see that in the RTL code at all), are you saying whether I send it or not, I am going to get some buffers trigger 0x0084 with zero length?
Yes, you will always get the pair of buffers, regardless of what the source device sends.
grimepoch wrote:(Notice the 4 in there as well)
Just means it's the end of the side info frame as well
grimepoch wrote:They definitely are coming though with size set, that's why I thought the ID was getting corrupted on occasion.
Mainly as I don't have an easy way to read the length on the current setup.
There are independent interrupts available for the image buffer vs the data buffer (as they're regarded in the peripheral), but the data ones haven't been used as normally the sensor embedded data comes through just before the image. The only way to determine the length is via the write pointer in the buffer, but that plays up sometimes.
grimepoch wrote:The only other thing I can imagine is the timing# settings maybe. But I've tried some pretty will ranges of values and haven't seen much of a difference.
What MIPI timing parameters have you set in your FPGA? If you give me those then I may be able to work out what the correct settings should be. AIUI most of them are irrelevant if you have continuous clocks.

You can try running "vcgencmd set_logging level=0xc0" before raspiraw, and then "sudo vcdbg log msg" afterwards (or during). The lines of interest are the ones starting "unicam_int_callback". Any signalled errors are logged in a readable form there. eg

Code: Select all

2263319.992: ip_process_buffer: Already processing a buffer
2263334.545: unicam_int_callback: instance=1 status = 0x8020
2263334.571: unicam_int_callback: bytes_written = 2547248, lines_done = 617
2263334.587: unicam_int_callback error: panic signalled
2263334.604: unicam_int_callback: Line Interrupt 617
(Not quite sure why my imx219 is signalling a panic (ie needs a higher priority to shift data to SDRAM) at the moment. That may just be a quirk of my setup).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Feb 08, 2018 7:16 am

Well I've purchased a 1Ghz LeCroy with the DPHY package so I'll take a look first to make sure my FPGA is actually outputting something that makes sense first before we get into timing. I figured that would be the best path here.

Is there a minimum speed that DCLK cannot go under for the Pi? Just want to make sure I am sending something reasonable. My pixel clock is 13.5Mhz for 24-bit, so I think that ends up at 160Mhz+

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5816
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Feb 08, 2018 9:59 am

grimepoch wrote:
Thu Feb 08, 2018 7:16 am
Well I've purchased a 1Ghz LeCroy with the DPHY package so I'll take a look first to make sure my FPGA is actually outputting something that makes sense first before we get into timing. I figured that would be the best path here.
Checking with an external device is always useful. Almost all hardware has quirks so it's hard to pin down issues when it could be either end at fault.
grimepoch wrote:Is there a minimum speed that DCLK cannot go under for the Pi? Just want to make sure I am sending something reasonable. My pixel clock is 13.5Mhz for 24-bit, so I think that ends up at 160Mhz+
The spec sheet doesn't list a minimum data rate - it's mostly clocked from the CSI bus into a FIFO, and then sent to RAM when threshold limits get hit.
Max is stated as 500MPix/s, or 1Gbit/s per lane on the CSI bus - the rest of the system isn't going to keep up at 500MPix/s, but you could write into RAM for about a second.
I've had it running with an analog TV capture chip (ADV7282M) which produces [email protected] UYVY, so that's 331Mbits/s, up to HDMI to CSI2 capturing 1080P50 UYVY or 1.66Gbit/s over 2 lanes. The camera modules generally keep the pixel clock the same for all modes and insert extended H and/or V blanking periods, so they aren't the most representative for pixel clock rates.

One odd one I have seen was failing to lock on to a signal, but setting "force_turbo=1" in config.txt bumps up the default VPU clock speed and made it happy. That was on a high speed mode so probably not relevant here, but it was a behaviour we couldn't understand.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Sat Feb 10, 2018 6:53 pm

I've read in a few white papers that CSI-2 problems are rarely the wiring and typically timing settings. Thought that was interesting.

Still waiting for the scope, so continuing to look at things.

I don't think my FPGA is in HS all the time. When I look at what I can see, it looks like it drops back to LP. I'm not sure if that's just between frames, or also between lines, but I see the signal raise up to LP voltage levels.

If I set the terminators to manual (1,1) and I run the code, raspiraw hangs and never leaves waiting for the buffers after receiving a handful. I have to restart the Pi to be able to run again. When I check the log output lots of parity, CRC, etc errors.

If I set the terminators to auto (0,0) I do see the signal change when the code starts checking, so it's attempting to terminate and then read. In this case it never sees anything.

Pertaining to timing, I have the RTL using the default values, these are in units of byte clock.

Code: Select all

    parameter LP01CLK_dly           = 6,
    parameter LP00CLK_dly           = 6,
    parameter HS00CLK_dly           = 6,
    parameter HSXXCLK_dly           = 6,
    parameter CLK2DATA_dly          = 6,
    parameter LP01DATA_dly          = 6,
    parameter LP00DATA_dly          = 6,
    parameter HS00DATA_dly          = 6,
    parameter HSXXDATA_dly          = 6
The byte clock I am using (single lane, 13.5Mhz pixel clock, RGB888) is 40.5Mhz. (And I could have my calculations wrong). This was derived using the FPGA calculations:

PPI = pixel clock rate (13.5Mhz)

DDR CLOCKS = PPI * word_width / (8 * lane_width) *4 = 13.5 * 24/8 * 4 = 162Mhz
(DDR clocks are 2 90 degrees out of phase)

BYTE CLOCK = PPI * word_width / (8 * lane_width) = 13.5 * 24/8 = 40.5Mhz

So with that, each unit of delay above is 24.69nS, so 6 units is 148nS.

With the MMAL RX TIMING settings, I am not sure what the units of measure are there. I thought I saw something that said 'nS' but now I can't find it.

For those timings for each of those delays, what would you recommend for the timing settings?

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Feb 11, 2018 7:56 pm

More progress, so I had a bad LDO on the 1.2V for the LP signals. Fixed and the output looks much better. Still just have my 100Mhz scope at the moment so cannot see a lot of detail in the DATA/CLK since it's at 168Mhz.

One thing I realize is the FPGA is NOT in continuous mode. How I was able to get anything before is beyond me, I was setting the termination manually and getting some of the data being sent.

Now that I've fixed everything best I can tell, now I cannot get any recognition of frames. I've played with the delay values above to see if any of that helps, but really it's a shot in the dark. Right now I have them set to '2' and here is a capture of Blue is clock and Yellow is Data (just one of the lanes) without termination applied.
SDS00005.png
SDS00005.png (39.71 KiB) Viewed 2424 times
When I set the driver to do auto termination (0,0) I see no change in the waveform when I run raspiraw. Clearly I don't understand how the auto detection works. (And I assume that is relative to the numbers we set on the timing).

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Feb 11, 2018 8:52 pm

Here is a better, view, I slowed down everything to 1/2 in the system since I am using an RTL pattern generator I can do this. 1 channel.

This is without the terminator active, green/magenta is the DCLK clock. Yellow/blue is the D0 lines.

These scope probes definitely help introduce some noise given the grounding and/or proximity to each other.

What I am wondering now is how the RTL is generating the starting frame message. It would seem to me there needs to be sufficient time to send that. Not sure if any of those delay values are helping with that and if I am dealing with a problem of never seeing a start frame because there isn't enough time to send it. Digging into that further. (or a partial start frame).

SDS00006.png
SDS00006.png (70.56 KiB) Viewed 2413 times

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Feb 11, 2018 10:09 pm

Also, do you know if the raspberry pi needs a frame number? Right now I can verify the frame start packet is sending zero for the frame number.

The spec is confusing on this. It says if frame is not used, to set it to zero, but that the value cannot be zero.

I'm also not seeing any exposure to the frame number in the RTL IP. Just want to make sure not looking at something that I do need need. (Thought process was, when the LDO was broken, it was generating spikes all over the place, and thinking if the spike happened during the frame data, that could allow it to think there was a frame number on occasion.)

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 12, 2018 5:03 am

Update, I am unsure that the MIPI output is within spec with the reference circuits they give, with the termination provided by the Pi.

I verified the proper data is being sent. What does not look right is the voltage levels with the termination on.
SDS00008.png
SDS00008.png (79.51 KiB) Viewed 2372 times
You can see from the scope (with the terminators active) that I am only getting a 100mV swing, and a common voltage of 100mV. This is about 1/2 of the recommended values. Here is a shot from another persons diagram on the DPHY requirements of HS mode.
Screen Shot 2018-02-11 at 11.32.05 PM.png
Screen Shot 2018-02-11 at 11.32.05 PM.png (221.86 KiB) Viewed 2372 times
You can see this person (I think they were working with DSI) measured a minimum common mode voltage of like 170mV.

Regardless, this is not what I see. I've written Lattice to find out what is going on. There are very few variables here in this working, the only thing I could suspect is their resistor choices on their reference design. They have 320R inline with the Data and Clk HS lines (the LP has a 50R, looks like LP is set to 0 in HS mode).

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5816
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 12, 2018 10:38 am

grimepoch wrote:
Mon Feb 12, 2018 5:03 am
Update, I am unsure that the MIPI output is within spec with the reference circuits they give, with the termination provided by the Pi.

I verified the proper data is being sent. What does not look right is the voltage levels with the termination on.

You can see this person (I think they were working with DSI) measured a minimum common mode voltage of like 170mV.

Regardless, this is not what I see. I've written Lattice to find out what is going on. There are very few variables here in this working, the only thing I could suspect is their resistor choices on their reference design. They have 320R inline with the Data and Clk HS lines (the LP has a 50R, looks like LP is set to 0 in HS mode).
I'm afraid that I can't really offer much more support on this. If you had a known good CSI2 data source then it'd be worth investing the extra effort, but debugging your FPGA isn't my remit.
A couple of options:
- there are others that have managed to get FPGAs to talk to the Pi. PiCatpure SD1 and HD1 are shipping product based on one. See also some details in viewtopic.php?t=154013. Use your prefered search engine to search with "site:raspberrypi.org fpga csi" for more users. Create a new thread in an appropriate forum and you may get some responses. (Please don't just ping all the old threads!)
- there are also a couple of other SoCs that have support for CSI2 receivers. The Freescale imx6 and NVidia Tegra are the two that spring to mind, and board variants with each are available. You do have to get involved in V4L2, Media Controller, and/or GStreamer to get them to work, but would enable you to validate things further.
- Can you set up another FPGA as CSI2 receiver that you can then debug? At least that way you have full control over both ends and can see what is going on.
- Produce parallel data out of your FPGA and use a Toshiba TC358746 or similar to convert to CSI2.

In answer to your other questions, no frame number isn't required. AIUI the receiver is only looking for frame start/end and line start/end short codes. Neither the frame number nor line number are used.
Your timing values look sensible and should match the defaults programmed in the hardware well enough.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 12, 2018 3:46 pm

Totally understand, and I really appreciate the massive amounts of help you've provided. Given the scope I have coming in will decode D-PHY, that will be my next step. I also have an email to my Lattice contact about what I am seeing. Hopefully something will come out of that.

I have done an extensive amount of searching on the MachX02/CSI-2 transmit bridge. Like you found, I've read through countless examples of people asking about the iMX6 and/or interfacing with the MachX02 and an intermediate chip (I saw your threads talking about the Toshiba variants). I've checked all the problems those others had as well (like FV to LV timing, and Word Count issues).

I'll of course report back when I get it all figured out. Hopefully for anyone else wanting to use a MachX02 this will help eventually!

One last question, I see that the CS interface is set to 120 in the vcgen log file. I remember reading in the iMX6 threads that the value of that clock was important to the transmitter talking to the iMX6. Is that different on the Pi?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5816
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 12, 2018 4:07 pm

grimepoch wrote:
Mon Feb 12, 2018 3:46 pm
One last question, I see that the CS interface is set to 120 in the vcgen log file. I remember reading in the iMX6 threads that the value of that clock was important to the transmitter talking to the iMX6. Is that different on the Pi?
??! "CS interface is set to 120"? I don't follow you on the log message you're seeing, and I don't have a suitable setup at present.
There is a low power clock that the SoC uses to clock the state machine for low speed to high speed transitions. That's set up internally at 100MHz IIRC which covers almost all situations we've encountered to date (adding "force_turbo=1" to config.txt solved the one odd-ball case, but the reasons for having issues in the first place still aren't quite clear).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 12, 2018 4:14 pm

Code: Select all

2877880.907: cm_deliver_buffer: New buffer 3e6d13dc. port->pBufferList is 3e6d1428
2877880.933: cm_new_input_frames: start
2877880.959: cm_new_input_frames: end - new frames locked 3e6d1428 and 3e6d13dc, head buffer now 0
2877880.969: cm_process_thread: Buffer swap 3 - task on new buffers
2877880.980: cm_set_buffers: start
2877880.998: set buf params ptr fd97b000-fd9eb800, stride 960
2877881.006: cm_set_buffers: end
2877881.014: cm_start_receiving: start
2877881.050: set pipeline to id 24, decode 0, unpack 0, encode 0, pack 0, enc block len 0
2877881.064: start with 1 datalanes, intfreq 120
2877882.173: cm_start_receiving: end
2877882.217: cm_process_thread: events 0001
2877882.237: cm_deliver_buffer: New buffer 3e6d1390. port->pBufferList is 3e6d1390
2877882.248: cm_new_input_frames: start
2877882.259: cm_new_input_frames: end - no new frames available
2877882.294: cm_process_thread: events 0001
2877882.314: cm_deliver_buffer: New buffer 3e6d1344. port->pBufferList is 3e6d1390
2877882.327: cm_new_input_frames: start
2877882.348: cm_new_input_frames: end - new frames locked 3e6d1390 and 3e6d1344, head buffer now 0
2877882.359: cm_process_thread: Buffer swap 3 - task on new buffers
2877882.369: cm_set_buffers: start
2877882.380: cm_set_buffers: buffers already swapped. Wait EOF
2878881.152: cm_set_port_state: setting port 350 to state 1 (was 4)
2878881.169: cm_process_thread: events 0004
2878881.185: cm_set_port_state: exiting with port state 4
2878881.198: rawcamRIL:cm_stop_peripheral: enter
This was part of the log I was looking at, the line in particular was this one:

2877881.064: start with 1 datalanes, intfreq 120

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5816
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 12, 2018 4:19 pm

grimepoch wrote:
Mon Feb 12, 2018 4:14 pm
This was part of the log I was looking at, the line in particular was this one:

2877881.064: start with 1 datalanes, intfreq 120
That is saying we'll produce an interrupt every 120 line end packets raised. Nothing to do with clocks.
It's a fairly arbitrary number of lines. IIRC we aim for 4 interrupts per frame, just so we get a good few chances to swap buffers and be ready for the next frame.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Feb 12, 2018 4:27 pm

Ahh, okay. As you can imagine, I was fine tooth combing over every detail. :) There is a lot of detail in that log, though it does give a good indication of most of what is going on.

I meant to say I also have the HDMI->CSI-2 interface board from Auvidea that once I get the scope in, I can peak at the CSI lines to compare.

grimepoch
Posts: 95
Joined: Thu May 12, 2016 1:57 am

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Feb 14, 2018 6:45 pm

Quick update, the MachX02 recommendations were wrong they initially gave me. I'm still trying to work out with them what I should be using. Their original design was for LVDS25E TX lines. They had told me LVDS25 or LVDS33 (non-emulated) would be fine, but that is not the case, they told me they were wrong. I am trying to use LVDS33E (since my VDDIO is 3.3V in this area of the FPGA) but so far still not working. (Trying not to use LVDS25E since I need 3.3V signals in that same bank).

Also, since I was not aware of this, the only difference between MachX02 and MachX03 they told me was the packaging. The silicon is the same, they were just more aggressive with their packages parasitic targets. Just putting this here for anyone else trying to interface with raspiraw with the MachX02.

Return to “Camera board”