samir_sogay
Posts: 17
Joined: Tue Jun 18, 2013 9:05 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Mon Jan 25, 2016 6:04 am

Dear Orbital6,

Thanks for your inputs. I did read the entire thread but it was not clear that the driver was for the prototype board. As far as taking control of TC is concerned through i2c, would you point me in the right direction as I am not proficient in this field. And would you suggest to use the driver mentioned by 6by9 for Toshiba rather than the firmware one?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Mon Jan 25, 2016 11:36 am

samir_sogay wrote:Dear Orbital6,

Thanks for your inputs. I did read the entire thread but it was not clear that the driver was for the prototype board. As far as taking control of TC is concerned through i2c, would you point me in the right direction as I am not proficient in this field. And would you suggest to use the driver mentioned by 6by9 for Toshiba rather than the firmware one?
The raspivid support works to some degree but is not going to be supported/updated.

Work is in progress to get things supported via the kernel drivers, but it's not trivial.

Current status of my work is all from userspace:
- There is a pure userspace app at https://github.com/6by9/userland/blob/h ... tc358743.c that includes all the registers being currently thrown at the Toshiba chip.
- You will also need to grab the test firmware from https://github.com/6by9/RPiTest/tree/master/rawcam as I haven't passed a few minor changes to Pi Towers as yet. Changes now in the official firmwares as of 8/4/16. Currently "sudo rpi-update" to pick it up. The next Raspbian release after that date should also include it.
- The final thing is I2C setup and pin muxing. The device is on i2c-0 and GPIOs 28&29, so you'll need to enable "dtparam=i2c_vc=on" in config.txt, and the camera_i2c script in https://github.com/6by9/userland/tree/hdmi_in will sort the pin muxing. (If you do run raspivid, you will have to rerun that script as it will fiddle with the muxing).

It works with 720P input only at present, and produces RGB888 output fed to the display. It does not support feeding to the video_encoder at the moment as I'm expecting that to only accept YUV420 formats.
I was in the process of dissecting the EDID to see what resolutions were being advertised as supported - I was expecting it to be 720P only, but from your comments it appears not. The userland code does NOT poll the Toshiba chip for resolution changes or update the display side on a change, so feeding anything else in will probably look garbled.
The main remaining part I'll be looking at from this is audio, but that may only be a superficial investigation depending on how easily things fall into place.

PLEASE DON'T ASK FOR SUPPORT ON ANY OF THIS AT PRESENT - it is there as a work in progress with flaws and almost no error handling. Feel free to hack it as you see fit. If you do something useful then push it to me for consideration if you can be bothered.

This userspace app is not my end goal - getting the CSI2 receiver integrated into a V4L2 kernel driver is. That way we should be able to use any of the V4L2 device drivers for devices that have a CSI2 output.
If you're interested and can help develop stuff, then please let me know (every appeal I've made so far has met with a wall of silence!).
Last edited by 6by9 on Fri Apr 08, 2016 8:10 pm, edited 2 times in total.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

samir_sogay
Posts: 17
Joined: Tue Jun 18, 2013 9:05 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Tue Jan 26, 2016 8:09 am

@6by9 Thanks for your inputs. I am not proficient in Linux so I will not be able to help in development but I can provide all inputs of my testing. It will take me lots of time to understand the app and the firmware and how to use it. Also,
https://github.com/6by9/userland/blob/hdmi_in/ url is not valid.

Anyway, did you get your hands on the prototype board yet?

I am linking the EDID file below
https://dl.dropboxusercontent.com/u/92299797/edid.txt

The TC supports 1080p60, so there should be a way to change the EDID register in TC to hand out 1080p resolution option to video source.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Tue Jan 26, 2016 11:56 am

samir_sogay wrote:@6by9 Thanks for your inputs. I am not proficient in Linux so I will not be able to help in development but I can provide all inputs of my testing. It will take me lots of time to understand the app and the firmware and how to use it. Also,
https://github.com/6by9/userland/blob/hdmi_in/ url is not valid.
Oops, trimmed a URL incorrectly - https://github.com/6by9/userland/tree/hdmi_in
I'll edit the earlier post too.
samir_sogay wrote:Anyway, did you get your hands on the prototype board yet?
Note my sig - I'm ex Broadcom and still support Raspberry Pi firmware development with the full blessing (gratitude!) of Pi Towers. I asked them nicely to borrow the prototype :D
The Auvidea board is going to be pretty close to that design seeing as it works with the firmware and my userland app, but I have no links with them as a company.
samir_sogay wrote:I am linking the EDID file below
https://dl.dropboxusercontent.com/u/92299797/edid.txt
Ta.
I'm seeing differences between decoding the EDID by hand, the Pi edidparser library, edid-decode under Ubuntu, and your results. I must have been doing something daft in my handling of the data block. The raw EDID is in my userland source repo at https://github.com/6by9/userland/blob/h ... 743.c#L298, so I'm trying to reverse engineer it to add 1080P30.
The Linux kernel driver cheats and passes full responsibility for setting the EDID up to userspace and the client app. I've yet to find an open source app that uses the driver - it was written by Cisco for one of their video conferencing systems, so the original app is not going to be open source.

You say you're not proficient in Linux, but If you felt like working on the EDID side then that contribution would be very welcome too.
samir_sogay wrote:The TC supports 1080p60, so there should be a way to change the EDID register in TC to hand out 1080p resolution option to video source.
1080P60 won't be supported. The standard Pi camera connector only exposes 2 CSI-2 data lanes at 1Gbit/s each. 1080P60 RGB888 is 1920*1080*24*60 = 2,985,984,000bits/s before overhead - it no fit! It would be possible on one of the 2 Compute Module camera ports, but you're not going to be able to do much with it as the video encoder can only support 1080P30 (H264 level 4.1)
1080P30 should be possible with a bit of work on the EDID (don't think it needs any further changes on that).
1080i and 576i I'm going to gloss over for now. The Toshiba chip sends the two fields on two different image IDs within the CSI stream. We can differentiate between 2 sets of IDs which are currently used for video vs non-video (audio in this case, though not fully handled yet). So we could drop audio in favour of the two fields, but the video encoder has no support for interlaced content so you'd have to deinterlace first. All possible, but not my priority.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

samir_sogay
Posts: 17
Joined: Tue Jun 18, 2013 9:05 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Mon Feb 01, 2016 7:49 pm

@6by9 Can you tell me how to use your userland app raspi_tc358743.c as I have failed at every attempt at making it into an application.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Mon Feb 01, 2016 10:29 pm

samir_sogay wrote:@6by9 Can you tell me how to use your userland app raspi_tc358743.c as I have failed at every attempt at making it into an application.

Code: Select all

git clone https://github.com/6by9/userland.git -b hdmi_in
cd userland
./buildme
./camera_i2c
./build/bin/raspi_tc358743
(that last line can probably be just "raspi_tc358743" as CMake is meant to install it into /opt/vc/, but I'm never certain if it actually does or not).

(Edited to correct the git clone line)
Last edited by 6by9 on Thu Feb 04, 2016 10:54 am, edited 1 time in total.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
vade
Posts: 6
Joined: Tue Dec 15, 2015 3:09 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Tue Feb 02, 2016 9:08 pm

My Auvidea boards arrived the other day, and some quick testing with OpenFrameworks on Raspberry Pi 2 just works, using this OpenMax / Raspberry Pi Camera 'add on' for OpenFrameworks

https://github.com/jvcleave/ofxRPiCameraVideoGrabber

https://vimeo.com/153959180

My original video I realize was slightly flawed, my output device was 854x480. I was able to test at 1280x800 and draw the full 720p image without issue.

Latency is pretty decent too:

https://twitter.com/_vade/status/694623089037070336

Thanks. This opens up a lot of possibilities. Getting hardware support opens up a lot of product capabilities, like software / GLSL driven effects pedals for video, mini video performance, cueing and live input devices, and possibly even video mixers assuming you can get enough CSI lanes.

Orbital6
Posts: 140
Joined: Sat Aug 08, 2015 6:32 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Tue Feb 02, 2016 9:18 pm

It'd be great if someone can help test an audio solution, i've been having issues rendering video and capturing i2s audio at the same time.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Tue Feb 02, 2016 9:41 pm

vade wrote:My Auvidea boards arrived the other day, and some quick testing with OpenFrameworks on Raspberry Pi 2 just works, using this OpenMax / Raspberry Pi Camera 'add on' for OpenFrameworks

https://github.com/jvcleave/ofxRPiCameraVideoGrabber
That is again using the firmware driver. Unsupported demo code. Don't expect support if it breaks, nor any updates.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
vade
Posts: 6
Joined: Tue Dec 15, 2015 3:09 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Tue Feb 02, 2016 11:55 pm

Understand about the demo code.

Does this mean however that anyone using a camera like the NOIR is forever using demo code? Whats the status of officially supported CSI access?

Apologies if this has been hashed out elsewhere, or even in this thread. I didn't see mention, except what I thought was 'unofficial' firmware for more EDID's and support for the Auvidea board.

Thank you.

If I can help, Im a fairly decent dev, however with little to zero linux experience.

Thanks again.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Feb 03, 2016 1:53 pm

vade wrote:Understand about the demo code.

Does this mean however that anyone using a camera like the NOIR is forever using demo code? Whats the status of officially supported CSI access?

Apologies if this has been hashed out elsewhere, or even in this thread. I didn't see mention, except what I thought was 'unofficial' firmware for more EDID's and support for the Auvidea board.
The Pi camera and PINoir are devices designed, made (under licence), and sold by Pi Foundation/Trading. They are fully supported and any issues found will be fixed.

The HDMI input setup was put together for a demo but has never been sold by Pi Foundation/Trading. Auvidea have made a board that happens to be close enough to the Foundation demo design that it works, but that doesn't mean Pi Foundation/Trading (or I) will be supporting it.

What you see as the board working is the combination of multiple layers of software stack. The camera stack is something similar to this horrid ASCII art:

Code: Select all

           userspace
               |
        OpenMAX IL / MMAL
               | 
         Buffer management    -      ISP tuner (AGC/AWB/etc)
        /         |       \
Sensor driver  ISP hw CSI2 receiver
and even that is a gross simplification. Throw in working efficiently with video encode as just one little bit.
The tuner and sensor driver for the HDMI input chip are the bits that are not supported. The equivalents for OV5647 (Pi Camera) are supported, as is the framework stuff.
vade wrote:If I can help, Im a fairly decent dev, however with little to zero linux experience.
Help always welcome, though there are a couple of low level changes to sort still to get the full functionality.

Official support is still unofficial at the moment - it's my work in progress. If you use it and find issues then I will debug them, but please appreciate that I am not employed by Raspberry Pi, and this is a spare time thing.

The app in my userland repo drives the TC358743 chip and sticks the images on the screen. No video encoding as yet, though I probably have all the bits in place to sort that fairly easily.
The first thing that could be usefully done by others is disassembling the EDID to work out what it advertises, and then add polling over I2C for changes in the resolution. On getting that event I'm happy to take over updating the parameters fed to the GPU to pick up the new resolution.
Updating the EDID to include 1080P24/25/30 would also be useful and doesn't have to be done by me. As I've said, I'm not looking at interlaced modes at the moment. That would need to be done anyway even with the V4L2 kernel driver, as it delegates EDID setup to userspace.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
vade
Posts: 6
Joined: Tue Dec 15, 2015 3:09 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Feb 03, 2016 3:11 pm

Thanks for taking the time to write out such a reply. I also support a lot of open source project and understand the 'officially supported' vs 'spare time' support. 100% get that.

In terms of EDID capture, does it make sense to try to capture the EDID values on an external machine, like via my Mac Book Pro. Ill experiment and report back any findings.

I'll also check out your repo and attempt to get it running.

Some questions you might be able to help, as I am interested in 'tuning' for stable frame rate and performance:

* On many platforms, there is an fast path texture submission pipeline, and faster formats. OS X for example tends to like GL_BGR / BGRA and UNSIGNED_INT_8_8_8_8_REV formats, as well as using GL_TEXTURE_STORAGE and GL_TEXTURE_RANGE extensions to avoid expensive memory copies from your app to the driver to the GPU.

Are there any docs I can look up for fast path upload? Ive noted the default format for the camera seems to be OMX_COLOR_Format32bitABGR8888 - at least - as per the previously posted Open Frameworks code. Do you know if the 3 channel variants work? Might save some memory bandwidth as alpha isn't going to come from anywhere :)

* As someone used to the niceties of desktop development (Xcode, har) - how does one debug OpenGL on Raspberry PI?

* Is there a mechanism to sync drawing calls to be driven by display updates for higher synchronization - something like a CVDisplaylink (apologies for the OS X terminology - this is a thread which calls a function you provide to draw on a higher priority and 'just in time' for display updates)?

* Is there any info on best practices for the GPU in terms of number of dependent texture fetches, fill rate, or other notes to help optimize GLSL processing (i.e. preferring lowp precision for texture units, etc etc).

Is there a better place to post findings? Github repo issues, or is this thread ok? Thanks again and apologies for the deluge of questions, I do appreciate the time required to answer them!

This is very exciting.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Feb 03, 2016 3:37 pm

vade wrote:In terms of EDID capture, does it make sense to try to capture the EDID values on an external machine, like via my Mac Book Pro. Ill experiment and report back any findings.
The raw EDID is in the source. samir_sogay already did some EDID grabbing earlier in this thread and posted to https://dl.dropboxusercontent.com/u/92299797/edid.txt I started reverse engineering it but didn't complete it.
vade wrote:Some questions you might be able to help, as I am interested in 'tuning' for stable frame rate and performance:

* On many platforms, there is an fast path texture submission pipeline, and faster formats. OS X for example tends to like GL_BGR / BGRA and UNSIGNED_INT_8_8_8_8_REV formats, as well as using GL_TEXTURE_STORAGE and GL_TEXTURE_RANGE extensions to avoid expensive memory copies from your app to the driver to the GPU.
GL is not an area I have much experience with.
The 3D hardware needs source data as RGB in a special tiled format. That is not exposed anywhere, and there are no hardware blocks that can create it so that conversion generally has to happen on the VPU (Vector Processing Unit - the main general purpose processor in the GPU). That is expensive.
There is a mechanism to get MMAL buffers into GL, but I can't remember the detail of it. raspistill and raspivid in the userland repo have an option to render via GL which uses it.
vade wrote:Are there any docs I can look up for fast path upload? Ive noted the default format for the camera seems to be OMX_COLOR_Format32bitABGR8888 - at least - as per the previously posted Open Frameworks code. Do you know if the 3 channel variants work? Might save some memory bandwidth as alpha isn't going to come from anywhere :)
Default camera format is actually YUV420PackedPlanar or YUV422PackedPlanar. All other formats (including RGB) are post processing steps done via the VPU.
vade wrote:* As someone used to the niceties of desktop development (Xcode, har) - how does one debug OpenGL on Raspberry PI?
Pass.
vade wrote:* Is there a mechanism to sync drawing calls to be driven by display updates for higher synchronization - something like a CVDisplaylink (apologies for the OS X terminology - this is a thread which calls a function you provide to draw on a higher priority and 'just in time' for display updates)?
dispmanx is the display backend. That already deals with synchronisation, gives callbacks when resources come off the screen, and all that sort of stuff.
If just looking at userspace multimedia stuff, then using MMAL and the video_render component deals with presenting images to the screen in an easy way. How to wrap that in with V4L2 (which is my intended platform for TC358743) may be trickier - the buffer will be allocated by dmabuf, but it's a question of mapping.
vade wrote:* Is there any info on best practices for the GPU in terms of number of dependent texture fetches, fill rate, or other notes to help optimize GLSL processing (i.e. preferring lowp precision for texture units, etc etc).
Pass.
vade wrote:Is there a better place to post findings? Github repo issues, or is this thread ok? Thanks again and apologies for the deluge of questions, I do appreciate the time required to answer them!
If specifically related to TC358743 then this thread is probably as good a place as any. If you start going off into GL land, then start a new thread.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

samir_sogay
Posts: 17
Joined: Tue Jun 18, 2013 9:05 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Feb 03, 2016 8:40 pm

6by9 wrote:
samir_sogay wrote:@6by9 Can you tell me how to use your userland app raspi_tc358743.c as I have failed at every attempt at making it into an application.

Code: Select all

git clone https://github.com/6by9/userland.git
cd userland
./buildme
./camera_i2c
./build/bin/raspi_tc358743
(that last line can probably be just "raspi_tc358743" as CMake is meant to install it into /opt/vc/, but I'm never certain if it actually does or not).
Tried many times the given commands but couldn't make it work. Then there was a forum about raspiraw and post by Jbeale about similar procedure and it worked. So added
git fetch origin; git checkout -b hdmi_in origin/hdmi_in

Also since I am on B rev 2
I modified camera_i2c as

gpio -g mode 0 in
gpio -g mode 1 in
gpio -g mode 0 in
gpio -g mode 0 alt0
gpio -g mode 1 in
gpio -g mode 1 alt0
#shutdown
gpio -g write 5 1
#LED
gpio -g write 21 1

Thanks for your support.

User avatar
vade
Posts: 6
Joined: Tue Dec 15, 2015 3:09 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Feb 04, 2016 12:19 am

hi 6x9

I'm able to check out and compile your 'user land' repo. One note for those reading, one needs to 'git checkout hdmi_in' as cloning sets the branch to master regardless what is set in github UI.

Code: Select all

git clone https://github.com/6by9/userland.git
cd userland
git checkout 'hdmi_in'
./buildme
./camera_i2c
./build/bin/raspi_tc358743
I seem to be getting a seg fault running raspi_tc358743:

Code: Select all

[email protected]:~/userland$ ./camera_i2c 
[email protected]:~/userland$ ./build/bin/raspi_tc358743 
mmal: Set pack to 0, unpack to 0
mmal: Set rx_timing
mmal: Enable rawcam....
mmal: output p_fmt_commit...
mmal: buffer size is 0 bytes, num 8
mmal: Create connection....
mmal: mmal_vc_port_info_set: failed to set port info (2:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0x1a37c70 (EINVAL)
mmal: mmal_connection_create: format not set on input port
mmal: mmal_connection_destroy_internal: connection vc.ril.rawcam:out:0/vc.ril.video_render:in:0 could not be cleared
mmal: All done. Start streaming...
mmal: Failed to set I2C address
mmal: View!
mmal: Stopping streaming...
mmal: Failed to set I2C address
Segmentation fault
I'm running a Raspberry Pi 2 B V 1.1. The edited camera_i2c posted by @samir_sogay didn't work for me.

Thanks!

samir_sogay
Posts: 17
Joined: Tue Jun 18, 2013 9:05 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Feb 04, 2016 8:30 am

From another post

B Rev1 - I2C 1 on GPIOs 2 & 3. GPIOs 5 & 27 for LED and power.
B Rev2 - I2C 0 on GPIOs 0 & 1. GPIOs 5 & 21 for LED and power.
A+, B+, and B2 all revisions - I2C 0 on GPIOs 28 & 29. GPIOs 32 & 41 for LED and power

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Feb 04, 2016 9:22 am

samir_sogay wrote:From another post

B Rev1 - I2C 1 on GPIOs 2 & 3. GPIOs 5 & 27 for LED and power.
B Rev2 - I2C 0 on GPIOs 0 & 1. GPIOs 5 & 21 for LED and power.
A+, B+, and B2 all revisions - I2C 0 on GPIOs 28 & 29. GPIOs 32 & 41 for LED and power
The HDMI board doesn't use the GPIOs for power or LED, that's only applicable to the Pi Camera (OV5647).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Feb 04, 2016 10:53 am

vade wrote:I'm able to check out and compile your 'user land' repo. One note for those reading, one needs to 'git checkout hdmi_in' as cloning sets the branch to master regardless what is set in github UI.

Code: Select all

git clone https://github.com/6by9/userland.git
cd userland
git checkout 'hdmi_in'
./buildme
./camera_i2c
./build/bin/raspi_tc358743
That's what I get for typing it out blind!
git clone https://github.com/6by9/userland.git -b hdmi_in
does the clone and checkout of the branch in one, and is what I meant to type. I think I went into the web browser, changed branch, and didn't check the clone URL I copied before pasting. I'll edit the earlier post for others.
vade wrote:I seem to be getting a seg fault running raspi_tc358743:

Code: Select all

[email protected]:~/userland$ ./camera_i2c 
[email protected]:~/userland$ ./build/bin/raspi_tc358743 
mmal: Set pack to 0, unpack to 0
mmal: Set rx_timing
mmal: Enable rawcam....
mmal: output p_fmt_commit...
mmal: buffer size is 0 bytes, num 8
mmal: Create connection....
mmal: mmal_vc_port_info_set: failed to set port info (2:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0x1a37c70 (EINVAL)
mmal: mmal_connection_create: format not set on input port
mmal: mmal_connection_destroy_internal: connection vc.ril.rawcam:out:0/vc.ril.video_render:in:0 could not be cleared
mmal: All done. Start streaming...
mmal: Failed to set I2C address
mmal: View!
mmal: Stopping streaming...
mmal: Failed to set I2C address
Segmentation fault
I'm running a Raspberry Pi 2 B V 1.1. The edited camera_i2c posted by @samir_sogay didn't work for me.
I'll try to support the older platforms, but all my development will be on a Pi2.
Can you "sudo apt-get install i2c-tools", "sudo i2cdetect -y 0" and confirm that address 0x0F shows up? That proves basic I2C comms to the HDMI chip.

A couple of guesses:
(a) /dev/i2c-0 doesn't exist (enable with "dtparam=i2c_vc=on" in /boot/config.txt. Add "disable-touchscreen=1" too if you have the Pi Display connected).
There's a bug in the code too - fopen returns NULL on failure, whilst open returns -1. Line 389 should be "if(fd == -1)", not "if(!fd)". I'm at work so can't update the repo at the moment, but will try to remember to do so tonight.
(b) You haven't grabbed either my test firwmare or done "sudo rpi-update". My test firmware is preferable as I haven't modified raspi_tc358743 to filter out the metadata buffers which video_render won't understand. That would explain the "mmal_vc_port_info_set: failed to set port info (2:0): EINVAL" log line, as BGR24 support for rawcam and video_render has only just been added.

I'm tempted to open a new thread solely for raspi_tc358743 support as this thread was intended to be about the kernel drivers for TC358743, and I am working to get those working. It'd also give me a chance to write the basic instructions in the first post on the thread!
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
vade
Posts: 6
Joined: Tue Dec 15, 2015 3:09 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Feb 04, 2016 8:44 pm

Thanks again for the detailed reply. Its appreciated.

To be clearI am running:
RPI2, Model B, v 1.1
Linux Infinity 4.1.17-v7+ #834 SMP Mon Feb 1 15:17:54 GMT 2016 armv7l GNU/Linux
latest firmware as of today.

Your guesses were right, I did not have /dev/i2c-0. Ive enabled that in my config.txt and installed i2c-tools.

My i2cdetect output:

Code: Select all

i2cdetect -y 0
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --            
attempting to run raspi_tc358743 results in:

Code: Select all

./raspi_tc358743 
mmal: Set pack to 0, unpack to 0
mmal: Set rx_timing
mmal: Enable rawcam....
mmal: output p_fmt_commit...
mmal: buffer size is 2764800 bytes, num 8
mmal: Create connection....
mmal: Enable connection...
mmal: buffer size is 2764800 bytes, num 8
mmal: All done. Start streaming...
mmal: i2c_wr: writing register 0x4 from 0xf failed

mmal: i2c_wr: writing register 0x2 from 0xf failed

mmal: i2c_wr: writing register 0x2 from 0xf failed

mmal: i2c_wr: writing register 0x6 from 0xf failed

mmal: i2c_wr: writing register 0x8 from 0xf failed

mmal: i2c_wr: writing register 0x14 from 0xf failed

mmal: i2c_wr: writing register 0x16 from 0xf failed

mmal: i2c_wr: writing register 0x20 from 0xf failed

mmal: i2c_wr: writing register 0x22 from 0xf failed

mmal: i2c_wr: writing register 0x4 from 0xf failed

mmal: i2c_wr: writing register 0x140 from 0xf failed

mmal: i2c_wr: writing register 0x144 from 0xf failed

mmal: i2c_wr: writing register 0x148 from 0xf failed

mmal: i2c_wr: writing register 0x14c from 0xf failed

mmal: i2c_wr: writing register 0x150 from 0xf failed

mmal: i2c_wr: writing register 0x210 from 0xf failed

mmal: i2c_wr: writing register 0x214 from 0xf failed

mmal: i2c_wr: writing register 0x218 from 0xf failed

mmal: i2c_wr: writing register 0x21c from 0xf failed

mmal: i2c_wr: writing register 0x220 from 0xf failed

mmal: i2c_wr: writing register 0x224 from 0xf failed

mmal: i2c_wr: writing register 0x228 from 0xf failed

mmal: i2c_wr: writing register 0x22c from 0xf failed

mmal: i2c_wr: writing register 0x234 from 0xf failed

mmal: i2c_wr: writing register 0x204 from 0xf failed

mmal: i2c_wr: writing register 0x518 from 0xf failed

mmal: i2c_wr: writing register 0x500 from 0xf failed

mmal: i2c_wr: writing register 0x8502 from 0xf failed

mmal: i2c_wr: writing register 0x8512 from 0xf failed

mmal: i2c_wr: writing register 0x8513 from 0xf failed

mmal: i2c_wr: writing register 0x8515 from 0xf failed

mmal: i2c_wr: writing register 0x8531 from 0xf failed

mmal: i2c_wr: writing register 0x8540 from 0xf failed

mmal: i2c_wr: writing register 0x8630 from 0xf failed

mmal: i2c_wr: writing register 0x8670 from 0xf failed

mmal: i2c_wr: writing register 0x8532 from 0xf failed

mmal: i2c_wr: writing register 0x8536 from 0xf failed

mmal: i2c_wr: writing register 0x853f from 0xf failed

mmal: i2c_wr: writing register 0x8543 from 0xf failed

mmal: i2c_wr: writing register 0x8544 from 0xf failed

mmal: i2c_wr: writing register 0x8545 from 0xf failed

mmal: i2c_wr: writing register 0x8546 from 0xf failed

mmal: i2c_wr: writing register 0x85c7 from 0xf failed

mmal: i2c_wr: writing register 0x85cb from 0xf failed

mmal: i2c_wr: writing register 0x8c00 from 0xf failed

mmal: i2c_wr: writing register 0x8c10 from 0xf failed

mmal: i2c_wr: writing register 0x8c20 from 0xf failed

mmal: i2c_wr: writing register 0x8c30 from 0xf failed

mmal: i2c_wr: writing register 0x8c40 from 0xf failed

mmal: i2c_wr: writing register 0x8c50 from 0xf failed

mmal: i2c_wr: writing register 0x8c60 from 0xf failed

mmal: i2c_wr: writing register 0x8c70 from 0xf failed

mmal: i2c_wr: writing register 0x8c80 from 0xf failed

mmal: i2c_wr: writing register 0x8c90 from 0xf failed

mmal: i2c_wr: writing register 0x8ca0 from 0xf failed

mmal: i2c_wr: writing register 0x8cb0 from 0xf failed

mmal: i2c_wr: writing register 0x8cc0 from 0xf failed

mmal: i2c_wr: writing register 0x8cd0 from 0xf failed

mmal: i2c_wr: writing register 0x8ce0 from 0xf failed

mmal: i2c_wr: writing register 0x8cf0 from 0xf failed

mmal: i2c_wr: writing register 0x8d00 from 0xf failed

mmal: i2c_wr: writing register 0x8544 from 0xf failed

mmal: i2c_wr: writing register 0x8544 from 0xf failed

mmal: i2c_wr: writing register 0x8544 from 0xf failed

mmal: i2c_wr: writing register 0x85d1 from 0xf failed

mmal: i2c_wr: writing register 0x8560 from 0xf failed

mmal: i2c_wr: writing register 0x8563 from 0xf failed

mmal: i2c_wr: writing register 0x8564 from 0xf failed

mmal: i2c_wr: writing register 0x8574 from 0xf failed

mmal: i2c_wr: writing register 0x8573 from 0xf failed

mmal: i2c_wr: writing register 0x8576 from 0xf failed

mmal: i2c_wr: writing register 0x8600 from 0xf failed

mmal: i2c_wr: writing register 0x8602 from 0xf failed

mmal: i2c_wr: writing register 0x8603 from 0xf failed

mmal: i2c_wr: writing register 0x8604 from 0xf failed

mmal: i2c_wr: writing register 0x8606 from 0xf failed

mmal: i2c_wr: writing register 0x8607 from 0xf failed

mmal: i2c_wr: writing register 0x8620 from 0xf failed

mmal: i2c_wr: writing register 0x8640 from 0xf failed

mmal: i2c_wr: writing register 0x8641 from 0xf failed

mmal: i2c_wr: writing register 0x8642 from 0xf failed

mmal: i2c_wr: writing register 0x8652 from 0xf failed

mmal: i2c_wr: writing register 0x8665 from 0xf failed

mmal: i2c_wr: writing register 0x8709 from 0xf failed

mmal: i2c_wr: writing register 0x870b from 0xf failed

mmal: i2c_wr: writing register 0x870c from 0xf failed

mmal: i2c_wr: writing register 0x870d from 0xf failed

mmal: i2c_wr: writing register 0x870e from 0xf failed

mmal: i2c_wr: writing register 0x9007 from 0xf failed

mmal: i2c_wr: writing register 0x854a from 0xf failed

mmal: i2c_wr: writing register 0x4 from 0xf failed

mmal: View!
mmal: Stopping streaming...
mmal: i2c_wr: writing register 0x8544 from 0xf failed

mmal: i2c_wr: writing register 0x8544 from 0xf failed

[email protected]:~/userland/build/bin$ 
Apologies if Ive missed anything obvious.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Feb 04, 2016 9:15 pm

vade wrote:Thanks again for the detailed reply. Its appreciated.

To be clearI am running:
RPI2, Model B, v 1.1
Linux Infinity 4.1.17-v7+ #834 SMP Mon Feb 1 15:17:54 GMT 2016 armv7l GNU/Linux
latest firmware as of today.

Your guesses were right, I did not have /dev/i2c-0. Ive enabled that in my config.txt and installed i2c-tools.
<snip>
Apologies if Ive missed anything obvious.
Probably the pin muxing.

Code: Select all

sudo raspi-gpio get
will dump out the muxing. I can't remember the specifics of where B v1.1 falls in the scheme of GPIO allocations. I think the camera is on i2c0 at GPIO 0&1, so samir_sogay's amended camera_i2c should probably sort it:

Code: Select all

gpio -g mode 0 in
gpio -g mode 0 alt0
gpio -g mode 1 in
gpio -g mode 1 alt0
(The only reason for setting and then resetting is to ensure the registers actually get modified). Make sure GPIOs 28&29 are NOT set to alt0.

If I've got it wrong, then you may find it is on i2c-1 and pins 2&3, so "i2cdetect -y 1" and pinmuxing should be automatically sorted on GPIOs 2&3 (should both be alt0).

Find your device with i2cdetect, then worry about getting the raspi_tc358743 working.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

flyingbaloon
Posts: 5
Joined: Tue Jan 12, 2016 6:08 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Sat Feb 13, 2016 7:09 pm

6by9 wrote:(This isn't one of the normal forums I lurk on - Camera Board, or Interfacing CSI/DSI/I2C are more likely for me to notice)

The answer may be paritially, but not because you're using kernel 4.1. There is no kernel support for the CSI2 receiver peripheral at the moment (partly due to IP issues around control registers).
However the Foundation had implemented the driver solely within the GPU. They perceived that there wasn't a sufficient market to productise the board. It is a reasonable guess that the Auvidea board would be compatible and just be recognised as the same, but I don't know of anyone who has tried it.

There is now a wrapper around the CSI2 receiver that has demoed it receiving the raw Bayer data from the standard camera - viewtopic.php?f=43&t=109137
It should be possible to write a suitable kernel driver to wrap that, in the same way that the V4L2 camera driver wraps the full camera component. The bit of information I haven't managed to determine is how to hook everything together to make the TC358743 driver and the CSI-2 receiver peripheral work together and form a system. At one point it looked like the soc-camera framework was the way, but that appears to have been deprecated in favour of the media framework.

If anyone has any experience of hooking these things together to make a working system, please let me know and it would motivate me to invest a bit more time in it. If there isn't anyone lurking here, I may go direct to the driver writer to get some assistance.

NB This driver only produces UYVY or RGB888 data, neither of which is currently consumable by the video encoder. Conversion on the ARM would be a heavyweight operation.
And yes, 1080P30 can be carried on 2 lane CSI2 - each lane is 1Gb/s, and 1920x1080x24bppx30 = 1.493Gb/s. YUYV is slightly less than that as it is only 16bpp.
I have a requirement to preview hdmi input (both video and audio) in my application , I have searched on the forum and found a couple of posts that refer to the Toshiba TC358743XBG chip and here it states that the present kernal supports it out of the box . So for my application I am trying to get a pcb made for the toshiba chip like the one shown in
http://blog.pi3g.com/2014/03/embedded-w ... bc-market/.

I am still unclear , Can you please guide me whether this approach will work , thanks in advance

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Sat Feb 13, 2016 9:55 pm

flyingbaloon wrote:I have a requirement to preview hdmi input (both video and audio) in my application , I have searched on the forum and found a couple of posts that refer to the Toshiba TC358743XBG chip and here it states that the present kernal supports it out of the box . So for my application I am trying to get a pcb made for the toshiba chip like the one shown in
http://blog.pi3g.com/2014/03/embedded-w ... bc-market/.

I am still unclear , Can you please guide me whether this approach will work , thanks in advance
Read the whole thread.
Some things will work via the firmware drivers, but it is NOT supported.
Work is in progress on a system which will be supported, though it may or may not include audio (the standard kernel driver routes it via I2S which may or may not work)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Orbital6
Posts: 140
Joined: Sat Aug 08, 2015 6:32 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Fri Feb 26, 2016 6:57 pm

I've been trying to make h264 recordings from the toshiba board. Both the userland code and raspivid preview (hdmi out) videos perfectly, but i've just noticed after trying commands like 'raspivid -o test.h264', my videos are recorded at ~0.75x speed of the source footage.

I haven't tried the userland app to make h264 files yet, but could any of you be nice enough to make a short recording using 'raspivid -o test.h264' to confirm that it's just my system being silly (or if there's something deeper).

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8083
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Fri Feb 26, 2016 11:31 pm

How are you playing it back? An h264 elementary stream has no timestamps, and some players will guess at 25fpa instead of 30 (or whatever your source material is).
You can use avconv to multiplex the stream into a container and tell it the correct framerate.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Orbital6
Posts: 140
Joined: Sat Aug 08, 2015 6:32 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Sat Feb 27, 2016 1:10 pm

6by9 wrote:How are you playing it back? An h264 elementary stream has no timestamps, and some players will guess at 25 fps instead of 30 (or whatever your source material is).
You can use avconv to multiplex the stream into a container and tell it the correct frame rate.
Using the same command with the tosh & camera generates different outputs.

This is the source video.

I use this command to capture a video from an HDMI source (pi and set to hdmi_mode=19 here, though this shouldn't have an influence on the frame rate):

Code: Select all

raspivid -t 30000 -fps 25 -o 30-seconds-25-fps-raspivid.h264
This is the output.

and with the camera:

Code: Select all

raspivid -t 30000 -fps 25 -o 30-seconds-25-fps-raspivid-camera.h264
This is the output.

Wrapping them using:

Code: Select all

 avconv -r 25 -i 30-seconds-25-fps-raspivid-camera.h264 -vcodec copy  30-seconds-25-fps-raspivid-camera-avconv.mp4
(same for the camera h264 file).

This is the output for the source video.

and for the camera.

You can see the playback difference and the difference in duration (tosh video is playing back at 0.75x). I've tried playing with different frame rates. The encoded preview (-e) showed the correct playback speed, as does normal preview.

It'd be helpful if you can make a short render yourself so i'd know if the problem was somewhere in my source.

Return to “Graphics, sound and multimedia”