I just read most of this thread, as I am very interested in the camera, as a HD video camera for aerial video (FPV). I currently use the GoPro, but would like to use higher bitrates (compression really costs a lot of image quality with fast-moving pictures...), and possibly different lenses. So far most embedded devices that can do HD video are locked down and don't do what I want.
I have a few questions regarding the camera:
- I know people have asked many times about replacing lenses... and also RAW image output. I understand that even if the existing lens could be removed and replaced with an M12/CS mount, the EDOF algorithm will ruin things. However, if we can get RAW output somehow, would the EDOF compensation already be applied, or is it just RAW pixel values from the bayer pattern?
- I am also really interested in RAW output for color-grading in post-production. There is an open source FPGA camera "Elphel" that does a neat trick to compress "RAW" without debayering. They call it JP4: http://wiki.elphel.com/index.php?title=JP4
As you can see in the wiki, there is already a linux streaming solution in place to work with this format.
Essentially they group the r, g, b pixel sites from the bayer mosaic into 4 blocks of 8x8 pixels, and do a (M)JPEG compression then. This way each JPEG 8x8 block is monochrome. The nice thing is that this technique doesn't first triple the data by converting the bayer into RGB values for each pixel, so the data rate remains pretty low. The JP4 files can be decompressed, and one can do a RAW conversion afterwards with various professional or OS RAW converters, which leads to much better image quality and allows for color-corrections.
Is there any possibility to do something similar on the RPi? With RAW output from the sensor I could easily implement it myself, but I guess the ARM might be too slow to compress MJPEG at 1080p 30fps. Would be best to run the grouping and compression on the GPU, but I guess that I cannot access that.
I really appreciate what you're doing, I have been looking for a small, affordable hardware solution for a long time and the RPi is amazing. Some people dismiss what to do with a camera - I think they have no trust in the imagination and creativity of a million people...
One remark I would like to make - I understand that you are bound by Broadcom NDAs and that they have good reason to protect their IP. However, I would like to point out that there are a few people out there that have a lot of experience and similar qualifications as you do. Even if parts of the image processing code are closed, it would therefore be great if you could think of ways how to get to unprocessed data, or keep some parts of the pipeline configurable (e.g. switch some parts off). Obviously most people won't need it, but some people do and they will figure out a way (and hopefully share it) as long as it is possible at all. E.g. many people seem to be interested to change the lens (for good reason I think). I wouldn't worry about the mechanical aspects, there's always a way... but it would require to bypass the EDOF algorithms, which won't be needed with properly focused quality optics. It would be great if you could keep "hackability" in mind
My usual problem is that most hardware is optimised for the "average consumer", which I am not... and in my view, there are truly amazing things that people do with new hardware that the original designer didn't think of (which I think does align with the RPi goals somehow...)