Chris D
Posts: 218
Joined: Tue Sep 20, 2016 11:27 am

Why no audio on the Pi camera module?

Sun Jun 21, 2020 11:10 am

This is a request for completing the PiCamera module.

Silent movies ended more that 100 years ago. In our little world, we have several cameras and PiCamera for us to use. These are modern inventions in the 21st century. We have a Raspberry Pi 4 - a very powerful computer - even designed with a purpose built port for interfacing to a camera. The PiCamera module is designed to record video - with many options - however, it can't record sound with the movie.

This seriously needs to be resolved. I get it that the camera doesn't have a microphone - no problem I can add a USB microphone for a few dollars. The Raspberry Pi and the PiCamera module SHOULD be able to record audio with the video.

What will it take to bring the Raspberry Pi and PiCamera into the 21st century?

Chris D

User avatar
thagrol
Posts: 3719
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Silent movies ended more than 100 years ago!

Sun Jun 21, 2020 11:40 am

Chris D wrote:
Sun Jun 21, 2020 11:10 am
Silent movies ended more that 100 years ago.
Time for a film history lesson.

The Hollywood silent movie era ended in 1927 (with the release of The Jaz Singer". So 97 years ago. But the vast majority of amateur movie cameras were silent up until the introduction of the home video camera. So were some of the 16mm cameras used in location work for TV production - there's a reason some location work doesn't have sync'd audio or the audio characteristics don't match the visual environment.

A few silent feature films have been made after the end of the silent era, for example The Artist (https://www.imdb.com/title/tt1655442) and Silent Movie (www.imdb.com/title/tt0075222)
In our little world, we have several cameras and PiCamera for us to use. These are modern inventions in the 21st century. We have a Raspberry Pi 4 - a very powerful computer - even designed with a purpose built port for interfacing to a camera. The PiCamera module is designed to record video - with many options - however, it can't record sound with the movie.

This seriously needs to be resolved. I get it that the camera doesn't have a microphone - no problem I can add a USB microphone for a few dollars. The Raspberry Pi and the PiCamera module SHOULD be able to record audio with the video.

What will it take to bring the Raspberry Pi and PiCamera into the 21st century?
Much the same as with any other feature requested by a single user: time, money, a willingness to pay more for the camera module and Pi, and a proven significant demand. Most of the camera based projects I've seen neither require nor need audio so I doubt the demand is there.

I'm not speaking for RPF/RPT just expressing my opinion based on responses to similar demands made in the past.
Arguing with strangers on the internet since 1993.

fruitoftheloom
Posts: 24463
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: Silent movies ended more than 100 years ago!

Sun Jun 21, 2020 11:47 am

Chris D wrote:
Sun Jun 21, 2020 11:10 am
This is a request for completing the PiCamera module.

Silent movies ended more that 100 years ago. In our little world, we have several cameras and PiCamera for us to use. These are modern inventions in the 21st century. We have a Raspberry Pi 4 - a very powerful computer - even designed with a purpose built port for interfacing to a camera. The PiCamera module is designed to record video - with many options - however, it can't record sound with the movie.

This seriously needs to be resolved. I get it that the camera doesn't have a microphone - no problem I can add a USB microphone for a few dollars. The Raspberry Pi and the PiCamera module SHOULD be able to record audio with the video.

What will it take to bring the Raspberry Pi and PiCamera into the 21st century?

Chris D

https://github.com/iizukanao/picam/blob ... /README.md
Thinking outside the box is better than burying your head in the sand...

User avatar
artisticforge
Posts: 35
Joined: Sat Sep 01, 2012 5:36 am
Location: Earth

Re: Silent movies ended more than 100 years ago!

Sun Jun 21, 2020 12:00 pm

hello

There is a need for audio in the capture of wildlife videos. The audio helps provide a context that the video was taken in.
This is true of CCTV systems. Video alone may tell you that someone was loitering about, with audio it would tell you their possible intent.

I am working on wildlife camera project. many of the newer model commercial wildlife cameras available from online megastores provide very good video and audio capture.

As I write this I am watching my squirrel friend raid the peanut house. unfortunately, he has come to rely on me for his food.

I need to get m hands on the 4B 8GB also need some better lenses for the RPI HQ Camera Module.
Need to checkout the thrift stores when they open again.
artisticforge ><>

Chris D
Posts: 218
Joined: Tue Sep 20, 2016 11:27 am

Re: Silent movies ended more than 100 years ago!

Sun Jun 21, 2020 12:34 pm

Does anyone know if this can be used through Python? It appears to be a stand alone program as opposed to an importable module.
fruitoftheloom wrote:
Sun Jun 21, 2020 11:47 am
Chris D wrote:
Sun Jun 21, 2020 11:10 am
This is a request for completing the PiCamera module.

Silent movies ended more that 100 years ago. In our little world, we have several cameras and PiCamera for us to use. These are modern inventions in the 21st century. We have a Raspberry Pi 4 - a very powerful computer - even designed with a purpose built port for interfacing to a camera. The PiCamera module is designed to record video - with many options - however, it can't record sound with the movie.

This seriously needs to be resolved. I get it that the camera doesn't have a microphone - no problem I can add a USB microphone for a few dollars. The Raspberry Pi and the PiCamera module SHOULD be able to record audio with the video.

What will it take to bring the Raspberry Pi and PiCamera into the 21st century?

Chris D

https://github.com/iizukanao/picam/blob ... /README.md

jim1961
Posts: 48
Joined: Sun Dec 17, 2017 8:31 pm

Re: Silent movies ended more than 100 years ago!

Sun Jun 21, 2020 12:38 pm

Hi Artistic Forge,

I use Raspberry PIs for monitoring our wren house and screech owl box. I struggled with audio as well. I settled on using UV4L as the video/audio capture software in the Pi, and then streaming out audio and video (RTP protocol) to iSpy (free) security camera software.

iSpy has setting for an "external" microphone and this works well with my RaspPi UV4L PiCam and USB microphone. iSpy will record too (manual or triggered by motion), and the recordings have video with audio!

I am struggling again though finding a USB (or analog/USB adapter) microphone that is small enough to fit in the box and provide decent audio. The ones I have tested recently either have really low volume or very high hum....

Jim

User avatar
mahjongg
Forum Moderator
Forum Moderator
Posts: 13368
Joined: Sun Mar 11, 2012 12:19 am
Location: South Holland, The Netherlands

Re: Silent movies ended more than 100 years ago!

Sun Jun 21, 2020 1:05 pm

adding unnecessary hardware for the few people that want it, and thereby increasing the cost for everybody is NOT the Raspberry PI way of doing things!

If you do want audio with your video recording, its easy enough!

The PI has a special interface for audio i/O, the inter IC sound channel (I2S) to which you can add a cheap codex chip, some modules combine the tiny Codex chip with an even tinyer MEMS Microphone. For example this solution from adafruit: can be directly connected to any RPI without needing anything but a few wires https://learn.adafruit.com/adafruit-i2s ... iring-test
Image

And obviously the RPI has USB, so you can also add an USB microphone (if you have room for it). These can also be very tiny, not much larger than s small USB plug. Image

No need to integrate sound and video, if you want sound input you can cheaply add it.

User avatar
thagrol
Posts: 3719
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Silent movies ended more than 100 years ago!

Sun Jun 21, 2020 2:27 pm

I suspect the assumption I and others have made that you were asking fro hardware changes comes from a lack of clarity in your initial post.

"PiCamera module" has two possible interpretations:
  1. The hardware module(s).
  2. The python picamera module.
To get #1 to record audio requires hardware changes to both the camera module and to the Pi, as well as software changes. All of wich adds complexity and cost.
While #2 does not require hardware changes it still requires software changes. Which is not free, adds complexity, takes time, and requires testing.

You didn't explictly request new hardware and/or cost but the implicationa of your request did so for you.



EDIT:
One more thing, the python pycamera module is open source (https://github.com/waveform80/picamera) so instead of shouting complaints at forum moderators you could get off your high horse and add this yourself. Then share it.

EDIT #2:
If you don't have the skils to do so, I suggest you open an issue on the github.
Last edited by thagrol on Sun Jun 21, 2020 2:33 pm, edited 1 time in total.
Arguing with strangers on the internet since 1993.

User avatar
hansotten
Posts: 357
Joined: Fri Jul 13, 2012 6:01 am
Contact: Website

Re: Silent movies ended more than 100 years ago!

Sun Jun 21, 2020 3:05 pm

I agree with the OP that recording audio when you have a Raspberry Pi camera is a 'modern' thing.
Having an USB audio adapter is one thing, the software side also needs to be able to do something with that.

So far so good, and I saw one answer pointing to a possible solution.
http://duinorasp.hansotten.com for Raspberry Pi and Arduino and ESP8266
http://retro.hansotten.nl for retrocomputing with the 6502 cpu

hippy
Posts: 8514
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Silent movies ended more than 100 years ago!

Sun Jun 21, 2020 3:13 pm

Chris D wrote:
Sun Jun 21, 2020 1:49 pm
How many video cameras can't record audio with the video?
You seem to be conflating "video sensor module" with "video camera". I doubt any video cameras have a microphone on their video sensor modules except where everything has been integrated onto product PCB's.

A better question might be how many webcams don't support audio but even there it's not an exact comparison.

I do accept there's legitimacy in asking for, or desiring, the Pi camera module to include a microphone or audio input capability but it's not an absolute necessity, to say it is is just opinion.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27398
Joined: Sat Jul 30, 2011 7:41 pm

Re: Why no audio on the Pi camera module?

Sun Jun 21, 2020 3:25 pm

Edited and deleted posts in this thread. Also changed the initial title to something actually appropriate and that won't rile people up.

Anybody ranting again like I saw in the post I deleted will be banned. This is nothing to do with the request, just with the tone on the post. It was unacceptable.

On to the point.

The original prototype of the camera board did have a microphone. But for reasons I cannot remember it was removed, I think it was a combination of things - the ribbon cable limitations, power supply and probably some other stuff. It may have required I2S to the board, which was not supported by the CSI plug.

So at present, the only interface up the ribbon cable that could be used for audio is the I2C interface to the camera. And I am not sure how good audio would be over than interface. Certainly I2C isn't mean for long distances like that over the ribbon cable, and at 100KHz (IIRC), the bandwidth isn't great.

So it would require a redesign of the Pi itself to provide I2S up the CSI connection,. making all previous camera module obsolete on any new Pi board - so this would need to be the Pi5 at a minimum.

So, not feasible I am afraid.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

hippy
Posts: 8514
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Silent movies ended more than 100 years ago!

Sun Jun 21, 2020 3:38 pm

jamesh wrote:
Sun Jun 21, 2020 3:25 pm
So it would require a redesign of the Pi itself to provide I2S up the CSI connection,. making all previous camera module obsolete on any new Pi board - so this would need to be the Pi5 at a minimum.

So, not feasible I am afraid.
It should be possible to design a camera module with audio capability which goes to a separate header which can then be connected by those who wish to use that.

That's not elegant nor ideal, and I am not saying the RPT must or even should do that, but it would provide a solution which is backwards compatible with the existing Pi range, making existing modules forward compatible, would provide a solution with only a camera module change.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27398
Joined: Sat Jul 30, 2011 7:41 pm

Re: Why no audio on the Pi camera module?

Sun Jun 21, 2020 4:03 pm

hippy wrote:
Sun Jun 21, 2020 3:38 pm
jamesh wrote:
Sun Jun 21, 2020 3:25 pm
So it would require a redesign of the Pi itself to provide I2S up the CSI connection,. making all previous camera module obsolete on any new Pi board - so this would need to be the Pi5 at a minimum.

So, not feasible I am afraid.
It should be possible to design a camera module with audio capability which goes to a separate header which can then be connected by those who wish to use that.

That's not elegant nor ideal, and I am not saying the RPT must or even should do that, but it would provide a solution which is backwards compatible with the existing Pi range, making existing modules forward compatible, would provide a solution with only a camera module change.
Just put a USB plug on it then. No need for an extra header. Or just plug in to the GPIO I2S. You could do that now with a custom OV5647 board since there is no need to deal with the crypto chip. Two cables needed of course. So you may as well just use a board as suggested by Mahjong above in that case and bolt it to the standard camera board.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

Tzarls
Authorised Reseller
Authorised Reseller
Posts: 256
Joined: Tue Feb 26, 2013 6:59 am

Re: Why no audio on the Pi camera module?

Sun Jun 21, 2020 4:25 pm

One of the great things about the Raspberry Pi is its modularity - people are free to choose the parts they want to build their project the way they want. Besides, if there was a microphone built into the camera module, there would be endless posts here asking why the mic didn't have this or that characteristics - not a good idea I think.

ethanol100
Posts: 652
Joined: Wed Oct 02, 2013 12:28 pm

Re: Why no audio on the Pi camera module?

Sun Jun 21, 2020 4:37 pm

I just want to give my two cents, too.

Video4Linux does not include audio, you need to have a separate source for audio. By design, the Raspberry Pi(or any other linux computer) receives the images from v4l2 and audio for example from alsa, pulseaudio or similar. The Raspberry Pi then takes both audio and video and saves it to a single file containing both video and audio. This is usually done by a container, like MP4, avi or similar.

If you buy a webcam, this will appear to the Raspberry Pi as two devices, one device for the video part, which usually uses the video4linux2 interface and a sound card which is exposed as an alsa device.

Therefore, it does not make any sense to build a microphone with a sound card into the raspberry pi camera. As this is only the image generating part of a "video" camera, not the camera is the the camcorder here, but the combination of Raspberry PI, Camera and audio source as a complete camcorder system.

The only issue I see, is that getting both sources to sync is not straight forward and it would be great if some tutorials would exists, which explain, how to produce a synchronized videos. It basically comes down to get correct timestamped audio and video frames.

CanPi
Posts: 64
Joined: Wed May 20, 2020 12:21 pm

Re: Why no audio on the Pi camera module?

Sun Jun 21, 2020 5:02 pm

I use this hat and the quality is outstanding

https://wiki.seeedstudio.com/ReSpeaker_2_Mics_Pi_HAT/

The USB mic I got looked similar to the picture in a previous post but nothing could be heard if it was more than a few cm from the mouth. It went back.

Tzarls
Authorised Reseller
Authorised Reseller
Posts: 256
Joined: Tue Feb 26, 2013 6:59 am

Re: Why no audio on the Pi camera module?

Sun Jun 21, 2020 5:43 pm

ethanol100 wrote:
Sun Jun 21, 2020 4:37 pm
The only issue I see, is that getting both sources to sync is not straight forward and it would be great if some tutorials would exists, which explain, how to produce a synchronized videos. It basically comes down to get correct timestamped audio and video frames.
Thw link posted above a few posts earilier (https://github.com/iizukanao/picam/blob ... /README.md) has a small system that can produce syncrinized video and audio. I've used it with the Pi camera and a simple USB mic with great results. It is a standalone program but you can cpntrol it from say a Python program easily.

ethanol100
Posts: 652
Joined: Wed Oct 02, 2013 12:28 pm

Re: Why no audio on the Pi camera module?

Sun Jun 21, 2020 6:07 pm

Tzarls wrote:
ethanol100 wrote:
Sun Jun 21, 2020 4:37 pm
The only issue I see, is that getting both sources to sync is not straight forward and it would be great if some tutorials would exists, which explain, how to produce a synchronized videos. It basically comes down to get correct timestamped audio and video frames.
Thw link posted above a few posts earilier (https://github.com/iizukanao/picam/blob ... /README.md) has a small system that can produce syncrinized video and audio. I've used it with the Pi camera and a simple USB mic with great results. It is a standalone program but you can cpntrol it from say a Python program easily.
Yes, this is a valid solution of generating an audio+video stream. However, it is only a full solution and you could not easily start to write your own code using it. Therefore, I think it would be better to have a standard way using for example v4l2 instead of a quite complicated program which uses OpenMAX IL.

Chris D
Posts: 218
Joined: Tue Sep 20, 2016 11:27 am

Re: Why no audio on the Pi camera module?

Sun Jun 21, 2020 6:44 pm

Chris D wrote:
Sun Jun 21, 2020 11:10 am
I get it that the camera doesn't have a microphone - no problem I can add a USB microphone for a few dollars. The Raspberry Pi and the PiCamera module SHOULD be able to record audio with the video.

Chris D
---------------------------------------------------------------------------------------------------

The original prototype of the camera board did have a microphone. But for reasons I cannot remember it was removed, I think it was a combination of things - the ribbon cable limitations, power supply and probably some other stuff. It may have required I2S to the board, which was not supported by the CSI plug.

So at present, the only interface up the ribbon cable that could be used for audio is the I2C interface to the camera. And I am not sure how good audio would be over than interface. Certainly I2C isn't mean for long distances like that over the ribbon cable, and at 100KHz (IIRC), the bandwidth isn't great.

So it would require a redesign of the Pi itself to provide I2S up the CSI connection,. making all previous camera module obsolete on any new Pi board - so this would need to be the Pi5 at a minimum.

So, not feasible I am afraid.


One more time to clarify -
I am not asking to have a microphone attached to the camera.
I am not asking that a microphone be included with the camera.
I am not asking for the hardware to be modified.


I am asking that the software library called PiCamera (https://picamera.readthedocs.io/en/release-1.13/), be able to record videos with sound into the video file. My understanding this is the suggested software library to work with the cameras from within Python.

My objective is develop higher functioning, special purpose camera(s). With the release of the High Quality Camera kit, this opens up many new possibilities with custom made cameras. Having the ability to record the video and audio at the same time and stored in the same file like other videos with sound, is a requirement to achieve this goal. From my ignorant point of view, I would think that this functionality can be added to that software library thus allowing a much greater appeal for the RPI and cameras.

Someone suggested that if I had the skill, I could rewrite the PiCamera software library . Unfortunately, I, (and I assume many other users of that software module) don't have the skills.

If that is not clear, please don't assume I mean something else, please ask for clarification.

Thank you in advance.

Chris D

User avatar
thagrol
Posts: 3719
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Why no audio on the Pi camera module?

Sun Jun 21, 2020 9:12 pm

Chris D wrote:
Sun Jun 21, 2020 6:44 pm
One more time to clarify -
I am not asking to have a microphone attached to the camera.
I am not asking that a microphone be included with the camera.
I am not asking for the hardware to be modified.


I am asking that the software library called PiCamera (https://picamera.readthedocs.io/en/release-1.13/), be able to record videos with sound into the video file. My understanding this is the suggested software library to work with the cameras from within Python.

My objective is develop higher functioning, special purpose camera(s). With the release of the High Quality Camera kit, this opens up many new possibilities with custom made cameras. Having the ability to record the video and audio at the same time and stored in the same file like other videos with sound, is a requirement to achieve this goal. From my ignorant point of view, I would think that this functionality can be added to that software library thus allowing a much greater appeal for the RPI and cameras.
Thanks for clearing that up. If only you'd been clearer in your first post...
Someone suggested that if I had the skill, I could rewrite the PiCamera software library . Unfortunately, I, (and I assume many other users of that software module) don't have the skills.
That was me. If you check back at that post you'll find I also suggested where you should raise your issue should you not have the skills. I'll point out again that software changes are not risk free, nor are they cost free. Though in this case they would likely be easier than hardware changes.

There are two further options that I didn't mention earlier:
  1. Learn the necessary skills - the Pi is all about education.
  2. Pay someone to do the work. This is the apropriate option if your " higher functioning, special purpose camera(s)" are going to be commercial products.
If that is not clear, please don't assume I mean something else, please ask for clarification.

Thank you in advance.

Chris D
Clarity in communication is required on all sides. Especially on technical subjects. And I'm still sensing something of an "attitude".
Arguing with strangers on the internet since 1993.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27398
Joined: Sat Jul 30, 2011 7:41 pm

Re: Why no audio on the Pi camera module?

Sun Jun 21, 2020 10:22 pm

And to add, the Picamera library is a third party library, not developed by Raspberry Pi....
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

hippy
Posts: 8514
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Why no audio on the Pi camera module?

Mon Jun 22, 2020 8:04 am

thagrol wrote:
Sun Jun 21, 2020 9:12 pm
Thanks for clearing that up. If only you'd been clearer in your first post...
And it might have helped if the OP had posted in a software related sub-section of the forum rather than the "Hardware and Peripherals / Camera board" sub-section which suggested they were talking about the Pi Camera Module hardware. But at least it's all clear now.

User avatar
thagrol
Posts: 3719
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Why no audio on the Pi camera module?

Mon Jun 22, 2020 9:11 am

hippy wrote:
Mon Jun 22, 2020 8:04 am
thagrol wrote:
Sun Jun 21, 2020 9:12 pm
Thanks for clearing that up. If only you'd been clearer in your first post...
And it might have helped if the OP had posted in a software related sub-section of the forum rather than the "Hardware and Peripherals / Camera board" sub-section which suggested they were talking about the Pi Camera Module hardware. But at least it's all clear now.
Indeed.
Arguing with strangers on the internet since 1993.

Chris D
Posts: 218
Joined: Tue Sep 20, 2016 11:27 am

Re: Why no audio on the Pi camera module?

Mon Jun 22, 2020 12:31 pm

Wow, I give up and am walking away from this subject.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27398
Joined: Sat Jul 30, 2011 7:41 pm

Re: Why no audio on the Pi camera module?

Mon Jun 22, 2020 12:59 pm

Chris D wrote:
Mon Jun 22, 2020 12:31 pm
Wow, I give up and am walking away from this subject.
Wow. You asked a question, then changed the parameters, then walk away. Despite that, your question HAS been answered, and you have chosen to ignore both the answer posts I made on the subject.

Even the title of this thread is "Why no audio on the Pi camera module?". That is a very specific question about the camera module HARDWARE. The content of the post does a little way to clarify but is still horrendously ambiguous - you mentioned the picamera MODULE without saying you actually mean the Picamera Python library, and not the HW itself. So many people misunderstood your post, and it's not surprising.

So, to answer again, the Python library known as Picamera was developed and is maintained by a third party, NOT Raspberry Pi. You really need to walk away from here and approach the maintainer.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

Return to “Camera board”