Hopding
Posts: 5
Joined: Wed Dec 30, 2015 12:16 pm

Java API for Accessing the RPi Cam

Wed Dec 30, 2015 12:41 pm

Hey all! I just recently picked up a RPi and a RPi Camera to tinker around with. I'm not new to programming or electronics, but haven't used a RPi before. I program in Java, primarily, and quickly noticed that there was no Java API for accessing the RPi Camera. So, I decided to make one myself.
JRPiCam is a very simple Java API that uses the ProcessBuilder class to invoke the raspistill software on your Pi. As long as raspistill is configured to work, so will JRPiCam. JRPiCam provides all the relevant functionality of raspistill to a Java app running on the RPi. I whipped up some examples and a javadoc and posted the project along with a .jar library to GitHub, https://github.com/Hopding/JRPiCam.
I should mention that because JRPiCam works by calling raspistill, it is not possible to read image data directly from the camera to a Java Buffer of any sort. I did however make it very easy to read the photos from the Pi into an internal buffer after they have been saved to the memory.
I hope you guys find it to be helpful. It's simple, but is quicker and easier to use rather than making your own ProcessBuilder based class. Thanks for reading!

EDIT
It is now possible to read an image directly into a java application, without saving it to memory at all (although the buffered image can be saved after it is in the Java buffer if desired). This allows an image to be accessed far faster than if it is saved to memory and then loaded into the Java app. Images can still be saved and read from memory just like before, if desired. Currently, buffered images are stored in a java.awt.image.BufferedImage object for easy manipulation (e.g. using in JavaCV, or converting with SwingFXUtils for use in a JavaFX application). I am working on a way to get an integer array of the image's RGB values as well, and should be ready to commit it soon.


- Hopding
Last edited by Hopding on Mon Jan 04, 2016 8:32 pm, edited 2 times in total.

clicky
Posts: 226
Joined: Thu Oct 25, 2012 7:34 am

Re: Java API for Accessing the RPi Cam

Thu Dec 31, 2015 5:23 pm

Really nice. :)

I can already see what are the next steps:

1. Allow spaces in the file name ;)
2. Stream jpg (and png, and other formats raspistill supports) directly from that process to a buffer in JVM mem.
3. Add raspivid in similar manner...

Hopding
Posts: 5
Joined: Wed Dec 30, 2015 12:16 pm

Re: Java API for Accessing the RPi Cam

Fri Jan 01, 2016 8:25 pm

clicky wrote:Really nice. :)

I can already see what are the next steps:

1. Allow spaces in the file name ;)
2. Stream jpg (and png, and other formats raspistill supports) directly from that process to a buffer in JVM mem.
3. Add raspivid in similar manner...
Thanks, clicky :)

I'm working on adding functionality to load the image directly into a buffer inside the Java app that's using the API. From there the image can be modified and saved in a number of different ways. It should also be loaded into the application faster than having it written to memory, and then read back, especially if the user doesn't wish to save it at all.

I wasn't initially aware that the '-o' option in raspistill spits out the image data to stdout if a save file isn't specified. Now that I discovered that, I should be able to simply read that into a buffer as it is printed out.

I plan to add raspivid features once I finish adding some things to the raspistill API, but I haven't played around to much with that. I'll also be sure to fix the spaces in the file name, it could be a major inconvenience in some situations ;)

clicky
Posts: 226
Joined: Thu Oct 25, 2012 7:34 am

Re: Java API for Accessing the RPi Cam

Sat Jan 02, 2016 12:14 pm

Oh, but that is just start. Sorry - I had so many ideas around Java + imaging on Raspberry Pi and it seems that you slowly started implementing it. When I get a moment I can write up here all the ideas and you can see how far you can/want to pushed them to.

daft
Posts: 4
Joined: Mon Jan 04, 2016 4:04 pm

Re: Java API for Accessing the RPi Cam

Mon Jan 04, 2016 4:07 pm

Hopding wrote:Hey all! I just recently picked up a RPi and a RPi Camera to tinker around with. I'm not new to programming or electronics, but haven't used a RPi before. I program in Java, primarily, and quickly noticed that there was no Java API for accessing the RPi Camera. So, I decided to make one myself.
Nice effort! I'm currently using Java and the RPiCam via OpenCV, would be curious to see what frame rates you are able to achieve via this system. Any benchmarks?

Hopding
Posts: 5
Joined: Wed Dec 30, 2015 12:16 pm

Re: Java API for Accessing the RPi Cam

Mon Jan 04, 2016 8:29 pm

daft wrote:
Hopding wrote:Hey all! I just recently picked up a RPi and a RPi Camera to tinker around with. I'm not new to programming or electronics, but haven't used a RPi before. I program in Java, primarily, and quickly noticed that there was no Java API for accessing the RPi Camera. So, I decided to make one myself.
Nice effort! I'm currently using Java and the RPiCam via OpenCV, would be curious to see what frame rates you are able to achieve via this system. Any benchmarks?
I don't haven't done any benchmark tests yet, although I probably will soon. I was initially only able to get an image into Java by saving it to the Pi's memory and then loading it from the file system. Obviously this wasn't ideal, and caused an undesirable lag between the time the image was taken and the time the Java app had it loaded and ready to work with. But I just completed a commit to the API that reads images directly from raspistill into the calling Java app, without touching the file system at all. Things work muchhh faster this way, since the image is loaded (more or less) as quickly as raspistill captures and encodes it.
I'm now working on getting my API to just load the raw RGB data from raspiyuv, skipping the encoding. This should be slightly faster than getting an encoded image from raspistill, and particularly useful for sending the image over a network to another (probably faster) computer for processing or storage. Or perhaps to use the unprocessed image in another library, such as JavaCV - although that should already be possible, since buffered images are stored in a BufferedImage object, and JavaCV can form images from a BufferedImage.
I haven't done much work with OpenCV before, so I don't know to much about it. Are you use JavaCV to get images into Java, or is there another way that I'm not aware of?

daft
Posts: 4
Joined: Mon Jan 04, 2016 4:04 pm

Re: Java API for Accessing the RPi Cam

Mon Jan 04, 2016 11:53 pm

Hopding wrote:I'm now working on getting my API to just load the raw RGB data from raspiyuv, skipping the encoding. This should be slightly faster than getting an encoded image from raspistill, and particularly useful for sending the image over a network to another (probably faster) computer for processing or storage.
This is where the benchmark will come in handy ;)
Hopding wrote:Or perhaps to use the unprocessed image in another library, such as JavaCV - although that should already be possible, since buffered images are stored in a BufferedImage object, and JavaCV can form images from a BufferedImage.
I haven't done much work with OpenCV before, so I don't know to much about it. Are you use JavaCV to get images into Java, or is there another way that I'm not aware of?
Nope, although I used to use JavaCV before, to my eyes it virtually died as soon as OpenCV started releasing proper (and therefore always up-to-date) Java bindings. Everything gets nicely packed into a .jar and a single all-inclusive {.dll|.so|.dylib}, no more need for external native/JNA bridges and libraries, a pure bless! Once in Java the conversion from OpenCV Mat to anything else is then pretty straightforward. Speed currently seems limited by the OS (I'm capping at 15fps on a 1280x720 Macbook Air integrated webcam).

Hopding
Posts: 5
Joined: Wed Dec 30, 2015 12:16 pm

Re: Java API for Accessing the RPi Cam

Tue Jan 05, 2016 9:36 pm

daft wrote:Nope, although I used to use JavaCV before, to my eyes it virtually died as soon as OpenCV started releasing proper (and therefore always up-to-date) Java bindings. Everything gets nicely packed into a .jar and a single all-inclusive {.dll|.so|.dylib}, no more need for external native/JNA bridges and libraries, a pure bless! Once in Java the conversion from OpenCV Mat to anything else is then pretty straightforward. Speed currently seems limited by the OS (I'm capping at 15fps on a 1280x720 Macbook Air integrated webcam).
Interesting. I'll have to look into those bindings, I wasn't aware of them before.
Perhaps I misunderstood, but I thought you said you were using OpenCV to work with the Raspberry Pi Camera. You mentioned 15 fps, but on a Mac's webcam, not the Pi. Are you talking about a VNC connection?

daft
Posts: 4
Joined: Mon Jan 04, 2016 4:04 pm

Re: Java API for Accessing the RPi Cam

Wed Jan 06, 2016 11:39 am

Hopding wrote:Perhaps I misunderstood, but I thought you said you were using OpenCV to work with the Raspberry Pi Camera. You mentioned 15 fps, but on a Mac's webcam, not the Pi. Are you talking about a VNC connection?
Sorry that was confusing indeed :)
I am playing around with OpenCV on both a Macbook Air (for development) and a Pi2+cam (for tests). The frame rate on the MBA navigates around 15-19 fps, while the frame rate on the Pi is rather around 6 fps for the same frame size. That is quite significantly slower than what the Pi cam is able to provide (up to 90 fps!), but I believe this is clearly due to the many interfaces along the way: camera => UV4L driver => OpenCV => Java, and I'm currently blaming UV4L for the slowness (as corroborated on this topic). I am therefore eager to know what frame rates you can achieve via raspivid to compare both approaches (I yet need to try the V4L2 driver as well).

RpiName
Posts: 709
Joined: Sat Jul 06, 2013 3:14 am

Re: Java API for Accessing the RPi Cam

Wed Jan 06, 2016 6:13 pm

daft wrote:I believe this is clearly due to the many interfaces along the way: camera => UV4L driver => OpenCV => Java, and I'm currently blaming UV4L for the slowness (as corroborated on this topic). I am therefore eager to know what frame rates you can achieve via raspivid to compare both approaches (I yet need to try the V4L2 driver as well).
UV4L is as efficient as other ways. Plus it's realtime. Your OpenCV layer is probably the bottleneck and cannot cope with higher frame rates from the driver.

daft
Posts: 4
Joined: Mon Jan 04, 2016 4:04 pm

Re: Java API for Accessing the RPi Cam

Fri Jan 08, 2016 12:16 am

RpiName wrote:UV4L is as efficient as other ways. Plus it's realtime. Your OpenCV layer is probably the bottleneck and cannot cope with higher frame rates from the driver.
Don't know what you mean by realtime (or rather, as opposed to what?), since I'm doing live video feed within my application...
Anyhow, after more digging I do have to apologise, I had some extra conversion code in my own Java layer within the acquisition loop that was imperceptible on the MBA's cpu, but suddenly not so fast on the poor little Pi :)
After removing this boilerplate code, I did reach the expected 30fps without trouble.

FYI I did also test the so-called "official" V4L driver via

Code: Select all

sudo apt-get install v4l-utils && sudo modprobe bcm2835-v4l2
and reached the same frame rate. By progressively re-inserting my own code into the loop, the fps (obviously) dropped down in a similar way, but I did notice that V4L was about 30% faster when actually asked for a camera grab+read (that's my own very limited _probably ugly_ tests, take with a grain of salt).

Side-note: OpenCV has a standard API to talk to the capture device and ask for things like picture size, and V4L was very happy with me playing around with this, although the same code crashed when using UV4L. Is there a way to change picture size on UV4L without having to stop/restart the service with different parameters? Perhaps an option in the config I have missed ?

Cheers

RpiName
Posts: 709
Joined: Sat Jul 06, 2013 3:14 am

Re: Java API for Accessing the RPi Cam

Fri Jan 08, 2016 12:50 am

daft wrote:Is there a way to change picture size on UV4L without having to stop/restart the service with different parameters? Perhaps an option in the config I have missed ?
UV4L implements the V4L2 API to set the resolution. I do not if /how your library and/or implementation is doing this though. Another way to change the resolution on the fly is through the Streaming Server from the Control Panel Web Page.

bhamail
Posts: 16
Joined: Fri Jan 24, 2014 7:04 am

Re: Java API for Accessing the RPi Cam

Tue Jan 12, 2016 2:49 am

@Hopding Neat project. Is the source code for JRPiCam.jar available?

Hopding
Posts: 5
Joined: Wed Dec 30, 2015 12:16 pm

Re: Java API for Accessing the RPi Cam

Wed Jan 13, 2016 8:53 pm

bhamail wrote:@Hopding Neat project.

Thanks!
bhamail wrote:Is the source code for JRPiCam.jar available?
It is, right here. Sorry about that. I appear to have gotten a bit carried away with my commits... :oops:, but it's there now.

RandomPotato26
Posts: 1
Joined: Sun Feb 12, 2017 12:43 am

Re: Java API for Accessing the RPi Cam

Sun Feb 12, 2017 1:19 am

I am trying to use this to process a video for a robot, but the delay for each image makes it impossible to show real time. Is there any functionality other than enable burst to speed it up? A target fps would be around 30. I am not to worried about having the sensor "settle". I ask this knowing that the python libraries allow fast processing and captures. Thanks if you get to this in time!

TheWlr9
Posts: 1
Joined: Tue Mar 06, 2018 4:53 am

Re: Java API for Accessing the RPi Cam

Tue Mar 06, 2018 5:02 am

Hello there! I'm currently trying to attatch a RPi 0 W with a RPi camera V2 to one of my model aircrafts to fly over my cabin in Ontario (which is in Canada if you didn't know), in order to stream a live video feed to my laptop screen via LAN.

I see that you were having some trouble with finding out how to receive the array or 2-D array of RGB values for each pixel in an image in Java, an I think I have the answer! I was doing a personal project and trying to implement a facial recognition software program, and I needed the same thing.
I came across in the Java API: Raster.
Using Rasters, you can convert any image (I forget if it's a BufferedImage, I think it is) into a 2-D array of RGB values, and a lot more!

So there's my little answer, but now I also ask a question of my own! Would it be better/faster to stream the RGB 2-D array of pixels via LAN to my laptop from the Pi Zero W, or would it be better/faster to send just the BufferedImage/ByteStream to my laptop?

I'm also a Java programmer at heart so both my server program and my client program are written in Java with Java 8 API.

Thanks and I hope this "Raster" class thing helps!

Return to “Java”