January 2010
Over the holiday break, I finally created an Android app that allows image processing on the camera’s raw data, and displays it back on the screen. It only uses Java on the CPU for now, but in my free time I’ll be porting the code to OpenGL ES to run on the GPU, and also exploring the OpenCV port for Android. The hard part so far was getting access to the raw pixel data and being able to display it back to the screen after modifying it. Now that this is done, any image processing effects I can think of will be easy to add to this app.
Download:
I provide the Source code as an example of Computer Vision on Android in Java.
Update 1 (Jan 7, 2011):
Why didn’t I know about MOTODEV Studio earlier!?! Through this Eclipse plug-in, getting JNI working was much less painful; I’m now processing vision code on the phone in C instead of Java now!
Update 2 (Jan 9, 2011):
Sobel filter on the phone in Java: 1-2 FPS… Sobel filter in C/jni: 15-20+ FPS!
Update 3 (Jan 11, 2011):
Implemented in native C:
-YUV420sp -> ARGB
-ARGB -> Grayscale
-Histogram generation/overlay
-Sobel Filtering
-Histogram normalization of the image
Update 4 (April 2011):
New: Android computer vision project with OpenCV – ViewerCV
thanks so much for posting about the motodev studio for android- i’ve been struggling with opencv jni stuff!
@ shani: You are welcome – good luck on your project!
Hello!
Thank you for your article.
I’m only getting 2-3 FPS with the sobel filter, what should I do to get the 15 fps you mentioned?
Thanks
Hi,
Thanks for your interest! The performance is heavily based on the input image resolution and camera capture settings. By default, when the app starts, it gets a list of supported camera capture resolutions, and chooses the second option in the list (on my Droid X, this is 320×240). The problem is probably due to your phone being initialized at a higher resolution, thus requiring more processing.
Check out the file ‘CameraWrapper.java’, and at about line 98 or so, there will be a function call:
params.setPreviewSize(previewSize.width, previewSize.height)
Simply set the preview width and height to be 320×240 manually, and see if that helps!
~Chris
That was indeed the problem! It was setting the resolution at 800 instead of 320, now it’s fast.
I’m learning about computer vision, so thank you for your help.
Hmm, the application crashes after a while, and I’ve traced it back to the JNI.
The crash is due to the JNI not releasing the arrays because of the flag JNI_COMMIT (updates Java heap, does not release in C) so I replaced JNI_COMMIT for 0 in all places, and doesn’t crash anymore.
Excellent!
Thanks for tracking that down (I haven’t had much time to work on Viewer lately).
The changes have been fixed in Google Code.