Computer Vision on Android with OpenCV

by

March 2011

With the help of Motodev Studio for Android, I’ve extracted the android-opencv JNI camera example and spawned a fork of my original computer vision app, Viewer, to an OpenCV version: ViewerCV. Both are available on Git Hub as open source software example of doing Computer Vision on Android with OpenCV.

ViewerViewerCV

Features:

  • *FAST Features (default android-opencv example)
  • *Square detection
  • *Sobel Filtering
  • *Histogram Equalization
  • *Gradient edge morphology (Neon mode)
  • *Local Contrast Enhancement (Vivid mode)
  • *Live HDR (high dynamic range + tonemapping) mode…

 

Quick GUI Readme:

  • *’Menu’ = changes processing effect
  • *’Mode’ = cycles options within an effect
  • *’Menu>Settings’ = adjusts resolution
  • *’Pic’ = saves a screenshot to the sdcard


Now on the Android Market!!! (please rate/comment!)

Also available as FOSS on GitHub (please play fair).

 

Screens:


38 thoughts on “Computer Vision on Android with OpenCV”

  7. hiii…thanks for information, but it’s not work on my galaxy Y.. i download from market, but this application not running, any help for me? please…

  9. It doesn’t work on my Wildfire S (Marvel) 2.3.5
    This is what I get with logcat. It seems that cannot load libandroid and provokes a SIGILL.

    Any clue?

    D/dalvikvm( 6947): Trying to load lib /mnt/asec/com.viewercv-1/lib/libandroid-opencv.so 0x4051e6a8
    I/DEBUG ( 6937): pid: 6947, tid: 6947 >>> com.viewercv <<<
    I/DEBUG ( 6937): signal 4 (SIGILL), code 1 (ILL_ILLOPC), fault addr 805763a2
    I/DEBUG ( 6937): r0 00000000 r1 8060cc18 r2 804a91a1 r3 804b1841
    I/DEBUG ( 6937): r4 000efe60 r5 8066fa90 r6 00000004 r7 00000000

  10. Hi Isma:

    Is this error coming from trying to run the version off the Android Market, or are you compiling the project yourself?

    If this is from the Market, try installing these OpenCV libs: code.google.com/p/mcclanahoochie/downloads/detail?name=OpenCV.apk, then re-installing the ViewerCV app.

    If you are compiling from source, make sure the OpenCV_src folder is properly sym-linked (see the README) and if you are using Eclipse, that the project properties are correctly linking to the android-opencv project.

    Good luck and thanks for your interest in ViewerCV!

  11. I am using the version from the market and I installed OpenCV.apk but the error still remains, 🙁

    If I get some time I will compile it and I will let you know.

    Thank you in advance.

  12. Isma:
    Hmm, my only other guess is that the HTC Wildfire’s camera doesn’t support the resolution ViewerCV requests at startup (160×120). The resolution is configurable inside ViewerCV, but I guess that doesn’t help here. I should add a “smarter” camera startup on the next release of the app maybe.
    Thanks again for trying.

  16. Very Cool…I was looking for some sample code to try out in my Computer Vision class…this would be very useful to play with live data.

  17. Hi – I just wanted to both congratulate you and ask for your advice… I’m thinking of writing an openCV app. I’ve noticed that many of the OpenCV apps on Android are very unstable, but yours seems to work great on most platforms. Do you have any advice on why yours has been so stable compared to other apps? Is it your language choices?

  18. Jeff:

    Thanks for your feedback! I’m not sure about the reasons for stability differences; I would like to think it is just my programming 😉 …

    I can list a few things that differentiate this app than others:
    – I am not using the “official” OpenCV for Android libraries (the ones in the opencv source now), but instead I’m using the “original” Android port (which is now depricated).
    – I try to keep all OpenCV calls standard and functions simple.
    – Via the help of other people, I have gotten viewercv tested on many different platforms.
    – Most of the actual opencv code was developed on a desktop first and then put on the phone (very cool to have almost no differences in api).
    – I keep the android version requirements to a minimum (android 2.3 I think).

    Good luck on your android opencv app!
    Feel free to use my project as a starting point (many people have), I just ask you give a little credit to viewercv somewhere if you do!

    Thanks!

  19. Hi
    I have an HTC hero (started as android 2.1) flashed to android 2.3.7. Googe Play claims its incompatible with my device. What’s confusing is that it also claims it requires android 2.1 and up… so where’s the incompatibility?

    Disclaimer: I am a Newbie!

  20. hello mcclanahoochie,

    I’m trying to compile and run your code using OpenCV 2.4.2 and is not compiling in eclipse! Which version did you used? Have you updated to the latest OpenCV version? Thanks for your help!

  21. Greg:

    I’ve included the ‘android-opencv’ ViewerCV uses in the git repo. I have not gotten around to updating ViewerCV to use the latest OpenCV; If you port the project to use the newer OpenCV, please share; otherwise, the easiest option is the android-opencv included in the project checkout. Thanks for the interest, and good luck!

    ~Chris

  22. Hi mcclanahoochie,

    I am software engineer from AMD.

    I’m curious about how the application implement the HDR.

    About the HDR, the usually used method is that we use multiple images with different exposure and compose them to one image, then, using tone mapping to generate a HDR image. Am i right?

    Do you use the OpenCV to implement it?

  23. gavinwang:

    The concepts of ‘exposure fusion’ and ‘tonemapping’ almost always get lumped into the term ‘HDR’. You are correct in that ViewerCV does fuse multiple expoesures into an HDR image, and then uses tonemapping on the HDR image for the final result.

    Yes, the old opencv-android is used in ViewerCV for all the image processing.

    The exposure fusion in ViewerCV is done by taking the sum of log images (hdr_img = log(img1)+log(img2)+…), and there are a few different options for tonemapping included.

    The source repo is linked to above, and you can see the use of OpenCV in http://code.google.com/p/viewercv/source/browse/ViewerCV/jni/Processor.cpp .

    Thanks for your interest!

    ~Chris

  24. Ok, thanks very much for your responding.

    Now, i am trying to implement the HDR functionality based on Camera application which released by the google.

    But i feel a little difficult to implement this because the original camera App is written by Java and the structure is totally different. I need do some research to dig out how i can integrate the HDR functionality into official camera application.

    Do you have any ideas or suggestion for this? I want to find a effective way to handle this.

  25. Hi mcclanahoochie,

    Merry X’mas, i’m so sorry to disturb your during your vacation.

    Would you please help to answer some question for me?

    So far, i have done the first step ‘exposure fusion’, but i have some questions for that.

    I used the same code logic as makehdr3log(&(imgbuff[0]), &(imgbuff[1]), &(imgbuff[2]), &(hdr)) , for one pixel, i used the code as below,

    int hdrPixel = (Math.log(img1Pixel) + Math.log(img2Pixel) + Math.log(img3Pixel)) / 3;

    I use the above code for all pixels in the image, actually, for each R,G,B channel of each pixel.

    In the end, i get the HdrImage, but the each pixel data of HdrImage are so weird. For example,

    The blue channel of img1 pixel is 0x81, the blue channel of img2 pixel is 0x45, the blue channel of img3 pixel is 0xdf, so i get the blue channel of hdr image pixel is as below,

    hdrBlueChannel = (Math.log(0x81) + Math.log(0x45) + Math.log(0xdf))/3 = 4

    so it means the data of blue channel in HDR image is just 0x4, then i release the img1, img2 and img3. After this, I use the HDR image to do the tonemapping.

    Am i right? Did i have some mistake or misunderstanding for this?

    Thanks in advance.

    GavinWang

  26. Hi mcclanahoochie,

    And i find that there three tonemapping mode for HDR image’s generation.

    In the photo taken cases, which tonemapping mode do you prefer? Which mode will get the better quality?

    thanks,

    GavinWang

  27. GavinWang:

    It sounds like you are well on your way. The raw HDR image created by adding the log images is not gauranteed to look correct on its own. This is where the tonemapping comes in.

    The raw HDR image (done in the fashion in ViewerCV) contains values anywhere from 0 to ~2.4, where a ‘normal’ image will range from 0-1 (normalized 0-255). Remeber this is in log space, so be sure to convert back to base 10 via exponent (such as cv::exp(hdr, hdr)).

    At minimum, the HDR image should be normalized from its min\max back to 0-1. Usually the image is transferred to a different colorspace for processing (ex. XYZ or YUV). Tonemapping is a way to compress the 0-? range down to 0-1, but in a way that tries to preserve shadows and highlights. There are many many ways of tonemapping.

    Of the methods in ViewerCV, the simplest is one I created: luminance scaling + lce (if _mode == 1). A more “traditional” method is the Drago method, then Mantiuk method (advanced tonemapping). The “best” algorithm is highly subjective, but of the 3 on the phone, drago or luminance scaling seem to work the best; mantiuk isn’t guaranteed to converge to a solution, but when it does it looks nice (nicer on a desktop though). I would go with the luminance scaling or drago to get started.

    Good luck!
    ~Chris

  28. Hi man …..

    Thanks for the source code, They are really useful for me 🙂
    Can you help me with more simple project source code to learn more?

    Thank you

  30. Well, I am a beginner (in android , not openCV) and have project to use the OpenCV library in android application.

    I need to control the “resolution” and the “focal lengths” of camera in my OpenCV application.

    Would it be possible for you to give me some samples to understand this part better?

    Best 🙂

  31. MC:

    In Android, in my experience, you get very little control over the camera. You can search for “setExposureValue” in the ViewerCV source here https://code.google.com/p/viewercv/source/browse/ViewerCV/src/com/viewercv/MainActivity.java for an example on how that may work, though, the newer OpenCV for Android is a bit different.

    On the desktop, OpenCV gives you a bit more control, but I don’t think you can change focal lengths, it really depends on your camera/driver. You can see my rttmo project and browse the source and see how to control exposure of usb and firewire cameras via OpenCV.

    Other than how to do exposure control (linked to above), you will have to look at OpenCV’s documentation for other camera parameters.

    Good luck!

  33. Thanks for this code , so Im a student and my PFE IT ABOUT THE logo recognition in android . any device? thanks

  36. Pingback: ViewerCV | Free Download APK for Android

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.