Local Contrast Enhancement with ArrayFire + OpenCV

25 thoughts on “Local Contrast Enhancement with ArrayFire + OpenCV”

1. Very informative post. You posted code for converted from OpenCV Mat to ArrayFire array. How can you convert the data from ArrayFire array back to OpenCV Mat?

2. Alex:
   Thanks for reminding me to update this post! I have since made new array< ->mat functions in a stanalone file that I have now updated/linked to here. Simply get the new af_helpers.cpp/.h and enjoy the new functionality in your app!
   ~Chris

3. Chris,
   thanks for the update! Much appreciated! I’m going to try it out soon.

4. Thanks! Array to Mat conversion works. The only thing is that array input in array_to_mat must be 1 channel. How can I make it work with multiple channels?

5. Alex:
   Excellent! Yeah, actually I was working on 3 channel support earlier, but didn’t have an elegant solution; until now… try updating again!
   Cheers,
   ~Chris

6. Chris, thanks your fast reply! array_to_mat gives incorrect output for multiple channels. Here’s my test code.

   int data[] = {1,2,3,4,5,6};
   // create Mat array of integers with 3 columns, 1 row and 2 channels
   Mat input(3,1,CV_32SC2, data);
   std::cout << input << std::endl; // input = [1, 2; 3, 4; 5, 6];
   Mat output;
   af::array afData;
   mat_to_array(input, afData);
   array_to_mat(afData, output);
   std::cout << output << std::endl; // output = [2, 4; 6, 1; 3, 5]

7. Ales:
   Ah, interesting. Can you try changing the “gfor” loop into a standard “for” loop in array_to_mat..? I think that may be the issue. If not, can you print me the output of af::info() ?
   ~Chris

8. Chris, got the same output using “for” loop. Are you getting the correct result with the above test?

   Here’s output from af::info()
   ArrayFire v1.9 (build c3cd513) on 64bit Windows.
   License: Standalone
   Addons: MGL4, DLA, SLA
   CUDA toolkit 5.0, driver 310.90
   GPU0 Tesla C2070, 5376 MG, Compute 2.0 (single, double) (current)
   Memory usage: 4887 MB free (5376 MB total)

9. Alex:
   Sorry about that, I should have tested it a bit more. I think I have fixed the error now; was due to OpenCV being ‘bgrbgrbgr’ vs ArrayFire ‘rrrgggbbb’ layout. Check the update again, please, and let me know if that helps!
   P.S. if this still doesn’t help try gfor/for replacement again (I’m working off a dev build of ArrayFire).
   ~Chris

10. Thanks Chris! I thought it had something to do with how Mat stores data. I’ll test the function on Monday.

11. Hello, Great post!
    I did encounter a few errors trying to run your code. I just installed ArrayFire v1.9, ran added your files and ran “make” and got these errors:
    make
    c++ `pkg-config –cflags opencv` -m64 -Wall -Werror -I/Users/salgarcia/Desktop/GPU/arrayfire/include -I/usr/local/cuda/include -O2 -DNDEBUG `pkg-config –libs opencv` -L/Users/salgarcia/Desktop/GPU/arrayfire/lib -laf -L/usr/local/cuda/lib -lcuda -lcudart -lcurand -lcusparse -lpthread -lstdc++ -lm -Wl,-rpath,/Users/salgarcia/Desktop/GPU/arrayfire/lib,-rpath,/Users/salgarcia/Desktop/GPU/arrayfire/lib,-rpath,/usr/local/cuda/lib -lafGFX -lopencv_core -lopencv_highgui -lopencv_imgproc webcam_demo.cpp -o webcam_demo
    webcam_demo.cpp:80:17: error: use of undeclared identifier ‘rgbtohsv’
    array hsv = rgbtohsv(img);
    ^
    webcam_demo.cpp:96:21: error: use of undeclared identifier ‘hsvtorgb’
    array lce_img = hsvtorgb(lce_v);
    ^
    webcam_demo.cpp:99:5: error: use of undeclared identifier ‘subfigure’
    subfigure(2, 2, 1); rgbplot(img); title(“Source”);
    ^
    webcam_demo.cpp:99:26: error: no matching function for call to ‘rgbplot’
    subfigure(2, 2, 1); rgbplot(img); title(“Source”);
    ^~~~~~~
    /Users/salgarcia/Desktop/GPU/arrayfire/include/af/gfx.h:442:15: note: candidate function not viable: requires 3
    arguments, but 1 was provided
    AFXAPI handle rgbplot(const float *d_X, const unsigned w, const unsigned h);
    ^
    webcam_demo.cpp:99:50: error: use of undeclared identifier ‘title’
    subfigure(2, 2, 1); rgbplot(img); title(“Source”);
    ^
    webcam_demo.cpp:100:5: error: use of undeclared identifier ‘subfigure’
    subfigure(2, 2, 3); rgbplot(lce_img); title(“LCE”);
    ^
    webcam_demo.cpp:100:26: error: no matching function for call to ‘rgbplot’
    subfigure(2, 2, 3); rgbplot(lce_img); title(“LCE”);
    ^~~~~~~
    /Users/salgarcia/Desktop/GPU/arrayfire/include/af/gfx.h:442:15: note: candidate function not viable: requires 3
    arguments, but 1 was provided
    AFXAPI handle rgbplot(const float *d_X, const unsigned w, const unsigned h);
    ^
    webcam_demo.cpp:100:50: error: use of undeclared identifier ‘title’
    subfigure(2, 2, 3); rgbplot(lce_img); title(“LCE”);
    ^
    webcam_demo.cpp:107:5: error: use of undeclared identifier ‘subfigure’
    subfigure(2, 2, 2); plot(seq(255), vhist/max(vhist)); title(“v hist”);
    ^
    webcam_demo.cpp:107:26: error: use of undeclared identifier ‘plot’; did you mean ‘plot2’?
    subfigure(2, 2, 2); plot(seq(255), vhist/max(vhist)); title(“v hist”);
    ^~~~
    plot2
    /Users/salgarcia/Desktop/GPU/arrayfire/include/af/gfx.h:239:15: note: ‘plot2’ declared here
    AFXAPI handle plot2(const array &X, const array &Y);
    ^
    webcam_demo.cpp:107:68: error: use of undeclared identifier ‘title’
    subfigure(2, 2, 2); plot(seq(255), vhist/max(vhist)); title(“v hist”);
    ^
    webcam_demo.cpp:108:5: error: use of undeclared identifier ‘subfigure’
    subfigure(2, 2, 4); plot(seq(255), shist/max(shist)); title(“v sharp hist”);
    ^
    webcam_demo.cpp:108:26: error: use of undeclared identifier ‘plot’; did you mean ‘plot2’?
    subfigure(2, 2, 4); plot(seq(255), shist/max(shist)); title(“v sharp hist”);
    ^~~~
    plot2
    /Users/salgarcia/Desktop/GPU/arrayfire/include/af/gfx.h:239:15: note: ‘plot2’ declared here
    AFXAPI handle plot2(const array &X, const array &Y);
    ^
    webcam_demo.cpp:108:68: error: use of undeclared identifier ‘title’
    subfigure(2, 2, 4); plot(seq(255), shist/max(shist)); title(“v sharp hist”);
    ^
    webcam_demo.cpp:111:5: error: use of undeclared identifier ‘draw’
    draw();
    ^
    webcam_demo.cpp:142:5: error: use of undeclared identifier ‘figure’
    figure();
    ^
    webcam_demo.cpp:148:30: error: no member named ‘tic’ in ‘af::timer’
    timer start = timer::tic();
    ~~~~~~~^
    webcam_demo.cpp:162:53: error: no member named ‘toc’ in ‘af::timer’
    printf(” demo fps:\t %f \n”, 1.f / (timer::toc(start)));
    ~~~~~~~^
    18 errors generated.
    make: *** [webcam_demo] Error 1

    Please let me know what I can do to resolve this. Thank you very much,
    Sal Garcia

12. Sal:
    Sorry about the confusion, ArrayFire has changed some of the syntax for functions from version 1.2 -> version 1.9, and I havent updated this code to 1.9 yet. I will try and update the code soon, but meanwhile, here are some of the simple changes to make to get you going..

    rgbtohsv(img) -> colorspace(img,”hsv”,”rgb”)
    plot -> plot2
    and

    subfigure(2, 2, 3); rgbplot(lce_img); title(“LCE”);
    becomes
    fig(“sub”, 2, 2, 3); image(lce_img); fig(“title”, “LCE”);

    Thanks for the interest and reminder to update the code, good luck!
    ~Chris

13. Hello Chris! I’ve been profiling your mat_to_array function. Essentially, mat_to_array is the bottleneck of my video processing application. Here’s a code I use to profile the function:

    timer start = timer::start();
    image = imread(“someimage.jpg”); // 480×640 RGB image
    array image_af;
    for (int j=0; j<1000; ++j) {
    mat_to_array(image, image_af);
    }
    printf("%g – mat_to_array\n", timer::stop(start));

    So, I call this function a 1000 times and it takes 7.3 seconds on my system. This is slower than my image processing loop (which takes about ~4.3 seconds for 1000 iterations). Can you think of a way to make this run faster? Thanks!

14. Did some more profiling. The most expensive operations in mat_to_array for a 3-channel image seem to be:

    0.00046 sec – clone input Mat
    0.00142 sec – convert Mat to float
    0.00160 sec – split Mat into three channels
    0.00340 sec – assign values to output array

    I think the first three steps can be optimized somehow.

15. Alex:
    Nice work profiling! Indeed, constantly transferring cpu/gpu memory is always a hassle.

    There are a few things that can be done here:

    – you can skip the clone if you are OK with after calling mat_to_array, the input mat is changed.
    – you can skip the floating point conversion if your mat is already 32 bit, just change the pointer to int* , uint*, float* or double* and arrayfire type (http://www.accelereyes.com/arrayfire/c/namespaceaf.htm#acfe99d230e216901bd782cc580e4e815a82ea90203678bdd0b547068f0a76524b) accordingly.
    – if you don’t convert the mat to 3 channels, and send the raw pointer directly into arrayfire, then you will need to handle the differences in mem layout by using some inverse of the 3-channel array_to_mat, such that arrayfire is column major (x,y) and each channel is separate in Z (3D volume), and opencv is row major and channels are packed xyzxyzxyz (2D).
    – there is really no way around the output assignment , unless you setup pinned memory (but that becomes tricky)

    Good luck!
    ~Chris

16. Thanks a lot Chris!

    – cloning. I need to keep my original Mat intact. So cloning stays.
    – my Mat elements are 1 byte uchar, so I guess I still have to convert Mat elements to 4 byte float.
    – I think I switch splitting data into channels and output assignment.
    Maybe I can first assign Mat data to array without worrying about memory layout or splitting Mat into channels. Then as a next step I can rearrange the elements of the output array as needed (to avoid costly splitting of the Mat object). What do you think?

17. Chris, that’s a very interesting point about pinned memory. Any pointers on how I can set it up? I may consider using it.

18. Chris:

    Just thought that the cloning of Mat may be skipped when we convert Mat elements to float (or double).

    input.convertTo(input2, CV_32F); // a modified copy is made here

19. Chris, I optimized mat_to_array(). The function now takes only 0.0023 seconds to run (it used to take 0.0070 seconds) for 480×640 RGB frame input. I followed your suggestion about pinned memory among other things. Thanks for your help!

20. Alex:
    Awesome!
    If you want to share what you are working on, I’m curious to hear about your project, or link me to your projects’s website when you are done if you have one.
    Thanks!
    ~Chris

21. Hi Chris!
    Here’s the link to my code for converting from Mat to array.
    http://code.google.com/p/opecv-to-arrayfire/source/browse/

22. Alex:
    Very cool, nice work!

23. Pingback: Image Processing with ArrayFire and OpenCV on the GPU | ArrayFire
24. Pingback: Image processing with LibJacket + OpenCV | mcclanahoochie's blog
25. Pingback: OpenCV vs. LibJacket: GPU Sobel Filtering | mcclanahoochie's blog