{"id":1896,"date":"2011-11-06T00:36:50","date_gmt":"2011-11-06T07:36:50","guid":{"rendered":"http:\/\/mcclanahoochie.com\/blog\/?post_type=portfolio&p=1896"},"modified":"2023-06-10T10:32:43","modified_gmt":"2023-06-10T17:32:43","slug":"gpu-tv-l1-optical-flow-with-libjacket","status":"publish","type":"post","link":"https:\/\/mcclanahoochie.com\/blog\/2011\/11\/gpu-tv-l1-optical-flow-with-libjacket\/","title":{"rendered":"GPU TV-L1 Optical Flow with ArrayFire"},"content":{"rendered":"

Update 1: LibJacket has been renamed to\u00a0\u00a0ArrayFire<\/a>.<\/em><\/h3>\n

Update 2<\/em>: Huang Chao-Hui was nice enough to port the LibJacket code mentioned here to ArrayFire – see his work here.<\/a><\/em><\/h3>\n

As one of my\u00a0Computer Vision<\/a>\u00a0class\u00a0projects, I decided to implement optical flow,<\/strong> because I wanted to learn more about optical flow, and also I wanted to code it on the GPU (because I love GPU programming<\/em>!).<\/p>\n

One of the more successful approaches to optical flow is\u00a0\u00a0total variation<\/a>\u00a0L1 regularization (TV-L1<\/strong>). \u00a0Pock et al<\/em>‘s An Improved Algorithm for TV-L1 Optical Flow<\/em><\/span>\u00a0[Paper]<\/a> describes a clever duality based method to solving L1 regularization\u00a0image optical flow. These\u00a0[slides]<\/a><\/em>\u00a0offer\u00a0a nice intro to variational methods and applications. Kudos to Pock et al<\/em> and their work!<\/p>\n

GPU4Vision hosts the reference MATLAB implementation: [download]<\/a> of the algorithm in the above paper (bundled in their Matlab Toolbox).\u00a0Having this as a reference, I chose to re-implement this code in LibJacket, because it makes porting MATLAB code into GPU C++ code incredibly easy<\/em>!<\/p>\n

\"\"<\/a>
TV-L1 Optical Flow running on the GPU via LibJacket<\/figcaption><\/figure>\n

While the ultimate goal is an OpenCL or CUDA custom\u00a0implementation\u00a0(coming here soon!<\/span>), I decided to start with AccelerEyes<\/a> LibJacket<\/strong>, because it super easy to translate from MATLAB M scripts to LibJacket C++ GPU code. I made use of what I learned earlier about combining LibJacket and OpenCV (see\u00a0Image Processing with LibJacket and OpenCV<\/a>) for this project.<\/p>\n

\"\"<\/a><\/p>\n

Performance<\/strong>: Using the same configuration settings for both my LibJacket version and the MATLAB version, there is over an order of magnitude improvement<\/span><\/em> in speed, making this fairly complex algorithm capapble of real-time performance.<\/p>\n

It’s still a work in progress<\/em> (a.k.a, probably still bugs), but I’m glad to share it as open source\u00a0software! [source code<\/a>] Enjoy!<\/p>\n

 <\/p>\n

\"\"<\/a>
Middlebury “rubber whale” image sequence<\/figcaption><\/figure>\n

\n","protected":false},"excerpt":{"rendered":"

Update 1: LibJacket has been renamed to\u00a0\u00a0ArrayFire. Update 2: Huang Chao-Hui was nice enough to port the LibJacket code mentioned here to ArrayFire – see his work here. As one of my\u00a0Computer Vision\u00a0class\u00a0projects, I decided to implement optical flow, because I wanted to learn more about optical flow, and also I wanted to code it … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[91,113,110,103,100,54,114,101,29],"class_list":["post-1896","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-arrayfire","tag-computer-vision","tag-cuda","tag-gpgpu","tag-gpu","tag-image-processing","tag-optical-flow","tag-programming","tag-projects"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pZdXI-uA","jetpack-related-posts":[{"id":1731,"url":"https:\/\/mcclanahoochie.com\/blog\/2011\/08\/image-processing-with-libjacket-opencv\/","url_meta":{"origin":1896,"position":0},"title":"Image processing with LibJacket + OpenCV","author":"mcclanahoochie","date":"August 24, 2011","format":false,"excerpt":"Update: one year later:\u00a0ArrayFire+OpenCV The OpenCV library is the de-facto standard for doing computer vision and image processing research projects. OpenCV includes several hundreds of computer vision algorithms, aimed for use in real-time vision applications. LibJacket is a matrix library built on CUDA. LibJacket offers hundreds of general matrix and\u2026","rel":"","context":"In \"arrayfire\"","block_context":{"text":"arrayfire","link":"https:\/\/mcclanahoochie.com\/blog\/tag\/arrayfire\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2011\/08\/Screen-shot-2011-08-24-at-2.42.52-PM-1024x640.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2011\/08\/Screen-shot-2011-08-24-at-2.42.52-PM-1024x640.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2011\/08\/Screen-shot-2011-08-24-at-2.42.52-PM-1024x640.png?resize=525%2C300 1.5x"},"classes":[]},{"id":1966,"url":"https:\/\/mcclanahoochie.com\/blog\/2011\/12\/computer-vision-learning-portfolio\/","url_meta":{"origin":1896,"position":1},"title":"Computer Vision Learning Portfolio","author":"mcclanahoochie","date":"December 12, 2011","format":false,"excerpt":"This page constitutes my required\u00a0external\u00a0learning portfolio for CS 7495, Computer Vision, taken in Fall 2011. In it, I discuss what I have learned throughout the course, my activities and findings, how I think I did, and what impact it had on me. About me I am a coffee fanatic that\u2026","rel":"","context":"In \"computer vision\"","block_context":{"text":"computer vision","link":"https:\/\/mcclanahoochie.com\/blog\/tag\/computer-vision\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2011\/12\/chris-raffertys-2-150x150.jpg?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1810,"url":"https:\/\/mcclanahoochie.com\/blog\/2011\/09\/opencv-vs-libjacket-gpu-sobel-filtering\/","url_meta":{"origin":1896,"position":2},"title":"OpenCV vs. LibJacket: GPU Sobel Filtering","author":"mcclanahoochie","date":"September 24, 2011","format":false,"excerpt":"Update: LibJacket has been renamed to\u00a0\u00a0ArrayFire. In response to a comment on a previous post about integrating LibJacket into an OpenCV project, below is just a simple FYI performance comparison of OpenCV's GPU Sobel filter versus LibJacket's conv2\u00a0convolution\u00a0filter (with a sobel kernel)... This is an evolutionary post, so be sure\u2026","rel":"","context":"In \"arrayfire\"","block_context":{"text":"arrayfire","link":"https:\/\/mcclanahoochie.com\/blog\/tag\/arrayfire\/"},"img":{"alt_text":"Sobel filter: OpenCV GPU vs. LibJacket","src":"https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2011\/09\/cv-versus-jkt.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1663,"url":"https:\/\/mcclanahoochie.com\/blog\/2011\/08\/cuda-connected-component-labeling\/","url_meta":{"origin":1896,"position":3},"title":"GPU Connected Component Labeling","author":"mcclanahoochie","date":"August 6, 2011","format":false,"excerpt":"Connected Component Labeling (CCL): \"is used in computer vision to detect connected regions in binary digital images\", and sometimes referred to as blob coloring. Motivation: To keep AccelerEyes'\u00a0ever expanding GPU library growing, over a few weeks of this summer\u00a0I took on the project of writing a CUDA version of connected\u2026","rel":"","context":"In \"arrayfire\"","block_context":{"text":"arrayfire","link":"https:\/\/mcclanahoochie.com\/blog\/tag\/arrayfire\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2011\/08\/coins-bwlabel-300x122.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2347,"url":"https:\/\/mcclanahoochie.com\/blog\/2012\/08\/local-contrast-enhancement-with-arrayfire-opencv\/","url_meta":{"origin":1896,"position":4},"title":"Local Contrast Enhancement with ArrayFire + OpenCV","author":"mcclanahoochie","date":"August 20, 2012","format":false,"excerpt":"About one year ago, I wrote about a simple example of Image Processing with LibJacket + OpenCV... and the trend continues today. In this post, I demonstrate how ArrayFire (an improved version of LibJacket) can easily interop with OpenCV, through a simple example of unsharp maksing (local contrast enhancement). \u00a0\u2026","rel":"","context":"In \"arrayfire\"","block_context":{"text":"arrayfire","link":"https:\/\/mcclanahoochie.com\/blog\/tag\/arrayfire\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2012\/08\/Screen-Shot-2012-08-20-at-9.19.21-AM_2-1024x640.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2012\/08\/Screen-Shot-2012-08-20-at-9.19.21-AM_2-1024x640.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2012\/08\/Screen-Shot-2012-08-20-at-9.19.21-AM_2-1024x640.png?resize=525%2C300 1.5x"},"classes":[]},{"id":1876,"url":"https:\/\/mcclanahoochie.com\/blog\/2011\/10\/gpu-convolution-opencv-gpu-and-libjacket-part-2\/","url_meta":{"origin":1896,"position":5},"title":"GPU Convolutions: OpenCV GPU and LibJacket – Part 2","author":"mcclanahoochie","date":"October 24, 2011","format":false,"excerpt":"This is a response to my earlier post comparing OpenCV's gpu::convolve() and LibJacket's jkt::conv2() convolution functions, at various image and kernel sizes. That post generated a lot of traffic, most notably from the OpenCV developer community. Taking note of this, it seems that the folks at Willow Garage have re-vamped\u2026","rel":"","context":"In \"arrayfire\"","block_context":{"text":"arrayfire","link":"https:\/\/mcclanahoochie.com\/blog\/tag\/arrayfire\/"},"img":{"alt_text":".","src":"https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2011\/10\/Screenshot-f2-2075.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2011\/10\/Screenshot-f2-2075.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/mcclanahoochie.com\/blog\/wp-content\/uploads\/2011\/10\/Screenshot-f2-2075.png?resize=525%2C300 1.5x"},"classes":[]}],"jetpack_likes_enabled":false,"_links":{"self":[{"href":"https:\/\/mcclanahoochie.com\/blog\/wp-json\/wp\/v2\/posts\/1896","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mcclanahoochie.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mcclanahoochie.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mcclanahoochie.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mcclanahoochie.com\/blog\/wp-json\/wp\/v2\/comments?post=1896"}],"version-history":[{"count":0,"href":"https:\/\/mcclanahoochie.com\/blog\/wp-json\/wp\/v2\/posts\/1896\/revisions"}],"wp:attachment":[{"href":"https:\/\/mcclanahoochie.com\/blog\/wp-json\/wp\/v2\/media?parent=1896"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mcclanahoochie.com\/blog\/wp-json\/wp\/v2\/categories?post=1896"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mcclanahoochie.com\/blog\/wp-json\/wp\/v2\/tags?post=1896"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}