Accordion Hands

About

Accordion Hands is a fun way to interactively “play” a virtual accordion!

The app was inspired by discovering the funny Life Accordion to Trump YouTube videos.

This app is not meant to be a real accordion simulator!

In its current form, the app is purely just a tech demo (just for fun), but I hope to address its shortcomings soon enough 🙂 and add many more features (see below).

*Free App Download!* | *New Web Demo!*

The app is powered by Google’s MediaPipe ML framework, and uses their hand detection + hand tracking machine learning models.

Since MediaPipe is cross-platform, I was able to very quickly prototype a demo on my macbook using a webcam (all in C++), and could also immediately test on Android. This allowed me to do quick dev iterations on my laptop (natively, not via emulator), and periodically test and tweak things on my phone.

How the app works:

The core logic piggybacks off of the MediaPipe multi-hand tracking example. I basically just had to do two things:
1) add one new calculator to the graph to calculate a rotated/stretched box between two hands, and manage the accordion sounds
2) extend the existing annotation calculator’s box drawing to support drawing a box filled with an image.

I was able to have a complete Android proof of concept pretty quickly (by hijacking existing demo apps) in about 1 weekend, and then spent 3+ weekends redesigning/customizing/polishing the Android app (b/c how overly complicated general Android development is, on top of that using only bazel instead of gradle, and waiting for app store approval..).

Demo

And finally: achieving the original goal, automatically via my app!
(click image for audio):

The app has a few rough edges, but I just couldn’t wait to release it! Updates will come later and improve many things, but in the meantime please have some fun 🙂

Another automatically created demo (click image for audio):

Download

Free Android App

*New: Web Demo!*

Release & Change Log

• 0.1 – 2020/04/12
  • Initial [beta] release
  • Interactive live camera demo
• 0.4 – 2020/05/17
  • Support loading a local video
  • Misc internal bug fixes
  • Improve speed performance
  • Misc UI tweaks
  • Significantly reduced app file size
• 0.5 – 2020/06/28
  • Improved tracking
  • Misc sound trigger tweaks
• 0.5.1 – 2020/07/01
  • Adjusted audio levels
  • Minor tracking improvement
• 0.6.0 – 2020/09/13
  • Rear/back camera support
  • Misc performance and memory improvements
  • Updated main menu
• 0.7.0 – 2022/03/19
  • Accordion graphic changes size based on ‘depth’
  • Misc performance improvements

Upcoming Features

• Option to load a video file
• Rear facing camera
• More sounds
• More accordion graphics
• Configurable settings
• Record/Share buttons
• …