Magic Hands: A New Generation of HCI Technology 
Magic Hands is a video-based application that has the capability to detect and identify human’s gestures. Besides, Magic hands has a user-friendly interface that can assign gestures to computer commands which can be used for web-browsing, presenting, game playing and so on. We believe Magic Hands will take place of traditional mouse and keyboard and become the next generation of HCI (Human-Computer-Interaction) technology.
Traditional HCI technology relies on mouse and keyboard, which are inconvenient for people who are disabled or have special needs such as doing a presentation. On the other hand, it’s natural for human beings to use gestures in human-to-human interactions. This fact leads us to think the potential of using gestures in human-to-computer interactions. The technical problem is how to let a computer understand human’s gestures. To solve this problem, Magic Hands uses Microsoft Kinect, whose prime sensor can estimate the 3D location information of 26 major joint positions on user’s body. Because of this feature, we can decompose human’s gestures into a set of data that represents the change of those joint positions. Magic Hands then uses algorithmic-based search and template-based search to decode those data and identify user’s gestures.
There’re more than 20 sets of gestures preloaded in our gesture library, which allows users to assign computer commands on them. Users are also free to add their own gestures into the library through a voice commander interface. The accuracy of recognition is as high as 95% for algorithmic-based search, which is designed for simple gestures, and close to 80% for template-based search, which handles the detection of complicated gestures. The latter accuracy is dynamic to improve because Magic Hands takes advantage of a machine-learning algorithm that will automatically raise the recognition rate when users reuse the same gestures. Magic Hands is an intuitive and easy-to-use HCI application and can help to improve the user experience of using computers in various scenarios. In fact, my senior design presentation was done with the help of Magic Hands and I even didn’t use the keyboard and mouse at all in the whole presentation.