Playing with Leapmotion for accessibility

Today I spent a few hours in the Leap motion offices talking about the web possibilities of this cool new technology. On the way back we had an internet connectivity outage, so it was a great opportunity to start playing with some of the ideas. Here is a video of what I am working on.

In essence, this means:

  • Swiping your finger right will scroll the page down, left will scroll up. The speed of the swipe defines the amount of pixels scrolled
  • Making a clockwise circular gesture will jump to the next link, making a counter-clockwise one to the previous link
  • A tap gesture in the air will activate the link

The code is actually amazingly simple, once you use the Leap.js library:

(function(){
var linkindex = -1;
var links = document.querySelectorAll('a');
myController = new Leap.Controller("ws://localhost:6437/");
myListener = new Leap.Listener();
myListener.onFrame = function(controller) {
  var frame = controller.frame();
  var gestures = frame.gestures();
  if (gestures[0] && gestures[0]._state === 'stop') {

    if (gestures[0]._type === 'swipe') {
      if(gestures[0]._direction.x > 0) {
        window.scrollBy(0,gestures[0]._speed*2);
      } else {
        window.scrollBy(0,-gestures[0]._speed*2);
      }
    }
    if (gestures[0]._type === 'keyTap') {
      if (linkindex) { 
        links[linkindex].click();
      }   
    }
    if (gestures[0]._type === 'circle') {
      if (gestures[0]._clockwise === true) {
          updateIndex(1);
          if(links[linkindex]) {
            links[linkindex].focus();   
          };
        } 
      if (gestures[0]._clockwise === false) {
          updateIndex(-1);
          if(links[linkindex]) {
            links[linkindex].focus();   
          };
      } 
    }
  }
}
function updateIndex(add) {
  linkindex += add;
  if (linkindex < 0) {
    linkindex = 0;
  }
  if (linkindex >= links.length) {
    linkindex = links.length - 1;
  }
}
myController.addListener(myListener);
myController.enableGesture("keyTap", true);
myController.enableGesture("swipe", true);
myController.enableGesture("circle", true);
})();

This is now very rough and ready and I will try to wrap this is a bookmarklet to apply it on any web site.