Hovercast VR Menu: a hand-based menu for Leap/VR

zachkinstner 2015-01-16 23:12:42 UTC #1

Control the world -- from the palm of your hand.

Henu is a powerful, customizable menu system that can be attached to a Leap Motion hand (and eventually, any three-dimensional input device). It features various buttons, sliders, checkboxes, radio sets, and nested levels.

The project includes a demo that is built for VR usage -- the Leap Motion controller mounted to an Oculus Rift headset.

The Henu project is only about two weeks old, and I'd love to hear feedback and ideas for making it better. I haven't had time to write proper documentation yet, but the demo application within the project provides several examples.

Watch the Henu demo video >

You can download a demo build (PC/Mac) using the links at the GitHub project:
https://github.com/aestheticinteractive/Henu

Komposten 2015-01-17 11:22:09 UTC #2

Woah! This is looking so great! Too bad I don't have a VR headset so I can't try it out.
This system definitely has great potential in all kinds of applications!

zachkinstner 2015-01-17 18:16:10 UTC #3

Thanks, @Komposten!

I have just put together a quick experimental build that uses only the Leap Motion controller. It doesn't require the Oculus Rift headset.

Mac: http://www.aestheticinteractive.com/clients/henu/Henu-LeapOnly-Mac-2015-01-17.zip
PC: http://www.aestheticinteractive.com/clients/henu/Henu-LeapOnly-PC-2015-01-17.zip

I have done very little testing with these. Please let me know if these work for you!

zachkinstner 2015-01-24 21:28:11 UTC #4

The “Henu” menu is now called...

Hovercast VR Menu

You can find more information, screenshots, and build links at Hovercast's GitHub project:
https://github.com/aestheticinteractive/Hovercast-VR-Menu

A new video showing the 2015-01-24 demo:

Video link

The latest build includes several improvements, such as:

Improved cursor features and visibility.
A line along the inner edge to highlight the nearest segment.
A new demo menu for customizing the menu's colors, sizes, and switching between left/right hands.
Reduced distance required to release the slider/sticky items.

Komposten 2015-01-25 00:07:31 UTC #5

The non-VR version doesn't work too well, sadly. The main problem is that in order for the menu to appear you need to hold your hand almost vertically which isn't tracked well by the Leap (since the fingers cover each other). The VR version looks really promising though. I hope to see this in things like future games and such!

Keep up the good work!

zachkinstner 2015-01-26 19:20:01 UTC #6

I really appreciate the feedback, @Komposten. Thanks again!

I agree that the Leap-only build doesn't work very well. As you've described, the table-mounted Leap Motion controller has a difficult time tracking the "palm toward eyes" hand orientation.

You might be able to approximate the VR experience by mounting the Leap Motion controller to a hat or headband. As far as I know, the VR demo builds will run without actually having a Oculus Rift headset (you would see the side-by-side output as shown in the video). I was able to use this setup when I was first experimenting with my Fragmental 3D VR demo, before my Oculus Rift arrived.

Komposten 2015-01-26 21:01:30 UTC #7

Actually, I've already tried it. By holding the Leap with my teeth.

Apart from a slight ache in my jaw, things worked perfectly.

zachkinstner 2015-01-29 19:05:56 UTC #8

Haha, that's great!

I started with a tooth-mount as well, but quickly realized it wasn't going to work well for hours of development and testing. I built a winter-hat-mount instead, using some binder clips and a rubber band

lolsmcfee 2015-02-08 00:41:10 UTC #9

would an android version be possible? like a google cardboard and a leap motion controller together to make this work

zachkinstner 2015-02-16 18:28:23 UTC #10

@lolsmcfee, from a development perspective, the Hovercast menu isn't dependent upon the headset/camera used (for example, the demo includes both VR and non-VR versions). Are you asking about development, or requesting a downloadable demo that supports Google Cardboard?

zachkinstner 2015-02-16 18:40:25 UTC #11

An update on the project...

I wrote two articles that appeared on the Leap Motion blog this weekend:

The first article provides an overview of the project, with discussion about why VR menu interfaces are important. The second article is more technical, and describes the decisions and considerations involved during the design/development process.

There have also been several other articles, blogs, and posts/tweets about Hovercast in the past week. Notably, articles at Mashable and Fast Company. See Hovercast's News and Feedback page for more...

billyboi23 2015-03-12 20:24:19 UTC #12

Perhaps a slightly modified gopro chest mount? Or even a helmet mount tightened down to your head. It seems like it would be fairly simple to manipulate some of the attachments in a manor that would support the leap.

zachkinstner 2015-03-12 21:40:59 UTC #13

@billyboi23 I've also seen a Leap Motion controller mounted to thick-rimmed sunglasses (or those 3D movie glasses). They popped out the lenses, and stuck the VR mount to the upper rim of the glasses.

zachkinstner 2015-03-12 21:43:03 UTC #14

I have added a new tool to the Hovercast family, currently called the Hoverboard VR Interface. In the first demo, I have created some traditional keyboard layouts for text entry.

You can find the first video, links, and discussion in this forum post: https://community.leapmotion.com/t/hoverboard-vr-interface-keyboard-panel-interfaces-for-leap-vr/2659

CerebralHack 2015-03-15 05:20:33 UTC #15

Is this still available?

zachkinstner 2015-03-15 16:33:59 UTC #16

Hi @CerebralHack, yes, the project is still active/online at https://github.com/aestheticinteractive/Hovercast-VR-Menu. You can also find licensing information in the Wiki pages there.

Are you looking to include Hovercast in an upcoming project?

marcodefcon 2015-04-07 08:23:31 UTC #17

I tried to fill the hovercast menu in an APK and have the error:

Shader error in 'LeapMotion/LeapRGBUndistorted': 'gl_' : reserved built-in name at line 123

any ideas on how to resolve it?

stacker3d 2015-04-17 06:25:04 UTC #18

@marcodefcon In the shader file (LeapRGBUndistorted.shader) you can replace all occurances of gl_FragColor by e.g. agl_FragColor this way the variable has a prefix which is not reserved,