While in the eye-doctor's waiting room the other day, waiting for my wife's exam to be over, I picked up a Popular Science magazine. I found an article on the surprising uses to which people are adapting the Microsoft Kinect game accessory. These clever folks are transforming this toy into a highly versatile user interface with amazing possibilities.
One that particularly intrigues me is the use of the Kinect to create a virtual keyboard that one can draw with pen and paper. The application uses the Kinect to image the paper, recognize the shapes on the paper as pushbuttons, and then determine when the user "presses" on the button. The machine then takes the appropriate action associated with that button.
I immediately imagined several potential uses for this capability. Say you have a machine in a high EMI environment that needs a user interface. You can place the machine in a Faraday cage with a transparent window that the Kinect looks through to see a printed keyboard outside the cage. Users can then type on that keyboard as though it were connected to the machine, without the need for any wires, which might cause an EMI leak, to penetrate the cage walls.
A similar application takes place in an underwater environment. Again, the machine is sealed inside a watertight chamber without the need for any cables or connectors to penetrate the walls, reducing the chances of a leak. Further, the keyboard, being just paint on a surface, would not need to be sealed, as there are no electronics to protect.
There are an amazing number of other applications using the Kinect as a user interface. One surgical system uses the Kinect to trigger a feedback force for surgeons performing robotic surgery, providing them with a sense of touch. Another uses the Kinect to create a real-time hologram of the object it is scanning, creating a 3D camera and projector. Popular Science keeps an ongoing list of such uses (which it calls "hacks") on its Website.
Is anyone in this community using or considering the Kinect as part of a user interface for their MCU design? If so, let us know!
Microp 2/14/2012 12:01:22 AM User Rank Program Manager
Re: Kinect Challenge
Ryszard, can you share the kinetic fun challenge link with the community, so that interested peoples can have a look for that. It seems that Kinetic have a vast potential for implementing in many applications.
but deadline for the application process was January 25th.
'Through this program, Microsoft is supporting entrepreneurs, engineers and innovators to bring to life a wide range of business ideas that leverage the limitless possibilities Kinect enables.'
It seems that they will work with Lenovo in order to play in the same market as Microsoft Kinect.
Anyway, I have a former colleague that is working in Belgium for SoftKinectic. I know that they have support in their technology for ARM (TI's OMAP) and Intel ATOM.
I can see a number of robotic applications for the Kinect. On YouTube and other sites there are robots being driven by people using the Kinect as the HUI "joystick". There are a number of postings showing obstacle avoidance using the depth image information and user code. Given the Kinect's camera, depth sensing, and microphone array there are too many applications to name. What do others think the most interesting or innovative application will be?
Rich Quinnell 2/14/2012 4:22:48 PM User Rank Blogger
Re: Kinect Application Possibilities
The one I like most is on the PopSci site, showing use of the the Kinect to make a shopping cart "heel" for a wheelchair-bound shopper. The cart follows the shopper around so the user can load groceries but not have to push the cart as his hands are full just moving through the aisles.
What a great idea! If only we could adapt it for the dogs :).
I was wondering when someone will come up with an aid for the visually impaired. Something like a machine version of a seeing eye dog but with the ability to read signs, packages, and speak / interact with the person. This would involve OCR and some horsepower to get it right and timely. Although the distance limits of the Kinect might preclude it from a greater acceptance, still having an audible means to navigate (around objects, over curbs, read isle numbers, contents of a package) would be really nice.
Rich Quinnell 2/14/2012 6:47:50 PM User Rank Blogger
Re: Kinect Application Possibilities
I can imagine a system with stereo headsets that give out a tone varying in pitch to indicate range and using the stereo effect to give direction. I wouldn't have to be too great a range if all it is for is to help navigate around obstacles. You could add another audio clue to indicate, say, height so one could distinguish between a low table trip hazard and a wall.
That sounds like a good project to me! Building a pair of ultrasonic range finders is easy (or you can just buy them built). Add a low power MCU with a counter/timer or so, a muxed comparator -- and I think you'll be in business!
Curt, I like the idea of using the low cost ultrasonic range finders! I have used a number of them and they can get confused by the reflecting material they encounter (say for example: Diamond Plate - wrecks havoc on the distancing measurement). The thing that was appealing to me with the Kinect is the IR sensor seems to be a little more robust than the ultrasonic rangers and has fairly decent resolution. Given the cost of the full up Kinect, I would hope that some enterprising company would purchase the sub-components - IR driver/sensor (if they can be bought) and create a low cost, high resolution seeing eye Kinect with voice feedback for reading signs. Simple color and shape recognition might be possible (maybe not fast enough to be real time but quick enough) using some of the high performance / low cost processors on the market. The neat thing is it would work in the dark better than in the light!
Rich Quinnell 2/21/2012 1:01:25 PM User Rank Blogger
Re: Kinect Application Possibilities: Rich
RD, shape recognition certainly should work, but color? If you are both illuminating and "seeing" in IR, then traditional color is out of the question since it depends on wavelengths in the visible. As I understand the Kinect, it uses IR as rangefinder and has a color camera for images. That would work for color in the daytime but not at night without a white light source.
By the way, most image sensors are also sensitive in the IR and camer makers have to put in special filters to prevent the IR from contaminating the photo. Early camers didnt have them and this let them "see" body heat signatures through clothes.
Rich, you are right about needing light for the color camera to work. I was refering to the distancing IR feature of the Kinect. If the natural sunlight is too bright the IR sensor tends to be "washed out". However, at night the IR sensor works great! Some of the early Sony Camcorders have the IR night vision option which caused quite a stir with some videos taken (aka the "body heat"). Some fabrics also were more transparent in the IR band (much to the surprise of the wearers).
Joe Stanganelli 2/22/2012 4:41:20 PM User Rank Program Manager
Re: Kinect Application Possibilities
There are even more professional and recreational uses yet. Making the assembly line more efficient. Controlling a hovercraft. Home automation and lighting control.
I had the pleasure of trying out the Kinect at a conference, and it was really neat. It's cool to see this technology being adapted for academic, military, and industrial purposes as well.
Rich Quinnell 2/17/2012 12:39:32 PM User Rank Blogger
Re: Cool application
Northstar, that is cool. It would be an interesting way of teaching map-reading in grade school, among other things.
All this with the Kinect harkens back to the theme of my blog Toys vs. Tools - the more versatile something is, the more powerful. Yet another "toy" becoming a tool.
Joe Stanganelli 2/29/2012 12:48:19 PM User Rank Program Manager
Re: Cool application
Wow, talk about "Sit back, relax, and we'll take care of everything for you."
If it only was only equipped with robotic arms (to grab items off the shelves) and NFC (to pay at the checkout line), you could run other errands while the grocery cart does the shopping for you, and pick your groceries up when it's done!
Rich Quinnell 2/29/2012 1:41:48 PM User Rank Blogger
Re: Cool application
Looks like a neat one. The blog associated with the video and many of the comments that follow make me think that this should be cross-posetd to my Luddite blog, as well.
Rich Quinnell 3/22/2012 9:06:12 PM User Rank Blogger
Now, the perfect fit
The Kinect has now made it into at least one commercial product. This one, from Bodymetrics, is used in shops to help folks find jeans that provide the perfect fit. I'm continually amazed by what imaginative people can do with new technology, and even more amazed at what becomes commercially viable.
Northstar, thanks for these additional links. The Kinect seems to have really hit a button with folks. I like the sandbox especially. It can help teach earth sciences really well (my wife's a teacher so I keep an eye out for educational apps) by visualizing things. It can even be used to teach topographic map reading.
Rich Quinnell 5/30/2012 3:58:23 PM User Rank Blogger
Re: More uses for Kinect
On the satellite application I wonder how flight-ready the Kinect is? I had to jump through flaming hoops to get an MCU qualified for space applications. Perhaps these guys aren't worried about the risk of failure due to temp, radiation, etc.
antedeluvian 5/30/2012 4:24:46 PM User Rank Blogger
Re: More uses for Kinect
I doubt that they have even started to consider the space environment yet. In addition I have some experience of NASA and capture of objects like the shuttle (at least in their specifications) with a robotic arm. It happens very slooooowly for safety reasons and is normally rehearsed many times. I wonder on the repeatabilty of the actions using the Kinect and the software would have to be able to adjust the speed of the motions. I suppose the host could record the actions and get them perfect, but there may still be safety concerns.
Rich Quinnell 5/31/2012 1:54:37 PM User Rank Blogger
Re: More uses for Kinect
Yes, slow is the key when maneuvering in space. I wonder how find a control over speed and such the Kinect-based designs will have? If it's not fine enough we may see our first game-based collision in space.
To save this item to your list of favorite Microcontroller Central content so you can find it later in your Profile page, click the "Save It" button next to the item.
If you found this interesting or useful, please use the links to the services below to share it with other readers. You will need a free account with each service to share an item via that service.
Tweet This
