So now that I’ve had a chance to play around with the Kinect SDK a lot more I discovered that you could also use the Kinect to detect sound recognition. At first I thought I would be a bit ambitious and I tried to get it to detect French numbers. I just thought that it would be brilliant to use for a vocabulary test with kids. I’m so excited about using the Kinect in my ICT lessons this term and can’t wait to come up with an application for the whole class to use!
Imagine if you had EAL students and you were trying to get them to pronounce words in English. You pictures which change colour when they have said it correct. There possibilities are endless and I’m very excited about teaching ICT this year. I’m even thinking about getting some of my students to build applications due to some of their coding backgrounds.
After a lot of research online, I discovered that the language / grammar pack wasn’t ready yet. I also noticed that even when I had the code ready, I was getting “null” reference errors. I found that the reason it wasn’t working was because I didn’t have the add-in’s installed. You will need the following installed if you wish to get the sound recognition working. Obviously you will need the Kinect, but here are the others.
- Kinect for Windows Runtime Language Pack
- Microsoft Speech Platform – Software Development Kit
- Microsoft Speech Platform – Server Runtime
Now when getting the sound recognition to run, I used WPF and C#. The reason I have used this is because I noticed there weren’t any examples on the internet and going through forums, it is what people are having the most difficulty with. It seemed much easier to pass it to a console window rather than using it in the application. However this simple example should help.
Rather than confuse you here with lots of code, I have attached the source code for a simple application that detects numbers and changes the numbers based on the input from the Kinect Mic. I have included comments in the program to help you understand them a little bit more. I know I work better from examples rather than trying to figure out what a class is and how you go about using it. Especially for some of you new to coding it may be more beneficially. If you follow the green, comments then you should be able to cope with the code.
Click here to download the source of the application. The application in the folder is called “SpeechRecognitionApplication”.
The last 17 seconds of the video below will show you it working and you can see what your program should do when you’re done.
If this blog post has been in any way useful please feel free to drop me an email to let me know your progress. Alternatively you could comment below. Thank you.