Microsoft has released an app for iPhone, designed to translate the visual world into an audible experience for those with vision impairments.

Making use of artificial intelligence for image recognition, and natural language processing, Seeing AI uses the smartphone’s camera to capture what’s happening around the user, then relay this through speech.

Point the app at a person and it will use facial recognition software to tell you that individual’s gender, age and emotion. Aim it at something that contains text – like a street sign or a document – and it will speak what it reads.

The app is also able to relay product information from scanned barcodes, while features are being developed for identifying currency – useful for handing over cash in a shop – and for describing wider scenes.

For example, in the promo below the user takes a picture, and the app says the image is of a young girl throwing a frisbee in the park.  


The app is the result of work from Microsoft Research, and looks to be a neat coupling of several areas of focus. The research wing’s site lists AI, computer vision and human language technologies as core areas of focus, with Seeing AI ticking off all three. Other projects the initiative is working on include a Minecraft-based testing ground for AI and means for neural networks to make their own software code.

Seeing AI certainly seems to have the potential to become a useful tool for those with vision impairments, although the app would presumably require some level of vision in order to line the camera up with a required object – particularly if it’s being used to scan documents. 

Those living in the US, Canada, Hong Kong, India, New Zealand and Singapore can try the iOS app now, while users in Australia will have to wait until it is made available.

Source link