Microsoft has released an app for iPhone, designed to translate the visual world into an audible experience for those with vision impairments.
Making use of artificial intelligence for image recognition, and natural language processing, Seeing AI uses the smartphone’s camera to capture what’s happening around the user, then relay this through speech.
Point the app at a person and it will use facial recognition software to tell you that individual’s gender, age and emotion. Aim it at something that contains text – like a street sign or a document – and it will speak what it reads.
The app is also able to relay product information from scanned barcodes, while features are being developed for identifying currency – useful for handing over cash in a shop – and for describing wider scenes.
For example, in the promo below the user takes a picture, and the app says the image is of a young girl throwing a frisbee in the park.
Seeing AI certainly seems to have the potential to become a useful tool for those with vision impairments, although the app would presumably require some level of vision in order to line the camera up with a required object – particularly if it’s being used to scan documents.
Those living in the US, Canada, Hong Kong, India, New Zealand and Singapore can try the iOS app now, while users in Australia will have to wait until it is made available.