Apps Modernisation for FSI with Azure Cognitive Services

04 Nov 2019, MOQdigital


MSFT Azure FSI - cognitive services

For teams wanting to innovate, application modernisation and innovation is being bolstered with a set of on-demand cognitive capabilities that previously would have been expensive to design, build and maintain.

As little as 3 years ago, embedding image recognition capabilities into traditional line of business applications was simply not an option for most teams.  Today, the simplicity of engaging these capabilities means that existing applications can be enhanced with new functions that ultimately improve experiences.

These might include:

  • Application forms recognition – eg. When applying for a new loan
  • Detecting user sentiment & face recognition - to personalise data entry

Microsoft Azure provides a collection of AI-infused Cognitive Services that simplify the ability to rapidly build advanced features, without the technical knowledge of a data scientist.

Cognitive Services span the categories of Vision, Speech, Language, Search, and Decision; offering the most comprehensive portfolio in the market for developers who want to embed the ability to see, hear, translate, decide (and more) into their apps. 

Exploring the ‘Vision’ Service Category

Vision gives the capability to identify, analyse content and extract data from within images and videos. It is further grouped into 5 sub-services:

  • Computer Vision
  • Form Recognizer
  • Ink Recognizer
  • Custom Vision
  • Face and Video Indexer.
Computer Vision

This function classifies images, identify scenes, celebrities, landmarks and activities. It can also recognise handwritten and printed words from images.

Figure 1 - Computer Vision analysis

Above: Computer Vision analysis in action. Right hand-side shows the results in JSON format.

Figure 2 Words from images

Above: Demonstrates the ability to extract words from images.*

This service is great for use cases where we want to count or recognise the presence of something. For example, this could assist with stocktake scenarios, or to allow retail customers to find an item in a catalogue simply by taking its picture.

Form Recognizer

This function converts form images into useable data by accurately translating form images into key/value pairs. The Form Recognizer can be tailored understand your unique form layout.

 Figure 3 - invoice data extract

Above:  An Invoice in PDF format and the output data extracted.*

Changing the way traditional form and character recognition is done, these capabilities rapidly improve paper to process scenarios. For example, when importing a loan application’s data, prefilling a client’s data set is vastly simplified. Common use cases are in Accounts Payable where recognising a supplier’s invoice and automating much of the process becomes the norm, resulting in less data entry and increased efficiency.

Ink Recognizer

Ink Recogniser lets us extract words and shapes from handwritten notes.

Figure 4 written words and shapes

Above: Image of written words and shapes with the interpreted result.*

This allows developers of traditional applications to Ink-enable them with little effort.

Custom Vision

Train your own image classifier to keep improving your image recognition algorithm. This is very useful when training on your own product catalogue.

Figure 5 image and probability analysis

Above: Algorithm trained to recognise a sample image and the probability analysis result.*


This function is face detection and identification with emotion recognition and classification. Allows developers to personalise the interaction experiences for users.

Figure 6 face recognition

Figure 6 Face detection determines that the person on the photos is the same person.*

Video Indexer

This is a Video and Audio utility that can extract spoken words, recognises and analyses images, faces, written text and more.  This is very useful for any media driven application, but its usage extends beyond the obvious.  For example, in the case of remote advice sessions given by financial planners, the functionality can be used to transpose draft notes for acceptance later into the CRM or tracking platform.

Other Cognitive Services Categories:


Multi-language capable text-to-speech and speech-to-text services.


Interprets chunk of text in order to detect the language, key phrases and sentiments to help recognise context, intent and provides links to related entities.


Leverages reinforcement learning to offer users specific recommendations to assist with decisions. This category includes Content Moderator, Anomaly Detector and a service called Personalizer, which uses reinforcement learning to provide users a specific recommendation to enable quick and informed decision-making.


Uses Bing search and applies powerful AI-powered algorithms capable of searching, comparing results, summoning billions of web pages, images and videos that are relevant to your inquiry.  

This enables customers to apply Cognitive Services algorithms to extract new insights from their structured and unstructured content. It also includes capability that enables developers to store AI insights gained from cognitive search, making it easier to create knowledge-rich experiences leveraging Power BI visualisations or machine learning models.

How to consume

The Azure Cognitive Services can either be consumed directly by application developers using the well-defined APIs directly in their code. Or more recently, these capabilities can be easily connected to by citizen developers in Power Platform’s low-code environment. Connecting these capabilities directly to PowerApps has been simplified so that anyone can rapidly prototype with these enhanced functions, and then deploy their solution without reaching out to developers to code up complex componentry.

As you can see, Cognitive Services have a lot to offer businesses wishing to modernise or innovate on their existing platforms or with new builds. MOQdigital provides expert design and build capabilities in innovation with Cognitive Services and can extend these to hyper scale scenarios.

To find out if your current solution can be modernised using Microsft Azure services, we are offering a free application modernisation and innovation workshop to help you prove out your ideas, fast.


*All images courtesy of Microsoft