AI Builder in Microsoft Power Apps

What is AI Builder?

AI Builder is a Microsoft Power Platform capability to improve business performance by automating processes and predicting outcomes.
You do not require any coding or data science skills to add intelligence to apps using AI Builder
AI Builder offers AI models that are intended to streamline your company's operations. With the aid of AI Builder, your company may apply intelligence to streamline procedures and extract knowledge from data using Power Apps and Power Automate.
With AI Builder, you can leverage the power of AI without having any coding or data science expertise.

You may either pick a prebuilt model that is prepared to use for many typical business scenarios, or you can develop bespoke models that are suited to your needs.
Microsoft AI Builder may assist you in improving the performance of your organization by automating processes and anticipating outcomes.
Using AI Builder, you can quickly include AI into the apps and procedures that interact with your business data stored in the underlying data platform (Microsoft Dataverse) or in a number of cloud data sources, like as SharePoint, OneDrive, and Azure.

Types of AI Model

Custom Model

These models can be built by choosing a model type in AI Builder and train it to do a specific AI task using the data
Your AI model must first be trained to function as you desire before you can utilize it. Once your model has been trained, publish it so that others can use it.
This model helps you configure a model in accordance with the demands of your company. AI Builder saves your progress as a draft each time you save modifications to your model. When you're finished, verify the settings you wish to use to train your model, then click Train to start the training process.

Prebuilt Model

These models are already trained and ready to use in the app.
These models are already built to perform specific task.
Prebuilt models from AI Builder make it easier to incorporate intelligence into apps and workflows without having to collect data, construct, train, and publish your own models.
For instance, you may add a component to Power Apps that uses a prebuilt model to identify contact information from business cards. If you want to determine if client feedback was favorable or unfavorable, you may utilize a prebuilt model in Power Apps and Power Automate.

Types of Custom Model

Prediction

The prediction models in AI Builder look for trends in the historical data you supply.
Prediction models pick up the ability to link such patterns to results.
Then, we apply AI's capacity to find previously learnt patterns in fresh data and utilise them to forecast future results.
Use the prediction model to investigate business queries that have one of the following possible outcomes:

from two possibilities (binary).
from a range of potential possibilities.
where a number is the solution.

Category

For businesses, the amount of text data is growing quickly. Text data is being added in more and greater quantities through channels including email, documents, and social media.
When important information is taken from this data and used, it enables you to provide your clients better goods and services.
Dealing with this constantly expanding volume of data is frequently time-consuming and prone to mistakes, which can result in lost business opportunities and higher expenses.
One of the primary challenges in natural language processing (NLP) is categorization. You may identify text entries with tags using category categorization for things like:

Sentiment analysis
Spam detection
Customer request routing
Other business needs

Entity

Based on your business requirements, AI Builder entity extraction models identify specific text data that you target.
The model locates important textual components and groups them into predetermined categories.
This can assist you in converting unstructured data into machine-readable structured data.
Processing can then be used to extract facts, retrieve information, and provide solutions.
AI Builder's entity extraction can utilize custom entity extraction models, by a process.
They must first be created, trained, and published. You may develop an entity extraction model that is especially suited to your particular needs by utilising your own training data and design criteria.

Object Detection

Using object detection, business procedures may be accelerated or automated.
It can simplify inventory management in retail, freeing up store executives to concentrate on building connections with customers in-person.
In the manufacturing industry, technicians can utilise it to expedite the repair procedure by immediately obtaining the instruction book for a piece of equipment whose UPC/serial number is not readily visible.
Any big company may utilize AI Builder object detection to give their apps these features for their own unique items.

Document Processing

When you process documents, you may read and store data from common papers like tax forms or invoices.
By utilizing Power Automate and Power Apps to evaluate, extract, organize, and store the data automatically during this process, you may save a lot of time.
Develop your model and specify the data that will be taken from your forms.
To get started, you simply need five form papers. Get precise results quickly that are suited to your particular content. You don't require a lot of manual efforts or data science knowledge while using AI Builder.
You may utilize your model in a flow in Power Automate or a canvas app in Power Apps once you train and publish it.

Image Classification

The process of labelling photos with categories that best describe their overall content is known as image classification.
Image classification algorithms learn from your photographs to identify patterns like textures, colours, and forms. Then, you may use these patterns to recognise your labels.

Setup a Custom Model and Implement in Power Apps

To train the custom model prefer below 5 sample invoices

For example : When we have some invoices to be stored in database we can use this AI feature to store information from the invoice using Form Processor in Power Apps and Document Processing Model

Step 1: Select the type of document- Preferred- Structured Document and hit next

Step 2: Choose the information to extract- Like Logo, From, To, Total and Item Table and hit next

Step 3: Now we have to train the AI by giving some sample invoices; for which we need to add collection and add at least 5 invoices and prefer giving different samples for our AI to get efficient with different invoices and hit next (Duplicate invoices will be auto discarded)

Step 4: Once we have uploaded all the samples we need to tag the field and table we want to extract. This is to be done for all the sample invoices you upload; below in green marked are the fields and data table selected; once done for all invoices you will get the next option enabled- hit next.

Step 5: Now finally you will be able to view all the setup and extraction you have done on this model. It will give you the review of model summary to edit if missed on something. Once confirmed hit Train and then the model will be trained and might take 3-5 minutes to do this.

It will display -

Once the model is trained- under AI Builder- model we will get the model published as below

We can then call this model using the inbuilt power apps -Form processor and on select of form processor it will ask you to select the model you have trained. We can select on the model and it will proceed further.

And after setting up the form processor we can take a form and store information extracted in DataSource and Final outcome would be:

Types of Prebuilt model

Business card reader The prebuilt model for business cards may be used to extract data from images of business cards. The AI model retrieves data such as the person's name, job title, address, email, firm, and phone numbers if it recognizes a business card in the image.
Category classification The ready-to-use AI model known as the prebuilt category classification model is set up to categorise your content into groups that are relevant to a particular business scenario. The first pre-built AI model for categorization was based on client feedback.
Entity extraction The prebuilt entity extraction model may identify particular text data that is relevant to your business. The model extracts important textual components and groups them into predetermined categories. This can assist in converting unstructured data into machine-readable structured data. Processing can then be used to extract facts, retrieve information, and provide answers.
ID reader Passports, US driving licenses, social security numbers, and green cards may all have their information extracted using the identification document (ID) reader prebuilt model. The model will extract details like the user's initial name, birthdate, or gender. Once the model has been processed, images including scans or photos of the identification papers are removed. Invoice processing The prebuilt AI model for invoice processing captures important invoice data to aid in automating the processing of invoices. The model for processing invoices is designed to identify typical invoice components including invoice ID, invoice date, due date, and more.
Key phrase extraction A text document's primary ideas are found using the key phrase extraction prebuilt model. The model, for instance, identifies key speaking points "food" and "excellent service" in response to the input text "The meal was amazing and there was great service!" This model can take an unstructured text content and extract a list of key words from it.
Language detection The prebuilt language detection model determines which language is used most frequently in a written document. The model examines the text and provides a number score between 0 and 1 along with the language that was found. Scores near 1 suggest a higher degree of confidence in the outcome. The language's "script" is what is returned when a language is discovered. For instance, it will return "en" rather than "en-US" for the phrase "I have a dog." Unknown is the outcome for undetectable languages.
Receipt processing Modern optical character recognition (OCR) technology is used in the prebuilt model of receipt processing to recognize printed and handwritten text and extract important data from receipts.
Sentiment analysis The prebuilt sentiment analysis model determines whether text data has a positive or negative sentiment. It can be used to examine social media, client evaluations, or any other text data that captures the attention your interest. Sentiment analysis rates and labels the text input at the sentence and document levels after evaluating it. The ratings and labelling may be favorable, unfavorable, or neutral. A "mixed" sentiment label with no score can likewise exist at the document level. By adding up the sentence ratings, the sentiment of the paper is calculated.
Text recognition The text recognition prebuilt model converts character streams into machine-readable textboxes from documents and images. Modern optical character recognition (OCR) technology is used to find printed and handwritten text in photographs. Lines of printed or handwritten text can be extracted using this technique by processing photos and document files.
Text translation Your text data is translated in real time across more than 60 languages by the text translation prebuilt model. Language barriers inside your firm might be eliminated with the aid of this prebuilt model. The language of the text data you want to translate is also detectable by the text translation model.