Extraction Model Overview
Extraction models enable structured data extraction based on fields, which can be configured for any document type.
What is an Extraction Model?
An Extraction Model in the Mindee platform is a type of model designed to extract structured data from documents. Extraction models process textual information using Optical Character Recognition (OCR) and process document structure using visual information (object detection).
This dual approach allows extracting textual and visual elements at the same time.
Each model defines a set of fields called a Data Schema Overview. Fields could include "Supplier Name," "Invoice Number," or "Total Amount" for an Invoice Model, as an example. The system will then identify and extract data from uploaded files according to these field definitions.
Extraction models can be used for any type of document including: invoices, receipts, passports, ID cards, financial statements, etc.
You can even extract data from objects having textual information: fuel dispensers, water gauges, odometers, electricity meters, etc.
Models allow you to automate the process of turning unstructured documents into actionable, structured data. They can be tailored to different document types and business needs, ensuring that only relevant information is captured for your workflows.
Each Extraction Model contains these dedicated tools and features:
Data Schema Overview: a model is also defined by a Data Schema, that sets the list of fields the API should extract for a given type of documents.
Continuous Learning (RAG): it allows the user to give additional instruction on some examples with unexpected behavior to durably improve the extraction performance of the model.
Model Settings: overall settings of the model such as processing zone, storage policy, ownership transfer, etc.
Create an Extraction Model
From the Mindee Platform main page, click on "Create your document AI model".
Create From the Model Catalog
Chances are the document you're trying to automate is a known type of document, and likely to be already present in our Model Catalog: Invoice, Receipt, Passport, Financial Document, ID Card... These are model templates with Data Schemas created by our Data Science team, allowing you to get started quickly by using a set of predefined fields.
You can search our catalog for suitable templates, then simply click on the one that best fits your needs. This will create a new model in your organization's account with its own model ID, allowing you to use and modify it.
Create a Custom Model
If none of the catalog models fit your needs, you can start from scratch. When creating a new Model, click on "Custom document".
Next, enter the required information to create the custom model:
Describe as precisely as possible the document(s) the model will process.
and / or
Upload up to 10 sample files representative of documents to process.
We highly recommend entering the document description and uploading at least 3 files.
With this approach the AI agent will be able to propose 3 different document scopes:
narrow, precisely matching a specific document, i.e. "US Driver License"
balanced, allowing some variance in documents, i.e. "US Photo ID"
wide, handling varied document types, i.e. "International ID"
Choose the one most representative of your use case.
As a general rule:
a narrower scope performs better on matching documents, at the expense of poorer performance on outlying documents.
a wider scope will be able to handle many different document types at the expense of lower precision on specific data.
The classic "expert versus generalist" conundrum!
When you need the extra precision of a narrow scope and the ability to handle multiple document types, use the chaining feature of a Classification model.
Creating the model will also generate the model's unique ID.
Modifying Your Extraction Data Schema
Once you have created a model, you'll likely want to modify its Data Schema to better suit your needs.
Even if the model was created from a model template in the Catalog, it is unique to your account and fully customizable.
Navigate to the Data Schema page where you can adjust fields, update configurations, and customize settings according to your requirements.
Using the AI Assistant
The Mindee AI Assistant is a powerful tool that can help you get the most of your model. It is the preferred method of modifying the Data Schema.
The Assistant is available in a dialog box in the model's Data Schema page.
Be as accurate as possible with field names, exact names and values should be in quotation marks. Example sentences to modify your data schema with the AI Assistant:
Add a new field: "document id" ⇒ Will create a new text field.
Add a new field: "is past due" ⇒ Will create a new boolean field Hint: boolean field names should start with "is" or "has".
Rename the "date" field to "invoice date"
Change the "document_type" field to a classification field. The expected classes are "INVOICE" and "RECEIPT"
Optimizing Model Results
In order to optimize the accuracy of a given model, the first step is to fine-tune the Data Schema.
In particular, start by asking the Mindee AI Assistant to automatically optimize your Data Schema.
If you started from a template in the catalog, it is usually necessary to adapt it to your specific documents and use case.
After this step, if there are still some lingering issues or unexpected behavior, there are further refinements available.
Problems With Specific Templates
When the overall accuracy is good, but there are problems on specific document templates.
For this you'll want to activate Continuous Learning (RAG). This allows adding guidance to specific templates, meaning you can target problem documents while leaving others unaffected.
High Accuracy Required
When you need very high precision for all the documents that you process.
Consider enabling Confidence Score and Accuracy Boost. This uses multiple models for higher accuracy and flags problematic fields.
Flagged fields can be sent for human review or the document can be rejected entirely, based on business rules defined on your side.
End User Review
Mindee provides the API tooling for the creation of sophisticated document review interfaces in your applications.
For example, allowing your end users to review and correct the data of documents they uploaded on your platform.
Enable Polygons (Bounding Boxes) so that the location of the extracted fields can be shown (this is always activated in the Live Test).
Combine this with confidence scores, to indicate directly in your user interface which fields require particular attention.
With both these features, it will be easy for your users to find erroneous fields and correct them.
Combined Benefits
These features work together to create a system that becomes more accurate as you use it, reducing manual corrections and improving automation success rates.
Last updated
Was this helpful?