> For the complete documentation index, see [llms.txt](https://docs.mindee.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.mindee.com/extraction-models/optional-features/improving-accuracy.md). # Continuous Learning (RAG) Enhance extraction accuracy with Retrieval-Augmented Generation (RAG) using your own documents. ## Overview When using an Extraction Model, sometimes the result accuracy is not satisfying on a given document or a particular template. If you have already followed the [Data Schema Best Practices](/extraction-models/data-schema-best-practices.md), and results are still not satisfactory, all hope is not lost. You can reliably improve the performance of the model for the next inferences made by using the RAG feature. {% embed url="" %} ## Basics of RAG Inner Workings Retrieval-Augmented Generation or RAG, is an approach in artificial intelligence that combines the strengths of retrieval-based models with generative LLM text production. Essentially, RAG leverages a rich database of source documents and embeddings to augment the knowledge base from which it draws responses. This means that the system retrieves specific and context-based data snippets and then uses them to generate semantic and fine-tuned responses that are both creative and factually accurate. ### Overall Steps of RAG 1. Annotating a given sample, which means giving the model the difference between the extracted data and the expected value(s). When activated, this example will be added to the RAG Database for the future. 2. Retrieval: When a next document will be send with the RAG option activated, the model will try to search for a similar example in the existing database. The question here is : "Maybe there is an existing example where I need to follow the instructions so that I'm not doing a same mistake again". If no example found, no need to augment the prediction. If an examples is matched in the RAG Database, here comes step 3. 3. Augmented Generation: A document was matched in the RAG Database. The model will use the instructions you gave on the RAG sample to make a better prediction this time. The prediction generated is augmented with an existing context helping the model to be better this time.
RAG process overview
## Set Up the RAG Database To use RAG on a given Extraction Model, you'll need to first set up the RAG database which will contain your documents and annotation data. You only need a single document to get started. Access the database by clicking on the "Continuous Learning (RAG)" link in the model's configuration section.\ \ From there, click the "Add a document" button to upload files to the RAG database. Only add files that have problematic responses which could not be resolved by [improving the Data Schema](/extraction-models/data-schema-best-practices.md). When uploading the file, you can add free-form tags to help identify or classify the document. For example you could tag based on geography, provider, problem type, etc. You need to annotate the document, selecting the fields you want to be covered by the RAG augmentation on this template. You can also add additional guidelines using plain language. {% hint style="success" %} Most of the time, the annotation is sufficient to make the model understand the issue.\ We recommend using the guideline only when the annotation doesn't solve the problem. {% endhint %} Once this document is annotated, **be sure to validate it.** Once validated, the document will be active. From now on, files sent for processing that match the document will have their results augmented with the annotations in the RAG database. To test this, go to the [Live Test](/models/live-test.md) page. Ideally, find a document with the same template (another invoice from the same supplier for instance), but **not** the exact same one added to the RAG database.\ \ Upload the document to the Live Test, leaving "Show RAG extraction" checked. You will see the before/after predictions, this allows validating that the RAG annotations were taken into account when processing the document.\ \ For documents that do not match the annotated document, the behavior remain the same, which means that RAG is improving the result with no regression on other documents. To see which documents are being used, in the database the columns "Matched" and "Last Matched" indicate how many times the document was used and the last time it was used, respectively. ## Activate RAG For best results, be sure to follow the [Data Schema Best Practices](/extraction-models/data-schema-best-practices.md) before activating this feature. If the Data Schema is not optimized, this feature may not significantly improve model accuracy. ### Activate RAG on the Platform When setting the activation state of a feature on the Platform, this will be the default.\ All API calls will use the default state unless explicitly set otherwise during the API call. This is useful for project managers, as it allows activating or deactivating a feature across all API calls on that model. Anyone with write access to the model can set the option's default value. Click on "Data Schema" then "Optional Features" tab. There you can activate the "RAG Processing" feature. ### Activate RAG via API Calls If you need finer-grained control over when the feature is used, you can activate or deactivate it when making API calls. This allows dynamically setting the activation state using your internal business or domain logic. Check the [Client Configuration](/integrations/client-libraries-sdk/configure-the-client.md#optional-features-configuration) section if using our [Client Libraries / SDKs](/integrations/client-libraries-sdk.md). ## Searching for RAG Documents You can filter the RAG documents displayed in the interface, based on a variety of criteria: * tags * filename * upload date * status This is done through the "Filters" button at the top of the document list. ## Frequently Asked Questions
How is my data protected during RAG processing? Your data remains private and isolated within your model. All documents and annotations added to your RAG database are encrypted in transit.
Is my RAG data shared with other users or sold to 3rd parties? No, never. RAG data is only accessible to the users of the organization that the model belongs to. Mindee never sells data to 3rd parties. All storage and retrieval of RAG data is done on Mindee's dedicated servers.
Does Mindee use my RAG data for training? No, all documents are the property of their organization and are not used for training Mindee models. We only use RAG data internally with your explicit knowledge and prior consent. This is limited to using your documents to help resolve a ticket you opened with our support team.
What happens to my RAG database if I downgrade my plan? We won't delete any of the files you've uploaded into your RAG document database if you downgrade, however you will not be able to modify or validate them. The guidelines you have given to your RAG documents will be taken into account in your inferences with the lower plan, even if that is above the plan's features.
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.mindee.com/extraction-models/optional-features/improving-accuracy.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.