# Multi Receipts Detector OCR

Mindee’s Multi Receipts Detector OCR API uses deep learning to automatically, accurately, and instantaneously parse your receipt coordinates. In under a second, the API extracts a set of data from your PDFs or photos of any document showing multiple receipts, including:

* Receipt Position by Bounding Box

It can be easily combined with Mindee's Receipts and Financial Documents OCR API to provide an end to end workflow from detecting to extracting key information on receipts.

## Set up the API

{% hint style="info" %}
**Create an API key**

To begin using the Mindee V1 OCR API, your first step is to [create your V1 API key](https://docs.mindee.com/v1/get-started/create-api-key).
{% endhint %}

1. You'll need a document including one or multiple receipts. You can use the sample document provided below.

<figure><img src="https://126655343-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2al1MDqAP9Dg9iDRjkWg%2Fuploads%2Fgit-blob-d33e3e08c9f5d98d8eeae4d46e312e8940c00408%2F1dd39d3-x_receipts.jpg?alt=media" alt=""><figcaption></figcaption></figure>

2. Access your API by clicking on the **Multi Receipts Detector** card in the **Utilities**

<figure><img src="https://126655343-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2al1MDqAP9Dg9iDRjkWg%2Fuploads%2Fgit-blob-9c412d49685b5e6e47b925dd7fa7bd57bbc56a29%2Fc73e284-multi-receipts-doc.png?alt=media" alt=""><figcaption></figcaption></figure>

3. From the left navigation, go to [**documentation**](doc:platform-tour#api---documentation) **> API Reference**, you'll find sample code in popular languages and command line.

{% tabs %}
{% tab title="Python" %}

```python
from mindee import Client, PredictResponse, product

# Init a new client
mindee_client = Client(api_key="my-api-key")

# Load a file from disk
input_doc = mindee_client.source_from_path("/path/to/the/file.ext")

# Load a file from disk and parse it.
result: PredictResponse = mindee_client.parse(
    product.MultiReceiptsDetectorV1,
    input_doc,
)

# Print a summary of the API result
print(result.document)

# Print the document-level summary
# print(result.document.inference.prediction)
```

{% endtab %}

{% tab title="Node.js" %}

```javascript
const mindee = require("mindee");
// for TS or modules:
// import * as mindee from "mindee";

// Init a new client
const mindeeClient = new mindee.Client({ apiKey: "my-api-key" });

// Load a file from disk
const inputSource = mindeeClient.docFromPath("/path/to/the/file.ext");

// Parse the file
const apiResponse = mindeeClient.parse(
    mindee.product.MultiReceiptsDetectorV1,
    inputSource
);

// Handle the response Promise
apiResponse.then((resp) => {
    // print a string summary
    console.log(resp.document.toString());
});
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php

use Mindee\Client;
use Mindee\Product\MultiReceiptsDetector\MultiReceiptsDetectorV1;

// Init a new client
$mindeeClient = new Client("my-api-key");

// Load a file from disk
$inputSource = $mindeeClient->sourceFromPath("/path/to/the/file.ext");

// Parse the file
$apiResponse = $mindeeClient->parse(MultiReceiptsDetectorV1::class, $inputSource);

echo $apiResponse->document;
```

{% endtab %}

{% tab title=".NET" %}

```csharp
using Mindee;
using Mindee.Input;
using Mindee.Http;
using Mindee.Parsing;

string apiKey = "my-api-key-here";
string filePath = "/path/to/the/file.ext";

// Construct a new client
MindeeClient mindeeClient = new MindeeClient(apiKey);

// Load an input source as a path string
// Other input types can be used, as mentioned in the docs
var inputSource = new LocalInputSource(filePath);

// Set the endpoint configuration
CustomEndpoint myEndpoint = new CustomEndpoint(
    endpointName: "multi_receipts_detector",
    accountName: "mindee",
    version: "1.0"
);

var response = await mindeeClient
    .ParseAsync(inputSource, myEndpoint);

// Print a summary of the predictions
System.Console.WriteLine(response.Document.ToString());

// Print the document-level predictions
// System.Console.WriteLine(response.Document.Inference.Prediction.ToString());
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
require 'mindee'

# Init a new client
mindee_client = Mindee::Client.new(api_key: 'my-api-key')

# Load a file from disk
input_source = mindee_client.source_from_path('/path/to/the/file.ext')

# Parse the file
result = mindee_client.parse(
  input_source,
  Mindee::Product::MultiReceiptsDetector::MultiReceiptsDetectorV1
)

# Print a full summary of the parsed data in RST format
puts result.document

# Print the document-level parsed data
# puts result.document.inference.prediction
```

{% endtab %}

{% tab title="Java" %}

```java
import com.mindee.MindeeClient;
import com.mindee.input.LocalInputSource;
import com.mindee.parsing.common.PredictResponse;
import com.mindee.product.custom.CustomV1;
import com.mindee.http.Endpoint;
import java.io.File;
import java.io.IOException;

public class SimpleMindeeClient {

  public static void main(String[] args) throws IOException {
    String apiKey = "my-api-key-here";
    String filePath = "/path/to/the/file.ext";
    
    // Init a new client
    MindeeClient mindeeClient = new MindeeClient(apiKey);

    // Load a file from disk
    LocalInputSource inputSource = new LocalInputSource(new File(filePath));

    // Configure the endpoint
    Endpoint endpoint = new Endpoint(
        "multi_receipts_detector",
        "mindee",
        "1.0"
    );
    
    // Parse the file
    PredictResponse<CustomV1> response =  mindeeClient.parse(
        inputSource,
        endpoint
    );

    // Print a summary of the response
    System.out.println(response.toString());
    
    // Print a summary of the predictions
//  System.out.println(response.getDocument().toString());

    // Print the document-level predictions
//    System.out.println(response.getDocument().getInference().getPrediction().toString());

    // Print the page-level predictions
//    response.getDocument().getInference().getPages().forEach(
//        page -> System.out.println(page.toString())
//    );
  }

}
```

{% endtab %}

{% tab title="Bash" %}

```bash
curl -X POST \\
  https://api.mindee.net/v1/products/mindee/multi_receipts_detector/v1/predict \\
  -H 'Authorization: Token my-api-key-here' \\
  -H 'content-type: multipart/form-data' \\
  -F document=@/path/to/your/file.png
```

{% endtab %}
{% endtabs %}

* Replace **my-api-key-here** with your new API key, or use the **select an API key** feature and it will be filled automatically.
* Copy and paste the sample code of your desired choice in your application, code environment, terminal etc.
* Replace `/path/to/the/file.ext` with the path to your input document.

{% hint style="warning" %}
Remember to replace with your V1 API key.
{% endhint %}

## API Response

Here is the full JSON response you get when you call the API:

```json
{
  "api_request": {
    "error": {},
    "resources": [
      "document"
    ],
    "status": "success",
    "status_code": 201,
    "url": "https://api.mindee.net/v1/products/mindee/multi_receipts_detector/v1/predict"
  },
  "document": {
    "id": "c68d69b3-3855-40b6-8dae-f53fd6198800",
    "inference": {
      "extras": {},
      "finished_at": "2023-05-06T16:37:29",
      "pages": [
        {
          "extras": {},
          "id": 0,
          "orientation": {
            "value": 0
          },
          "prediction": {...}
        }
      ],
      "prediction": {...},
      "processing_time": 1.755,
      "started_at": "2023,-05-06T16:37:28"
    },
    "is_rotation_applied": true,
    "n_pages": 1,
    "name": "sample_receipt.jpg"
  }
}
```

You can find the prediction within the `prediction` key found in two locations:

* **In `document > inference > prediction` for document-level predictions**: it contains the different fields extracted at the document level, meaning that for multi-pages PDFs, we reconstruct a single *List of Receipts* object using all the pages.
* **In `document > inference > pages[ ] > prediction` for page-level predictions**: it gives the prediction for each page independently. With images, there is only one element on this array, but with PDFs, you can find the extracted data for each PDF page.

Each predicted field may contain one or several values:

* a `confidence` score
* a `bounding_box` highlighting the information location
* a `page_id` where the information was found (document level only)

```json
{
  "extras": {},
  "finished_at": "2023-08-30T09:04:43.856494",
  "is_rotation_applied": true,
  "pages": [
    {
      "extras": {},
      "id": 0,
      "orientation": {
        "value": 0
      },
      "prediction": {
        "orientation": {
          "confidence": 0.69,
          "degrees": 0
        },
        "receipts": [
          {
            "bounding_box": [
              [
                0.016,
                0.538
              ],
              [
                0.114,
                0.538
              ],
              [
                0.114,
                0.783
              ],
              [
                0.016,
                0.783
              ]
            ],
            "confidence": 1
          },
          {
            "bounding_box": [
              [
                0.156,
                0.389
              ],
              [
                0.284,
                0.389
              ],
              [
                0.284,
                0.843
              ],
              [
                0.156,
                0.843
              ]
            ],
            "confidence": 1
          },
          {
            "bounding_box": [
              [
                0.337,
                0.381
              ],
              [
                0.466,
                0.381
              ],
              [
                0.466,
                0.765
              ],
              [
                0.337,
                0.765
              ]
            ],
            "confidence": 1
          },
          {
            "bounding_box": [
              [
                0.491,
                0.296
              ],
              [
                0.617,
                0.296
              ],
              [
                0.617,
                0.861
              ],
              [
                0.491,
                0.861
              ]
            ],
            "confidence": 1
          },
          {
            "bounding_box": [
              [
                0.684,
                0.294
              ],
              [
                0.827,
                0.294
              ],
              [
                0.827,
                0.756
              ],
              [
                0.684,
                0.756
              ]
            ],
            "confidence": 1
          },
          {
            "bounding_box": [
              [
                0.87,
                0.435
              ],
              [
                0.971,
                0.435
              ],
              [
                0.971,
                0.723
              ],
              [
                0.87,
                0.723
              ]
            ],
            "confidence": 1
          }
        ]
      }
    }
  ],
  "prediction": {
    "receipts": [
      {
        "bounding_box": [
          [
            0.016,
            0.538
          ],
          [
            0.114,
            0.538
          ],
          [
            0.114,
            0.783
          ],
          [
            0.016,
            0.783
          ]
        ],
        "confidence": 1,
        "page_id": 0
      },
      {
        "bounding_box": [
          [
            0.156,
            0.389
          ],
          [
            0.284,
            0.389
          ],
          [
            0.284,
            0.843
          ],
          [
            0.156,
            0.843
          ]
        ],
        "confidence": 1,
        "page_id": 0
      },
      {
        "bounding_box": [
          [
            0.337,
            0.381
          ],
          [
            0.466,
            0.381
          ],
          [
            0.466,
            0.765
          ],
          [
            0.337,
            0.765
          ]
        ],
        "confidence": 1,
        "page_id": 0
      },
      {
        "bounding_box": [
          [
            0.491,
            0.296
          ],
          [
            0.617,
            0.296
          ],
          [
            0.617,
            0.861
          ],
          [
            0.491,
            0.861
          ]
        ],
        "confidence": 1,
        "page_id": 0
      },
      {
        "bounding_box": [
          [
            0.684,
            0.294
          ],
          [
            0.827,
            0.294
          ],
          [
            0.827,
            0.756
          ],
          [
            0.684,
            0.756
          ]
        ],
        "confidence": 1,
        "page_id": 0
      },
      {
        "bounding_box": [
          [
            0.87,
            0.435
          ],
          [
            0.971,
            0.435
          ],
          [
            0.971,
            0.723
          ],
          [
            0.87,
            0.723
          ]
        ],
        "confidence": 1,
        "page_id": 0
      }
    ]
  },
  "processing_time": 0.672,
  "product": {
    "features": [
      "receipts"
    ],
    "name": "mindee/multi_receipts_detector",
    "type": "standard",
    "version": "1.0"
  },
  "started_at": "2023-08-30T09:04:43.184661"
}
```

## Extracted data

Using the above example the following are the basic fields that can be extracted.

* [List of Receipts](#receipts)

### List of Receipts

* **receipts**: This field outputs the coordinates for each receipt detected.

```json
{
  "receipts": [
    {
      "bounding_box": [
        [
          0.016,
          0.538
        ],
        [
          0.114,
          0.538
        ],
        [
          0.114,
          0.783
        ],
        [
          0.016,
          0.783
        ]
      ],
      "confidence": 1,
      "page_id": 0
    },
    {
      "bounding_box": [
        [
          0.156,
          0.389
        ],
        [
          0.284,
          0.389
        ],
        [
          0.284,
          0.843
        ],
        [
          0.156,
          0.843
        ]
      ],
      "confidence": 1,
      "page_id": 0
    },
    {
      "bounding_box": [
        [
          0.337,
          0.381
        ],
        [
          0.466,
          0.381
        ],
        [
          0.466,
          0.765
        ],
        [
          0.337,
          0.765
        ]
      ],
      "confidence": 1,
      "page_id": 0
    },
    {
      "bounding_box": [
        [
          0.491,
          0.296
        ],
        [
          0.617,
          0.296
        ],
        [
          0.617,
          0.861
        ],
        [
          0.491,
          0.861
        ]
      ],
      "confidence": 1,
      "page_id": 0
    },
    {
      "bounding_box": [
        [
          0.684,
          0.294
        ],
        [
          0.827,
          0.294
        ],
        [
          0.827,
          0.756
        ],
        [
          0.684,
          0.756
        ]
      ],
      "confidence": 1,
      "page_id": 0
    },
    {
      "bounding_box": [
        [
          0.87,
          0.435
        ],
        [
          0.971,
          0.435
        ],
        [
          0.971,
          0.723
        ],
        [
          0.87,
          0.723
        ]
      ],
      "confidence": 1,
      "page_id": 0
    }
  ]
}
```