# Nutrition Facts Data OCR

Mindee’s Nutrition Facts Data API uses deep learning to automatically, accurately, and instantaneously parse your documents details. In a few seconds, the API extracts a set of data from your PDFs or photos of Nutrition Facts, including:

* Serving per Box
* Serving Size
* Calories
* Total Fat
* Saturated Fat
* Trans Fat
* Cholesterol
* Total Carbohydrate
* Dietary Fiber
* Total Sugars
* Added Sugar
* Protein
* Sodium
* Nutrients

## Set up the API

{% hint style="info" %}
**Create an API key**

To begin using the Mindee V1 OCR API, your first step is to [create your V1 API key](https://docs.mindee.com/v1/get-started/create-api-key).
{% endhint %}

1. You'll need a Nutrition Facts document. You can use one of the sample documents provided below.

   <figure><img src="https://126655343-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2al1MDqAP9Dg9iDRjkWg%2Fuploads%2Fgit-blob-aa9b88d94bf252e9e9d596755c6c589d3b4fcee8%2Fb9da03c2747df3f19d3f5a24c097621ec5d1305b5427c3d40692645f5a0aea0d-image.png?alt=media" alt=""><figcaption></figcaption></figure>
2. Access your Nutrition Facts Data API by clicking on the corresponding product card in the **Document Catalog**

   <figure><img src="https://126655343-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2al1MDqAP9Dg9iDRjkWg%2Fuploads%2Fgit-blob-13eb0621dbdbcaa7f2c9038dfb357bb1f44b0e0d%2F2ef03ee2574bd15e420a2434659bb2f31f82dd968bf9ad9f83cfcada8a2c4f07-image.png?alt=media" alt=""><figcaption></figcaption></figure>
3. From the left navigation, go to [**documentation**](https://docs.mindee.com/v1/platform-ui-tour/api-documentation#api-reference) **> API Reference**, you'll find sample code in popular languages and command line.

{% tabs %}
{% tab title="Python" %}

```python
from mindee import Client, product, AsyncPredictResponse

# Init a new client
mindee_client = Client(api_key="my-api-key-here")

# Load a file from disk
input_doc = mindee_client.source_from_path("/path/to/the/file.ext")

# Load a file from disk and enqueue it.
result: AsyncPredictResponse = mindee_client.enqueue_and_parse(
    product.NutritionFactsLabelV1,
    input_doc,
)

# Print a brief summary of the parsed data
print(result.document)
```

{% endtab %}

{% tab title="Node.js" %}

```javascript
const mindee = require("mindee");
// for TS or modules:
// import * as mindee from "mindee";

// Init a new client
const mindeeClient = new mindee.v1.Client({ apiKey: "my-api-key-here" });

// Load a file from disk
const inputSource = mindeeClient.docFromPath("/path/to/the/file.ext");

// Parse the file
const apiResponse = mindeeClient.enqueueAndParse(
  mindee.v1.product.NutritionFactsLabelV1,
  inputSource
);

// Handle the response Promise
apiResponse.then((resp) => {
  // print a string summary
  console.log(resp.document.toString());
});
```

{% endtab %}

{% tab title=".NET" %}

```csharp
using Mindee;
using Mindee.Input;
using Mindee.Product.NutritionFactsLabel;

string apiKey = "my-api-key-here";
string filePath = "/path/to/the/file.ext";

// Construct a new client
MindeeClient mindeeClient = new MindeeClient(apiKey);

// Load an input source as a path string
// Other input types can be used, as mentioned in the docs
var inputSource = new LocalInputSource(filePath);

// Call the product asynchronously with auto-polling
var response = await mindeeClient
    .EnqueueAndParseAsync<NutritionFactsLabelV1>(inputSource);

// Print a summary of all the predictions
System.Console.WriteLine(response.Document.ToString());

// Print only the document-level predictions
// System.Console.WriteLine(response.Document.Inference.Prediction.ToString());
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
#
# Install the Ruby client library by running:
# gem install mindee
#

require 'mindee'

# Init a new client
mindee_client = Mindee::Client.new(api_key: 'my-api-key')

# Load a file from disk
input_source = mindee_client.source_from_path('/path/to/the/file.ext')

# Parse the file
result = mindee_client.parse(
  input_source,
  Mindee::Product::NutritionFactsLabel::NutritionFactsLabelV1
)

# Print a full summary of the parsed data in RST format
puts result.document

# Print the document-level parsed data
# puts result.document.inference.prediction
```

{% endtab %}

{% tab title="Java" %}

{% endtab %}

{% tab title="undefined" %}

```java
import com.mindee.MindeeClient;
import com.mindee.input.LocalInputSource;
import com.mindee.parsing.common.AsyncPredictResponse;
import com.mindee.product.nutritionfactslabel.NutritionFactsLabelV1;
import java.io.File;
import java.io.IOException;

public class SimpleMindeeClient {

  public static void main(String[] args) throws IOException, InterruptedException {
    String apiKey = "my-api-key-here";
    String filePath = "/path/to/the/file.ext";

    // Init a new client
    MindeeClient mindeeClient = new MindeeClient(apiKey);

    // Load a file from disk
    LocalInputSource inputSource = new LocalInputSource(new File(filePath));

    // Parse the file asynchronously
    AsyncPredictResponse<NutritionFactsLabelV1> response = mindeeClient.enqueueAndParse(
        NutritionFactsLabelV1.class,
        inputSource
    );

    // Print a summary of the response
    System.out.println(response.toString());

    // Print a summary of the predictions
//  System.out.println(response.getDocumentObj().toString());

    // Print the document-level predictions
//    System.out.println(response.getDocumentObj().getInference().getPrediction().toString());

    // Print the page-level predictions
//    response.getDocumentObj().getInference().getPages().forEach(
//        page -> System.out.println(page.toString())
//    );
  }

}
```

{% endtab %}

{% tab title="Bash" %}

```bash
API_KEY='my-api-key-here'
ACCOUNT='mindee'
ENDPOINT='nutrition_facts'
VERSION='1'
FILE_PATH='/path/to/your/file.png'

# Maximum amount of retries to get the result of a queue
MAX_RETRIES=10

# Delay between requests
DELAY=6

# Enqueue the document for async parsing
QUEUE_RESULT=$(curl -sS --request POST \
  -H "Authorization: Token $API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F "document=@$FILE_PATH" \
  "https://api.mindee.net/v1/products/$ACCOUNT/$ENDPOINT/v$VERSION/predict_async")

# Status code sent back from the server
STATUS_CODE=$(echo "$QUEUE_RESULT" | grep -oP "[\"|']status_code[\"|']:[\s][\"|']*[a-zA-Z0-9-]*" | rev | cut --complement -f2- -d" " | rev)

# Check that the document was properly queued
if [ -z "$STATUS_CODE" ] || [ "$STATUS_CODE" -gt 399 ] || [ "$STATUS_CODE" -lt 200 ]
then
  if [ -z "$STATUS_CODE" ]
  then
    echo "Request couldn't be processed."
    exit 1
  fi
  echo "Error $STATUS_CODE was returned by API during enqueuing. "

  # Print the additional details, if there are any:
  ERROR=$(echo "$QUEUE_RESULT" | grep -oP "[\"|']error[\"|']:[\s]\{[^\}]*" | rev | cut --complement -f2- -d"{" | rev)
  if [ -z "$ERROR" ]
  then
    exit 1
  fi

  # Details on the potential error:
  ERROR_CODE=$(echo "$ERROR" | grep -oP "[\"|']code[\"|']:[\s]\"[^(\"|\')]*" | rev | cut --complement -f2- -d"\"" | rev)
  MESSAGE=$(echo "$QUEUE_RESULT" | grep -oP "[\"|']message[\"|']:[\s]\"[^(\"|\')]*" | rev | cut --complement -f2- -d"\"" | rev)
  DETAILS=$(echo "$QUEUE_RESULT" | grep -oP "[\"|']details[\"|']:[\s]\"[^(\"|\')]*" | rev | cut --complement -f2- -d"\"" | rev)
  echo "This was the given explanation:"
  echo "-------------------------"
  echo "Error Code: $ERROR_CODE"
  echo "Message: $MESSAGE"
  echo "Details: $DETAILS"
  echo "-------------------------"
  exit 1
else

  echo "File sent, starting to retrieve from server..."

  # Get the document's queue ID
  QUEUE_ID=$(echo "$QUEUE_RESULT" | grep -oP "[\"|']id[\"|']:[\s][\"|'][a-zA-Z0-9-]*" | rev | cut --complement -f2- -d"\"" | rev)

  # Amount of attempts to retrieve the parsed document were made
  TIMES_TRIED=1

  # Try to fetch the file until we get it, or until we hit the maximum amount of retries
  while [ "$TIMES_TRIED" -lt "$MAX_RETRIES" ]
  do
    # Wait for a bit at each step
    sleep $DELAY

    # Note: we use -L here because the location of the file might be behind a redirection
    PARSED_RESULT=$(curl -sS -L \
      -H "Authorization: Token $API_KEY" \
      "https://api.mindee.net/v1/products/$ACCOUNT/$ENDPOINT/v$VERSION/documents/queue/$QUEUE_ID")

    # Isolating the job (queue) & the status to monitor the document
    JOB=$(echo "$PARSED_RESULT" | grep -ioP "[\"|']job[\"|']:[\s]\{[^\}]*" | rev | cut --complement -f2- -d"{" | rev)
    QUEUE_STATUS=$(echo "$JOB" | grep -ioP "[\"|']status[\"|']:[\s][\"|'][a-zA-Z0-9-]*" | rev | cut --complement -f2- -d"\"" | rev)
    if [ "$QUEUE_STATUS" = "completed" ]
    then
      # Print the result
      echo "$PARSED_RESULT"

      # Optional: isolate the document:
      # DOCUMENT=$(echo "$PARSED_RESULT" | grep -ioP "[\"|']document[\"|']:[\s].*([\"|']job[\"|'])" | rev | cut -f2- -d"," | rev)
      # echo "{$DOCUMENT}"

      # Remark: on compatible shells, fields can also be extracted through the use of tools like jq:
      # DOCUMENT=$(echo "$PARSED_RESULT" | jq '.["document"]')
      exit 0
    fi
    TIMES_TRIED=$((TIMES_TRIED+1))
  done
fi

echo "Operation aborted, document not retrieved after $TIMES_TRIED tries"
exit 1
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php

use Mindee\Client;
use Mindee\Product\NutritionFactsLabel\NutritionFactsLabelV1;

// Init a new client
$mindeeClient = new Client("my-api-key-here");

// Load a file from disk
$inputSource = $mindeeClient->sourceFromPath("/path/to/the/file.ext");

// Parse the file asynchronously
$apiResponse = $mindeeClient->enqueueAndParse(NutritionFactsLabelV1::class, $inputSource);

echo $apiResponse->document;
```

{% endtab %}
{% endtabs %}

* Replace **my-api-key-here** with your new API key, or use the **select an API key** feature and it will be filled automatically.
* Copy and paste the sample code of your desired choice in your application, code environment, terminal etc.
* Replace `/path/to/the/file.ext` with the path to your input document.

{% hint style="warning" %}
Remember to replace with your V1 API key.
{% endhint %}

4. Run your code. You will receive a JSON response with the Nutrition Facts Data details.

## API Response

Here is the full JSON response you get when you call the API:

```json
{
  "api_request": {
    "error": {},
    "resources": [
      "document",
      "job"
    ],
    "status": "success",
    "status_code": 200,
    "url": "https://api.mindee.net/v1/products/mindee/nutrition_facts/v1/documents/4205474c-cabe-4732-918c-03d3d5eac15f"
  },
  "document": {
    "id": "4205474c-cabe-4732-918c-03d3d5eac15f",
    "inference": {
      "extras": {},
      "finished_at": "2024-10-21T12:20:44.224000",
      "is_rotation_applied": true,
      "pages": [
        {
          "extras": {},
          "id": 0,
          "orientation": {
            "value": 0
          },
          "prediction": {}
        }
      ],
      "prediction": {
        "added_sugars": {
          "daily_value": 20,
          "per_100g": null,
          "per_serving": 10
        },
        "calories": {
          "daily_value": null,
          "per_100g": null,
          "per_serving": 230
        },
        "cholesterol": {
          "daily_value": 0,
          "per_100g": null,
          "per_serving": 0
        },
        "dietary_fiber": {
          "daily_value": 14,
          "per_100g": null,
          "per_serving": 4
        },
        "nutrients": [
          {
            "daily_value": 10,
            "name": "Vitamin D",
            "per_100g": null,
            "per_serving": 2,
            "unit": "mcg"
          },
          {
            "daily_value": 20,
            "name": "Calcium",
            "per_100g": null,
            "per_serving": 260,
            "unit": "mg"
          },
          {
            "daily_value": 45,
            "name": "Iron",
            "per_100g": null,
            "per_serving": 8,
            "unit": "mg"
          },
          {
            "daily_value": 6,
            "name": "Potassium",
            "per_100g": null,
            "per_serving": 240,
            "unit": "mg"
          }
        ],
        "protein": {
          "daily_value": null,
          "per_100g": null,
          "per_serving": 3
        },
        "saturated_fat": {
          "daily_value": 5,
          "per_100g": null,
          "per_serving": 1
        },
        "serving_per_box": {
          "value": 8
        },
        "serving_size": {
          "amount": 55,
          "unit": "g"
        },
        "sodium": {
          "daily_value": 7,
          "per_100g": null,
          "per_serving": 160,
          "unit": "mg"
        },
        "total_carbohydrate": {
          "daily_value": 13,
          "per_100g": null,
          "per_serving": 37
        },
        "total_fat": {
          "daily_value": 10,
          "per_100g": null,
          "per_serving": 8
        },
        "total_sugars": {
          "daily_value": null,
          "per_100g": null,
          "per_serving": 12
        },
        "trans_fat": {
          "daily_value": null,
          "per_100g": null,
          "per_serving": 0
        }
      },
      "processing_time": 11.178,
      "product": {
        "features": [
          "serving_per_box",
          "serving_size",
          "calories",
          "total_fat",
          "saturated_fat",
          "trans_fat",
          "cholesterol",
          "total_carbohydrate",
          "dietary_fiber",
          "total_sugars",
          "added_sugars",
          "protein",
          "sodium",
          "nutrients"
        ],
        "name": "mindee/nutrition_facts",
        "type": "standard",
        "version": "1.0"
      },
      "started_at": "2024-10-21T12:20:32.861000"
    },
    "n_pages": 1,
    "name": "nutrition-facts-sample.jpg"
  },
  "job": {
    "available_at": "2024-10-21T12:20:44.245000",
    "error": {},
    "id": "4058bb0f-2dd6-461b-80b2-5a0cbc96d297",
    "issued_at": "2024-10-21T12:20:32.861000",
    "status": "completed"
  }
}
```

You can find the prediction within the `prediction` key found in **`document > inference > prediction` for document-level predictions**: it contains the different fields extracted at the document level, meaning that for multi-pages PDFs, we reconstruct a single receipt object using all the pages.

```json
{
  "document": {
    "id": "4205474c-cabe-4732-918c-03d3d5eac15f",
    "inference": {
      "extras": {},
      "finished_at": "2024-10-21T12:20:44.224000",
      "is_rotation_applied": true,
      "pages": [
        {
          "extras": {},
          "id": 0,
          "orientation": {
            "value": 0
          },
          "prediction": {}
        }
      ],
      "prediction": {
        "added_sugars": {
          "daily_value": 20,
          "per_100g": null,
          "per_serving": 10
        },
        "calories": {
          "daily_value": null,
          "per_100g": null,
          "per_serving": 230
        },
        "cholesterol": {
          "daily_value": 0,
          "per_100g": null,
          "per_serving": 0
        },
        "dietary_fiber": {
          "daily_value": 14,
          "per_100g": null,
          "per_serving": 4
        },
        "nutrients": [
          {
            "daily_value": 10,
            "name": "Vitamin D",
            "per_100g": null,
            "per_serving": 2,
            "unit": "mcg"
          },
          {
            "daily_value": 20,
            "name": "Calcium",
            "per_100g": null,
            "per_serving": 260,
            "unit": "mg"
          },
          {
            "daily_value": 45,
            "name": "Iron",
            "per_100g": null,
            "per_serving": 8,
            "unit": "mg"
          },
          {
            "daily_value": 6,
            "name": "Potassium",
            "per_100g": null,
            "per_serving": 240,
            "unit": "mg"
          }
        ],
        "protein": {
          "daily_value": null,
          "per_100g": null,
          "per_serving": 3
        },
        "saturated_fat": {
          "daily_value": 5,
          "per_100g": null,
          "per_serving": 1
        },
        "serving_per_box": {
          "value": 8
        },
        "serving_size": {
          "amount": 55,
          "unit": "g"
        },
        "sodium": {
          "daily_value": 7,
          "per_100g": null,
          "per_serving": 160,
          "unit": "mg"
        },
        "total_carbohydrate": {
          "daily_value": 13,
          "per_100g": null,
          "per_serving": 37
        },
        "total_fat": {
          "daily_value": 10,
          "per_100g": null,
          "per_serving": 8
        },
        "total_sugars": {
          "daily_value": null,
          "per_100g": null,
          "per_serving": 12
        },
        "trans_fat": {
          "daily_value": null,
          "per_100g": null,
          "per_serving": 0
        }
      }
    }
  }
}
```

## Detailed Field Information

Using the above Nutrition Facts Data example the following are the basic fields that can be extracted.

* [Serving per Box](#serving-per-box)
* [Serving Size](#serving-size)
* [Calories](https://github.com/mindee/legacy-docs/blob/main/off-the-shelf-products/calories/README.md)
* [Total Fat](#total-fat)
* [Saturated Fat](#staurated-fat)
* [Trans Fat](#trans-fat)
* [Cholesterol](#cholesterol)
* [Total Carbohydrate](#total-carbohydrate)
* [Dietary Fiber](#dietary-fiber)
* [Total Sugars](#total-sugars)
* [Added Sugars](#added-sugars)
* [Protein](#protein)
* [Sodium](https://github.com/mindee/legacy-docs/blob/main/off-the-shelf-products/sodium/README.md)
* [Nutrients](#nutrients)

### Serving per Box

* **serving\_per\_box**: The number of servings in each box of the product.

```json
{
  "serving_per_box": {
    "value": 8
  }
}
```

### Serving Size

* **serving\_size**: The size of a single serving of the product.
  * **amount**: The amount of a single serving.
  * **unit**: The unit for the amount of a single serving.

```json
{
  "serving_size": {
    "amount": 55,
    "unit": "g"
  }
}
```

### Calories

* **calories**: The amount of calories in the product.
  * **per\_100g**: The amount of calories per 100g of the product.
  * **per\_serving**: The amount of calories per serving of the product.
  * **daily\_value**: DVs are the recommended amounts of calories to consume or not to exceed per day..

```json
{
  "calories": {
    "daily_value": null,
    "per_100g": null,
    "per_serving": 230
  }
}
```

### Total Fat

* **total\_fat**: The amount of total fat in the product.
  * **per\_100g**: The amount of total fat per 100g of the product.
  * **per\_serving**: The amount of total fat per serving of the product.
  * **daily\_value**: DVs are the recommended amounts of total fat to consume or not to exceed per day..

```json
{
  "total_fat": {
    "daily_value": 10,
    "per_100g": null,
    "per_serving": 8
  }
}
```

### Saturated Fat

* **saturated\_fat**: The amount of saturated fat in the product.
  * **per\_100g**: The amount of saturated fat per 100g of the product.
  * **per\_serving**: The amount of saturated fat per serving of the product.
  * **daily\_value**: DVs are the recommended amounts of saturated fat to consume or not to exceed per day..

```json
{
  "saturated_fat": {
    "daily_value": 5,
    "per_100g": null,
    "per_serving": 1
  }
}
```

### Trans Fat

* **trans\_fat**: The amount of trans fat in the product.
  * **per\_100g**: The amount of trans fat per 100g of the product.
  * **per\_serving**: The amount of trans fat per serving of the product.
  * **daily\_value**: DVs are the recommended amounts of trans fat to consume or not to exceed per day..

```json
{
  "trans_fat": {
    "daily_value": null,
    "per_100g": null,
    "per_serving": 0
  }
}
```

### Cholesterol

* **cholesterol**: The amount of trans fat in the product.
  * **per\_100g**: The amount of cholesterol per 100g of the product.
  * **per\_serving**: The amount of cholesterol per serving of the product.
  * **daily\_value**: DVs are the recommended amounts of cholesterol to consume or not to exceed per day..

```json
{
  "cholesterol": {
    "daily_value": 0,
    "per_100g": null,
    "per_serving": 0
  }
}
```

### Total Carbohydrate

* **total\_carbohydrate**: The amount of total carbohydrate in the product.
  * **per\_100g**: The amount of total carbohydrate per 100g of the product.
  * **per\_serving**: The amount of total carbohydrate per serving of the product.
  * **daily\_value**: DVs are the recommended amounts of total carbohydrate to consume or not to exceed per day..

```json
{
  "total_carbohydrate": {
    "daily_value": 13,
    "per_100g": null,
    "per_serving": 37
  }
}
```

### Dietary Fiber

* **dietary\_fiber**: The amount of total dietary fiber in the product.
  * **p**

```json
{
  "dietary_fiber": {
    "daily_value": 14,
    "per_100g": null,
    "per_serving": 4
  }
}
```

### Total Sugars

* **total\_sugars**: The amount of total sugars in the product.
  * **per\_100g**: The amount of total sugars per 100g of the product.
  * **per\_serving**: The amount of total sugars per serving of the product.
  * **daily\_value**: DVs are the recommended amounts of total sugars to consume or not to exceed per day..

```json
{
  "total_sugars": {
    "daily_value": null,
    "per_100g": null,
    "per_serving": 12
  }
}
```

### Added Sugars

* **added\_sugars**: The amount of added sugars in the product.
  * **per\_100g**: The amount of added sugars per 100g of the product.
  * **per\_serving**: The amount of added sugars per serving of the product.
  * **daily\_value**: DVs are the recommended amounts of total sugars to consume or not to exceed per day..

```json
{
  "added_sugars": {
    "daily_value": null,
    "per_100g": null,
    "per_serving": 12
  }
}
```

### Protein

* **protein**: The amount of protein in the product.
  * **per\_100g**: The amount of protein per 100g of the product.
  * **per\_serving**: The amount of protein per serving of the product.
  * **daily\_value**: DVs are the recommended amounts of protein to consume or not to exceed per day..

```json
{
  "added_sugars": {
    "daily_value": null,
    "per_100g": null,
    "per_serving": 12
  }
}
```

### Sodium

* **sodium**: The amount of sodium in the product.
  * **per\_100g**: The amount of sodium per 100g of the product.
  * **per\_serving**: The amount of sodium per serving of the product.
  * **daily\_value**: DVs are the recommended amounts of sodium to consume or not to exceed per day..
  * **unit**: The unit of measurement for the amount of sodium.

```json
{
  "sodium": {
    "daily_value": 7,
    "per_100g": null,
    "per_serving": 160,
    "unit": "mg"
  }
}
```

### Nutrients

* **nutrients**: The amount of nutrients in the product.
  * **name**: The name of the nutrients of the product.
  * **per\_100g**: The amount of sodium per 100g of the product.
  * **per\_serving**: The amount of sodium per serving of the product.
  * **unit**: The unit of measurement for the amount of nutrients.
  * **daily\_value**: DVs are the recommended amounts of sodium to consume or not to exceed per day..

```json
{
  "nutrients": [
    {
      "daily_value": 10,
      "name": "Vitamin D",
      "per_100g": null,
      "per_serving": 2,
      "unit": "mcg"
    },
    {
      "daily_value": 20,
      "name": "Calcium",
      "per_100g": null,
      "per_serving": 260,
      "unit": "mg"
    },
    {
      "daily_value": 45,
      "name": "Iron",
      "per_100g": null,
      "per_serving": 8,
      "unit": "mg"
    },
    {
      "daily_value": 6,
      "name": "Potassium",
      "per_100g": null,
      "per_serving": 240,
      "unit": "mg"
    }
  ]
}
```