Document recognition

Algorithm of the API /recognize method

1. The algorithm looks for rectangular shapes on the incoming image that look like documents and cuts them out.

2. The Classifier assigns a class to each cut out area: Passport, Driver’s License, and so on.

3. The algorithm evaluates the orientation of the document in space. If necessary, the classifier rotates or mirrors the document.

4. The algorithm finds and cuts out information fields from the document. In a document like the passport, for example, this could be just the First Name, the Last Name, Passport Series, MRZ, etc.

5. The OCR algorithm reads and extracts information from the cut out information fields.

6. The OCR algorithm assigns a "confidence level" to the extracted information from each of the fields.

7. If manual OCR mode is enabled, the HITL feature processes the "cut out field + extracted text" pair.

8. The extracted text is verified with masks and dictionaries.

Confidence level

The confidence parameter in the answer shows the level of confidence of the algorithm in the correctness of character recognition:

• 0.90-1.00 - absolutely sure;

• 0.70-0.89 - quite sure;

• 0.50-0.69 - possible mistake in the answer;

• 0.01-0.49 - there must be an error in the field;

• 0 - there is definitely an error in the field.

The algorithm will return an empty answer with zero confidence if the digitized text does not pass the mask and dictionary checks. For example, the date of birth "56.12.1988" will not be returned as an answer.

The function compares the fields with an external file

This function compares the results of field detection with the text in your file. This is useful when you want to reconcile data from image documents with data from other sources. To use the function, optionally specify a JSON file in the verify_fields parameter.

An example JSON file for comparing series-number and full name from RF passport with recognition results is shown below:

{
  "series_and_number": "1111 222222",
  "surname": "Söze",
  "first_name": "Keyser",
  "other_names": "Roger"
}

To compose your JSON file, copy the field names from the API specification.

The reconciliation function returns a "valid" attribute for each field of the document. Available attribute values are:

• true - the field text in the JSON file and in the recognition results match;

• false - the text of the field does not match;

• null - the field does not exist in JSON-file.

In addition, the reconciliation function returns the levenshtein attribute - Levenshtein distance for the recognition result and the similar field from the external JSON file.

API specification

Below is the API specification for the document recognition method. See Connecting and testing for more information on how to put together a recognition request.

recognize

POST https://latest.handl.ai/recognize

Query Parameters

Name	Type	Description
external_check_fake	boolean	true - when using checks against external databases (external_check_...) no actual checking is performed, but a template response is returned (necessary for debugging) false - the check imitation is disabled
external_check_vehicle_dtp_and_restrict	boolean	true - search available Government Automobile databases for accidents involving the given vehicle and encumbrance by certificate of registration (vehicle_registration_certificate_front) false - the access to the database is disabled for this check
external_check_vehicle_wanted_list	boolean	true - search in the external databases if the vehicle is wanted by using the vehicle_registration_certificate_front or pts_front values. false - the access to the database is disabled for this check
external_check_vehicle_restrict_list	boolean	true - search in the external databases for the vehicle's encumbrance using the vehicle_registration_certificate_front or pts_front values. false - the access to the database for this check is disabled
external_check_fico	boolean	true - returns FICO financial scoring for an individual by passport (passport_main). false - this check is disabled for the database
external_check_pledges_list	boolean	true - searches external databases for outstanding debts of an individual using the given passport information (passport_main) false - database access is disabled for this check
external_check_inn	boolean	true - searches external databases to find the INN number using the given passport information (passport_main). false - reference to external databases by INN number is disabled
external_check_is_valid	boolean	true - checks document authenticity via available government databases. At the moment only passport authentication is available (doc_type="passport_main"). false - document authentication by databases is disabled.
doc_type	array	A list of document types to be recognized in the incoming file. It is used for deterministic processes, for example, if only the main passport spread needs to be processed in the flow, and all other types do not need to be answered.
priority	integer	>0, the default value is 1. Priority of asynchronous task in the queue for processing
simple_cropper	boolean	false (default) - the simplified algorithm of cutting documents from images is not used. true - the simplified algorithm of cutting documents from images is used: it is faster but less accurate. Documents in images with complex backgrounds may be cut out less accurately.
async	boolean	true - asynchronous mode of request processing. false - synchronous mode of processing requests.
check_fake_experimental	boolean	Out of date and not used.
check_fake	boolean	true - the algorithm looks for signs of modification in the file metadata via digital editors, the result is returned in a separate field called “fake”. false - the metadata checking algorithm is disabled.
use_internal_api	boolean	true - normalizes the "Issued by" field in the passport according to the embedded database. This allows you to get the text without misprints even from low-quality images, but may lead to inconsistency between the document and the result false — normalization is disabled
use_external_api	boolean	Out of date and not used.
pdf_raw_images	boolean	true - the algorithm leaves the decision of PDF files’ rasterization to the auto_pdf_raw_images parameter. false - all PDF files will be rasterized, the value of the auto_pdf_raw_images parameter will be ignored.
auto_pdf_raw_images	boolean	true - the algorithm looks for a text layer in PDF files. If it is found, the PDF will be rasterized. false - the algorithm will never rasterize PDF files.
dpi	integer	>0, , the default value is 300 - sets the number of pixels per inch for PDF rasterization. We recommend 300. Higher values usually do not increase the quality, but increase the weight of the image.
quality	integer	0-100, the default value is 75 - sets the degree of JPEG compression for PDF rasterization. The recommended value is 75 for balance between the weight of the image and its quality.
gauss	number	Out of date and is not used.
mode	string	default - the whole recognition pipeline is used; recognize_only - disables the cutting and orienting algorithms, the original image goes to the recognition stage
with_hitl	boolean	true - sends the document fields to human validators for verification and manual recognition. false - manual recognition is disabled
hitl_async	boolean	true - allows the HITL module to return the document information field values asynchronously, without waiting for the whole set of document information fields to be completed. The parameter works only when using manual document recognition mode with_hitl=true. The response with an incomplete set of fields is followed by code 202, the complete set of fields is followed by by code 200. false - switches off asynchronous HITL mode, the method will return the answer only after all the fields in the document are processed
hitl_required_fields	array	Allows for identification of required information fields in the document within the array - this tells the HITL module that it can return fields asynchronously only after the required information fields have been processed. Only works with with_hitl and hitl_async enabled.
hitl_sla	string	Not used

Request Body

Name	Type	Description
verify_fields	string	Activates the function of checking the fields against information in an external file (see above)
image	object	File whose contents must be p

200 The request is successful. The "doc_type" attribute specifies the type of the found document.

{
  "detail": [ // technical information
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ],
  "items": [
    {
      "doc_type": "passport_main", // document type
      { "fields": {
        "date_of_birth": { // document field name
          "text": "01.04.2004", // value of the document field
          "confidence": 0.5262104272842407 // parameter Confidence
          "valid": null, // verification result if verify_fields is present in the request
          "levenshtein": null, // levenshtein distance after verification if present in verify_fields parameter
          "coords": [ // field coordinates in the input file
            [
              [
                873,
                1468
              ],
              [
                1187,
                1468
              ],
              [
                1187,
                1520
              ],
              [
                873,
                1520
              ]
            ]
          ]
        },
        "date_of_issue": { // document field name
          "text": "01.04.2018", // value of the document field
          "confidence": 0.5271461009979248 // parameter Confidence
          "valid": null, // verification result if verify_fields is present in the request
          "levenshtein": null, // levenshtein distance after verification if present in verify_fields parameter
          "coords": [ // field coordinates in the input file
            [
              [
                752,
                1142
              ],
              [
                947,
                1142
              ],
              [
                947,
                1196
              ],
              [
                752,
                1196
              ]
            ]
          ]          
        }
      },
      "color": true, // the color of the image within the object ("true" if in color; "false" if in b/w)
      "other": 
      {
        "external_check_results": 
        {
          { "inn": "1234567890",
          { "fico":
          {
            { "errorCode": 0,
            { "status": "SUCCESS",
            { "exclusionCode": 0,
            { "score": 620,
            "reasonCode1": "D9",
            "reasonCode2": "M1",
            "reasonCode3": "T5",
            "reasonCode4": "A6",
            "reasonCode1Desc": "Too little time since last delinquency",
            "reasonCode2Desc": "Number of delinquent accounts",
            "reasonCode3Desc": "Too many recent credit history inquiries on the subject."
            "reasonCode4Desc": "Amount owed on delinquent accounts",
            "scoreSource": "nbki"
          },
          { "pledges_list": [],
          { "vehicle_restrict_list": [],
          "vehicle_wanted_list": [],
          "vehicle_dtp_and_restrict": 
          {
            { "restrict_list": [],
            { "dtp_list": []
          },
          }, "is_valid": true
        },
        }, "external_check_errors":
        {
          { "inn": "string",
          { "fico": "string",
          { "pledges_list": { "string",
          { "vehicle_restrict_list": "string",
          "vehicle_wanted_list": "string",
          "vehicle_dtp_and_restrict": "string",
          "is_valid": "string"
        }
      }
      "error": null // = "Recognition for requested document not implemented" if the model is not trained to recognize the document
    }
  ],
  "task_id": null, //task's internal id
  "code": null, //error code
  "message": null, // error message within the object
  "errno": null, // error number
  "traceback": null, // error message within the limits of object
  "fake": true, // response at the parameter check_fake = "true"
  "pages_count": 1, // number of pages in the input file
  "docs_count": 1 // number of documents in the input file
}

422 The request contains invalid input parameters

{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}