Image Built-ins

Image Built-ins are used to:

• Computer image structural similarity

• Extract 7-segment digits from pictures of displays

• Detects the image display margins

Additional functions try to recognize the picture's display based on user-specified configurations.

Configuration

The DeviceDisplayProfiles configuration has two sections:

• displayDetectionOptions

• sevenSegmentChars

displayDetectionOptions

The displayDetectionOptions parameters are used by the algorithm to detect the display in the image. The algorithm first resizes the image to a lower resolution for a better display contour detection accuracy. Then it will apply a Gaussian blur filter to reduce noise. Next, using the Canny edge detection algorithm, it will detect the display edges and extract the display from the picture. After this step, the detected contours and display images are attached to the report, allowing for inspection of what was detected.

sevenSegmentChars

These parameters specify digit coordinates on the display. The coordinate values are specified relative to the display's width/height. Finding the right values is case-specific, for example, some experimentation is required to calibrate the algorithm with the optimal configuration for certain setups.

Note: Blur is only needed to reduce noise that might be otherwise detected as an edge in the image. Applying a blur with too large a kernel size might result in the digit edges no longer being detected reliably.

Syntax

DeviceDisplayProfiles {
    <name String>: {
        displayDetectionOptions: {
            enabled: <Boolean>
            imageResizeWidth: <Number>
            displayCropMethod: <String>
        }
}
        sevenSegmentChars: [{
            xPercentage: <Number>
            yPercentage: <Number>
            widthPercentage: <Number>
            heightPercentage: <Number>
            segmentMinAreaThreshold: <Number>
            font: <String>
        }]
    }
}

Parameters The following configuration parameters are available within the displayDetectionOptions block:

• name - The name assigned to the configuration

• enabled -

  • false skips the display detection step
  • true (default) otherwise

• imageResizeWidth (Optional) - The width in pixels to which the image will be scaled before applying the edge detection algorithm on it

  • Default is set to 500

• displayCropMethod (Optional) - The method for cropping the display after its contour detection

  • May be one of:
    • plain (default) - A plain crop without any other transformations
    • The default is the value when the display is centered straight in the image (no rotation, tilt or recification)
    • perspective-warp - Crops the display by performing a perspective warp transformation
      • This value is useful for scenarios where the display is not straight in the image
      • In such cases, the algorithm will apply a corrective perspective transformation to bring the display into a straight position

The following configuration parameters are available within the sevenSegmentChars block:

• xPercentage and yPercentage - The approximate position of the digit (starting from top-left corner)

  • The values are numbers between 0 and 100
  • The coordinates are specified (as a percentage) to the display width, respective height
  • For example, xPercentage = 33 and yPercentage = 33 means the top corner of the digit is located on the display ⅓ from left of the screen and ⅓ from the top

• widthPercentage and heightPercentage - Specifies the width and height of the digit as a percentage of the display size

• segmentMinAreaThreshold (optional) A threshold used to filter out contours that are too small to be a candidate segment of the digit

  • This means all contours that have an area less than the given threshold are not interpreted as digit segments in the image
  • Default is set to 50

• font - May be one of DSEG-7 (default) or DSEG-14

Example

DeviceDisplayProfiles {
    "device-1": {
        displayDetectionOptions: {
            enabled: true
            imageResizeWidth: 500
        }

        sevenSegmentChars: [{
            xPercentage: 10.2
            yPercentage: 10.1
            widthPercentage: 30
            heightPercentage: 50
            segmentMinAreaThreshold: 400
            font: DSEG-14
        }]
    }
}

Open Image

Opens an image and supports png and jpg format. This function must be used before calling any additional Image Built-in functions on the picture.

Syntax

<image Image> := Image.openImage(<url String> OR <file File>)

Returns
The image.

Parameters

• url - The path to the file

  • If a remote URL is given, intaQt will download the image

• file - A file object created by File Built-ins

Example 1 - Opening the Image via a URL

img := Image.openImage("http://localhost:8080/tmp/test.png")

Example 2 - Opening the Image via File Built-ins

file := File.fromProject("a.jpg")
ing := Image.openImage(file)

Compare Image Structural Similarity

Computes image structural similarity score by comparing two images.

Syntax

<score Number> := Image.structuralSimilarity(<img1 Image>, <img2 Image>)

Returns
A number between 0 and 1, representing the similarity score for the input images. The higher the score, the more similar the two images are. A score is 1 would result from comparing an image with itself.

Parameters

• img1 - The first image used for comparison, as obtained from the #open function

• img2 - The second image used for comparison, as obtained from the #open function

Example

img1 := Image.openImage(File.fromProject("img1.jpg"))
img2 := Image.openImage(File.fromProject("img2.jpg"))

score := Image.structuralSimilarity(img1, img2)

assert score > 0.8 ! "img1 differs from img2 too much"

Extract Image

Extracts the 7-segment digits from the image.

Syntax

<result Object> := Image.extractSevenSegmentText(<image Image>, <deviceProfile String>)

Returns
An object that has the following fields/methods available:

• result.isOk() - A boolean stating the operation status

• result.result - A string containing the extracted text

If result.isOk() returns false, this means the text extraction failed and the result.result will be an empty string.

Parameters

• image - The image object created by the openImage function

• deviceProfile - The device profile name to use for the extraction, as configured in the DeviceDisplayProfiles configuration section

Example

img := Image.openImage("http://device-host:8080/pic")
Image.extractSevenSegmentText(img, "device-1")

Attach Image to Report

Attaches the image to the test report in png format.

Syntax

Image.attachToReport(<image Image>, <name String>)

Parameters

• image - The image object created by the openImage function

• name - The name assigned to the attachment

Example

img := Image.openImage(File.fromProject("a.png"))
Image.attachToReport(img, "my-image")