EBT Installation Guide, Getting Started, and Developer API V2

EBT Installation Guide, Getting Started, and Developer API V2

Important:

There have been significant changes between V1 and V2, in particular around the installation process of the Trueface software onto the iryx camera. In order to take advantage of V2 using an older iryx camera, you must ensure that your firmware version on the camera is at least version E1.2-010521 or newer.

You can download the the firmware here:

Upgrading to this new firmware version will wipe the previous Trueface installation, so you must reinstall the Trueface software after upgrading (see below for instructions).

For existing clients, if you prefer to stick to the old firmware and old Trueface application V1, please refer to our now deprecated documentation below.

EBT Installation Guide, Getting Started, and Developer API V1 (Deprecated)EBT Installation Guide, Getting Started, and Developer API V1 (Deprecated)
image

The Trueface Elevated Body Temperature (EBT) software uses the inner canthus (eyeduct) to measure a person's temperature.

The EBT software is currently supported by the iryx thermal camera and FLIR A400/A500 thermal cameras (which will be referred to as AX00 in this document). With the iryx thermal camera, The Trueface software runs directly onboard the iryx thermal camera at the edge. Contrastingly, the AX00 camera requires the software to run in a docker container on a computer that is connected to the same network as the camera.

Regardless of which camera you use, the usage interface is the exact same. The Trueface Software consists of three main interfaces: a WebSocket stream which sends all the event data for every frame, an HTTP configuration server used to modify the behavior of the app, and a token validation micro app which can be used to obtain and validate the token. Additionally, all important information such as facial bounding boxes and temperature is drawn directly on the RTSP stream (iryx) or snapshot frame (AX00).

This document describes the installation process and developer APIs for the solution.

If you encounter any bugs / mistakes in this documentation, please reach out to cyrus@trueface.ai and the issues will be fixed promptly.

Table of Contents

Installation

iryx

The Trueface iryx solution runs directly onboard the camera. Therefore, the software and license token need to be installed on the device.

Start by powering up your camera by plugging it into a Power Over Ethernet switch. You must use a standard 48V 802.3af PoE switch. Here is an example switch that will work.

After determining the IP address of your camera, navigate to the IP address in your web browser. You will be prompted to set a username and password. If the username and password were already set prior to receiving the camera, then the default credentials will be:

username: admin

password: admin

Next, on the top navigation panel, click Thermal > Thermal Overlay

Note, if you recently upgraded your firmware version and don't see the navigation panel, you are probably getting served the cached version of the web page. If this is the case, clear your cache and / or use an incognito browser.

This page should display a live video stream of the visual data overlayed with the thermal data. Click anywhere on the video feed and it should display the temperature at the selected point in the bottom left. Check that the reported temperature is roughly correct. If the temperature is obviously incorrect (ex: Displays 50 degrees c in your office, shows the same temp no matter where you click, etc) then please contact Trueface and do not proceed with installing the software.

Installation using the web interface

On the top navigation panel, click on Apps > Trueface

You should be presented with a screen that looks like this:

image

Checking to see if the software is installed

The first step is to install the Trueface software to the camera. You can check to see if it is already installed by looking at the first two fields:

image

Installing the Trueface software from the web

As you can see in the image above, the software is not installed to the camera. There are two ways to install the software; the first is to download it directly from the web. To do this, choose your desired version from the drop down list (default is latest) then press download. We advise always using the latest version:

image

Install the Trueface software from a tar archive

The second option is to upload the archive (.tgz file) to the camera from a local computer. This is best if you do not have internet access, or if you are installing to multiple cameras since you only have to download the archive once. Visit the releases page and download your desired version, then choose the file and upload it.

image

Ensure software has been installed correctly

After installing the software, you should ensure that the software was successfully installed:

image

The Trueface application will automatically start after installation. Note, after starting the service, it will take 30 seconds for the application to boot up.

Generating a device fingerprint for camera locked Trueface license tokens

We are not quite finished at this point. We need to obtain a Trueface token for the camera and upload that to the camera. To obtain a token, first press the "get fingerprint" button. This will produce a unique fingerprint for the camera:

image

Copy this fingerprint and send it to your representative at Trueface and they will generate a unique token for you, or if you are a new client, email sales@trueface.ai. If you are dealing with a large number of cameras, it is best if you place these fingerprints into a spreadsheet so we can efficiently generate tokens for you. We will also go over how to do all these steps programmatically below to automate this process if dealing with large batches of cameras.

Uploading the token to the device

Once you have received your token, paste the token in the New Token input box then press the upload token button. Once you upload the token, ensure that the Token has been installed and that it is valid.

image

If you token is not valid, then navigate to the Token Validation section below to get more information on why your token is not valid.

At this point, press the stop button, then press the start button and you are ready to go.

Note: If the Trueface application is running and the camera is rebooted, it should automatically start up again after booting the camera - the Trueface application will take an additional 30 seconds to start up after boot. After a cold boot, the camera will take up to 20 minutes for the temperature measurements to stabilize due to the thermal lens heating up. With a reboot, this will only take 5 minutes.

Automating the installation from the command line

If you are dealing with a large number of cameras, then you may choose to automate the installation processes using the command line or a bash script. Everything which was achieved above using the web interface can also be done from the command line. In the following commands, replace <IP> with the IP address of your camera, and <username> and <password> with your camera username and password.

To download and install the Trueface software from the web:

curl --anyauth -u <username>:<password> "http://<ip>/priv/cgi-bin/trueface_control.cgi?download=latest"

To upload and install the Trueface software from a local computer:

curl --anyauth -u <username>:<password> -X POST -F 'file=@trueface_dist_IRYX_latest.tgz' "http://<ip>/priv/cgi-bin/trueface_upload.cgi"

To get the device fingerprint:

curl --anyauth -u <username>:<password> "http://<ip>/priv/cgi-bin/trueface_control.cgi?fingerprint"

To upload the token string:

curl --anyauth -u <username>:<password> "http://<IP>/priv/cgi-bin/trueface_control.cgi?set_token=<token string>"

You can find a list of all command line controls, including accessing the logs, stopping and starting the service, etc, by clicking on For Developers > Config API then scrolling down to trueface_control.cgi

Viewing the live stream

You can ensure the application was installed and is working correctly by viewing the RTSP stream. After starting the Trueface app then waiting 30 seconds for the Trueface application to boot up, navigate to an RTSP client such as VLC then connect to the following RTSP URL, replacing <IP> with the camera IP address, and <username> and <password> with your username and password.

rtsp://<username>:<password>@<IP>/stream6

If you change the stream_number configuration parameter to something other than stream6 (default), then you will need to specify the RTSP stream appropriately.

rtsp://<username>:<password>@<IP>/<stream_number>

If the application is working, you should see the bounding box and temperature drawn directly on the stream.

image

FLIR AX00

The FLIR AX00 software runs on in a docker container on a (x86) computer which is connected to the same network as the camera. This device can be running any OS, as long as it has docker installed.

Start by navigating to the web interface (enter the camera IP address in the browser). Use the login credentials provided by FLIR. They can be found inside the FLIR getting started manual.

On the left hand tab, under the “Data presentation modes” section, click on “Image mode”, then ensure “Visual” is selected. Refer to the following image:

image

Note: You must have docker installed to proceed.

A400

To obtain Trueface software, run the following command. For a list of all releases, please visit https://hub.docker.com/r/trueface/flir-a400/tags

docker pull trueface/flir-a400:latest

A500

To obtain Trueface software, run the following command. For a list of all releases, please visit https://hub.docker.com/r/trueface/flir-a500/tags

docker pull trueface/flir-a500:latest

To run the Trueface software, run the following command. You will need to replace ${YOUR_TOKEN} with your license token, ${AX00_IP} with the IP address of the AX00 camera, and ${AX00_VERSION} with your camera type, either a400 or a500.

 docker run -e TOKEN=${YOUR_TOKEN} -e IP=${AX00_IP} -p 8090:8090 -p 8091:8091 -p 8092:8092 -v $(pwd)/trueface_ax00:/trueface_ax00 -d --name tf-flir-${AX00_VERSION} trueface/flir-${AX00_VERSION}:latest

Other things to note: in the above command, we mount the /trueface_ax00 directory in the container to the $(pwd)/trueface_ax00 directory on the host. This is necessary so that the configuration options and log file persist even after the container is removed. It also allows the log files to be viewed while outside of the container. Additionally, we detach the process using the docker -d flag and we assign the container the name tf-flir-${AX00_VERSION}.

Configuration Server

The Trueface application has a configuration HTTP server running on port 8090. This configuration server can be used to both configure the application and also to generate face recognition templates. For the iryx camera, the configuration server will be running on the camera. For the FLIR AX00 solution, the configuration server will be running in a docker container on the same computer which is running the Trueface software.

For full documentation of all the endpoints (with sample CURL requests) please refer to our postman collection.

iryx

Once changes are made using the configuration API, the Trueface application must be stopped then restarted. Remember, it will take 30 seconds after the app is started for the Trueface application to fully boot.

FLIR AX00

Once changes are made using the configuration API, the application must be restarted. To do so, stop and restart the container.

To stop the container, run docker kill tf-flir-${AX00_VERSION}

To start the container again (with the new config options), run docker start tf-flir-${AX00_VERSION}

If you need to enter the running container (to debug, view logs, etc), run

docker exec -it tf-flir-${AX00_VERSION} bash

Event Data Streaming

The Trueface application will stream event data over WebSocket on default port 8091 (which can be changed from config server).

WebSocket URL: ws://<IP>:<port>

For the iryx camera, the WebSocket server will be running on the camera. For the FLIR AX00 solution, the WebSocket server will be running on the same computer which is running the Trueface software (localhost).

At first glance, the iryx and AX00 WebSocket packets may look quite similar; however, there are some notable differences. With the AX00 application, the temperature detection is running asynchronously and is separate from the main face detection loop. Therefore, there can be instances where a face and eye duct are detected in the main loop, but no face is detected in the asynchronous temperature detection loop. Therefore, there is an additional field called temp_detected which should be checked.

Because the Trueface software cannot draw directly on the AX00 RTSP streams like it can with the iryx RTSP streams, there is an additional field in the WebSocket packet called snapshot. This is a base64 encoded RGB jpg image of the entire frame with the bounding box and overlay temperature text. Display this image to simulate a video stream. Refer to the sample frontend app for an example of how to do this.

iryx

iryx Websocket Packet Formatiryx Websocket Packet Format

FLIR AX00

FLIR AX00 Websocket Packet FormatFLIR AX00 Websocket Packet Format

Video Stream

As mentioned, all important information such as facial bounding boxes and temperature are drawn directly on the RTSP stream (IRYX) or snapshot frame (AX00).

The green bounding box will surround either the left or right eye duct, based on the orientation of the head:

image

If the bounding box around the face is red, it indicates that the eye duct is not fully visible or its position could not be computed accurately. This can be resolved by facing the camera directly, and ensuring to stay in the center of the frame.

image

If no bounding box is drawn, it indicates that the face did not pass the liveness test (and there may be a spoof attempt). Please note that the liveness test may fail (and consequently no bounding box is drawn) if the face is moving quickly in the frame. For best results, ensure the face is centered in the frame and is still for at least one second.

Sample Apps

There are currently three sample apps:

  1. Front end app demonstrating how to connect to the WebSocket stream and display the live video in a browser.
  2. App demonstrating how to set up a face recognition pipeline with the EBT solution.
  3. App demonstrating how to set up temperature-based alerts and send SMS and email notifications when the alert qualifications are met.

The sample apps can be found here.

The following is a demo of the sample front end app which demonstrates how to display the RTSP / snapshot in a web browser.

image

Locking token to camera

The majority of license agreements require you to lock each individual token to a specific camera. If this is the case, then you must generate a unique fingerprint ID for the camera. This fingerprint ID must be provided to Trueface and then the token will be locked to this fingerprint ID.

Generating fingerprint ID for iryx camera

Refer to this section for instructions for generating the camera fingerprint.

Generating fingerprint ID for AX00 camera

In order to generate a fingerprint ID for the AX00 camera, start by pulling the trueface/flir-ax00-generate-fingerprint dockerhub image. Next, run the following command, substituting ${AX00_IP} for your camera's IP address.

docker run -e IP=${AX00_IP} trueface/flir-ax00-generate-fingerprint

If the executable runs successfully, it will print the device fingerprint to the console. Provide this to Trueface and the appropriate token will be generated.

Mounting the iryx camera

The iryx camera has a standard ¼ - 20 mounting thread on the backside and is therefore compatible with most camera mounts / tripods.

Calibrating the iryx camera

For best performance, be sure to specify the operating distance of your camera. Using the navigation bar at the top of the web interface, click on Thermal > Thermal Settings.

Specify the operating distance using the following input box.

image

If the camera requires further tuning, you can add a "manual temperature" to the temperature from the same tab on the web interface. Alternatively, you can achieve this using the /config endpoint and setting the offset_temperature value appropriately.

Additionally, you should ensure the thermal and visual frames are calibrated correctly. To calibrate this, start by stopping the Trueface service if it is running:

image

Next, use the navigation panel at the top to go to Thermal > Thermal Overlay. From here, click the "adjust alignment" button at the bottom left. This should bring you to a screen that looks like this:

image

Use the up / down / left / right buttons until the thermal and visible frames are aligned. Do not press the bigger / smaller buttons as this interfere with the Trueface software.

Once you are happy with the alignment, go back to the Trueface page and start the Trueface application again. Below are examples of poor and good alignment:

image

Poor alignment

image

Good alignment

Reporting Errors or Bugs

If you encounter any bugs or unexpected behavior while running the application, please send the log files to the Trueface engineering team. On the iryx device, the log files can be obtained from the web interface by clicking on the the logs button:

image

In the AX00 docker container, the log files can be found in /trueface_ax00/trueface_logs/.

If possible, please also provide steps to reproduce the error and provide as much detail as possible. The team will help you debug the problem.

Token Validation

Version v1.10+ adds a micro server for token validation. This server runs on port 8092 and supports a single endpoint GET /token

This endpoint can be used to retrieve the token on the camera. This is helpful as this same token can be used for the Trueface SDK, thus more complex applications which utilize both the EBT solution and SDK can share a single token. In addition, the endpoint can be used to validate the token. This is helpful as this endpoint will run even when the main temperature app itself may have crashed (due to an invalid token for example).

Sample Request

curl --location --request GET "http://{camera_ip}:8092/token"

{
    "days_remaining": 667,
    "message": "Success",
    "token": "YOU_TOKEN_HERE",
    "valid": true
}

Explanation of Fields

valid (boolean) - Returns true of the token is of the correct format, or false if the token has been corrupted, is missing fields, has an incorrect signature, etc. Note, an expired token (which is still a valid token) will return true.

message (string) - Will return "Success" if the token has passed all the checks and has not expired. Otherwise, will return the error message.

token (string) - The token. Will be an empty string if no token was found on the device.

days_remaining (int) - The number of days remaining for the token. An expired token will return 0.

So to check if a token is usable, you should check that valid is true and that days_remaining > 0.

Changelog

V2.4

Added

  • Added support for LITE_V2 model, only available for IRYX. Learn more about the model here and here.

V2.3

Fixed

  • Added rate limit to logger to avoid printing duplicate log messages within 5 minute period.

V2.2

Fixed

  • Works with newer versions of iryx firmware.

V2.1

Fixed

  • Added the eyeduct bounding box to the IRYX snapshot frame.
  • Added mask_score and eyeglasses_score to IRYX websocket packet.

V2.0

Changed:

  • Changed the entire installation process for installing the Trueface software onto the iryx camera

Added:

  • Added a web interface to the Trueface iryx software
  • Added support for FLIR A500

v1.10.1

Fixed

  • Bug that prevented token files with newline character at end of file from being read properly (for example pasting the token into token.txt using vim or echo automatically generates this \n and was causing issues previously).

v1.10

Added

Fixed

  • Bug in iryx implementation which causes database to get corrupted when starting app without valid token.

v1.9

Added

  • Optional snapshot field to iryx websocket stream. snapshot contains the base64 encoded 640x360 frame (jpg) that is in sync with the websocket packet data. When a face is detected, the bounding box is drawn on the frame. It was added because the RTSP stream can lag the websocket stream by a second or two.
  • enable_snapshot payload field added to /config endpoint. Used to turn on and off snapshot streaming on iryx camera. Streaming the snapshot does slow things down so it is advised you only enable it if required.

v1.8.1

Changed

  • Changed scheme for locking token to individual cameras.

v1.8

Changed

  • A400 bug fix with visible frame capture (previously the app only worked while the A400 web interface was also open. This is no longer the case).
  • iryx minor improvements and optimizations

v1.7

Added

  • Eyeglass detection to iryx and A400 solutions.
  • eyeglasses_label to websocket packet
  • Liveness check
  • liveness_check_passed to websocket packet
  • liveness_check configuration option

v1.6

Fixed

  • Bug in /fr-template and /fr-template-lite endpoints

v1.5

Added

  • Mask detection to iryx and A400 solutions
  • mask_label to websocket packet
  • draw_mask option to config server which allows enabling and disabling the mask / no mask text drawn on video

v1.4

Added

  • Corridor mode support for A400 camera.

Changed

  • A400 displays instantaneous temp instead of average temp on stream.
  • Changed default frame_count to 20 frames.

Fixed

  • Bug in A400 docker app which caused crashes with cameras with new firmware version.

v1.3

Fixed

  • Bug in version v1.2 install script which over rights the thermal to visible calibration values.
  • /corridor-enable and /corridor-disable now send response before rebooting.

v1.2

Fixed

  • Bug with corridor mode bounding box location for iryx.
  • Disabled auto rotation for corridor mode for iryx.

v1.1

Breaking

  • Face detection is now always performed using stream 3 as the source buffer, regardless of the selected stream_number. This improves performance significantly for corridor mode, and when displaying stream 1 (HD stream). The stream_number option still dictates which stream the results (bounding box, temp, etc) are drawn on. Because of this, the smallest_face_height option now needs to be set relative to stream 3. We therefore suggest setting the smallest_face_height to 40 px. Since the configuration options are not overwritten when upgrading the trueface software, the user must update this smallest_face_height value themselves by either running GET /restore-defaults or POST /config with the new value. The previous default value of 100px will likely not work with stream 3 as the resolution is smaller (640x360).

Added

  • Corridor mode support for iryx camera.
    • Can be enabled / disabled using /corridor-enable /corridor-disable endpoints.

Changed

  • /restore-defaults from POST to GET (in order to resolve a strange bug with curl).

v1.0

Added

  • FLIR A400 support
  • display_text and display_text_color configuration options.

Changed

  • Major code refactor (API interface remains the same).
  • Documentation refactor.

v0.9

Added

  • timestamp to websocket packet. Format day-month-year hour:minute:second.millisecond

v0.8

Added

  • median_temperature_measured to the websocket packet. The median measured temperature over the last n (frame_count) frames of the eye duct region, reported in the chosen unit. If a face is not detected in a frame, this median will be reset.

v0.7

Added

  • temp_unit to websocket packet, either C or F.

Changed

  • Average temperature drawn on stream instead of temperature of last frame.

Fixed

  • Fixed bug with temperature averaging which reported incorrect temperature. This bug only occurred briefly after a face goes from the eyeduct not visible to eyeduct visible again state (red bounding box to blue bounding box).

v0.6

Added

  • GET /fr-template-lite endpoint which returns LITE model face feature vector for the face detected in the last 5 frames.
  • Added face_chip_aligned field to Websocket packet which contains the base64 encoded cropped and aligned 112x112 RGB face chip.

v0.5

Added

  • Configuration option to turn off temperature display text.

Data Collection and Privacy

The Trueface Elevated Body Temperature software works as follows:

The software first grabs a frame from the visual light camera then uses a face detection algorithm to detect the largest face in the frame (if any) and the corresponding facial landmarks of that face. Those landmarks are then used to compute the coordinates of the inner canthus region, which are then used to obtain the temperature of the corresponding region from the thermal frame. Once the temperature has been extracted, the face landmarks information is deleted. After streaming any data (such as the canthus temperature) via websocket on the private network, the data is immediately deleted. No information is sent back to Trueface or any third parties.

The Trueface Elevated Body Temperature software does not store any personally identifiable information, face images, face biometrics, face landmarks, or temperature data. A special token can be provided which disables the use of all face recognition functionality.