EBT Installation Guide, Getting Started, and Developer API V1 (Deprecated)

EBT Installation Guide, Getting Started, and Developer API V1 (Deprecated)

image

The Trueface Elevated Body Temperature (EBT) software uses the inner canthus (eyeduct) to measure a persons temperature.

The EBT software is currently supported by the iryx thermal camera and FLIR A400 thermal camera. With the iryx thermal camera, the Trueface software runs directly onboard the camera at the edge. With the A400 camera, the software runs on a computer which is connected to the same network as the camera.

Regardless of which camera you use, the usage interface is the exact same. The Trueface Software consists of three main interfaces: a websocket stream which sends all the event data for every frame, an http configuration server used to modify the behaviour of the app, and finally a token validation micro app which can be used to obtain and validate the token. Additionally, all important information such as facial bounding boxes and temperature is drawn directly on the RTSP stream (iryx) or snapshot frame (A400).

This document describes the installation process and developer APIs for the solution.

If you encounter any bugs / mistakes in this documentation, please reach out to cyrus@trueface.ai and the issues will be fixed promptly.

Table of Contents

Installation

iryx

The Trueface iryx solutions runs directly onboard the camera. Therefore, the software and license token must be copied and installed to the device. To receive a token, please contact sales@trueface.ai

If this is your first time using a newly purchased IRYX camera, please ensure that the thermal camera is working correctly. To do so, navigate to the web interface (enter the camera IP address in a web browser) then log in with the following web interface credentials:

Username: root

Password: admin

Next, navigate to the thermal overlay tab (one of the tabs on the left), which should display a live video of the thermal stream overlaid on the visible stream. Click anywhere on the video and it should display the temperature at the selected point in the bottom left. Check that the reported temperature is roughly correct. If the temperature is obviously incorrect (ex. Displays 50 degrees c in your office, shows the same temp no matter where you click, etc) then please contact Trueface and do not proceed with installing the software.

Note: Please avoid changing any of the default settings from the IRYX web interface as it may interfere with the Trueface software.

The default SSH username and password are as follows:

Username: root

Password: !admin This is different from the web interface password, note the extra !.

New versions of the IRYX camera come with the Trueface software already installed. However, you will need to check that the installed version is up to date, as new features and improvements are continuously being added.

Checking for Existing Software and Determine the Version

To determine if the Trueface software is already installed and if so, what version, run the following command (replace <ip_addr> with your camera IP address):

ssh root@<ip_addr> grep -oh "\w*_v\w.\w" /var/trueface_logs/trueface_temp_reader.log | tail -n 1

The output should look something like this, where X.X indicates the version number:

trueface_temp_reader_vX.X

If the output is blank or there is a “No such file or directory” error, then the software was either not installed, or it was not installed correctly. If that is the case, refer to the Installation Instructions section below and perform all the steps.

If the output is present, then check that the installed library is most up to date. Compare the library version (indicated by the X.X) to our releases page and ensure that it is the most up to date. The releases page can be found here:

EBT Embedded API Releases EBT Embedded API Releases

If the Trueface software version on the camera is already up to date, then you will only need to perform step 3 below, then reboot the camera. If the version is not up to date, then perform all the steps below (installing a new version of the Trueface software will remove the old version).

Installation Instructions

The following video demonstrates how to perform the instructions below:

In the following instructions, X.X should be replaced by your library version number, and <ip_addr> should be replaced by your camera IP address.

1. Navigate to the releases page and download the latest release.

2. Copy the trueface_dist_IRYX_vX.X.tgz to /mnt/data on the camera 

scp ./trueface_dist_IRYX_vX.X.tgz root@<ip_addr>:/mnt/data

3. Paste your Trueface token into a file called token.txt if you did not already receive it in this format (only the token itself, no quotations around the token, no json, nothing else).

Next, copy token.txt to /mnt/data on the camera

scp ./token.txt root@<ip_addr>:/mnt/data

4. SSH into the camera

ssh root@<ip_addr>

5. Navigate to /mnt/data

cd /mnt/data

6. Extract the tarball

tar -xzvf ./trueface_dist_IRYX_vX.X.tgz

7. Enter the trueface_dist_IRYX_vX.X directory

cd ./trueface_dist_IRYX_vX.X

8. Run the installation shell script

./install.sh

Note: Ensure the installation is successful based on the output from the shell script. If it is not successful, notify our engineers.

At this point, the camera will automatically reboot.

It will take 30 seconds after rebooting the camera for the application to start running. It will generally take 2 minutes after a reboot for the application to run at full speed (before this it will run at reduced speed while things initialize on the camera).

Note: After a cold boot, the camera will take up to 20 minutes for the temperature measurements to stabilize due to the thermal lens heating up. With a reboot, this will only take 5 minutes.

To view the stream, open a web browser and enter the IP address of the camera. Next, click settings and (after entering the username and password) navigate to the thermal overlay tab on the left. It should display a video with the bounding box and temperature drawn onto the video stream.

Alternatively, you can view the stream from VLC or another RTSP client by connecting to rtsp://root:admin@<ip_addr>/stream<stream_number> (learn more about stream_number here)

If the app installed correctly, the stream should look something like this:

image

FLIR A400

The FLIR A400 software runs on a (x86) computer which is connected to the same network as the camera. This device can be running any OS, as long as it has docker installed.

Start by navigating to the web interface (enter the camera IP address in the browser). Use the login credentials provided by FLIR. They can be found inside the getting started manual provided by FLIR.

On the left hand tab, under the “Data presentation modes” section, click on “Image mode”, then ensure “Visual” is selected. Refer to the following image:

image

Note: You must have docker installed to proceed.

To obtain Trueface software, run the following command. For a list of all releases, please visit https://hub.docker.com/r/trueface/flir-a400/tags

docker pull trueface/flir-a400:latest

To run the Trueface software, run the following command. You will need to replace ${YOUR_TOKEN} with your license token and ${A400_IP} with the IP address of the A400 camera.

docker run -e TOKEN=${YOUR_TOKEN} -e IP=${A400_IP} -p 8090:8090 -p 8091:8091 -p 8092:8092 -v $(pwd)/trueface_a400:/trueface_a400 -d --name tf-flir-a400 trueface/flir-a400:latest

Some other things to note. In the above command, we mount the /trueface_a400 directory in the container to the $(pwd)/trueface_a400 directory on the host. This is necessary so that the configuration options and log file persist even after the container is removed. It also allows the log files to be viewed while outside of the container. Additionally, we detach the process using the docker -d flag and we assign the container the name tf-flir-a400.

Configuration Server

The Trueface application has a configuration HTTP server running on port 8090. This configuration server can be used to both configure the application, and also to generate face recognition templates. For the IRYX camera, the configuration server will be running on the camera. For the FLIR A400 solution, the configuration server will be running on the same computer which is running the Trueface software.

For full documentation of all the endpoints (with sample CURL requests) please refer to our postman collection.

IRYX

Once changes are made using the configuration API, the camera must be rebooted for the changes to take effect. Alternatively, you can run the following command from on the camera to restart the service:

/etc/init.d/S98trueface restart

It will take 30 seconds after rebooting the camera / restarting the service for the application to start.

Note: After a cold boot, the camera will take up to 20 minutes for the temperature measurements to stabilize due to the thermal lens heating up. With a reboot, this will only take 5 minutes.

FLIR A400

Once changes are made using the configuration API, the application must be restarted. To do so, stop and restart the container.

To stop the container, run docker kill tf-flir-a400

To start the container again (with the new config options), run docker start tf-flir-a400

If you need to enter the running container (to debug, view logs, etc), run docker exec -it tf-flir-a400 bash

Event Data Streaming

The Trueface application will stream event data over websocket on default port 8091 (can be changed from config server).

Websocket URL: ws://<ip_address>:<port>

For the IRYX camera, the websocket server will be running on the camera. For the FLIR A400 solution, the websocket server will be running on the same computer which is running the Trueface software (localhost).

At first glance, the IRYX and A400 websocket packets may look quite similar; however, there are some notable differences. With the A400 application, the temperature detection is running asynchronously and is separate from the main face detection loop. Therefore, there can be instances where a face and eyeduct are detected in the main loop, but no face is detected in the asynchronous temperature detection loop. Therefore, there is an additional field called temp_detected which should be checked.

Additionally, since Trueface cannot draw directly on the A400 RTSP streams like is done with the IRYX RTSP streams, there is an additional field in the websocket packet called snapshot. This is a base64 encoded RGB jpg image of the entire frame with the bounding box and overlay temperature text. Display this image to simulate a video stream. Refer to the sample frontend app for an example of how to do this.

IRYX

iryx Websocket Packet Formatiryx Websocket Packet Format

FLIR A400

FLIR AX00 Websocket Packet FormatFLIR AX00 Websocket Packet Format

Video Stream

As mentioned, all important information such as facial bounding boxes and temperature is drawn directly on the RTSP stream (IRYX) or snapshot frame (A400).

The green bounding box will surround either the left or right eye duct, based on the orientation of the head:

image

If the bounding box around the face is red, it indicates that the eye duct is not fully visible or its position could not be computed accurately. This can be resolved by facing the camera directly, and ensuring to stay in the center of the frame.

image

If no bounding box is drawn, it indicates that the face did not pass the liveness test (and there may be a spoof attempt). Note, the liveness test may fail (and consequently no bounding box is drawn) if the face is moving in the frame quickly. For best results, ensure the face is centered in the frame and not moving for at least 1 second.

Sample Apps

There are currently three sample apps:

  1. Frontend app demonstrating how to connect to the websocket stream and display the live video in a browser.
  2. App demonstrating how to set up a face recognition pipeline with the EBT solution.
  3. App demonstrating how to set up temperature based alerts and send sms and email notifications when the alert qualifications are met.

The sample apps can be found here.

The following is a demo of the sample front end app which demonstrates how to display the RTSP / snapshot in a web browser.

image

Locking token to camera

Some license agreement schemes may required you to lock each individual token to a specific camera. If this is the case, then you must run an executable to generate a fingerprint ID for the camera. This fingerprint ID must be provided to Trueface then the token will then be locked to this fingerprint ID.

Generating fingerprint ID for IRYX camera

If the Trueface software has not already been installed to the IRYX camera then be sure to first install the Trueface software (instructions here).

To generate the fingerprint, ssh into the camera and navigate to /mnt/data/trueface_dist_IRYX_vX.X , then run the executable called generateFingerprintIRYX. Note, this executable is only available in v1.8.1 a newer, so upgrade your version if necessary.

Running the executable will print the device fingerprint to the console. Provide this to Trueface and the appropriate token will be generated.

Generating fingerprint ID for A400 camera

In order to generate a fingerprint ID for the A400 camera, start by pulling the trueface/flir-a400-generate-fingerprint dockerhub image. Next, run the following command, substituting ${A400_IP} for your camera's IP address.

docker run -e IP=${A400_IP} trueface/flir-a400-generate-fingerprint

If the executable runs successfully, it will print the device fingerprint to the console. Provide this to Trueface and the appropriate token will be generated.

Mounting the IRYX camera

The IRYX camera has a standard ¼ - 20 mounting thread on the backside and is therefore compatible with most camera mounts / tripods.

Calibrating the IRYX camera

We advise you to calibrate your camera to work best at your operating distance (we suggest ~2 feet). To do so, start by running the re-enable_overlay_for_calibration.sh shell script which comes in the distribution package. If the Trueface software has been installed to the camera, this script can be found at /mnt/data/trueface_dist_IRYX_vX.X/ . This will re-enable the thermal overlay (and disable the Trueface software ... we'll cover re-enabling it later) and will allow you to calibrate the thermal and visible offset. Next, navigate to http://<ip_addr>/priv/toverlay_adjust.html and then press the up / down / left / right buttons until the thermal and visible frames align correctly (note you may need to refresh the page to see live changes). Do not press the bigger / smaller buttons. Once you are happy with the changes, re-install the Trueface software on the camera by running install.sh (which is also in the distribution package) and you should be good to go.

image

Poor Alignment of thermal an visual images, adjustment buttons are in the top-left corner.

image

Better Alignment

Next, you should also be sure to specify the operating distance of the camera for best performance.

Navigate to the thermal tab, then set the distance to your operating distance:

image

If the camera requires further tuning, you can add a "manual temperature" to the temperature from the same tab on the web interface. Alternatively, you can achieve this using the /config endpoint and setting the offset_temperature value appropriately.

Reporting Errors or Bugs

If you encounter any bugs or unexpected behaviour while running the application, please send the log files to the Trueface engineering team. On the IRYX device, the log files can be found in /var/trueface_logs/. In the A400 docker container, the log files can be found in /trueface_a400/trueface_logs/. The team will help you debug the problem.

Token Validation

Version v1.10 adds a micro server for token validation. This server runs on port 8092 and supports a single endpoint GET /token

This endpoint can be used to retrieve the token on the camera. This is helpful as this same token can be used for the Trueface SDK, thus more complex applications which utilize both the EBT solution and SDK can share a single token. In addition, the endpoint can be used to validate the token. This is helpful as this endpoint will run even when the main temperature app itself may have crashed (due to an invalid token for example).

Sample Request

curl --location --request GET "http://{camera_ip}:8092/token"

{
    "days_remaining": 667,
    "message": "Success",
    "token": "YOU_TOKEN_HERE",
    "valid": true
}

Explanation of Fields

valid (boolean) - Returns true of the token is of the correct format, or false if the token has been corrupted, is missing fields, has an incorrect signature, etc. Note, an expired token (which is still a valid token) will return true.

message (string) - Will return "Success" if the token has passed all the checks and has not expired. Otherwise, will return the error message.

token (string) - The token. Will be an empty string if no token was found on the device.

days_remaining (int) - The number of days remaining for the token. An expired token will return 0.

So to check if a token is usable, you should check that valid is true and that days_remaining > 0.

Changelog

v1.10.1

Fixed

  • Bug that prevented token files with newline character at end of file from being read properly (for example pasting the token into token.txt using vim or echo automatically generates this \n and was causing issues previously).

v1.10

Added

Fixed

  • Bug in IRYX implementation which causes database to get corrupted when starting app without valid token.

v1.9

Added

  • Optional snapshot field to IRYX websocket stream. snapshot contains the base64 encoded 640x360 frame (jpg) that is in sync with the websocket packet data. When a face is detected, the bounding box is drawn on the frame. It was added because the RTSP stream can lag the websocket stream by a second or two.
  • enable_snapshot payload field added to /config endpoint. Used to turn on and off snapshot streaming on IRYX camera. Streaming the snapshot does slow things down so it is advised you only enable it if required.

v1.8.1

Changed

  • Changed scheme for locking token to individual cameras.

v1.8

Changed

  • A400 bug fix with visible frame capture (previously the app only worked while the A400 web interface was also open. This is no longer the case).
  • IRYX minor improvements and optimizations

v1.7

Added

  • Eyeglass detection to IRYX and A400 solutions.
  • eyeglasses_label to websocket packet
  • Liveness check
  • liveness_check_passed to websocket packet
  • liveness_check configuration option

v1.6

Fixed

  • Bug in /fr-template and /fr-template-lite endpoints

v1.5

Added

  • Mask detection to IRYX and A400 solutions
  • mask_label to websocket packet
  • draw_mask option to config server which allows enabling and disabling the mask / no mask text drawn on video

v1.4

Added

  • Corridor mode support for A400 camera.

Changed

  • A400 displays instantaneous temp instead of average temp on stream.
  • Changed default frame_count to 20 frames.

Fixed

  • Bug in A400 docker app which caused crashes with cameras with new firmware version.

v1.3

Fixed

  • Bug in version v1.2 install script which over rights the thermal to visible calibration values.
  • /corridor-enable and /corridor-disable now send response before rebooting.

v1.2

Fixed

  • Bug with corridor mode bounding box location for IRYX.
  • Disabled auto rotation for corridor mode for IRYX.

v1.1

Breaking

  • Face detection is now always performed using stream 3 as the source buffer, regardless of the selected stream_number. This improves performance significantly for corridor mode, and when displaying stream 1 (HD stream). The stream_number option still dictates which stream the results (bounding box, temp, etc) are drawn on. Because of this, the smallest_face_height option now needs to be set relative to stream 3. We therefore suggest setting the smallest_face_height to 40 px. Since the configuration options are not overwritten when upgrading the trueface software, the user must update this smallest_face_height value themselves by either running GET /restore-defaults or POST /config with the new value. The previous default value of 100px will likely not work with stream 3 as the resolution is smaller (640x360).

Added

  • Corridor mode support for IRYX camera.
    • Can be enabled / disabled using /corridor-enable /corridor-disable endpoints.

Changed

  • /restore-defaults from POST to GET (in order to resolve a strange bug with curl).

v1.0

Added

  • FLIR A400 support
  • display_text and display_text_color configuration options.

Changed

  • Major code refactor (API interface remains the same).
  • Documentation refactor.

v0.9

Added

  • timestamp to websocket packet. Format day-month-year hour:minute:second.millisecond

v0.8

Added

  • median_temperature_measured to the websocket packet. The median measured temperature over the last n (frame_count) frames of the eye duct region, reported in the chosen unit. If a face is not detected in a frame, this median will be reset.

v0.7

Added

  • temp_unit to websocket packet, either C or F.

Changed

  • Average temperature drawn on stream instead of temperature of last frame.

Fixed

  • Fixed bug with temperature averaging which reported incorrect temperature. This bug only occurred briefly after a face goes from the eyeduct not visible to eyeduct visible again state (red bounding box to blue bounding box).

v0.6

Added

  • GET /fr-template-lite endpoint which returns LITE model face feature vector for the face detected in the last 5 frames.
  • Added face_chip_aligned field to Websocket packet which contains the base64 encoded cropped and aligned 112x112 RGB face chip.

v0.5

Added

  • Configuration option to turn off temperature display text.