Two samples are provided in the C++ version of the Cochl.Sense SDK: sense-file and sense-stream.

• sense-file performs a prediction on an audio file (wav or mp3).

• sense-stream performs a prediction on an audio buffer.

1. Check the Requirements

Follow our Getting started section to set up the environment.

2. C++ Requirements

Install the packages below to use our Cochl.Sense C++ SDK.

$ sudo apt update
$ sudo apt install unzip
$ sudo apt install libpulse-dev pulseaudio pulseaudio-utils

3. Prepare the Sample

The samples can be found here

$ git clone https://github.com/cochlearai/sense-sdk-cpp-tutorials.git

Unzip the SDK

$ unzip path/to/sdk/sense-sdk-<version>-cpp.zip -d path/to/sample/sense-sdk-cpp-tutorials/

Your sample directory should now look like this

└── sense-sdk-cpp-tutorials
    ├── audio_files          # audio samples
    ├── examples
    |   ├── AudioFile.h
    |   ├── sense-file.cc    # sense-file sample
    │   └── sense-stream.cc  # sense-stream sample
    └── sense                # Sense SDK
        ├── include
        └── lib

4. Parameters

There are a few parameters you can set to your liking while initializing the sense.

// sense.hpp

...
struct Parameters {
  /// The name you want to set for this device in order to find it
  /// on your Dashboard.
  std::string device_name = "";
  
  // This list depends on your platform. Cf sense.hpp.
  ModelDelegate model_delegate = ModelDelegate::Default;
  
  /// Number of threads available to the tflite interpreter.
  int num_threads = 0;

  /// See below.
  Metrics metrics;
...

Every time a frame is processed by the SDK, a metric will be created, representing the result of this frame. Metrics are meant to be sent to the server and visible on your project dashboard.

Possible Metrics options

// sense.hpp
struct Metrics {
  size_t retention_period = 0;   // range, 1 to 31 days
  size_t free_disk_space = 100;  // range, 0 to 1,000,000 MB
  size_t push_period = 30;       // range, 1 to 3,600 seconds
};

retention_period

• The sense SDK tries to push the metrics to our servers right after it was created. If the push fails, the SDK stores the data locally and will try again later.
• retention_period determines the number of days that the SDK will keep the metrics data locally.
• Default value is 0 (metrics data is not saved locally).
• If an invalid value is passed (e.g. greater than 31 days), it will be set to 31 (days).
• If a user has set this period, metrics that are older than the retention period will be removed immediately.
• It is project-specific; e.g. retention period of project X can be 7 days, while the retention period of project Y is 14 days.

free_disk_space

• When the available disk space is less than free_disk_space (unit MB), the SDK will stop storing the metrics locally. In that case, the retention period set by the user will be overridden and set to 0 (days).
• If an invalid value is passed (e.g. greater than 100 MB), it will be set to 100 (MB).
• If the retention period is 0 and there are metrics stored locally, the SDK will attempt to push them one more time before removing them permanently.

push_period

• push_period determines how often metrics are pushed to the server.
• If an invalid value is passed (0 or greater than 3,600 seconds), it will be set to 30 (seconds).

5. Build

// examples/sense-file.cc

...
if (sense::Init("YOUR_PROJECT_KEY_HERE", sense_params) < 0) {
  return -1;
}
...
sense::Terminate();
...

$ g++ -fopenmp examples/sense-file.cc -I./sense/include/ -lsense-core -L./sense/lib -o sense-file -lm -std=c++11 -ldl -lstdc++ -Wl,-rpath -Wl,./sense/lib

// examples/sense-stream.cc

...
if (sense::Init("YOUR_PROJECT_KEY_HERE", sense_params) < 0) {
  return -1;
}
...
sense::Terminate();
...

$ g++ -fopenmp examples/sense-stream.cc -I./sense/include/ -lsense-core -L./sense/lib -o sense-stream -lm -std=c++11 -ldl -lstdc++ -lpulse -lpulse-simple -Wl,-rpath -Wl,./sense/lib

6. Run

$ LD_LIBRARY_PATH=. ./sense-file <PATH_TO_AUDIO_FILE>

$ LD_LIBRARY_PATH=. ./sense-file audio_files/babycry.wav

$ LD_LIBRARY_PATH=. ./sense-stream

(Note) sense-stream troubleshooting

Try the steps below when you have trouble running sense-stream

(1) Check connection

• Check if the input device is connected

(2) Check power

• The board may lack of power when an external device is connected
• Check if the board is stable and has enough power

(3) Check pulseaudio

# start pulseaudio
$ pulseaudio -D

# check your device index from the pulseaudio command result
$ pacmd list-sources | grep -e 'index:' -e device.string -e 'name:'

# set default source with the index above
$ pacmd set-default-source <DEVICE_INDEX>

(Note) CPU and memory usage in Cochl.Sense stream mode

Test environment specification:

• Sense SDK v1.2.0

  • Parameters:

    sense::Parameters sense_params;
    sense_params.metrics.retention_period = 0;
    sense_params.metrics.free_disk_space = 100;
    sense_params.metrics.push_period = 30;
    

• Raspberry Pi 3 Model B with TensorFlow Lite

  • OS(Raspberry Pi OS)
    • Release date: April 4th 2022
    • System: 64-bit
    • Kernel version: 5.15
    • Debian version: 11 (bullseye)
  • Quad Core 1.2GHz Broadcom BCM2837 64bit CPU(ARM Cortex A53)
  • 1GB RAM
  • BCM43438 wireless LAN
  • Storage(SanDisk 16GB Ultra Micro SD HC Class 10)

Result:

• In idle, CPU usage: 2.2526 (%), memory usage: 229,578 (kB)

CPU usage (%):

Memory usage (kB):