C++ Sample

Two samples are provided in the C++ version of the Cochl.Sense SDK: sense-file and sense-stream.

sense-file performs a prediction on an audio file (wav or mp3).
sense-stream performs a prediction on an audio buffer.

1. Check the Requirements

Follow our Getting started section to set up the environment.

2. Prepare the Sample

The samples can be found here

$ git clone https://github.com/cochlearai/sense-sdk-cpp-tutorials.git

Unzip the SDK

$ unzip path/to/sdk/sense-sdk-<version>-cpp.zip -d path/to/sample/sense-sdk-cpp-tutorials/

Your sample directory should now look like this

└── sense-sdk-cpp-tutorials
    ├── audio_files          # audio samples
    ├── examples
    |   ├── AudioFile.h
    |   ├── sense-file.cc    # sense-file sample
    │   └── sense-stream.cc  # sense-stream sample
    └── sense                # Sense SDK
        ├── include
        └── lib

3. Parameters

There are a few parameters you can set to your liking while initializing the sense.

// sense.hpp

...
struct Parameters {
  /// The name you want to set for this device in order to find it
  /// on your Dashboard.
  std::string device_name = "";
  
  // This list depends on your platform. Cf sense.hpp.
  ModelDelegate model_delegate = ModelDelegate::Default;
  
  /// Number of threads available to the tflite interpreter.
  int num_threads = 0;

  /// See below.
  Metrics metrics;
...

Everytime a frame is processed by the SDK, a metric will be created, representing the result of this frame. Metrics are meant to be sent to the server and visible on your project dashboard.

Possible `Metrics` options

// sense.hpp
struct Metrics {
  size_t retention_period = 0;   // range, 1 to 31 days
  size_t free_disk_space = 100;  // range, 0 to 1,000,000 MB
  size_t push_period = 30;       // range, 1 to 3,600 seconds
};

`retention_period`

The sense SDK tries to push the metrics to our servers right after it was created. If the push fails, the SDK stores the data locally and will try again later.
retention_period determines the number of days that the SDK will keep the metrics data locally.
Default value is 0 (metrics data is not saved locally).
If an invalid value is passed (e.g. greater than 31 days), it will be set to 31 (days).
If a user has set this period, metrics that are older than the retention period will be removed immediately.
It is project-specific; e.g. retention period of projext X can be 7 days, while there retention period of project Y is 14 days.

`free_disk_space`

When the available disk space is less than free_disk_space (unit MB), the SDK will stop storing the metrics locally. In that case, the retention period set by the user will be overridden and set to 0 (days).
If an invalid value is passed (e.g. greater than 100 MB), it will be set to 100 (MB).
If the retention period is 0 and there are metrics stored locally, the SDK will attempt to push them one more time before removing them permanently.

`push_period`

push_period determines how often metrics are pushed to the server.
If an invalid value is passed (0 or greater than 3,600 seconds), it will be set to 30 (seconds).

4. Build

sense-file
sense-stream

Put your Project Key into the sample code. Note that it’s important to invoke the sense::Terminate() function at the end if sense::Init() was successful. Otherwise, an undefined behavior may occur while cleaning up the memory allocated by the sense during sense::init().

// examples/sense-file.cc

...
if (sense::Init("YOUR_PROJECT_KEY_HERE", sense_params) < 0) {
  return -1;
}
...
sense::Terminate();
...

We are now ready to build the sense-file.

$ g++ -fopenmp examples/sense-file.cc -I./sense/include/ -lsense-core -L./sense/lib -o sense-file -lm -std=c++11 -ldl -lstdc++ -Wl,-rpath -Wl,./sense/lib

// examples/sense-stream.cc

...
if (sense::Init("YOUR_PROJECT_KEY_HERE", sense_params) < 0) {
  return -1;
}
...
sense::Terminate();
...

We are now ready to build the sense-stream.

$ g++ -fopenmp examples/sense-stream.cc -I./sense/include/ -lsense-core -L./sense/lib -o sense-stream -lm -std=c++11 -ldl -lstdc++ -lpulse -lpulse-simple -Wl,-rpath -Wl,./sense/lib

5. Run

sense-file
sense-stream

$ LD_LIBRARY_PATH=. ./sense-file <PATH_TO_AUDIO_FILE>

The repository contains audio files that can be used as well.

$ LD_LIBRARY_PATH=. ./sense-file audio_files/babycry.wav

Make sure the input device is properly connected before you run sense-stream

$ LD_LIBRARY_PATH=. ./sense-stream

(Note) `sense-stream` troubleshooting

Try the steps below when you have trouble running sense-stream

(1) Check connection

Check if the input device is connected

(2) Check power

The board may lack of power when an external device is connected
Check if the board is stable and has enough power

(3) Check `pulseaudio`

# start pulseaudio
$ pulseaudio -D

# check your device index from the pulseaudio command result
$ pacmd list-sources | grep -e 'index:' -e device.string -e 'name:'

# set default source with the index above
$ pacmd set-default-source <DEVICE_INDEX>

(Note) CPU and memory usage in Cochl.Sense stream mode

Test environment specification:

Raspberry Pi 3 Model B with TensorFlow Lite
- OS(Raspberry Pi OS)
  - Release date: April 4th 2022
  - System: 64-bit
  - Kernel version: 5.15
  - Debian version: 11 (bullseye)
- Quad Core 1.2GHz Broadcom BCM2837 64bit CPU(ARM Cortex A53)
- 1GB RAM
- BCM43438 wireless LAN
- Storage(SanDisk 16GB Ultra Micro SD HC Class 10)
- Parameters:
```
sense::Parameters params;
params.audio_format = sense::AF_FLOAT32;
params.model_delegate = ModelDelegate::Default;
params.service = Service::Human_Interaction;
params.metrics.retention_period = 0;  // days
params.metrics.free_disk_space = 100;  // MB
```
Result:
- In idle, CPU usage: 2.2526 (%), memory usage: 229,578 (kB)
- CPU usage (%):

Memory usage (kB):