Multi-stream inference status on Jetson AGX and Qualcomm IQ-9075

This benchmark measures how throughput and latency scale as channels are added, running iQS-Streampipe on an NVIDIA Jetson AGX Orin 32GB and an EXMP-Q911 — useful for sizing a deployment (how many cameras per board), not as a general hardware ranking. A recording of live multi-stream inference is available here.

• Application: iQS-Streampipe
• Task: object detection with an INT8 model
• Input: 1080p, 30 FPS H.264 video streams
• Platforms: Jetson AGX Orin 32GB, EXMP-Q911 (Qualcomm IQ-9075-based module)
• Metrics:
  • Output average end-to-end FPS per channel
  • Average CPU loading (%)
  • Average CPU memory usage (%)
  • Output end-to-end FPS percentiles (P95, P75, P50, P25)
• Channel counts: 1, 4, 9, 16

Hardware

The benchmark uses two edge AI platforms with the configurations shown below.

AI model

Videos

Results

Measurement summary

• The FPS plot (left) shows that both platforms keep per-channel FPS close to 30 at lower channel counts and gradually lose per-channel throughput as channels increase; EXMP-Q911 stays nearer to real-time at medium and higher channel counts, while Jetson AGX Orin drops earlier.
• The CPU loading plot (middle) shows low utilization at 1 channel and a steady increase as more channels are added, with both platforms near or at high CPU usage at 9 and 16 channels; the points where CPU load approaches saturation align with the visible per-channel FPS reductions.
• The CPU memory plot (right) shows that memory usage rises with channel count on both platforms but remains in a moderate range; in this benchmark, scaling behavior is dominated by CPU load and per-channel FPS rather than memory capacity limits.

Conclusion

Under the aligned 8-core configuration, both systems show the same pattern: as channel count increases and CPU usage approaches saturation, per-channel FPS drops from near real-time to lower values. In this workload, near real-time FPS per channel is sustained up to 9 channels on EXMP-Q911, while on NVIDIA Jetson AGX Orin 32GB the per-channel FPS is already below real-time at higher channel counts as the load increases.

How to Use

Below is an overview of how we conduct platform benchmark testing.

EXMP-Q911

1. 1
   Install iQ Studio

   {{ mc1.fname }} bash

   $ git clone https://github.com/InnoIPA/iQ-Studio.git
   $ cd iQ-Studio
   $ ./install.sh

2. 2
   From the repository root on EXMP-Q911, change to the benchmark directory:

   {{ mc2.fname }} bash

   $ cd benchmarks/iq-9075/multi-stream/iqs-streampipe

3. 3
   Edit config.json so that the streams array contains 1, 4, 9, or 16 entries, depending on the channel count being tested.
4. 4
   Run the benchmark script with the Qualcomm platform selected:

   {{ mc3.fname }} bash

   $ ./scripts/auto_benchmark.sh \
       --platform qcom \
       --test_time 300 \
       --warmup_time 180 \
       --output ./logs/qcom_chX.txt

   

NVIDIA Jetson AGX Orin 32GB procedure

1. 1
   Download the archive using the following command.

   {{ mc4.fname }} bash

   $ wget https://github.com/InnoIPA/iQ-Studio/releases/download/v0.0.6/benchmark_nvidia_sources.tar.gz

2. 2
   Under benchmarks/iq-9075/multi-stream/iqs-streampipe, extract the NVIDIA benchmark archive:

   {{ mc5.fname }} bash

   $ tar xzf benchmark_nvidia_sources.tar.gz

   This provides config_nv.json, the videos/ directory, and the nvidia/streampipe_nv binary with its engine file.

3. 3
   Set the power mode to MAXN:

   {{ mc6.fname }} bash

   $ sudo nvpmodel -m 0

4. 4
   Run the benchmark script with the NVIDIA platform selected:

   {{ mc7.fname }} bash

   $ ./scripts/auto_benchmark.sh \
       --platform nv \
       --test_time 300 \
       --warmup_time 180 \
       --output ./logs/nv_chX.txt