160 FPS at the Edge: Deploying YOLO26s on the SiMa Modalix DevKit

A practical, no-nonsense walkthrough for getting a YOLO model running on dedicated NPU silicon.

We've all seen the flashy edge inference benchmarks boasting massive FPS numbers. But if you've ever actually tried taking a trained model checkpoint and squeezing genuine real-time throughput out of a dedicated NPU, you know the truth: the SDK quickstart guides conveniently skip the messy parts. They don't tell you how to export the model, where to split it before compression, which hyper-specific compiler flags your hardware actually demands, or how to map the raw output back into something usable.

This is a guide to getting YOLO26s running on the SiMa Modalix DevKit—taking a stock Ultralytics checkpoint and turning it into a compiled package doing real-time object detection on-device. The payoff? Roughly 160 frames per second at about 6.2 milliseconds of latency per frame. To get there, we have to navigate four specific bottlenecks where the wrong default setting will either break your build entirely or quietly sabotage your results. Let's break down the pipeline, the reasoning behind the fixes, and the actual commands you need to run.

How the Pipeline Is Organized

The workflow splits cleanly across two machines: your host computer and the DevKit itself. Think of your host machine as the workshop. This is where all the heavy, iterative prep work happens (exporting, restructuring, compressing, and compiling). Mistakes here are cheap and easy to fix. The DevKit (the edge device with the NPU) is just the showroom—it never sees the prep work; it only runs the finished, validated artifact.

# Start the SiMa SDK container
sima-cli sdk neat

# Inside the container, activate the model-compilation environment
activate-model-compilor

Everything from Step 2 onward assumes you are in this activated environment. If you get a "command not found" or import error, check that you're actually in the container before tearing your hair out over the pipeline.

Step 1: Gathering a Sample Set of Images

Before we can compress the model, we need a representative handful of real-world images. We don't need thousands—just a few dozen from a standard public dataset will do. These serve a dual purpose: calibrating the compression process and sanity-checking the deployed model later.

# Download COCO val2017 images
wget http://images.cocodataset.org/zips/val2017.zip
unzip val2017.zip

# Download annotations
wget http://images.cocodataset.org/annotations/annotations_trainval2017.zip
unzip -j annotations_trainval2017.zip 'annotations/instances_val2017.json'

# Pull out a 50-image subset for calibration + evaluation
mkdir -p calib_images
ls val2017/ | head -50 | while read f; do cp "val2017/$f" calib_images/; done

Step 2: Exporting the Model in Its Raw Form

Your first big decision is how to export the trained model. YOLO detectors usually give you the option to bake the final cleanup steps (like filtering overlapping detections) right into the exported model. Don't do it. Keep the model's internal structure simple by exporting only the raw prediction output. More importantly, leaving it raw keeps the predicted box positions and the class confidences bundled together for now, which sets us up for the most critical fix in this entire guide (see Step 4).

Step 3: Locking the Model to a Fixed Input Size

Hardware compilers despise flexibility. A model that can theoretically accept images of any shape or size is a nightmare to optimize. Do yourself a favor and mechanically lock your model to a single, fixed input size before compression. Stripping out the conditional logic leftover from training will save you from cryptic compiler errors down the road.

Step 4: The Fix That Matters Most

If you only take one idea from this guide, make it this: separate the two kinds of output. A YOLO detector does not produce one clean, uniform result. Instead, it emits a bundled set of values for every candidate box: the box coordinates, which describe where the object is and how large it is, plus the class confidence scores, which describe how likely each class is. On paper, they arrive together. In practice, they are very different kinds of numbers.

That difference matters because the two outputs live on completely different scales. Box coordinates are usually measured in pixels, so they can reach into the hundreds or even higher depending on the image size. Confidence scores, by contrast, are tiny probabilities or logits that often sit between zero and one. They carry very different meaning, and they behave very differently when you compress them.

Once you compress a model to a lower-precision format for the NPU, the hardware toolchain typically applies one shared scale across an entire output tensor. That works only when the values in that tensor are numerically similar. But here, the coordinates and the confidence scores are not similar at all. If the scale is widened enough to preserve the large coordinate values, the small confidence values lose their detail and collapse into nearly useless noise.

In other words, the model may still run, but it stops being trustworthy. The detections start to look unstable because the confidence values no longer have enough resolution to express meaningful differences between classes. A score that should clearly separate a strong prediction from a weak one can get flattened into something too coarse to use well.

This is a tiny rewrite of the output structure, but it has an outsized effect. It allows the compiler and quantization process to treat each kind of data appropriately instead of forcing both through the same numeric bottleneck. The result is simple: the detector can still fit the hardware constraints without sacrificing the quality of its predictions.

Step 5: Compressing and Compiling for the Hardware

With the outputs already separated into clean, hardware-friendly groups, we can finally move into the part that turns a trained model into something the Modalix chip can actually execute efficiently: compression and compilation.

Compression reduces numerical precision so the model uses less memory and fits the chip's execution constraints. Compilation goes a step further—it rewrites the model into the exact instruction flow the NPU expects. On a chip like Modalix, this matters a lot because the hardware is optimized for aggressive parallelism, fast on-chip routing, and specialized internal layouts that are much more rigid than a general-purpose GPU or CPU.

That flexibility is also where the pain comes from. Modalix supports advanced compute patterns, including attention-style operations and other nonstandard detection-head behaviors, but those patterns do not automatically map cleanly onto the chip's default assumptions. If the compiler is left to guess, it may place tensors in the wrong geometry or choose a layout the runtime cannot execute efficiently. The result is the dreaded geometry mismatch error—usually discovered only after a long compile cycle, and often with very little context to explain what went wrong.

This is why the process has to be explicit. You are not just shrinking the model; you are teaching the compiler how to respect the chip's internal structure. The more accurately you describe the model's layout and data flow, the more likely the compiler is to generate a build that is both correct and performant.

python3 quantize_compile.py \
  --model_path yolo26s_noe2e_split.onnx \
  --model_layout NHWC \
  --device modalix \
  --build_dir ./build \
  --real_data \
  --dataset_images ./calib_images \
  --num_calib_samples 50 \
  --calib_method mse \
  --any_shape_on_mla \
  --auto_layout \
  --mla-tesselation \
  --compile

Why these flags matter

Step 6: Moving the Compiled Package to the Device

The hard part is over. Now, we just transfer the compiled package and our test images over the network to the DevKit.

DEVKIT_IP="${1:-192.168.0.27}"
DEVKIT_USER="${2:-sima}"
REMOTE_DIR="/home/${DEVKIT_USER}/yolo_eval"

# Prepare remote directory
ssh "${DEVKIT_USER}@${DEVKIT_IP}" "mkdir -p ${REMOTE_DIR}/test_images"

# Copy the compiled package
scp build/yolo26s_noe2e_split/yolo26s_noe2e_split_mpk.tar.gz \
  "${DEVKIT_USER}@${DEVKIT_IP}:${REMOTE_DIR}/"

# Copy the same sample images used for calibration
scp calib_images/*.jpg "${DEVKIT_USER}@${DEVKIT_IP}:${REMOTE_DIR}/test_images/"

# Copy the dataset's annotation file (vital for Step 7)
scp instances_val2017.json "${DEVKIT_USER}@${DEVKIT_IP}:${REMOTE_DIR}/"

Step 7: Running Inference and Making Sense of the Output

Now we run the pipeline end-to-end on the hardware. Here is what happens under the hood to get usable data back:

Kick off the loop on your device with:

source ~/pyneat/bin/activate
cd ~/yolo_eval
python3 evaluate_devkit.py \
  yolo26s_noe2e_split_mpk.tar.gz \
  test_images \
  instances_val2017.json \
  50

Step 8: The "Sanity Check" (Optional but Recommended)

Before you blame the NPU or the compiler for weird results, run the exported model on your standard host computer using a basic CPU inference script. We aren't looking for performance here; we are isolating variables. If your bounding boxes are drawing in the wrong places, running it on the CPU will tell you if you have a bug in your Python resizing math, rather than sending you on a wild goose chase debugging hardware quantization.

Comparing CPU vs NPU Outputs

Once the CPU baseline looks right, compare it with the NPU output on the same image. Small differences are normal, but the overall result should still be recognizable and stable.

Objectives Achieved

Practical Debugging Tips

• Use one simple test image first, preferably with large, obvious objects.
• Keep a copy of the exact preprocessing code used for both CPU and NPU runs.
• Log image shape, scale ratio, padding values, and output tensor shape.
• Test with NMS off once to see whether the issue is decoding or suppression.
• Verify that the same label file and class order are used in every stage.
• If the CPU result is wrong too, fix the Python pipeline before touching the hardware path.

A good sanity check saves hours: it turns vague "the accelerator is broken" symptoms into a short list of concrete places to inspect and fix.

The Numbers That Matter

When everything is working, the jump in performance validates all the headaches.