Model Compiler Activation Inside the MCP Server Container

How the Model Compiler (afe package) is installed, activated, and referenced inside the MCP server's Docker container environment.

This document covers how the Model Compiler (afe package) is installed and activated within the MCP server's Docker container environment. It explains the three Python environments involved, the different activation methods available in the container, and how the MCP server references, loads, and uses the Model Compiler during execution.

Overview

The MCP server (mcp-server/docker_mcp.py) runs on the host and proxies every command into the sima-neat-sandbox container via the Docker SDK. The Model Compiler (sima-frontend / afe package) is installed inside the container at /root/sdk-extensions/model-compiler/ — a dedicated Python virtual environment.

This architecture matters because it separates the control plane from the execution environment: the host MCP server decides what to run, while the container provides the isolated runtime where the compiler, dependencies, and filesystem state actually live. Understanding that boundary is key to debugging command routing, locating installed packages, and knowing where changes take effect.

The next sections will walk through the host-to-container flow, how the Docker SDK is used to invoke commands, where the Model Compiler code lives inside the sandbox, and how to trace a request from the MCP server all the way to execution in the virtual environment.

Container Python Environments

There are three distinct Python environments inside the container because they serve different layers of the workflow and have different dependency needs. The system Python is the baseline runtime used for default NEAT SDK operations. The sima-cli virtual environment is dedicated to the CLI tooling that handles package downloads and authentication, so it can manage its own dependencies without affecting the rest of the container. The Model Compiler must stay in its own isolated virtual environment because afe depends on a specialized stack for quantization and compilation that may conflict with the system runtime or CLI packages if they were installed together. Keeping these environments separate reduces dependency collisions, makes upgrades safer, and lets each tool be maintained independently while still running inside the same container.

For the MCP server, this separation means command routing has to be precise: each task must invoke the correct interpreter or venv rather than assuming a single shared Python installation. The server can safely proxy commands into the container, but it must preserve environment boundaries so CLI operations, SDK defaults, and compilation workflows do not interfere with one another. In practice, that makes the MCP server more reliable, because failures in one environment are less likely to break the others, and troubleshooting becomes easier when each tool has a clearly defined runtime.

How the Model Compiler Was Installed

The Model Compiler was set up by first downloading the required wheel packages through sima-cli, then creating an isolated Python virtual environment, and finally installing the compiler's core packages and their runtime dependencies in a controlled sequence. This approach ensures the compiler runs in its own environment without interfering with the system Python installation, while also avoiding build issues from optional dependencies that are not needed for this setup.

# 1. Download the Model Compiler wheel packages via sima-cli
sima-cli install -v 2.1.2 tools/model-compiler/amd64
# This downloaded 19 resources (183.6 MB) to /workspace/
# including:
# sima_frontend-2.1.0.dev0+master.388-cp312-cp312-linux_x86_64.whl
# sima_tvm, sima_mlc, sima_ml_kernels, sima_utils, etc.

# 2. Create the virtual environment
python3 -m venv /root/sdk-extensions/model-compiler

# 3. Install the core packages with --no-deps (avoiding llama_cpp_python build)
/root/sdk-extensions/model-compiler/bin/pip3 install \
  --no-deps \
  sima_frontend-2.1.0.dev0+master.388-cp312-cp312-linux_x86_64.whl \
  sima_tvm*.whl sima_mlc*.whl sima_ml_kernels*.whl \
  sima_utils*.whl sima_accelerator*.whl \
  sima_ev_transforms*.whl swml_qat*.whl

# 4. Install runtime dependencies for those packages
/root/sdk-extensions/model-compiler/bin/pip3 install \
  numba jax jaxlib decorator pytorch-lightning onnx onnxoptimizer \
  neural_compressor mpk_parser

# 5. Downgrade numpy (the model compiler requires numpy < 2)
/root/sdk-extensions/model-compiler/bin/pip3 install 'numpy<2'

Each step serves a specific purpose: step 1 retrieves the prebuilt compiler wheels, step 2 creates a clean environment for the install, step 3 installs the compiler's own packages without triggering dependency resolution, step 4 adds the libraries those packages expect at runtime, and step 5 pins numpy to a compatible version. The --no-deps flag is important because it prevents pip from trying to fetch and build extra transitive dependencies such as llama_cpp_python, which could introduce unnecessary compilation work or installation failures. numpy must be downgraded because the model compiler is not compatible with numpy 2; keeping it below version 2 avoids API incompatibilities and ensures the compiler stack functions correctly.

Activation Methods

Activating the model compiler environment is important because it ensures the correct Python interpreter, dependencies, and environment variables are available before you run commands. In an interactive shell, activation can persist for the duration of your session; in a non-interactive shell, each command starts fresh, so activation must be applied differently or bypassed entirely.

source activate-model-compiler

# Use the venv Python directly (no activation needed)
/root/sdk-extensions/model-compiler/bin/python3 \
  -c "import afe; print(afe.__version__)"

For the MCP server, the direct-path method is recommended because it avoids dependence on shell persistence and guarantees the model compiler runs from the intended virtual environment on every call.

How the MCP Server References the Model Compiler

The MCP server's command execution flow is intentionally simple but has an important consequence for environment setup: in mcp-server/docker_mcp.py, the _exec() function launches every command through container.exec_run(cmd=f"bash -c '{command}'", workdir="/workspace"). That means each tool invocation starts in a fresh shell rather than continuing a previously activated session. Understanding this reference method is critical because the model compiler lives inside a Python virtual environment, and whether commands are launched through the venv's interpreter or through the system Python determines if imports like afe succeed. In other words, the server's architecture makes activation state temporary, so tools must either re-establish the environment every time or bypass activation entirely by calling the correct interpreter directly.

Current MCP tools that interact with the Model Compiler

Overall, check_model_compiler() is useful as a lightweight presence check but does not validate runtime readiness. activate_model_compiler() is the closest to a correct approach because it explicitly activates the environment before testing afe, but it still depends on a shell session that disappears after the call ends. quantize_compile_model() is the most fragile of the three because it runs without activation and therefore can fail whenever the system Python does not include the model compiler dependencies. The practical takeaway is that the direct-venv pattern is the most reliable way to run model-compiler-backed commands in this MCP architecture.

The Activation Gap

The MCP server's quantize_compile_model() tool calls quantize_compile.py with the system python3 rather than the Model Compiler virtual environment's Python interpreter. That creates an activation gap: the tool assumes the environment is ready, but the script depends on packages that are only available after the Model Compiler venv has been activated.

What the activation gap is

The gap is the mismatch between how the tool is launched and what the script requires. quantize_compile.py imports afe directly, so it expects the Model Compiler environment to already be active. When the tool uses the system interpreter, that environment is missing, along with the package path and dependencies that activation normally provides.

Why it happens

• quantize_compile_model() invokes python3 skills/quantize_compile/scripts/quantize_compile.py without first running activate-model-compiler.
• The python3 on $PATH is often the system Python, not the venv Python.
• Activation is usually what sets PYTHONPATH, updates $PATH, and makes afe importable.
• Because the activation step is skipped, the script starts in the wrong runtime context.

What happens when the script runs without activation

When quantize_compile.py starts, it immediately tries to import modules from afe. If the Model Compiler venv is not active, Python cannot resolve that import and the script fails very early, before any meaningful compile work begins.

In practice, the run will usually terminate with an import error such as:

Traceback (most recent call last):
  File "skills/quantize_compile/scripts/quantize_compile.py", line 1, in <module>
    from afe import ...
ModuleNotFoundError: No module named 'afe'

Depending on the environment, users may also see related errors such as:

• ImportError: cannot import name ... from afe
• ModuleNotFoundError for other Model Compiler dependencies loaded by afe
• Failing subprocess calls if the script partially starts but cannot find the expected tooling

Consequences

This issue is more than a convenience problem. It makes the tool unreliable because it only works in sessions where the caller manually activated the Model Compiler environment beforehand.

• Users get nondeterministic failures depending on which Python interpreter is chosen.
• Compile jobs fail immediately, wasting time and making the tool look broken.
• Automation becomes fragile because the tool depends on hidden shell state.
• Debugging becomes confusing, since the failure appears inside quantize_compile.py even though the root cause is the missing activation step.

Because the activation requirement is implicit, the tool violates the principle of self-contained execution. That means every caller has to know extra setup details, and any missed step causes a hard failure.

Proposed solution

The fix is to make quantize_compile_model() activate the Model Compiler environment before invoking the script, or to invoke the script directly through the venv's Python interpreter. Either approach ensures afe is importable every time.

• Preferred: run activate-model-compiler and then execute quantize_compile.py in the same shell context.
• Alternative: call the venv Python explicitly instead of the system python3.
• Add a preflight check so the tool fails fast with a clear message if the environment is not active.

Verification Commands

These commands provide a fast way to confirm the Model Compiler environment is wired correctly inside the running container. The first check verifies that the venv-backed Python interpreter can import afe and report its version, which confirms the expected runtime is available. The second command lists the installed packages so you can quickly spot whether the Model Compiler stack, including sima, afe, tvm, and mlc, is present. The final check confirms that the activation script exists at the expected location, which helps isolate whether the issue is caused by a missing install step, a bad image build, or an activation mismatch. Together, these commands make it easier to debug environment problems without guessing.

# Quick health check (from host)
docker exec sima-neat-sandbox \
  /root/sdk-extensions/model-compiler/bin/python3 \
  -c "import afe; print(afe.__version__)"

# Full package list
docker exec sima-neat-sandbox \
  /root/sdk-extensions/model-compiler/bin/pip3 list \
  | grep -iE "sima|afe|tvm|mlc"

# Check activation script exists
docker exec sima-neat-sandbox \
  ls -la /usr/local/bin/activate-model-compiler

If everything is working, the health check should print an afe version instead of an import error, the package list should return matching entries for the expected tools, and the activation script should appear in the directory listing with normal file permissions. Missing output, empty results, or errors like ModuleNotFoundError usually indicate the environment is not activated correctly or the container image is incomplete. Use the three checks together: import success confirms Python access, package output confirms installation, and the script check confirms the activation path.

Credential Dependency

Credential management is critical for the Model Compiler installation because sima-cli uses OAuth tokens to authenticate every install and update operation. If those tokens expire, sima-cli install can no longer retrieve or refresh the Model Compiler package, which means the MCP server may be unable to reinstall the compiler after a reset, recover from a broken environment, or apply future updates. To refresh access, sign in again with sima-cli on the host so a new token is written to ~/.sima-cli/.tokens.json, then copy the updated file into /root/.sima-cli/.tokens.json inside the sandbox or container. Keeping both token stores in sync ensures the MCP server retains the ability to reinstall or update the Model Compiler when needed.

The Model Compiler installation was downloaded through sima-cli, which authenticates via OAuth tokens cached at /root/.sima-cli/.tokens.json. These tokens were copied from the host (~/.sima-cli/.tokens.json) and must remain valid for sima-cli install operations.

3-Python Environment Summary

This environment is organized around three Python entry points inside the container: the system Python at /usr/bin/python3 for NEAT SDK, tutorials, and general operations; the sima-cli virtual environment at /root/.sima-cli/.venv/bin/python3 for authenticated package download and installation; and the Model Compiler virtual environment at /root/sdk-extensions/model-compiler/bin/python3 for afe v2.1.0 and related compiler packages such as sima-frontend, sima-tvm, sima-mlc, sima-ml-kernels, and swml-qat. The host-side MCP server bridges user requests into the container workflow, while OAuth tokens cached at /root/.sima-cli/.tokens.json enable sima-cli install operations and must remain valid for the toolchain to function end to end.

Container
│
├── /usr/bin/python3 (system)
│   └── NEAT SDK, tutorials, generic ops
│
├── /root/.sima-cli/.venv/bin/python3
│   └── sima-cli tool (download/install packages)
│
└── /root/sdk-extensions/model-compiler/bin/python3
    ├── afe v2.1.0 ← MODEL COMPILER
    └── sima-frontend, sima-tvm, sima-mlc,
        sima-ml-kernels, swml-qat

Key Files

Request Flow and Activation Steps

A user request begins at the MCP server on the host, where docker_mcp.py receives the job and translates it into container-based execution. If the request requires package retrieval or installation, the server first relies on the sima-cli environment, which depends on valid OAuth tokens in /root/.sima-cli/.tokens.json. Once dependencies are available, the workflow activates the Model Compiler environment using either /usr/local/bin/activate-model-compiler or the underlying venv activation script at /root/sdk-extensions/model-compiler/bin/activate. From there, the compilation pipeline runs through /workspace/skills/quantize_compile/scripts/quantize_compile.py, which uses the Model Compiler Python runtime and its related packages to perform quantization and compilation. In practice, the activation order is: host MCP server context first, sima-cli authentication if package access is needed, then Model Compiler environment activation, and finally the compile script execution. The system Python remains available throughout for general utilities, tutorials, and SDK-side support tasks that do not require the dedicated compiler venv.

Best Practices and Recommendations

• Keep the OAuth token file valid and monitor its expiry date before running installs.
• Use the activation wrapper script when possible to reduce mistakes in environment setup.
• Reserve the system Python for generic tooling and use the Model Compiler venv for compiler-specific work.
• Verify package availability before launching quantize and compile jobs to avoid mid-run failures.
• Document any changes to the MCP server or skill scripts so the request flow remains reproducible.