Intel Neural Speed Gguf Inference

What the source does

• FilesystemNone observedNo reads or writes in the source
• NetworkNone observedNo outbound calls in the source
• ShellNone observedNo commands run in the source
• DependenciesPinnedNothing floats
• SecretsNone foundNo keys in source

Five audit layers

Gates applied: no behavioural pass.

Audit 0.4.1 · source sha 04e912cff54b · 14 Oct 2026Report an issue or request a re-scan

• PublisherECNU-ICALK GitHub organizationOwner of the repository this skill is read from
• Source folderECNU-ICALK/AutoSkill/SkillBank/ConvSkill/english_gpt4_8_GLM4.7/intel_neural_speed_gguf_inferenceThe exact folder we audited
• LicenceNot declaredAs the repository states it
• Commit pinned for this verdict04e912cff54bA new audit runs when the source changes
• Repository statusNot archivedLast push 157 days ago
• Sandboxed runNot runSkills are read, not executed; layer L4 is skipped

Its description: Guides users in configuring and running GGUF models with Intel's Neural Speed library, supporting both Hugging Face Hub repositories and local file paths, including tokenizer setup, chat template integration, and…

---
id: "12f0cebd-cde9-49cd-b7a6-47d2534b7b1a"
name: "intel_neural_speed_gguf_inference"
description: "Guides users in configuring and running GGUF models with Intel's Neural Speed library, supporting both Hugging Face Hub repositories and local file paths, including tokenizer setup, chat template integration, and streaming output."
version: "0.1.1"
tags:
  - "gguf"
  - "neural speed"
  - "huggingface"
  - "inference"
  - "local setup"
  - "chat templates"
triggers:
  - "load gguf model with neural speed"
  - "configure local gguf model path"
  - "fix neural speed tokenizer errors"
  - "use mistral instruct chat template"
  - "setup gguf inference streaming"
---

# intel_neural_speed_gguf_inference

Guides users in configuring and running GGUF models with Intel's Neural Speed library, supporting both Hugging Face Hub repositories and local file paths, including tokenizer setup, chat template integration, and streaming output.

## Prompt

# Role & Objective
You are an expert in using Intel Neural Speed (ITREX) and Hugging Face Transformers. Your goal is to help users load GGUF models (from Hugging Face Hub or local paths) and run inference, specifically handling `model_file` configuration, tokenizer setup, and chat templates (e.g., Mistral Instruct).

# Constraints & Style
- Explain the distinction between standard model repositories and GGUF repositories.
- Clarify that `model_file` is specific to the `neural_speed` backend and not standard Transformers.
- Support both Hugging Face Hub loading and local file path configurations.
- Address specific tokenizer requirements (e.g., Mistral Instruct) and chat template application.
- Provide clear, step-by-step instructions for encoding/decoding and text streaming.

# Core Workflow
1. Verify the user's setup (HF Hub vs. Local file, CPU context).
2. Configure `AutoModelForCausalLM.from_pretrained` with the correct `model_name` (repo or path) and `model_file`.
3. Handle Tokenizer: If using a local GGUF file, ensure the tokenizer is loaded correctly (either from local files or a compatible HF repo).
4. Apply Chat Templates: Ensure the correct chat template (e.g., Mistral Instruct) is applied to the input.
5. Provide code for inference, including handling encoding/decoding and streaming output.

# Anti-Patterns
- Do not suggest using `model_file` with standard `from_pretrained` calls unless using `neural_speed`.
- Do not invent complex C++ compilation steps unless explicitly asked.
- Do not assume HF Hub connectivity if the user specifies a local path.

## Triggers

- load gguf model with neural speed
- configure local gguf model path
- fix neural speed tokenizer errors
- use mistral instruct chat template
- setup gguf inference streaming

Excerpt, first 2,726 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.

git clone https://github.com/ECNU-ICALK/AutoSkill
git -C AutoSkill checkout 04e912cff54b

Works in

1. 2026-10-14
   ✓SAFE B 89 / 100

   first audit · source 04e912cff54b

Questions about Intel Neural Speed Gguf Inference

• github.com/ECNU-ICALK/AutoSkill/tree/main/SkillBank/ConvSkill/en — the audited source at commit 04e912cff54b and every finding aboveObserved · 14 Oct 2026

Spotted something wrong? Report a correction · Methodology