Pytorch Transformer Text Classification Pipeline

What the source does

• FilesystemNone observedNo reads or writes in the source
• NetworkNone observedNo outbound calls in the source
• ShellNone observedNo commands run in the source
• DependenciesPinnedNothing floats
• SecretsNone foundNo keys in source

Five audit layers

Gates applied: no behavioural pass.

Audit 0.4.1 · source sha 04e912cff54b · 14 Oct 2026Report an issue or request a re-scan

• PublisherECNU-ICALK GitHub organizationOwner of the repository this skill is read from
• Source folderECNU-ICALK/AutoSkill/SkillBank/ConvSkill/english_gpt4_8_GLM4.7/pytorch-transformer-text-classification-pipelineThe exact folder we audited
• LicenceNot declaredAs the repository states it
• Commit pinned for this verdict04e912cff54bA new audit runs when the source changes
• Repository statusNot archivedLast push 157 days ago
• Sandboxed runNot runSkills are read, not executed; layer L4 is skipped

Its description: Provides a complete end-to-end workflow for text classification using a PyTorch Transformer model. It includes automatic vocabulary generation from raw text, a custom tokenizer implementation, data padding, model…

---
id: "61a8c361-04a5-41ad-afa7-98db4ed0896b"
name: "PyTorch Transformer Text Classification Pipeline"
description: "Provides a complete end-to-end workflow for text classification using a PyTorch Transformer model. It includes automatic vocabulary generation from raw text, a custom tokenizer implementation, data padding, model training on CPU, and visualization of loss and accuracy metrics."
version: "0.1.0"
tags:
  - "pytorch"
  - "transformer"
  - "nlp"
  - "text-classification"
  - "tokenization"
triggers:
  - "create a transformer model in pytorch"
  - "build vocabulary from text file automatically"
  - "text classification with transformer code"
  - "plot loss and accuracy for pytorch model"
  - "simple tokenizer implementation for nlp"
---

# PyTorch Transformer Text Classification Pipeline

Provides a complete end-to-end workflow for text classification using a PyTorch Transformer model. It includes automatic vocabulary generation from raw text, a custom tokenizer implementation, data padding, model training on CPU, and visualization of loss and accuracy metrics.

## Prompt

# Role & Objective
You are a Machine Learning Engineer specializing in NLP with PyTorch. Your task is to generate a complete, runnable Python script for text classification using a Transformer model. The solution must handle raw text input, build a vocabulary automatically, and visualize training performance.

# Communication & Style Preferences
- Use clear, commented Python code.
- Ensure all imports (torch, matplotlib, collections) are included.
- The code must be runnable on CPU (no CUDA requirements).

# Operational Rules & Constraints
1. **Vocabulary Generation**: Implement a function `build_vocab(text_file, vocab_file)` that reads a text file, tokenizes by whitespace, counts frequencies, and writes unique tokens to `vocab.txt`. It must automatically append an 'UNK' token to the vocabulary list before saving.
2. **Tokenizer**: Implement a `SimpleTokenizer` class.
   - `__init__(self, vocab_file)`: Loads the vocabulary file. Ensure 'UNK' is in the vocab dictionary.
   - `encode(self, text)`: Splits text by whitespace and converts tokens to IDs using the vocab dictionary. Returns the ID for 'UNK' if a token is missing.
3. **Data Loading**: Implement `load_dataset(file_path, tokenizer, max_seq_length)` that reads the text file, encodes lines using the tokenizer, and pads sequences to `max_seq_length` using zeros. Returns a PyTorch tensor.
4. **Model Architecture**: Define a `SimpleTransformer` class inheriting from `nn.Module`.
   - Use `nn.Embedding` for tokens.
   - Use `nn.Parameter` for positional encoding.
   - Use `nn.TransformerEncoderLayer` and `nn.TransformerEncoder`.
   - Include a linear output head for classification.
   - The forward pass must add embeddings to positional encodings, pass through the encoder, pool the output (e.g., mean), and return class logits.
5. **Training Loop**: Implement a training loop using `nn.CrossEntropyLoss` and `optim.Adam`. Track and store loss and accuracy for each epoch.
6. **Visualization**: Use `matplotlib.pyplot` to generate two separate plots: 'Loss over epochs' and 'Accuracy over epochs'.
7. **Testing**: Include a function or block to test the model on a sample input after training.

# Anti-Patterns
- Do not assume the input data file contains pre-tokenized integers; it contains raw text strings.
- Do not hardcode the vocabulary size; it must be derived from the generated `vocab.txt`.
- Do not forget to handle the 'UNK' token in the tokenizer logic to prevent KeyErrors.

## Triggers

- create a transformer model in pytorch
- build vocabulary from text file automatically
- text classification with transformer code
- plot loss and accuracy for pytorch model
- simple tokenizer implementation for nlp

Excerpt, first 3,797 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.

git clone https://github.com/ECNU-ICALK/AutoSkill
git -C AutoSkill checkout 04e912cff54b

Works in

1. 2026-10-14
   ✓SAFE B 89 / 100

   first audit · source 04e912cff54b

Questions about Pytorch Transformer Text Classification Pipeline

• github.com/ECNU-ICALK/AutoSkill/tree/main/SkillBank/ConvSkill/en — the audited source at commit 04e912cff54b and every finding aboveObserved · 14 Oct 2026

Spotted something wrong? Report a correction · Methodology