Ddia SystemsCAUTION

Wondel.ai Agent Skills — Business, Marketing, UX & Coding Frameworks from Bestselling Books. 50 skills + 12 guided journeys for Claude Code, Codex, Cursor & other agentskills.io agents.

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: ddia-systems
description: 'Design data systems by understanding storage engines, replication, partitioning, transactions, and consistency models. Use when the user mentions "database choice", "which database should I use", "SQL or NoSQL", "replication lag", "partitioning strategy", "consistency vs availability", "stream processing", "ACID transactions", "eventual consistency", "my queries are slow at scale", or "data is inconsistent across replicas". Also trigger when choosing a datastore, designing data pipelines, or debugging distributed-system consistency issues. Covers data models, batch/stream processing, and distributed consensus. For system design, see system-design. For resilience, see release-it.'
license: MIT
metadata:
  author: wondelai
  version: "1.4.0"
---

# Designing Data-Intensive Applications Framework

A principled approach to building reliable, scalable, and maintainable data systems. Apply these principles when choosing databases, designing schemas, architecting distributed systems, or reasoning about consistency and fault tolerance.

## Core Principle

**Data outlives code.** Applications are rewritten and frameworks come and go, but data persists for decades -- prioritize the long-term correctness, durability, and evolvability of the data layer. Most applications are data-intensive, not compute-intensive: the hard problems are data volume, complexity, and rate of change, and explicit consistency/availability/latency trade-offs separate robust systems from fragile ones.

## Scoring

**Goal: 10/10.** Score a data architecture by the seven Quick Diagnostic rows below: award ~1.4 points per row answered "yes" with evidence (deliberate, documented trade-off), 0 where the answer is "no" or unknown.

- **9-10:** every domain choice -- data model, storage engine, replication, partitioning, isolation, derived-data, fault handling -- is deliberate, documented, and matched to actual read/write/consistency requirements; failover tested.
- **5-6:** core choices made but two or three diagnostic rows fail -- e.g. default isolation level unknown, hot-key risk unhandled, or failover untested.
- **<=3:** choices driven by familiarity, not requirements; ignored failure modes (replication lag, write skew, hot partitions) and accidental complexity dominate.

Report the current score, which diagnostic rows failed, and the improvements needed to reach 10/10.

## The DDIA Framework

Seven domains for reasoning about data-intensive systems:

### 1. Data Models and Query Languages

**Core concept:** The data model shapes how you think about the problem. Relational, document, and graph models each impose different constraints and enable different query patterns.

**Why it works:** Choosing the wrong data model forces application code to compensate for representational mismatch, adding accidental complexity that compounds over time.

**Key insights:**
- Relational models excel at many-to-many relationships and ad-hoc queries; document models at one-to-many relationships and locality; graph models at recursive traversals over interconnected data
- Schema-on-write (relational) catches errors early; schema-on-read (document) offers flexibility
- Polyglot persistence -- different stores for different access patterns -- is often the right answer
- Object-relational impedance mismatch is a real cost; document models reduce it for self-contained aggregates

**Code applications:**

| Context | Pattern | Example |
|---------|---------|---------|
| **User profiles with nested data** | Document model for self-contained aggregates | Profile, addresses, and preferences in one MongoDB document |
| **Social network connections** | Graph model for relationship traversal | Neo4j Cypher: `MATCH (a)-[:FOLLOWS*2]->(b)` for friend-of-friend |
| **Financial ledger with joins** | Relational model for referential integrity | PostgreSQL foreign keys between accounts, transactions, entries |

See [references/data-models.md](references/data-models.md) when picking relational vs document vs graph or evaluating schema-on-read -- adds the full trade-off matrix and query-language comparisons.

### 2. Storage Engines

**Core concept:** Storage engines trade off read performance against write performance. Log-structured engines (LSM trees) optimize writes; page-oriented engines (B-trees) balance reads and writes.

**Key insights:**
- LSM trees: append-only writes, periodic compaction, excellent write throughput, higher read amplification
- B-trees: in-place updates, predictable read latency, write amplification from page splits
- Write amplification (one logical write causing multiple physical writes) matters for SSDs with limited write cycles
- Column-oriented storage dramatically improves analytical queries through compression and vectorized processing
- In-memory databases are fast because they avoid encoding overhead, not because they avoid disk

**Code applications:**

| Context | Pattern | Example |
|---------|---------|---------|
| **High write throughput** | LSM-tree engine | Cassandra or RocksDB for time-series ingestion at 100K+ writes/sec |
| **Mixed read/write OLTP** | B-tree engine | PostgreSQL B-tree indexes for transactional point lookups |
| **Analytical queries** | Column-oriented storage | ClickHouse or Parquet for scanning billions of rows, few columns |

See [references/storage-engines.md](references/storage-engines.md) when a workload is read/write-bound or you must choose indexes -- adds write/read-path diagrams, compaction strategies, column storage, and a benchmark-driven decision procedure.

### 3. Replication

**Core concept:** Replication keeps copies of data on multiple machines for fault tolerance, scalability, and latency reduction. The core challenge is handling changes consistently.

**Why it works:** Every replication strategy trades off consistency, availability, and latency. Making the trade-off explicit prevents subtle anomalies that surface only under load or failure.

**Key ins

CAUTIONgrade B · trust 89/100 Install with care. The audit found things worth knowing before you trust its output.

What the source does

Filesystem

none-observed

Network

none-observed

Shell

none-observed

Dependencies

pinned

Secrets in source

none-found

Findings (5)

.agents/skills/37signals-way

.agents/skills/architecture-optimization

.agents/skills/blue-ocean-strategy

.agents/skills/clean-architecture

.agents/skills/clean-code

Gates applied: no_behavioural_pass.