Juicebox ObservabilitySAFE

Model-agnostic agent-skills platform with a harness-free canonical layer, verified adapters, and the ccpi package manager. Explore at tonsofskills.com.

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: juicebox-observability
description: 'Set up Juicebox monitoring.

  Trigger: "juicebox monitoring", "juicebox metrics".

  '
allowed-tools: Read, Write, Edit, Grep
version: 1.16.0
license: MIT
author: Jeremy Longshore <[email protected]>
tags:
- saas
- recruiting
- juicebox
compatibility: Designed for Claude Code
---
# Juicebox Observability

## Overview

Juicebox provides AI-powered people search and analysis where query performance, dataset ingestion rates, and quota consumption are the primary observability concerns. Monitor analysis completion times to ensure interactive UX, track ingestion pipeline health for data freshness, and watch quota usage to prevent mid-workflow cutoffs. Slow queries or failed ingestions degrade recruiter productivity and data accuracy.

## Key Metrics

| Metric | Type | Target | Alert Threshold |
|--------|------|--------|-----------------|
| Search latency p95 | Histogram | < 2s | > 5s |
| Analysis completion time | Histogram | < 10s | > 30s |
| Dataset ingestion rate | Gauge | > 100 records/s | < 50 records/s |
| API error rate | Gauge | < 1% | > 5% |
| Quota usage (daily) | Gauge | < 70% | > 85% |
| Query result relevance | Gauge | > 80% precision | < 60% |

## Instrumentation

```typescript
async function trackJuiceboxCall(operation: string, fn: () => Promise<any>) {
  const start = Date.now();
  try {
    const result = await fn();
    metrics.histogram('juicebox.api.latency', Date.now() - start, { operation });
    metrics.increment('juicebox.api.calls', { operation, status: 'ok' });
    return result;
  } catch (err) {
    metrics.increment('juicebox.api.errors', { operation, error: err.code });
    throw err;
  }
}
```

## Health Check Dashboard

```typescript
async function juiceboxHealth(): Promise<Record<string, string>> {
  const searchP95 = await metrics.query('juicebox.api.latency', 'p95', '5m');
  const errorRate = await metrics.query('juicebox.api.error_rate', 'avg', '5m');
  const quota = await juiceboxAdmin.getQuotaUsage();
  return {
    search_latency: searchP95 < 2000 ? 'healthy' : 'slow',
    error_rate: errorRate < 0.01 ? 'healthy' : 'degraded',
    quota: quota.pct < 0.7 ? 'healthy' : 'at_risk',
  };
}
```

## Alerting Rules

```typescript
const alerts = [
  { metric: 'juicebox.search.latency_p95', condition: '> 5s', window: '10m', severity: 'warning' },
  { metric: 'juicebox.api.error_rate', condition: '> 0.05', window: '5m', severity: 'critical' },
  { metric: 'juicebox.quota.daily_pct', condition: '> 0.85', window: '1h', severity: 'warning' },
  { metric: 'juicebox.ingestion.rate', condition: '< 50/s', window: '15m', severity: 'critical' },
];
```

## Structured Logging

```typescript
function logJuiceboxEvent(event: string, data: Record<string, any>) {
  console.log(JSON.stringify({
    service: 'juicebox', event,
    operation: data.operation, duration_ms: data.latency,
    result_count: data.resultCount, query_length: data.queryLen,
    // Redact candidate PII — log only aggregate counts
    timestamp: new Date().toISOString(),
  }));
}
```

## Error Handling

| Signal | Meaning | Action |
|--------|---------|--------|
| 429 rate limit | Quota exhausted for period | Pause queries, check daily allocation |
| Search timeout > 5s | Complex query or service load | Simplify filters, retry with narrower scope |
| Ingestion stall | Dataset too large or format error | Check upload logs, validate schema |
| Empty result set | Index gap or query mismatch | Verify dataset freshness, adjust search params |

## Prerequisites

- An approved telemetry schema, sandbox synthetic fixture, redaction policy, source/destination allowlists, suppression controls, retention window, and an owner for alert response.

## Instructions

1. Instrument aggregate operational signals only; reject raw queries, contact fields, enrichment values, credentials, and unapproved exports.
2. Validate a sandbox canary, confirm suppression, retention, redaction, and `contacts_exported=0`, then compare it with the approved baseline.
3. Pause ingestion or downstream delivery on scope, policy, quota, or retention drift and return to the last approved configuration.
4. Keep only redacted aggregate evidence and delete test telemetry after its approved window.

## Output

Produce an observability receipt with environment, event classes, aggregate volume/error/latency signals, redaction/suppression/no-export outcomes, alert owner, retention/deletion proof, and rollback reference.

## Examples

`env=ci-synthetic; events=aggregate-only; latency_p95=within-baseline; suppression=pass; contacts_exported=0; retention=24h; cleanup=verified` is a valid canary record.

## Resources

- Juicebox Dashboard

## Next Steps

See `juicebox-incident-runbook`.

SAFEgrade B · trust 89/100 Nothing in the source contradicts what it says it does. Grade A is reserved for packages that have also passed the behavioural sandbox.

What the source does

Filesystem

none-observed

Network

none-observed

Shell

none-observed

Dependencies

pinned

Secrets in source

none-found

Findings (0)

No findings outside the package's declared scope.

Gates applied: no_behavioural_pass.