Measure prompt cache hits to verify cost savings
Anthropic prompt caching silently fails in four ways (misplaced breakpoints, prefix drift, TTL expiration, unmeasured hit rates); wrap the SDK with explicit cache metrics to catch regressions.
Cache setup looks successful on first call but regresses silently—without measuring cache_read_input_tokens per request, you're likely paying full price on stale cached prefixes. Visibility prevents unexpected billing spikes.
Replaces manual response parsing and guesswork with a drop-in SDK wrapper that tracks hit rate, cost savings, and emits passive warnings on cache failures. Zero dependencies, ~50KB. Worth installing immediately if you're using Anthropic's cache API.
- “the only way to verify a call had hit the cache was to manually parse cache_read_input_tokens from the response usage on every request”
- “The cache only hits if the cacheable prefix is byte-identical to what was cached”
- “Anthropic recently dropped the default cache TTL from 1 hour to 5 minutes”
- “A chatbot answering 1000 questions/day with a 10K-token system prompt easily hits 70%+ cost reductions”
anthropicprompt-cachingcost-optimizationtypescriptmonitoring
elementary-data PyPI package publishes credential stealer
Version 0.23.3 compromised via GitHub Actions script injection; malware harvests dbt profiles, cloud credentials, SSH keys, and secrets at interpreter startup using .pth file execution.
If you installed elementary-data==0.23.3, the malware activated on every Python startup before your code ran, exfiltrating all accessible credentials (AWS, GCP, Azure, Snowflake, Kubernetes, SSH) regardless of whether you explicitly imported the package. This affects data engineers running the CLI against connected warehouses and any developer machine where the package was installed.
Immediately uninstall elementary-data==0.23.3 and upgrade to 0.23.4. Rotate all credentials that could have been on affected machines: dbt profiles, cloud provider keys, SSH keys, .env files, API tokens, Kubernetes configs. Scan CI/CD runners and developer machines for $TMPDIR/.trinny-security-update (Linux/macOS) or %TEMP%\.trinny-security-update (Windows) as evidence of execution. This is not optional—the payload is confirmed live on PyPI and requires immediate action.
- “elementary-data is a dbt-native data observability CLI tool used by data and analytics engineers to monitor pipeline health, detect anomalies, and track test failures across data warehouses like Snowflake, BigQuery, Redshift, and Databricks”
- “over 1.1 million per month”
- “The vulnerable run: block directly interpolated ${{ github.event.comment.body }} into a shell script before bash parsing occurred”
- “Any line in a .pth file that begins with import is executed as Python code at interpreter startup, before your own code runs”
- “Installing it is sufficient”
- “the decoded payload: Harvested credentials and secrets across the filesystem, targeting a broad set of material: dbt profiles (~/.dbt/profiles.yml) and data warehouse credentials (Snowflake, BigQuery, Redshift, Databricks). Cloud provider credentials: AWS ~/.aws/credentials plus live role credentials fetched from the IMDSv2 metadata endpoint”
- “Left a marker file at $TMPDIR/.trinny-security-update (Linux/macOS) or %TEMP%\.trinny-security-update (Windows), indicating the malware executed at least once”
supply-chain-securitygithub-actions-injectioncredential-theftpypidbt
HELLoRA targets MoE experts for efficient adaptation
Attach LoRA modules only to frequently activated experts per layer, reducing trainable parameters to 15.7% of vanilla LoRA while improving accuracy 9.2% on OlMoE.
MoE model fine-tuning is now cheaper: less memory, faster training (1.9x throughput gain), and better task performance without full-model adaptation overhead. Matters if you're scaling PEFT across sparse architectures.
Replaces vanilla LoRA for Mixtral, DeepSeekMoE, OlMoE workloads. Requires activation pattern tracking at inference and modified adapter placement logic. Credible enough to test on existing MoE pipelines—the parameter reduction is measurable and the approach is straightforward to implement.
- “Mixture-of-Experts (MoE) models scale parameters at near-constant per-token compute, and their sparse activation patterns create untapped opportunities for more efficient adaptation”
- “Relative to vanilla LoRA on OlMoE, HELLoRA uses 15.7% of the trainable parameters, reduces adapter FLOPs by 38.7%, achieves 1.9x the training throughput, and improves accuracy by 9.2%”
- “activation-aware adapter placement is an effective and practical route to scaling PEFT for MoE language models”
moe-modelsloraparameter-efficient-finetuningsparse-activationefficient-training
Route cheap work away from expensive models
Agent cost explodes not from reasoning calls but from using Claude Opus for heartbeat checks, status validation, and retry logic—move those to cheaper models or simple code.
Long-running agents become expensive when supervision logic retries on expensive models. Separating task routing by complexity cuts spend to one-third while improving reliability through explicit state and hard retry limits.
Replaces all-Claude-Opus architectures and prompt-based loop prevention. Requires explicit state storage (Redis/Postgres), coded retry limits, and task triage logic. Worth implementing immediately—the pattern is proven across n8n, Make, Zapier, and custom agents.
- “moving heartbeat checks, cron pings, and other low-value supervision off Claude Opus cut spend to about one-third”
- “The expensive part usually isn't the main reasoning step. It's the invisible scaffolding around it.”
- “heartbeat checks, cron-trigger validation, retry bookkeeping, simple routing, status classification, watchdog logic, 'did this step finish?' checks”
- “Agents rarely become expensive because one prompt was huge. They become expensive because a workflow can't confidently tell whether it succeeded.”
- “A lot of teams pay premium model costs to compensate for weak state handling. That's backwards. Better state is cheaper than better prompting.”
agent-cost-optimizationmodel-routingstate-managementlong-running-workflowsretry-logic
DecisionBench measures router fidelity across agentic delegation
New benchmark suite isolates delegation routing quality (7.5%–29.5% fidelity-at-1) from end-task quality, revealing that delivery channel beats description content for model selection.
If you're building multi-model orchestration systems, you need to measure routing decisions independently from task outcomes—quality-only evals hide whether your router is actually picking the right model. DecisionBench gives you the substrate to test learned routers, adaptive profiles, and delegation strategies against 23k task instances with normalized metrics.
This replaces ad-hoc delegation benchmarking with a standardized reference harness covering GAIA, tau-bench, BFCL. Requires instrumenting your agentic workflow with a call_model interface and optional peer profiles. Worth adopting now if you're evaluating orchestration methods; the released substrate and 220 run archives let you baseline immediately without reproducing their sweep.
- “mean end-task quality is statistically indistinguishable across the four awareness conditions (|beta| <= 0.010, p >= 0.21), so quality-only evaluation would miss the orchestration signal”
- “routing fidelity-at-1 ranges from 7.5% to 29.5% across conditions at near-equal mean quality, with delivery channel (on-demand tool vs. preloaded description) dominating description content”
- “a counterfactual ceiling places perfect delegation 15-31 percentage points above measured performance on every suite, locating large unrealized headroom for future orchestration methods”
- “We release the substrate, annotation layer, reference intervention suite, analysis pipeline, and 220 per-condition run archives”
multi-model-routingagentic-workflowsdelegation-benchmarkingorchestration