I Built an AI News Digest with n8n and Claude API — Here's Everything That Went Wrong (and How I Fixed It)

I was spending about 45 minutes every morning doing the same thing: opening Hacker News, scrolling through r/LocalLLaMA, checking the Anthropic and OpenAI blogs, skimming Ars Technica, reading Simon Willison's latest post, glancing at TechCrunch AI. Twelve-plus sources, most of them noisy, trying to figure out which 5-10 stories actually mattered.

That is a bad use of a human brain. So I built a workflow to do it for me.

This is the chronological story of building an AI-powered news digest with n8n and the Claude API — from the first line of SQL to the first Slack message landing in my channel. It took about 7 hours across an evening and a day. I hit 11 distinct errors. One of them cost me 90 minutes. Here is what actually happened.

Why Not Use an Existing Template?

Before building anything, I spent time looking at what already exists. There are dozens of free n8n RSS-to-AI templates on the community hub. I tried several. Every single one had at least half of these problems:

1. No full-text extraction. They summarize RSS excerpts — the 2-3 sentence blurb that feeds include. You cannot meaningfully analyze an article from its RSS excerpt.
2. All OpenAI-based. Zero templates using Claude. Every one hardcodes GPT-3.5 or GPT-4.
3. No deduplication. The same story appears across multiple feeds daily. Without dedup, your digest has duplicates.
4. No relevance scoring. Articles are either all included or randomly truncated. No intelligence about what actually matters.
5. No feed health monitoring. When a feed URL dies, the workflow silently breaks.
6. No cost tracking. No visibility into what each run costs.
7. No professional formatting. Plain text output, not Block Kit or Discord embeds.
8. Single delivery channel. Discord OR Slack OR email. Never multiple.
9. No persistent history. No database, no audit trail, no "what did the digest look like last Tuesday?"
10. No documentation. A bare JSON file with no explanation of why anything is configured the way it is.

I wanted all ten of these solved. So I built it from scratch.

Architecture Overview

The workflow runs daily at 7 AM and follows a linear pipeline:

Schedule Trigger (7 AM)
  -> Fetch 10 RSS/API feeds
  -> Parse 4 different formats (RSS, Atom, HN API, Reddit JSON)
  -> 24-hour freshness filter
  -> 2-tier deduplication (URL hash + fuzzy title matching)
  -> 7-day database dedup against previous runs
  -> Select top 25 articles by social score
  -> Extract full text via Jina Reader (free tier)
  -> Analyze each article with Claude Haiku 4.5 (structured JSON)
  -> Score and rank with configurable topic weights
  -> Select top 12, classify as lead/top/quick
  -> Compile digest narrative with Claude Sonnet 4.5
  -> Format for Discord embeds, Slack Block Kit, and markdown
  -> Deliver to channels
  -> Archive everything to Postgres

27 nodes total. Two Claude models. One Jina Reader integration. Four database tables.

Phase 1: Database and Feed Parsing

The Schema

I started with the database because everything downstream depends on it. Four tables:

• digest_feeds — A registry of RSS sources with health tracking (consecutive failures, last success timestamp).
• digest_runs — An execution log. Every run gets a row with timestamps, article counts, token usage, and dollar cost.
• digest_articles — The archive. Every article the system has ever processed, with its URL hash for dedup.
• digest_config — Key-value pairs for topic weights, article limits, and other tunables. Change your priorities without touching workflow code.

Parsing Four Feed Formats

My 10 feeds use four completely different formats:

Here is the first gotcha: n8n Code nodes run in a sandboxed VM. DOMParser is not available. fetch() is not available. cheerio is not available. You parse XML with regex or you do not parse XML at all.

So I wrote regex parsers for RSS and Atom that handle CDATA wrappers, nested tags, and the common XML variations you find in the wild (one feed wrapped its <link> tags in CDATA for no discernible reason — I still don't understand why). It is not elegant. It works.

Each format also has different date fields and social signal data. HN gives you points and num_comments. Reddit gives you score and num_comments. RSS and Atom give you nothing — just a publication date. The parser normalizes all of this into a single article format with a socialScore field.

The 24-Hour Filter

Every article is checked against Date.now() - 86400000. Anything older than 24 hours gets dropped. This prevents stale content from feeds with slow update cycles — MIT Tech Review, for example, sometimes has week-old articles in their RSS feed.

Error Resilience from Day One

I set onError: continueRegularOutput on the HTTP Request node that fetches feeds. This is critical. If TechCrunch's RSS is temporarily down, the workflow continues with the other 9 feeds instead of crashing. Each feed fetch is wrapped in try/catch during parsing too — one malformed response does not prevent other feeds from being processed.

This "one failure must not kill the whole run" philosophy became the design principle for the entire workflow.

Phase 2: Deduplication

Deduplication happens in three tiers, which is probably the most over-engineered part of this workflow and also the part I am most proud of.

Tier 1: URL Hash

A djb2 hash of each article URL. Fast, deterministic. Two articles with the same URL get the same hash. This catches exact duplicates across feeds — when both TechCrunch and The Verge link to the same announcement.

Tier 2: Fuzzy Title Matching (Levenshtein)

This is the interesting one. I normalize titles (lowercase, strip punctuation, collapse whitespace) and compute Levenshtein distance between every pair. If the edit distance is less than 20% of the longer title's length, it is a duplicate.

The Levenshtein implementation itself was a minor adventure. n8n Code nodes run in a sandboxed VM — no npm packages, no external libraries. I wrote the full dynamic programming matrix from scratch in vanilla JavaScript. It is O(n*m) per pair and O(n^2) across all articles, which means 25 articles produce 300 pairwise comparisons. On a test run with 67 articles that number jumped to 2,211 comparisons and still finished in under 200ms. Not a bottleneck yet, but I can see it becoming one at 200+ articles.

The first version was too aggressive. It flagged "OpenAI Announces GPT-5" and "OpenAI Announces Safety Board" as duplicates because both started with "OpenAI Announces" and the threshold was 30%. Dropped it to 20% and the false positives disappeared.

Why 20%? I tested with real headlines:

• "OpenAI Releases GPT-5" vs "OpenAI Has Released GPT-5" = ~8% distance. Caught.
• "OpenAI Releases GPT-5" vs "Google Announces Gemini 3" = ~60% distance. Not caught.

20% is the threshold that catches minor headline variations without false positives.

Tier 3: Database Lookback

After in-memory dedup, the remaining URL hashes are checked against digest_articles from the last 7 days. This catches the scenario where Anthropic publishes a blog post on Monday that appears in Monday's digest, but the RSS feeds still include it on Tuesday.

The 7-day window is a pragmatic choice. It covers weekly cycles without needing an ever-growing lookup table.

Phase 3: Jina Reader and the Budget Error

Jina Reader (r.jina.ai) converts any URL into clean markdown text. You prepend https://r.jina.ai/ to any URL and it returns the article content stripped of navigation, ads, and layout. The free tier requires no API key.

I initially set the X-Token-Budget header to 4000, thinking this would keep extractions lean and fast.

Error #9 — Jina 409 BudgetExceededError. Most articles are 10,000 to 40,000 tokens. A 4,000 token budget causes Jina to return a 409 BudgetExceededError on the majority of articles instead of extracting anything at all. In the first test run, Jina only extracted 13,000 tokens total across 25 articles — most of them failed.

The fix: Remove the X-Token-Budget header entirely. Let Jina extract the full article, then truncate to 8,000 characters in the next Code node. The free tier has no per-token cost, so there is no financial penalty. Jina tokens per run jumped from 13K to ~95K, but that is the expected cost of actually getting full article text.

Silent failures. The worst kind. The header exists, it seems like a good optimization, and it silently breaks the majority of your extractions.

Phase 4: Claude Analysis — The $0.25 Mistake

The per-run cost was supposed to be $0.07. It was $0.25.

The Architecture: Two Models, Two Jobs

The idea is straightforward. Use the cheap model (Claude Haiku 4.5) for the repetitive work — analyzing 25 articles individually to extract structured JSON (summary, importance score, categories, sentiment, key entities). Use the expensive model (Claude Sonnet 4.5) once at the end for the creative work — compiling the top 12 articles into a coherent digest with a lead analysis and trend detection.

Haiku at $1/$5 per million tokens for 25 calls. Sonnet at $3/$15 per million tokens for 1 call. The math said ~$0.07/run.

The AI Agent Trap

I initially used n8n's AI Agent node with a Haiku LLM sub-node for the article analysis. This is the native n8n pattern — you add an AI Agent, attach a language model, and it processes items.

What I did not realize is that n8n's AI Agent (conversationalAgent) accumulates conversation history across batch items. When you process 25 articles sequentially:

• Call 1: 777 input tokens (system prompt + 1 article)
• Call 5: 3,800 input tokens (system + 5 articles + 5 responses)
• Call 15: 10,200 input tokens
• Call 25: 15,905 input tokens (all 25 articles + all 25 responses as context)
• Total: 203,656 input tokens when it should have been ~32,500

Per run.

The 25th API call was sending every previous article and response as conversation history. The workflow cost $0.25/run instead of $0.06/run. Ten times the designed cost. I stared at the token usage numbers for a solid minute trying to figure out where I had made a decimal error. I hadn't.

The fix: Replace the AI Agent + LLM sub-node with a direct HTTP Request to the Claude Messages API. Build the full API request body (system prompt, user message, model, temperature, max tokens) in a Code node, and send each one as a completely independent API call. No conversation memory. No accumulation.

This dropped the per-run cost from ~$0.25 back to ~$0.10 (slightly higher than the original $0.07 estimate due to the full-text Jina extractions giving Claude more context to process, which is a quality improvement worth paying for).

Lesson for anyone using n8n's AI nodes for batch processing: the conversationalAgent is designed for chat. If you are doing independent analyses across multiple items, use a Basic LLM Chain or direct HTTP requests to the API. The AI Agent will quietly multiply your costs.

Prompt Engineering: Getting Honest Scores

The analysis prompt went through three iterations:

v1 — "Analyze this article and return JSON." Problem: every article scored 7 or 8. No differentiation.

v2 — Added a scoring guide with examples at each level (10 = industry-reshaping, 4-5 = incremental). Problem: scores shifted to 6-7. Still too compressed.

v3 — Added one sentence: "Be ruthlessly honest about importance. Most articles are 4-6. Reserve 8+ for genuinely significant events." Also changed "why it matters" to require actionable insight: "means X for developers" not "this is interesting."

The result across test runs: scores ranged from 3 to 9, with an average of 6.48. Genuine differentiation. The distribution looked right — most articles at 4-6, with 8+ reserved for things like major model releases and regulatory announcements.

The compilation prompt for Sonnet worked well from the first version. The key instruction was voice: "Write like a knowledgeable colleague summarizing what they read today, not a news anchor." Sonnet 4.5 nailed the tone immediately with no iteration needed.

Scoring and Ranking

After Claude analyzes each article, a Code node applies topic weight multipliers:

ai-models: 1.5x    ai-agents: 1.5x    ai-tools: 1.3x
security: 1.3x     ai-research: 1.2x  open-source: 1.2x
devops: 1.1x

These are stored in the digest_config database table, not hardcoded. If my interests shift, I change a database row, not a workflow node.

The formula: weightedScore = (importance * maxTopicWeight) + socialBoost, where social boost is min(socialScore / 200, 2) — capped at +2 points so that a viral Reddit post does not automatically become the lead story over a genuinely significant announcement.

The top 12 articles become the digest: 1 lead story, 4 top stories, 7 quick hits.

Phase 5: The 90-Minute Slack Debugging Session

Delivery should have been the easy part. It was not.

Error #8 — Slack "no_text"

The Slack node threw Slack error response: "no_text". The Slack API requires a text field on every message, even when you are sending Block Kit blocks. The text is used for push notifications and screen reader accessibility.

Simple fix, right? Add text: subjectLine. That stopped the error. But the blocks were not rendering. The message showed up as plain text.

I tried messageType: "text" with blocks in otherOptions.blocks. The Slack API accepted it, but n8n's otherOptions.blocks field is not actually wired to the API payload for message type "text". The blocks were silently dropped.

I tried reading n8n's Slack node source code on GitHub. The parameter definitions are spread across three files and the block handling logic is buried inside a conditional chain that checks messageType before deciding which fields to read from the node config. That is where I found ensureType: 'object' — n8n internally parses the blocksUi string into an object before sending it to the Slack API. This is not documented anywhere I could find.

The actual solution: set messageType: "block" and use the blocksUi field with JSON.stringify({blocks: $json.slackBlocks}).

It was past midnight at this point. I had been staring at the same Slack node configuration panel for an hour and a half, trying every permutation of messageType, text, blocksUi, and otherOptions.blocks. I was genuinely ready to just send a plain text message and call it done. The moment those Block Kit blocks rendered correctly I just sat there for a minute before moving on.

Phase 6: Cron Triggers and the Activation Dance

With the workflow built and tested via manual triggers, the last step was activating the daily 7 AM cron.

Error #11 — Cron Never Fires

The workflow showed "Active" in the n8n logs. The cron expression was correct (0 7 * * *). But it never fired.

Two problems stacked on top of each other:

Problem 1: The workflow settings object was missing timezone: "America/Chicago". Without an explicit timezone, n8n uses the container's GENERIC_TIMEZONE environment variable, which was set correctly — but the workflow-level timezone takes precedence if present, and its absence introduced ambiguity.

Problem 2: Workflows with langchain AI Agent nodes sometimes fail to register cron triggers during n8n container startup. The workflow activates, but the cron scheduler does not pick it up. The fix is to deactivate the workflow, restart n8n, then reactivate via the API (not during startup). API-based activation after the container is fully running properly registers the cron.

Active. Green. Dead. This one took an hour to diagnose because the workflow genuinely said "Active" and the logs showed no errors.

The Numbers

Test Results Across 4 Runs

Run 1 was before the Jina budget fix (13K tokens = mostly failed extractions). Runs 2-4 show the healthy state: ~91-96K Jina tokens (full extraction), all 26 nodes succeeding, about 2 minutes end-to-end.

Cost Per Run

For context, using Sonnet for everything (analysis + compilation) would cost ~$0.45/run or ~$13.50/month. The dual-model strategy saves 78%.

What the Digest Actually Looks Like

The Slack message arrives as a Block Kit formatted message with:

• A header with the date and article count
• A stats line: "10 feeds | 67 articles scanned | 12 selected"
• The lead story with a 2-paragraph analysis and a "Why It Matters" callout
• Four top stories with 2-sentence summaries and linked titles
• Seven quick hits as single-line bullet points
• A trend note if the AI detected a pattern across stories
• A footer with the API cost for that run

The Discord version uses color-coded embeds: red for importance 9-10, orange for 7-8, blue for 5-6, grey for quick hits.

Everything is also saved as markdown and archived to Postgres with full metadata — so I can query what the digest looked like on any given day.

What I Would Do Differently

Test with more articles earlier. The AI Agent context accumulation bug only appeared when processing 25 articles in sequence. My initial tests with 3-5 articles looked fine because the cost increase was negligible. Testing at production scale sooner would have caught this.

Use direct API calls for batch AI work from the start. The AI Agent node is great for interactive chat workflows. For batch processing where each item should be independent, skip the agent abstraction and call the API directly. You get cleaner control over prompts, tokens, and error handling.

Set up Slack delivery first, not last. I built the entire pipeline before testing delivery. If I had set up the Slack node in phase 1 with a simple test message, I would have discovered the messageType: "block" + blocksUi pattern early instead of debugging it at midnight.

Budget more time for n8n cron activation quirks. This is my third major n8n workflow and I still got surprised by the cron registration behavior. If you are building scheduled workflows with AI nodes, assume you will need the deactivate-restart-reactivate dance and plan accordingly.

The Full Error Log

For reference, here is every error I hit in order, with the fix:

1. Wrong Code node type — Used @n8n/n8n-nodes-langchain.code instead of n8n-nodes-base.code. Changed all references. (15 min)
2. Null analysis crash — AI response with no JSON object caused null reference. Added explicit null check with safe defaults. (20 min)
3. Empty digest_markdown — Missing field in the UPDATE query. Added it. (5 min)
4. Manual API execution fails in queue mode — Known n8n limitation. Used temporary cron triggers for testing. (30 min)
5. Cron time confusion — n8n uses CST via GENERIC_TIMEZONE, system clock is UTC. Calculate cron times in your configured timezone. (15 min)
6. Session cookie expires after restart — Re-login after each n8n container restart. (5 min)
7. Token usage shows 0 — Token data lives in tokenUsageEstimate in execution metadata, not in $json output. Cosmetic issue, not fixed. (0 min)
8. Slack "no_text" + blocks not rendering — Set messageType: "block", use blocksUi with JSON.stringify({blocks: ...}), always include text field. (90 min)
9. Jina 409 BudgetExceededError — Removed X-Token-Budget header entirely. Free tier has no token cost. (10 min)
10. Anthropic 529 Overloaded — Added retry (3 attempts, 5s wait) and onError: continueRegularOutput on the analysis node. (15 min)
11. Cron never fires despite "Active" status — Added workflow timezone setting, use API-based deactivate/reactivate after restart. (60 min)

Total debugging time: roughly 4.5 hours out of a 7-hour build. That ratio is normal for n8n workflows with AI integration. The happy path is fast. The edge cases are where you live.

This workflow is available as a complete product kit with the full 27-node workflow JSON, database migrations, prompt templates, 30+ curated feeds, and a troubleshooting guide covering every error above. Check the nxsi.io store for details.