Portfolio project · Python · RAG · agents · n8n

AI Market Intelligence
& Product Strategy Agent

A scheduled pipeline that turns public competitor discussions into an evidence-backed market report — with every claim traced to a source post.

Python · SQLite RAG · bge-small + sqlite-vec 5 agents · typed JSON FastAPI · Docker n8n · verified live CI · retrieval quality gate Multi-user · self-learning queries

01 · how it fits together

Four layers, each testable on its own

Sources → orchestration → retrieval → agents → deliverables

Sources

Hacker News API

public · no auth

Reviews / Reddit

source-agnostic fetch

Orchestration

n8n · schedule

06:00 refresh · 07:00 report

SQLite + reports

corpus & artifacts

Pipeline & retrieval

clean → chunk → embed

bge-small · 384d · local

vector index · sqlite-vec

cosine k-NN · citations

Agents

research → competitor

typed JSON claims

customer → strategy

opportunities + risks

critic

claim-by-claim verdicts

Output

market report

markdown + sources

Slack / Notion

verdicts & citation health

HTTP API between them: n8n never imports the code · the code never knows n8n exists

02 · collect & index

Idempotent, so it can run on a schedule

$ python run_pipeline.py

queries : 15 (3 competitors × 5 angles)

kept : 169 posts after dedupe + brand filtering

$ python run_index.py

posts : 169

chunks : 217 (overlapping windows)

model : BAAI/bge-small-en-v1.5 (384 dims, local, no API key)

$ python run_corpus.py export

exported → tests/fixtures/ci_corpus.json (network-free CI)

→ CI rebuilds this corpus with zero network: same numbers, every push

$

03 · five agents, one job each

Grounded in retrieval — never raw text

$ python run_report.py "fees and developer payouts"

research : 9 queries planned → 15 posts retrieved

competitor : shopify 5 · woocommerce 5 · bigcommerce 5

customer : pain points · praise · switching themes

strategy : ranked opportunities + risks, cited

critic : 5 supported, 6 partial, 9 unsupported

citations : 15 in-range, 0 out-of-range

warning : 1 competitor claim has no citation

report : data/reports/market_report.md

Anti-hallucination, in layers

1 · model sees numbered evidence only

2 · every claim must cite [n]

3 · critic judges support claim by claim

4 · code verifies each [n] was really retrieved

Output is typed, not prose

{ "claim": "...", "citations": [2,6] }
verdicts keyed by claim id · retried once
invalid JSON → free text, flagged in the audit

04 · the deliverable

Every claim points at a post you can open

Developers are unhappy with the revised revenue share and are considering building for other platforms [3, 11]

critic · SUPPORTED — both sources describe the payout change and the reaction

Shopify offers a generous payout program, twice a month [3]

critic · PARTIAL — [3] is a sarcastic complaint about cuts, not a program description

15

evidence posts, all cited in range

5

claims judged fully supported

6

partial — flagged for a human

9

unsupported — dropped, not shipped

The critic is a small local model and it is not always right — which is exactly why verdict counts are computed from typed data in code, never from the model's own arithmetic.

05 · measured, not asserted

Labeled questions score retrieval and answers

10 labeled questions · top-5 retrieval · citation scores with a local LLM

MRR

1.000

nDCG@5

1.000

recall@5

0.820

citation recall

0.733

citation precision

0.578

Measured against hand-labeled questions over the 83-post CI fixture corpus. Retrieval needs no LLM, so run_eval.py gates CI on every push.

CI gate · recall@5 0.820 >= 0.700 PASS · MRR 1.000 >= 0.800 PASS · 142 tests

06 · live ops

Scheduled runs you can audit — success and failure

n8n · self-hosted · executions

06:00 corpus refresh → job 96843cae · succeeded · 6m 47s

07:00 market report → 43 polls · 22 evidence posts · 22m 55s

06:00 refresh, next day → fetch OOM → failed → alert

each run = one API job, tagged source: "n8n"

console · /console — live jobs, source badges, report viewer

$

Corpus refreshed — 0 new posts (169 total)

Slack · #market-intel · 06:47

Market report ready — 22 evidence posts · verdicts: 2 PARTIAL / 20 UNSUPPORTED · 22m

Slack · #market-intel · 07:23

Corpus refresh failed — ONNX out of memory · check the job log

Slack · #market-intel · the failure path is tested too

07 · shipped

Python for the intelligence.
n8n for the business workflow.

Service

POST /reports → 202 job
GET /jobs/{id} → status
POST /search · /ask · /eval
GET /health → corpus + provider
GET /console → browser UI

Deploy

docker compose up --build
model baked into the image
sqlite-vec smoke-tested at build
secrets from env, never in git

Schedule

06:00 refresh corpus
07:00 generate report
poll → find a failed report
Slack receipts: success + failure

Read reports critically: a 3B local model writes generic prose and over-flags some supported claims. The architecture is provider-agnostic — swap in a bigger model and the same numbers get better.

github.com/Aabyyss/AI-Market-Intelligence-Product-Strategy-Agent · v1.2.0