Skip to content
Jefferson Gonçalves
← Back to projects

Deep Research

Proof of concept research agent using Browserbase Search, Fetch, and OpenAI

#Research Agent

A research agent that discovers broad web coverage with Browserbase Search, retrieves pages with Browserbase Fetch, and synthesizes a structured research brief.

Proof of concept only. This repository is intentionally lightweight and should be treated as an example implementation, not production-ready research infrastructure.

Built with Browserbase Search API, Browserbase Fetch API, and OpenAI.

No browser sessions. No Playwright. No Puppeteer. Just API calls.

#API Spotlight (For The Video)

  1. Search API — discovery starts with Browserbase Search (bb.search.web) across multiple query angles for broad source coverage.
  2. Fetch API — retrieval runs through Browserbase Fetch (bb.fetchAPI.create) with concurrency, retries, redirect handling, and optional proxy retry for resilience.
  3. Search + Fetch stay primary — the pipeline is intentionally API-first, with no browser-session dependency in the default flow.

#How It Works

  1. Discover (Search-first) — runs multiple diversified Browserbase Search queries for breadth, then canonicalizes + dedupes URLs.
  2. Select — picks a diverse candidate set so one domain does not dominate.
  3. Fetch (robust) — fetches candidates concurrently with retries, status/content-type checks, and optional proxy retry path.
  4. Synthesize — summarizes in chunks, then produces a final citation-grounded brief.
  5. Diagnostics — appends retrieval metrics (queries, fetch success/failure, and included sources) to the report.

#Setup

npm install
cp .env.example .env
# Add your API keys to .env, then run the agent

You'll need:

For local development:

  • .env stays local and should not be committed
  • .env.example is the safe template to publish
  • generated research-*.md output is ignored by default
  • editor metadata like .idea/ is ignored by default

#Usage

# Default topic
npm run research

# Custom topic
npm run research -- "your research topic here"

Examples:

npm run research -- "latest developments in local-first software"
npm run research -- "state of WebAssembly in 2026"
npm run research -- "best practices for AI agent architecture"

Output is printed to the console and saved as a markdown file.

#Configuration

Tune breadth and reliability with environment variables:

  • SEARCH_RESULTS_PER_QUERY (default: 15, max: 25)
  • MAX_SEARCH_QUERIES (default: 5)
  • EXTRA_SEARCH_QUERIES (pipe-separated query list, e.g. q1|q2|q3)
  • MAX_FETCH_CANDIDATES (default: 30)
  • MAX_RESULTS_PER_DOMAIN (default: 2)
  • FETCH_CONCURRENCY (default: 5)
  • MAX_FETCH_ATTEMPTS (default: 3)
  • SEARCH_RETRY_ATTEMPTS (default: 3)
  • RETRY_BASE_DELAY_MS (default: 500)
  • FETCH_PROXY_RETRY (default: true)
  • FETCH_PROXY_RETRY_ATTEMPT (default: 2)
  • MIN_CONTENT_CHARS (default: 200)
  • MAX_CHARS_PER_PAGE (default: 5000)
  • MAX_SOURCES_FOR_SUMMARY (default: 18)
  • SUMMARY_CHUNK_SIZE (default: 6)
  • OPENAI_MODEL (default: gpt-4o)

Example (wider discovery for a topic run):

MAX_SEARCH_QUERIES=7 \
SEARCH_RESULTS_PER_QUERY=20 \
MAX_FETCH_CANDIDATES=40 \
FETCH_CONCURRENCY=8 \
npm run research -- "state of local-first architecture"

You can also validate the project locally with:

npm run typecheck

#What You Could Add

  • Stream the final synthesis output instead of waiting for completion
  • Persist retrieval metadata in JSON for later analysis
  • Add optional recency filters in query templates
  • Add lightweight reranking before fetch selection
  • Add optional Browserbase session escalation only for repeated Fetch misses

#Limits

  • Fetch: 1MB content cap, 10s timeout, no JavaScript execution
  • Search: 25 results max per query, 120 requests/min rate limit
  • If a page needs JS to render, optionally escalate to a Browserbase browser session

New version available.