Engagement: Contract, approximately 120–160 hours. Full-time focus preferred.
Location: Remote
The Role
We're rebuilding a deal-sourcing pipeline. The system is currently an opaque third-party AI agent on Tasklet. We need inspectable, version-controlled code.
We need one strong app and automation developer to build it: a Playwright-driven scraper across approximately 113 Canadian restaurant chains, LLM-based extraction, deterministic validation and scoring, two-way synchronization with Supabase, and a weekly orchestrated maintenance pipeline with real logs and per-run reports.
What You'll Build
Chain scraper
Visit every chain's deal page, extract structured deals, and perform a two-way sync with the database. This includes inserting new deals, updating changed deals, and deactivating stale deals. The process must be idempotent and re-runnable.
LLM extraction layer
Fetch page HTML and screenshots, then use Claude to extract structured deal objects guided by per-chain configuration.
Deterministic core
Build pure, unit-tested TypeScript functions for validation rules, pricing math using dp, op, and pct, category mapping, and 0–100 scoring. The same input must produce the same output every time.
Audit steps
Build link-health checks, content verification to confirm that database records match live pages, data-quality audits for issues such as fake free offers, incorrect categories, math errors, and duplicates, and CTA audits.
Orchestration
Create a weekly scheduled pipeline using GitHub Actions cron that runs every step in order and produces both machine-readable and human-readable run reports.
Observability
Store structured logs in Postgres for every run and every chain so we can always answer questions such as, “What did the pipeline do last Monday, and why?”
Submission enrichment
Build a Supabase Edge Function that turns free-text user submissions into structured and scored deals.
Required Skills
Playwright
This is the primary scraping mechanism and the hardest part of the job. You should be comfortable driving headless browsers against JavaScript-rendered and bot-resistant sites and debugging scrapers when a site changes its markup.
TypeScript and Node.js
The entire pipeline will be a TypeScript and Node.js package. You should write strong, idiomatic TypeScript and be comfortable structuring a production codebase rather than a collection of scripts.
SQL and Postgres
You will design several tables and confidently read, write, compare, and update rows. Supabase Postgres is the backing store.
REST APIs
You should be comfortable consuming and reasoning about REST and PostgREST endpoints.
Automated testing
You should write unit tests for pure functions as a standard part of development, particularly for validation, scoring, and pricing calculations.
Web-scraping judgment
You understand that scrapers are inherently fragile, plan for website changes, and know how to isolate configuration so repairs are inexpensive.
Strongly Preferred
LLM API integration
Experience using Claude or a similar model for structured extraction with deterministic post-processing. You should understand prompt and schema design and be conscious of API costs.
Supabase
Experience with Edge Functions, pg_cron, migrations, and service-role versus anonymous key handling.
CI/CD automation
Experience with GitHub Actions, including scheduled workflows, secrets, and artifacts.
ETL and data-pipeline experience
Experience with two-way synchronization, idempotency, diffing, and data-quality auditing.
Nice to Have
Experience building self-healing or configuration-driven scrapers that can adapt to site changes with minimal code edits.
Experience integrating Slack or webhook notifications for pipeline reporting.
Who This Is Right For
This role is for someone who is genuinely comfortable with architecture and automation depth, not just SQL.
The value is not in writing queries. It is in building a robust, observable, and testable pipeline while taming a fragile scraping layer. If scraping changing websites sounds like a familiar, solvable but fiddly problem rather than a mystery, you may be a strong fit.
You should be self-directed. The brief is detailed, the requirements are clear, and we want someone who can take the project from beginning to end while making sound decisions about open questions such as runtime, extraction engine, and project phasing.
Scope and Timeline
Estimate
Approximately 120–160 hours of focused work. A capable developer using modern AI tooling may be able to establish the basic end-to-end loop in roughly five to seven days. The remaining work will involve making Playwright reliable across approximately 113 sites and ensuring the complete pipeline runs smoothly.
Reality Check on Scraping
The scraper will not be a one-time build. Websites change, and the Playwright scripts will occasionally require adjustments. The initial implementation should make ongoing maintenance as inexpensive as possible through configuration-driven design and detailed logging.
Phase 1
Build the core loop, including configuration, extraction, scraping and two-way synchronization, data-quality auditing, deterministic scoring, run tables and logs, a command-line interface with a dry-run option, and unit tests.
Phase 2
Build link-health, content-verification, and CTA audits; news and DROPs scrapers; submission enrichment; the complete Monday orchestration schedule; and report delivery.
How We'll Evaluate
Demonstrated Playwright experience with real, messy websites through a portfolio, repository, or short technical conversation.
Clean and tested TypeScript. We will want to understand how you structure pure functions and tests.
Sound judgment about scraper fragility, including how you would respond when a restaurant chain changes its page layout or blocks headless browsers.