Client Project
An AI tools and agent marketplace — a searchable directory of 10,000+ tools with categories, rankings, and a live leaderboard. Built for scale and discovery.
Wavel is an AI tools and agent marketplace indexing more than 10,000 tools, with category rankings, a live leaderboard, and multi-axis filtering. I built the platform for the client to handle continuous ingestion and search at a scale where a static directory stops working.
This was a discovery product, which makes it fundamentally an SEO and search-performance problem rather than a feature problem. If the category pages do not rank and the search does not feel instant, the tool count is irrelevant.
New AI tools launch weekly, and the directories competing in this space were manually curated — which means perpetually stale, and expensive to keep even slightly current. The client needed continuous ingestion with no human bottleneck in the loop.
Two constraints pulled against each other. Discovery traffic comes from search engines, so thousands of category and tool pages had to be server-rendered and fast for crawlers. But the leaderboard and rankings change on every ingestion run, so those same pages could not simply be built once and cached forever.
Search was the third constraint. Users filter and re-filter while comparing tools, so search has to feel like typing rather than like submitting a query. Naive `LIKE` queries across a 10,000-row table with joins to categories and scores do not deliver that, and the obvious fix — bolt on a hosted search service — would have added cost and a second source of truth that drifts from the database.
I kept search in Postgres rather than adding a dedicated search service. Postgres full-text search with a generated `tsvector` column, GIN indexes, and trigram matching for typo tolerance was more than sufficient at this corpus size, and it kept one source of truth. The result is sub-200ms search responses without a second system to sync, pay for, or debug when it drifts.
Category pages use incremental static regeneration rather than per-request rendering. Crawlers and users get a static-speed HTML response; the page revalidates on a window matched to ingestion frequency. That gave the SEO characteristics of a static site with the freshness of a dynamic one, and kept database load flat regardless of crawl volume — which matters when a search engine is indexing thousands of pages.
Ingestion computes the leaderboard rather than the page render doing it. A weighted scoring model runs as part of the ingestion pipeline and writes materialized ranks, so serving the leaderboard is a simple indexed read instead of an aggregate over the whole corpus on every request.
Postgres full-text search over Algolia or Elasticsearch is the decision I would defend hardest here. At 10,000 documents with structured filters, a properly indexed `tsvector` plus trigram similarity is genuinely fast, and it avoids the failure mode that hurts directory sites most: a search index that has quietly diverged from the database, so users find tools that no longer exist and miss ones that do. If the corpus grew by an order of magnitude I would revisit it, and the query layer is isolated enough that swapping it would be contained.
ISR over server-side rendering was about who the audience really is. Most requests to a directory's category pages are crawlers and first-time visitors arriving from search — neither needs per-request freshness, both are extremely sensitive to time to first byte. Rendering per request would have meant paying database cost for every crawl of every page to produce output that was, in almost every case, identical to the last one.
Computing ranks at ingestion instead of at read time is the same trade applied to the leaderboard. Rankings change when data arrives, not when someone looks at them, so that is where the work belongs.
I'm available for full-time startup roles and select freelance projects.