# ApnaBazaar

A Next.js storefront that demonstrates **semantic product search over a JSON file, with no
database anywhere**. Products live in `data/products.json`. Every product is embedded into
a vector once, the vectors are stored in `data/product-vectors.json`, and search ranks them
by cosine similarity blended with keyword relevance.

It runs with **zero configuration** — no API key, no model download, no network, no
database. Set an API key and it will use a neural embedding model instead.

```bash
npm install
npm run dev
# http://localhost:3000
```

---

## Table of contents

- [Technology stack](#technology-stack)
- [Requirements](#requirements)
- [Installation](#installation)
- [Environment variables](#environment-variables)
- [Running the application](#running-the-application)
- [The product data](#the-product-data)
- [Search, in practice](#search-in-practice)
- [Vector search architecture](#vector-search-architecture)
- [Regenerating product vectors](#regenerating-product-vectors)
- [API documentation](#api-documentation)
- [Project structure](#project-structure)
- [Performance notes](#performance-notes)
- [Design decisions](#design-decisions)
- [Future improvements](#future-improvements)

---

## Technology stack

| Concern     | Choice                                                        |
| ----------- | ------------------------------------------------------------- |
| Framework   | Next.js 16.3.8 (App Router, React 19.2, Turbopack)            |
| Runtime     | Node.js 20.9+ (20 LTS)                                        |
| Language    | TypeScript 5 (strict)                                         |
| Styling     | Tailwind CSS 4                                                |
| Linting     | ESLint 9 (flat config, `eslint-config-next`)                  |
| Product data| `data/products.json` — a plain JSON file                      |
| Vectors     | `data/product-vectors.json` — a plain JSON file                |
| Vector maths| Implemented in-project: TF-IDF + truncated SVD + cosine similarity |
| Cart        | React context + `localStorage`, frontend only                 |
| Database    | **None.** No MongoDB, MySQL, PostgreSQL, Prisma, or any ORM. |

---

## Requirements

- **Node.js 20.9.0 or newer.** Next.js 16 dropped Node 18 and requires 20.9+ as the
  minimum LTS. The `engines` field in `package.json` declares this.
- npm 10+ (ships with Node 20).

Verify with `node -v`. On Windows use [nvm-windows](https://github.com/coreybutler/nvm-windows)
if you need to switch versions.

No database, no Docker, no API key required.

---

## Installation

```bash
npm install
```

Two generated artefacts are committed, so the first run needs no extra steps:

- `data/product-vectors.json` — the prebuilt offline vector index
- `public/products/*.svg` — one placeholder image per product

If either is missing, the app regenerates what it needs at runtime (see
[Regenerating product vectors](#regenerating-product-vectors)).

---

## Environment variables

All variables are **optional**. Copy the template if you want to change anything:

```bash
cp .env.example .env.local
```

`.env.local` is git-ignored. `.env.example` is committed.

| Variable                 | Default                    | Purpose                                            |
| ------------------------ | -------------------------- | -------------------------------------------------- |
| `EMBEDDING_API_KEY`      | *(empty)*                  | Enables the remote provider. Blank = offline LSA.  |
| `EMBEDDING_MODEL`        | `text-embedding-3-small`  | Model name sent to the embeddings API.             |
| `EMBEDDING_API_BASE`     | `https://api.openai.com/v1` | Base URL of any OpenAI-compatible endpoint.      |
| `EMBEDDING_BATCH_SIZE`   | `64`                       | Texts per embeddings request.                       |
| `EMBEDDING_TIMEOUT_MS`   | `10000`                    | Request timeout.                                    |
| `SEARCH_WEIGHT_SEMANTIC` | `0.7`                      | Semantic component of the hybrid score.            |
| `SEARCH_WEIGHT_KEYWORD`  | `0.3`                      | Keyword (BM25) component.                          |
| `SEARCH_WEIGHT_PRICE`    | `0.15`                     | Price component; only active for price-intent queries. |

**Key handling:** the API key is read only in server-side modules
(`src/lib/embeddings.ts`, `src/lib/vector-index.ts`). No variable is prefixed with
`NEXT_PUBLIC_`, so nothing in this list reaches the browser bundle. The `/api/search`
response contains no vectors and no credentials.

Weights are renormalised to sum to 1, so the blended score always stays in `0..1`.

---

## Running the application

```bash
npm run dev     # development server on http://localhost:3000
npm run build   # production build
npm run start   # serve the production build
npm run lint    # ESLint
npm run typecheck # tsc --noEmit
```

### Maintenance scripts

| Command                        | What it does                                                        |
| ------------------------------ | ------------------------------------------------------------------- |
| `npm run build:images`         | Regenerate `public/products/*.svg` from `data/products.json`.        |
| `npm run build:vectors`        | Rebuild `data/product-vectors.json` with the offline LSA provider.  |
| `npm run build:vectors:remote` | Rebuild using the remote provider (needs `EMBEDDING_API_KEY`).      |
| `npm run search:check`         | Run the five reference queries and print the ranking.               |

---

## The product data

`data/products.json` is the single source of truth for the catalogue. 40 products across
8 categories. It is read-only: there is no CRUD, no write path, and no database.

```json
{
  "id": 1,
  "name": "Wireless Noise Cancelling Headphones",
  "description": "AudioPro over-ear wireless headphones with adaptive noise cancelling, 40-hour battery life and a folding travel case built for long flights, train commutes and everyday music listening.",
  "category": "Electronics",
  "subcategory": "headphones",
  "price": 149.99,
  "brand": "AudioPro",
  "rating": 4.5,
  "stock": 25,
  "image": "/products/wireless-noise-cancelling-headphones.svg",
  "tags": ["wireless bluetooth", "noise cancelling", "travel", "commute", "long battery life", "music", "gift for music lovers"]
}
```

| Field         | Type       | Notes                                                          |
| ------------- | ---------- | -------------------------------------------------------------- |
| `id`          | `number`   | Unique, integer. Used in the `/products/[id]` URL.             |
| `name`        | `string`   | Display name.                                                  |
| `description` | `string`   | One sentence; contributes most of the searchable text.         |
| `category`    | `string`   | One of the eight categories. Drives the filter facet.          |
| `subcategory` | `string`   | Finer grouping; drives suggestions.                            |
| `price`       | `number`   | Two decimals. Filters and price intent read this.              |
| `brand`       | `string`   | Reused across related products so brand filtering is useful.    |
| `rating`      | `number`   | 0–5, one decimal.                                              |
| `stock`       | `number`   | `0` means out of stock.                                        |
| `image`       | `string`   | Path under `/public`.                                         |
| `tags`        | `string[]` | 4–7 lowercase keywords or short phrases.                       |

### Adding a product

1. Append an object with the fields above to `data/products.json`. Give it the next
   unused `id`; keep the array ordered by `id`.
2. Write descriptive text. **This is what the search reads.** The indexed document is
   built from the name, subcategory, category, brand (twice), description and tags —
   so a product whose description says *"built for long flights and train commutes"*
   will be reachable by a traveller who never uses the word "travel".
3. Generate its placeholder image:

   ```bash
   npm run build:images
   ```

   That reads every product's `image` field and writes the matching SVG.
4. Rebuild the vectors:

   ```bash
   npm run build:vectors
   npm run search:check
   ```

**You do not strictly need step 4.** The index carries a fingerprint of the catalogue;
when it no longer matches, the server rebuilds the index in memory automatically and logs
it. Regenerating explicitly just moves the cost from the first request to the CLI.

For an external embedding model, use `npm run build:images` then
`EMBEDDING_API_KEY=... npm run build:vectors:remote`, which writes
`data/product-vectors.remote.json` and leaves the offline index untouched.

---

## Search, in practice

Run these against the running app:

| Query                            | Top result                                   | Why                                                             |
| -------------------------------- | -------------------------------------------- | --------------------------------------------------------------- |
| `headphones for travelling`      | Wireless Noise Cancelling Headphones         | Query expands to travel/commute/flight, all in the description.  |
| `something for working out`     | Adjustable Dumbbell Set                      | Expands to gym/workout/exercise/training.                        |
| `comfortable shoes for running`  | Men's Cushioned Running Shoes                | All three terms are in the text; keyword score confirms it.       |
| `gift for someone who loves music` | Vinyl and Velvet Music History, then audio  | `gift` expansion plus a strong `music` signal.                   |
| `cheap wireless headphones`      | Budget True Wireless Earbuds ($34.99)       | "cheap" triggers the price component, which lifts the budget item. |

Confirm from the command line:

```bash
npm run search:check
```

Which prints, per query, the ranking with its similarity, keyword score and blended score.

### Supported filters

Category, brand, minimum price, maximum price, minimum rating, in-stock-only. All are
applied **before** scoring, and all are reflected in the URL, so a filtered search is
shareable and the back button works.

Sort options: best match, price low→high, price high→low, top rated, name A→Z.

---

## Vector search architecture

Full walkthrough with code references: **[docs/vector-search.md](docs/vector-search.md)**.

```
Build time — runs once
───────────────────────
data/products.json
     ↓  preprocessing            buildProductSearchText()      src/lib/text.ts
        text representation      one string per product
     ↓  embedding generation      embedMany()                   src/lib/embeddings.ts
     ↓  vector index              data/product-vectors.json     src/lib/vector-index.ts
        loaded into memory once per process


Query time — runs per search
───────────────────────────
user query "headphones for travelling"
     ↓  query understanding      analyzeQuery()                src/lib/search-utils.ts
     ↓  query embedding          projectQuery()                src/lib/lsa.ts
     ↓  similarity calculation   cosine similarity             src/lib/vector-search.ts
     ↓  keyword relevance        BM25                          src/lib/keyword-search.ts
     ↓  filters + weighted blend                              src/lib/search.ts
ranked products → /products grid
```

### Ranking

```
score = semantic × 0.70 + keyword × 0.30 [+ price × 0.15 if the query has price intent]
```

- **Semantic** — cosine similarity against the precomputed product vectors.
- **Keyword** — BM25 with per-field weights (name 3.0, tags 2.5, brand 2.0,
  subcategory 1.75, category 1.5, description 1.0).
- **Price** — activates only for `cheap`/`budget`/`affordable` and
  `premium`/`luxury` queries, and honours an explicit ceiling like `under 100`.

### The two providers

| Provider    | Needs                | Index file                          | Notes                                                        |
| ----------- | -------------------- | ----------------------------------- | ------------------------------------------------------------ |
| `local-lsa` | nothing              | `data/product-vectors.json`         | Default. TF-IDF + exact truncated SVD over the corpus.       |
| `remote`    | `EMBEDDING_API_KEY`  | `data/product-vectors.remote.json`  | Any OpenAI-compatible `/embeddings` endpoint.                |

The offline provider factorises the catalogue into a shared latent space and derives
**term vectors**, so new text can be projected into the same space without re-embedding
anything:

```
X = X · Xᵀ  =  U · Σ² · Uᵀ          (exact Jacobi eigendecomposition)
document vectors = U · Σ
term vectors     = Xᵀ · U · Σ⁻¹     (orthonormal columns)
```

Because `cos(document, query)` reduces to a projected tf-idf score, a query costs one
matrix-vector product plus a `30`-term stem, not 40 embedding API calls.

**Automatic fallback:** if a configured remote provider fails — offline, invalid key,
timeout — the search is served by the offline index and the response reports
`meta.providerFallback: true`. Search never goes dark because a third party is
unreachable.

---

## Regenerating product vectors

```bash
npm run build:vectors              # offline LSA index (no key needed)
npm run build:vectors:remote       # remote provider (needs EMBEDDING_API_KEY)
```

Both go through the same code path the server uses
(`regenerateVectorIndex()` in `src/lib/vector-index.ts`), so the CLI and the app cannot
drift.

Output:

```
[build:vectors] 40 products -> 24 dimensions (provider local-lsa/lsa-tfidf) in 0.06s
[build:vectors] vocabulary size: 497 terms
[build:vectors] source: a6929caa
```

The `source` value is a fingerprint of the catalogue. On startup the server compares it
against `data/products.json`; a mismatch triggers an automatic in-memory rebuild, so the
index is never silently stale.

When to regenerate:

- You added, removed, edited or reordered products.
- You changed how product text is built (`buildProductSearchText`).
- You changed the LSA dimension count.

Regenerating is **not** needed after changing ranking weights — those are applied at
query time.

---

## API documentation

### `GET /api/search`

| Parameter  | Type    | Default    | Notes                                              |
| ---------- | ------- | ---------- | -------------------------------------------------- |
| `q`        | string  | `""`       | Natural-language query. Max 200 characters.        |
| `category` | string  | —          | Exact category match.                              |
| `brand`    | string  | —          | Exact brand match.                                 |
| `minPrice` | number  | —          | `0`–`100000`.                                      |
| `maxPrice` | number  | —          | `0`–`100000`. Must be ≥ `minPrice`.                |
| `minRating`| number  | —          | `0`–`5`.                                           |
| `inStock`  | `true`  | off        | `inStock=true` restricts to `stock > 0`.           |
| `sort`     | enum    | `relevance` | `relevance`, `price-asc`, `price-desc`, `rating-desc`, `name-asc`. |
| `limit`    | number  | `24`       | `1`–`50`.                                          |

```bash
curl "http://localhost:3000/api/search?q=wireless+headphones&limit=3"
```

### `POST /api/search`

```bash
curl -X POST http://localhost:3000/api/search \
  -H "content-type: application/json" \
  -d '{
    "query": "cheap wireless headphones",
    "filters": { "minPrice": 0, "maxPrice": 100, "inStockOnly": true },
    "sort": "price-asc",
    "limit": 5
  }'
```

`q` is also accepted as an alias for `query`. Unknown fields are ignored.

### Response

```json
{
  "query": "headphones for travelling",
  "count": 3,
  "results": [
    {
      "id": 1,
      "name": "Wireless Noise Cancelling Headphones",
      "description": "AudioPro over-ear wireless headphones with adaptive noise cancelling…",
      "category": "Electronics",
      "subcategory": "headphones",
      "price": 149.99,
      "brand": "AudioPro",
      "rating": 4.5,
      "stock": 25,
      "image": "/products/wireless-noise-cancelling-headphones.svg",
      "tags": ["wireless bluetooth", "noise cancelling", "travel", "…"],
      "score": 0.9165,
      "similarity": 0.8808,
      "keywordScore": 1,
      "priceScore": 0,
      "matchedTerms": ["headphone", "travel"]
    }
  ],
  "suggestions": ["noise cancelling", "wireless bluetooth", "travel"],
  "meta": {
    "total": 12,
    "returned": 3,
    "took": 17,
    "provider": "local-lsa",
    "providerFallback": false,
    "sort": "relevance",
    "weights": { "semantic": 0.7, "keyword": 0.3, "price": 0 },
    "priceIntent": "none"
  }
}
```

| Field                | Meaning                                                     |
| -------------------- | ----------------------------------------------------------- |
| `score`              | Final blended relevance, `0..1`. The ranking key.            |
| `similarity`         | Cosine similarity between query and product vectors, `0..1`. |
| `keywordScore`       | Normalised BM25 relevance, `0..1`.                           |
| `priceScore`         | Price affinity, `0..1`. Non-zero only for price intent.     |
| `matchedTerms`       | Query terms that matched lexically.                         |
| `meta.took`          | Server-side search time in milliseconds.                     |
| `meta.provider`      | `local-lsa` or `remote`.                                    |
| `meta.providerFallback` | `true` if a remote provider failed and the offline index took over. |

**Raw vectors are never serialised.** The API returns the derived similarity score only.

### Errors

`400` with a field-keyed payload:

```json
{
  "error": {
    "code": "INVALID_INPUT",
    "message": "One or more search parameters are invalid.",
    "details": { "sort": ["must be one of relevance, price-asc, …"] }
  }
}
```

| Status | Code             | Cause                                        |
| ------ | ---------------- | -------------------------------------------- |
| `400`  | `INVALID_INPUT`  | A parameter failed validation.               |
| `400`  | `INVALID_JSON`   | `POST` body was not valid JSON.             |
| `500`  | `SEARCH_FAILED`  | The index could not be built or loaded.      |

### `GET /api/products`

Returns a minimal catalogue snapshot (`id`, `name`, `price`, `image`, `brand`, `stock`)
for client-side reconciliation. It exists because the cart runs in the browser and cannot
read `node:fs`.

---

## Project structure

```
apnabazzar/
├── data/
│   ├── products.json              # 40 products — the source of truth
│   └── product-vectors.json       # generated vector index
├── docs/
│   └── vector-search.md           # developer documentation
├── public/
│   └── products/*.svg             # generated placeholder imagery
├── scripts/
│   ├── generate-vectors.ts        # npm run build:vectors
│   ├── generate-placeholders.mjs  # npm run build:images
│   └── check-search.ts            # npm run search:check
├── src/
│   ├── app/
│   │   ├── layout.tsx             # root layout, fonts, cart provider
│   │   ├── page.tsx               # home
│   │   ├── error.tsx              # route error boundary (Next 16 `retry`)
│   │   ├── not-found.tsx
│   │   ├── globals.css
│   │   ├── products/
│   │   │   ├── page.tsx           # /products
│   │   │   └── [id]/page.tsx      # /products/[id]
│   │   ├── cart/page.tsx          # /cart
│   │   └── api/
│   │       ├── search/route.ts    # GET + POST
│   │       └── products/route.ts  # catalogue snapshot
│   ├── components/
│   │   ├── cart/                  # CartProvider, CartView, add-to-cart, quantity
│   │   ├── home/                  # Hero, CategoryGrid, FeaturedProducts, CtaBanner
│   │   ├── layout/                # Header, Footer
│   │   ├── products/              # ProductCard, ProductGrid, ProductSearch,
│   │   │                          # ProductFilters, ProductSort, ProductsExplorer
│   │   └── ui/                    # RatingStars, Skeleton, Spinner, icons, states
│   ├── hooks/
│   │   ├── useProductSearch.ts    # request lifecycle, debounce, abort
│   │   └── useDebouncedValue.ts
│   ├── lib/
│   │   ├── products.ts            # load + validate products.json
│   │   ├── text.ts                # tokenise, stem, build searchable text
│   │   ├── matrix.ts              # dense matrix ops + Jacobi eigensolver
│   │   ├── lsa.ts                 # TF-IDF + truncated SVD + query projection
│   │   ├── embeddings.ts          # provider interface, local + remote
│   │   ├── vector-index.ts        # build / persist / load / memoise
│   │   ├── vector-search.ts       # cosine similarity
│   │   ├── keyword-search.ts      # BM25
│   │   ├── search.ts              # hybrid orchestration
│   │   ├── search-config.ts       # weights and limits
│   │   ├── search-utils.ts        # query analysis, validation, suggestions
│   │   └── format.ts              # currency, stock level
│   └── types/
│       ├── product.ts
│       ├── search.ts
│       └── result.ts              # client-safe API shapes
├── .env.example
├── next.config.ts
└── tsconfig.json                  # "@/*" → "./src/*"
```

### Module responsibilities

| Module              | Responsibility                                                      |
| ------------------- | ------------------------------------------------------------------- |
| `products.ts`       | Read and validate the JSON catalogue. Server-only.                  |
| `text.ts`           | Tokenisation, stop words, stemming, searchable-text construction.   |
| `matrix.ts`         | Dense matrix maths and the symmetric eigensolver.                   |
| `lsa.ts`            | The offline embedding: TF-IDF → truncated SVD → query projection.   |
| `embeddings.ts`     | `EmbeddingProvider` interface; local and remote implementations.    |
| `vector-index.ts`   | Build, persist, load and memoise the index; provider fallback.      |
| `vector-search.ts`  | Cosine similarity over the in-memory index.                         |
| `keyword-search.ts` | BM25 with per-field weights.                                        |
| `search.ts`         | Filters → score → sort → respond. The single entry point.            |
| `search-utils.ts`   | Intent phrases, price detection, input validation, suggestions.     |
| `search-config.ts`  | Weights, limits, sort options. Env-overridable.                     |

---

## Performance notes

- **Product embeddings are generated once.** They live in
  `data/product-vectors.json` and in memory. A search never re-embeds the catalogue.
- **The index is memoised on `globalThis`**, so dev hot reloads don't rebuild it.
- **Query embeddings are cached** per `(provider, text)`, bounded at 256 entries.
- **A query costs one matrix-vector product** against the term vectors, then 40 dot
  products of 24 dimensions. Measured end-to-end: **2–35 ms** per search, including
  validation and serialisation.
- **Filters run before scoring**, so scoring work is spent only on products that can appear.
- **Server Components by default.** The homepage, product pages and cart shell ship zero
  client JavaScript for the catalogue data. Client components exist only where
  interaction requires them.
- **Client-side search state.** Typing updates the URL with `history.replaceState`
  instead of a navigation, so the products page is not re-rendered on the server per
  keystroke.
- **Debounced at 300 ms** with `AbortController`, and out-of-order responses are discarded.
- **`next/image`** everywhere, with `fill` + `sizes` so the browser picks the right source
  for each breakpoint. SVG sources are served unoptimized by `next/image` automatically.
- **Never loads the catalogue in the browser** beyond what a card renders.

---

## Design decisions

**Why the catalogue is never rebuilt at request time.** Product embeddings are expensive
relative to a search. They are computed by a CLI, persisted, fingerprinted, and memcised.

**Why the URL uses `replaceState`.** A `router.push` per keystroke would re-run the
products Server Component on every debounce. The URL stays shareable; the server does no
work.

**Why the products page reads `searchParams` at all.** Only to seed initial state, so a
shared link opens on the right results with the right cards.

**Why filters are controlled.** No duplicated state means the URL, the request and the
visible controls cannot disagree.

**Why the remote provider has a separate vector file.** Vectors from different models are
not comparable. Two files mean switching providers never invalidates the index you already
have, and never silently mixes vector spaces.

**Why `local-lsa` rather than a bundled transformer.** `@xenova/transformers` would give
better semantics but downloads ~90 MB of model weights on first run. LSA is exact,
deterministic, zero-dependency, and already answers every query in the brief correctly.
The `EmbeddingProvider` interface means swapping in a neural model is a one-file change.

**Why query expansion exists.** A 40-document corpus teaches a latent space only the words
it contains. `"something for working out"` has no lexical overlap with "gym" until someone
maps the intent. A short phrase table is transparent, tunable, and honest about being a
heuristic.

---

## Future improvements

- **Neural embeddings by default.** Ship `@huggingface/transformers` with a small model
  (all-MiniLM-L6-v2, ~23 MB quantised) and drop the expansion table for real semantics.
- **Incremental indexing.** Add or edit one product and re-embed only that document
  instead of the whole catalogue.
- **Reciprocal rank fusion.** Blend the semantic and keyword *rankings* rather than their
  scores; more robust when the two scales are not comparable.
- **Product-to-product similarity.** The same index gives "more like this" for free.
- **Vector persistence.** SQLite FTS5 with a vector extension would add hybrid lexical
  search with incremental updates while staying file-based.
- **Faceted counts.** Return per-facet result counts so the filter panel can show what each
  option would yield.
- **Query analytics.** Log `(query, result count, clicked id)` to a local JSONL file to
  identify queries that return nothing.
- **Multi-embedding.** Index name, description and tags as separate vectors and fuse with
  RRF, which handles multi-field relevance better than one weighted document.
- **Streaming search.** `Server Components` with a Suspense boundary for progressive
  rendering of large result sets.
- **Tests.** Property tests asserting that the documented reference queries keep their
  expected top result, guarding the ranking against regressions.
- **Internationalisation.** The expansion table and stop-word list are the natural seam.

---

## License

Sample catalogue and code provided for demonstration purposes.