# TokNext AI assistant operations

TokNext uses the official OpenAI Node SDK and the Responses API only. The feature is disabled by default and the API key is server-only. The browser receives newline-delimited JSON (NDJSON) events from `POST /api/ai/chat`; it never receives the provider response object, raw tool payloads, credentials, or SQL.

## Configuration

Set these values in the server environment, never in client code:

```dotenv
OPENAI_ASSISTANT_ENABLED="false"
OPENAI_API_KEY=""
OPENAI_MODEL="gpt-5-nano"
OPENAI_MODEL_PREMIUM="gpt-5-mini"
OPENAI_MAX_OUTPUT_TOKENS="900"
OPENAI_PREMIUM_MAX_OUTPUT_TOKENS="1600"
OPENAI_REQUEST_TIMEOUT_MS="30000"
OPENAI_RATE_LIMIT_PER_MINUTE="12"
OPENAI_MAX_ACTIVE_REQUESTS_PER_USER="2"
OPENAI_PREMIUM_DAILY_PRO_LIMIT="20"
OPENAI_PREMIUM_DAILY_AGENCY_LIMIT="100"
```

Database migrations are fail-closed. Before using `npm run db:migrate:safe`, set `AI_STAGING_DATABASE_NAME`, `EXPECTED_DATABASE_HOST`, `EXPECTED_DATABASE_PORT`, `EXPECTED_DATABASE_NAME`, and `ALLOW_DATABASE_MIGRATIONS=true` only for a confirmed isolated database. The guard rejects the known VPS host, the historical SSH-tunnel endpoint `127.0.0.1:5433`, and any unexpected port. Use `npm run db:migrate:status:safe` for a guarded status check; direct migration commands are not approved for staging operations.

Recommended isolated workflow:

```dotenv
DATABASE_URL="postgresql://...@localhost:5432/toknext_ai_staging?schema=public"
AI_STAGING_DATABASE_NAME="toknext_ai_staging"
EXPECTED_DATABASE_HOST="127.0.0.1"
EXPECTED_DATABASE_PORT="5432"
EXPECTED_DATABASE_NAME="toknext_ai_staging"
ALLOW_DATABASE_MIGRATIONS="true"
AI_STAGING_POSTGRES_USER="..."
AI_STAGING_POSTGRES_PASSWORD="..."
```

With Docker available, start only the isolated service with `npm run db:staging:up`; the script explicitly loads the ignored `.env.local` file so Compose does not fall back to `.env`. Then run `npm run db:identity`, `npm run db:migrate:safe`, and `npm run db:migrate:status:safe`. Never reuse a VPS tunnel for this workflow.

After migration, run the real isolation suite with `RUN_AI_DB_INTEGRATION=true npm test -- src/lib/ai/isolation.integration.test.ts` (PowerShell: `$env:RUN_AI_DB_INTEGRATION='true'; npm test -- src/lib/ai/isolation.integration.test.ts`). The test creates and cleans its own users/products and requires the same safety identity guard.

The only allowed model values are `gpt-5-nano` and `gpt-5-mini`. Invalid environment values fail closed. The browser may send only `analysisMode: auto | fast | deep`.

Keep `OPENAI_ASSISTANT_ENABLED=false` until staging verification and owner approval. To disable immediately, set the flag to false and restart the application; unauthenticated and authenticated requests then stop before OpenAI configuration or usage reservation.

## Stream contract

The route emits one JSON object per line:

- `request.started` — request correlation ID.
- `routing.completed` — depth, request class, fallback and selected model.
- `tool.started` / `tool.completed` — safe progress text and result count only.
- `text.delta` — incremental assistant text.
- `usage.updated` — sanitized token totals.
- `response.completed` — response ID, source cards, routing and totals.
- `response.error` — one of the safe error categories.

The stop button aborts the browser request and the server signal. The Responses stream, tool loop and pending rounds stop on cancellation; the durable `AiRun` record is marked `CANCELLED` rather than completed.

## Durable controls and retention

`UsageCounter.AI_GENERATION` remains the monthly message quota. `AiPremiumBudget` stores the UTC-day premium reservation atomically, so it survives PM2 restarts and multiple instances. `AiRateLimitBucket` stores the per-user fixed-minute request bucket. `AiRun` stores sanitized operational metadata: request ID, user, model, class, mode, status, fallback, tool names/count, token counts, latency, error category and timestamps.

No prompts, full answers, raw tool payloads, product records, TikTok tokens, Stripe values or API keys are stored. Retain `AiRun` records for the shortest period required by support/cost auditing (recommended 30–90 days), then delete them through a bounded, scheduled housekeeping job. The additive migration is `20260727130000_add_ai_operational_records`; apply it only to an explicitly isolated local/staging database after verifying `DATABASE_URL` does not target production.

## Readiness and diagnostics

`GET /api/ai/readiness` is admin-only and checks the flag, key presence, model allowlist, database tables and tool registry without making a paid model request. `paidConnectivityVerified` is always false until a separately approved live test is run. `/api/health` remains a general application health check.

The readiness response also reports database identity/migration verification and explicit evidence flags for live Responses API, tool calls, text deltas, cancellation, isolation, Playwright, email flow, API-key rotation, clean migration rebuild, dependency-audit review, Auth.js chain review, reachable/mitigated high vulnerabilities and a formal exception. It includes only aggregate counts and reports OpenAI rotation/live revalidation as pending until the owner records new-key evidence. These flags are false unless an operator records the corresponding controlled validation; configuration presence alone is never treated as live evidence. The endpoint remains admin-only and never exposes environment values.

For a local/staging manual provider check, run `npm run ai:diagnostic` after configuring a test key. It makes one minimal Responses API request, reports only a sanitized status/model/latency/response ID, and never prints the key or customer data. Do not run it with a production key without owner approval.

### Final blocker audit

The disposable empty-database rebuild exposed a historical ordering defect: Prisma applies `20260724_add_phase6` before `20260724134204_add_product_intelligence`, while Phase 6 creates a foreign key and GIN index that require `products`. A clean `prisma migrate deploy` therefore stops with `P3018` / PostgreSQL `42P01` (`relation "products" does not exist`). The current staging database was not changed during this audit.

No historical migration was renamed, edited, or deleted. A corrective migration after the failing migration cannot repair a brand-new database, while renaming/reordering the historical directory would diverge `_prisma_migrations` for existing databases and require a coordinated checksum/history repair. The least-risk strategy is to keep history immutable, prepare a reviewed baseline/history repair plan, and gate rollout until a disposable rebuild and an existing-database rehearsal both pass. No production migration repair is approved by this repository change.

The production dependency audit still reports the Auth.js/Nodemailer high-severity advisory with no non-breaking fix. The application does not use Nodemailer's raw message option and now disables file and URL access on the transport, but the advisory remains a production readiness blocker until the Auth.js-compatible dependency path is upgraded or formally replaced.

The migration repair preserves the historical `20260724_add_phase6` checksum. `20260724_add_phase5_compat` creates a marked two-column placeholder only on an empty database, `20260724_add_phase6_cleanup` removes only that marked placeholder after Phase 6, and `20260724150000_restore_phase6_product_dependencies` restores the Phase 6 FK and FTS index after `products` is created. Existing databases take the three migrations as no-ops (or idempotent repairs); fresh databases run the same ordered history through `prisma migrate deploy`. Roll back by disabling the feature and restoring the database/application backup; do not manually delete migration rows or product data.

For an existing isolated database, take a backup, run `npm run db:identity`, then `npm run db:migrate:safe` and `npm run db:migrate:status:safe`. A failed historical row must be reviewed against `_prisma_migrations` before any `migrate resolve`; this repair does not require changing the old checksum. For a fresh environment, create an empty database and run `npx prisma migrate deploy --schema prisma/schema.prisma`, followed by `npx prisma migrate status` and `npx prisma migrate diff --from-url <isolated-url> --to-schema-datamodel prisma/schema.prisma --exit-code`.

Email authentication remains enabled. Auth.js uses Credentials and optional Google providers; TokNext does not register Auth.js `EmailProvider`. Nodemailer is used by the server-only transactional mailer for verification, password reset, password-change, welcome, subscription and payment notifications. Recipients and subjects are validated, CR/LF header injection is rejected, templates escape user values, callback URLs are single-slash in-app paths, SMTP requires TLS 1.2 on ports 465/587, file/URL access is disabled, attachments are not accepted, and mock logs redact addresses and never print token-bearing bodies. Registration, forgot-password and verification-resend flows are rate limited.

### Auth.js/Nodemailer risk exception

The current Auth.js-compatible dependency path pins Nodemailer 8.x, while the audited advisory is `GHSA-p6gq-j5cr-w38f` (high, Nodemailer `<=9.0.0`, raw-message file read/SSRF behavior). Nodemailer 9.0.3 is outside the current `next-auth@5.0.0-beta.32` peer range (`^7.0.7 || ^8.0.5`), and no compatible patched Auth.js release is available in this dependency set. TokNext never accepts Nodemailer's `raw`, `attachments`, file paths or remote URLs; all application call sites pass fixed `{from,to,subject,html,text}` fields and the transport disables file/URL access. This is a temporary, evidence-based exception—not a claim of zero vulnerabilities—with owner review required when Auth.js or Nodemailer publishes a compatible patched chain. The readiness endpoint exposes only aggregate counts and exception flags.

Exception owner: application security owner. Review date: 2026-08-27. Expiration: 2026-10-27 or the first compatible patched Auth.js/Nodemailer release, whichever comes first. The issue automatically reopens when a patched compatible chain is available, when a new email call site accepts raw/attachment/URL input, or when an SMTP provider error exposes credentials or tokens.

The complete exception record is [docs/security/nodemailer-authjs-risk-exception.md](security/nodemailer-authjs-risk-exception.md).

### Deferred OpenAI verification

After the owner rotates the previously exposed key: revoke the old key, create a project-scoped replacement, add it only to the server environment (never `NEXT_PUBLIC_OPENAI_API_KEY`), restart the local server, run `npm run ai:diagnostic`, perform one authenticated streamed `/api/ai/chat` request and one cancellation request, verify `AiRun`/usage persistence, then record the readiness flags. Do not paste the key into chat, source, client bundles or logs.

PowerShell checklist after rotation (run only with the replacement key installed server-side):

```powershell
npm run ai:diagnostic
# Sign in with the approved local test account, then stream:
Invoke-WebRequest -Method Post -Uri http://localhost:3000/api/ai/chat -WebSession $session -ContentType 'application/json' -Body '{"message":"Busca los tres productos con mayor Opportunity Score y compáralos.","analysisMode":"auto"}'
# Repeat once, abort the request after progress begins, and verify CANCELLED AiRun/quota accounting.
```

Do not set `AI_API_KEY_ROTATION_VERIFIED`, `AI_AUTHENTICATED_STREAM_VERIFIED` or cancellation flags until those checks and persistence queries succeed.

## Routing and cost policy

Routing is deterministic and happens before the Responses API call. Nano handles searches, details, short explanations, trends, categories, hooks, captions and ordinary lookups. Mini is reserved for Pro/Agency requests that clearly need full scripts, LIVE/UGC, detailed strategies, three-plus-product comparisons or deep multi-dataset analysis. Free users always stay on nano. Premium exhaustion falls back to nano and is shown in the UI. No extra model-classification call is made.

## Security and data rules

- Auth.js identity, active status and plan are derived server-side; client `userId` and model fields are rejected.
- Tools are a strict registry with Zod validation, bounded outputs and ownership-aware repository calls.
- Demo, provider, TikTok and TokNext-derived values remain explicitly labeled.
- Opportunity Score is never described as an official TikTok metric; null and partial score states are reported as unavailable/partial.
- Sales guidance cannot invent commissions, discounts, reviews, scarcity, product capabilities, health outcomes or guaranteed income.
- Model prompts cannot request arbitrary URLs, SQL, filesystem access, secrets or another customer's private data.

## Rollout

1. Local: run static checks, migration status, mocked stream tests and the manual diagnostic only with approved credentials.
2. Staging: enable for internal test users; verify real authenticated streaming, tool progress, trend periods, cancellation, usage records and isolation.
3. Admin-only production: keep a daily premium budget cap and per-user rate limits enabled; monitor errors, latency, 429s, tool failures, token/premium usage and disconnects.
4. Selected Pro/Agency cohort: expand only when no critical data-isolation, cost or accuracy regression is observed.
5. General availability: document owner approval and rollback contact; avoid irreversible database changes during the rollout.

Rollback is the feature flag first, followed by application restart. Investigate before re-enabling if there is any cross-customer access issue, fabricated metric, cost spike, timeout surge, stream disconnect spike, database regression or token anomaly.

## Observability

Structured logs include `requestId`/`traceId`, one-way user hash, plan, locale, analysis mode, request class, selected model, fallback, tool names/count, latency, token totals, status and safe error category. Search logs by `requestId`; never log raw prompts or provider payloads. Alert on provider errors/429s, timeouts, latency, premium/token spikes, tool failures, auth failures, isolation denials, database latency, disconnects, cancellations and empty responses.

## Approved live verification

Only after the owner configures a test key, enables the flag locally/staging, provides a test account and approves spend: verify a simple real tool request, nano hooks, mini UGC strategy, cancellation, cross-customer denial, secret/prompt-injection refusal and forced provider/tool failure. Routine automated tests use mocks and never call OpenAI. A successful health check alone is not OpenAI connectivity proof.