Capture AI crawlers on Vercel.

Last reviewed 2026-10-19

A Log Drain sends every request, including CDN-cached pages, without running your code. Middleware works on any plan but only sees requests that reach your app.

Choose a method#

Method	Plans	Coverage
Log Drains (recommended on Pro+)	Pro and Enterprise	Best coverage: includes CDN-cached pages without running application code
Middleware	Any plan	Runs per matched request in Next.js 16+ `proxy.ts`; does not replace CDN request logs

You need your site ID and server secret from Settings → Connection (see Hosting integrations). In the dashboard, Connect analytics → Hosting integrations → Vercel shows these steps with your values filled in.

Log Drains#

Set up with your coding agent

Hand the setup to Claude Code, Codex or another coding agent. Replace the site ID and example.com with your own; the agent asks you for the server secret. In the dashboard, Install with AI fills them in.

Prompt

Configure a Vercel Log Drain so example.com sends request logs to Virent AI crawler analytics.

Site ID: <your-site-id>
Use the full Virent server secret from the Connect dialog. Do not use the masked preview "vha_sk_••••" as the real value.

Steps:
1. In the Vercel dashboard, add a Logs drain for the project serving example.com.
2. Name it Virent Crawler Logs. Select that specific project.
3. Sources: Functions, Edge Functions, Static Files, Rewrites, Redirects. Environment: Production.
4. Destination: Custom Endpoint https://stage.virent.app/api/ingest/vercel
5. Add an Authorization header with value: Bearer <paste-your-server-secret>
6. Keep delivery at 100% with no sampling rules, create the drain, then send a crawler request to example.com.

Do not commit the server secret.

Choose data to drain#

Open your team's drain settings in the Vercel dashboard. Click Add Drain, select Logs, then click Next.

Configure the drain#

Setting	Value
Drain name	`Virent Crawler Logs`
Projects	Specific Projects: pick the project serving your site
Sources	Functions, Edge Functions, Static Files, Rewrites, Redirects
Environments	Production

You do not need sampling rules. Continue with the defaults after selecting the project, sources and production environment.

Configure the destination#

Select Custom Endpoint, paste this URL, add a custom Authorization header with the value below (your server secret after Bearer), then click Test.

Custom endpoint

https://stage.virent.app/api/ingest/vercel

Authorization header value

Bearer <paste-your-server-secret>

Create the drain#

Keep delivery at 100% with no sampling rules, then click Create Drain at the end of the wizard. That is everything on Vercel's side. Send a test crawler request to your site to confirm.

Middleware#

For Next.js 16+ on any Vercel plan. The Next.js guide covers the same proxy together with Human Analytics.

Set up with your coding agent

Prompt

Install Virent AI crawler analytics on example.com with Vercel middleware (Next.js proxy.ts).

Site ID: <your-site-id>
Use the full Virent server secret from the Connect dialog. Do not use the masked preview "vha_sk_••••" as the real value.

Steps:
1. Run npm install @virent.app/sdk@^0.3.0 in the project serving the site.
2. Add proxy.ts at the project root, or src/proxy.ts if the app uses a src directory. Use createVirentBotProxy with endpoint: process.env.VIRENT_INGEST_ENDPOINT and report in event.waitUntil so the response is never blocked. Do not rely on the SDK default endpoint; use the exact Virent environment below.
3. Set these environment variables locally and in Vercel:
VIRENT_SITE_ID=<your-site-id>
VIRENT_INGEST_SECRET=<paste-your-server-secret>
VIRENT_INGEST_ENDPOINT=https://stage.virent.app/v1/ingest/bot
4. Deploy the site to Vercel, then send a crawler request to example.com. Confirm the ingest request is accepted and appears in this Virent environment; an HTTP 200 from the website alone does not verify analytics delivery.

This is server-side crawler tracking, not browser analytics. Do not hard-code the secret.

Install the package#

Run this in the project serving your site:

npm install @virent.app/sdk

Add the proxy#

Create proxy.ts at the project root, or inside src if the project uses a src directory. The helper reports in event.waitUntil, after the response path is released.

proxy.ts

// proxy.tsimport { createVirentBotProxy } from "@virent.app/sdk/next"export const proxy = createVirentBotProxy({	  siteId: "<your-site-id>",	  writeKey: process.env.VIRENT_INGEST_SECRET!,	  endpoint: process.env.VIRENT_INGEST_ENDPOINT || "https://stage.virent.app/v1/ingest/bot",	  trustedProxy: "vercel",	})export const config = {  matcher: [    "/((?!api|_next/static|_next/image|favicon.ico|robots.txt|sitemap.xml|.*\\.(?:png|jpg|jpeg|gif|svg|ico|webp|css|js|map)$).*)",  ],}

Set environment variables#

Add these values locally and in Settings → Environment Variables on Vercel, so the deployed site can report securely.

.env.local

VIRENT_SITE_ID=<your-site-id>VIRENT_INGEST_SECRET=<paste-your-server-secret>VIRENT_INGEST_ENDPOINT=https://stage.virent.app/v1/ingest/bot

Deploy#

Push to GitHub or deploy from the Vercel CLI, then send a test crawler request.

vercel deploy --prod

Integrations overview

Connect Vercel, Cloudflare, AWS CloudFront, Akamai, Netlify, Google Cloud, Fastly, Framer or Webflow to Virent AI Crawler Analytics, with the same steps as the dashboard.

Cloudflare

Connect a Cloudflare zone to Virent AI Crawler Analytics with a Worker (most sites) or Logpush (Enterprise).