Making AI Catalog Chat Safer Without Losing Merchant Visibility

An AI-powered catalog chat can make a Shopify storefront easier to explore. Instead of forcing a shopper to translate a need into filters and keywords, it can answer questions such as “Which patio sets fit a small balcony?” or “Show me a dining table that seats eight and is in stock.”

That convenience also creates an exposed public endpoint. Automated traffic can submit large volumes of prompts, consume model and search capacity, probe catalog behavior, and distort engagement metrics. A practical solution must protect the expensive parts of the request path without turning legitimate shoppers into collateral damage—or leaving merchants unable to understand how the protection is performing.

This project addressed that balance in a configurable Next.js Shopify Catalog Chat prototype. The application connects AI responses to store-aware tools through the Model Context Protocol (MCP), while a layered reCAPTCHA Enterprise implementation evaluates each chat submission before OpenAI or catalog processing begins. The result is a safer conversational shopping experience with reporting that helps merchants tune decisions using real data.

The central design principle: stop risky traffic early

The most important architectural choice is where verification occurs. Browser-only checks are useful for collecting a signal, but they should not be treated as the final authority. A public client can be manipulated, scripted, or bypassed. The server must make the enforcement decision.

For each chat submission, the application first resolves the tenant and then validates the reCAPTCHA token through a server-side Google CreateAssessment request. Only a token that is present, valid, and associated with an accepted risk score can proceed to the AI orchestration layer.

flowchart LR
  A["Shopper submits a chat message"] --> B["Resolve merchant tenant"]
  B --> C["Check honeypot and browser token"]
  C --> D["Server verifies token with reCAPTCHA Enterprise"]
  D --> E{"Score meets tenant threshold?"}
  E -->|"No"| F["Return HTTP 403 and stop"]
  E -->|"Yes"| G["Invoke OpenAI and MCP catalog tools"]
  G --> H["Store turn analytics and accepted score"]
  H --> I["Show merchant reporting"]

This order matters operationally. A rejected request receives an HTTP 403 response and does not create a conversation-turn analytics row. That prevents low-quality or automated requests from spending downstream AI and MCP capacity, while also avoiding polluted engagement data that could make a storefront appear busier or less effective than it really is.

Why risk scores are better than a single yes-or-no gate

Modern bot protection produces a risk signal rather than a simplistic verdict. In this implementation, requests must meet a configurable application threshold; the default is 0.8. Scores below that threshold are rejected, while accepted scores travel with the request into streaming analytics.

This gives the platform a useful middle ground between an open endpoint and a high-friction challenge flow. Legitimate shoppers can continue using an invisible browser-side check, while the server applies a consistent policy before sending work to the model or store tools.

A score threshold should not be treated as a magic number. A stricter threshold can reduce suspected automation, but it may also block more genuine customers in unusual browsing environments. A lower threshold may admit more traffic, including some requests that are less likely to be human. The right setting is therefore a product and operational decision, not merely a security setting.

Multi-tenant security requires tenant-aware configuration

A single-store prototype can put configuration in local environment variables. A multi-tenant application needs a more deliberate model because each merchant may have separate reCAPTCHA identifiers, project settings, and tolerance for risk.

The platform supports both local configuration and database-backed tenant settings. Each tenant can hold its public site and project identifiers, while the API key remains server-only. The operational workflow also supports configuration upserts, deployment, and credential verification so a value being saved is not confused with a tenant being truly ready to process protected chat traffic.

That distinction is essential. Provisioning has at least two stages:

1. Configuration exists: the expected identifiers and secret references have been saved.
2. Configuration works: the deployed environment can safely access the credentials and successfully verify real requests.

For teams building multi-merchant conversational experiences, verifiable tenant provisioning for AI commerce is a more dependable operational model than stopping at a successful configuration write.

Protecting an AI commerce request path

The chat experience uses the OpenAI Responses API with native MCP tools that can access product catalog data, blog content, policies, FAQs, and—where enabled—OpenSearch queries. Those tools turn a general-language interaction into a store-aware shopping assistant, but every call may incur latency, API usage, and infrastructure cost.

Placing bot-risk verification before that work creates a clean control boundary. Requests that fail do not reach model inference, product discovery, policy lookup, or optional search infrastructure. Requests that pass are eligible to use the same store-specific capabilities that make conversational commerce valuable.

This is also why MCP deserves intentional engineering rather than being treated as a simple connector. A well-designed tool layer should define the data an assistant can access, apply authorization and tenant boundaries, expose observable behavior, and avoid uncontrolled access to business systems. Organizations exploring this architecture can learn more about MCP server development and conversational search systems built around secure tools, retrieval workflows, and trustworthy customer experiences.

Visibility is part of the security control

Blocking traffic is only half of the job. Merchant teams need enough context to assess whether the current policy is effective, whether it is too aggressive, and how accepted conversations contribute to discovery and conversion.

For accepted requests, the application stores the reCAPTCHA score with conversation-turn events. Reporting can then expose both the score for an individual turn and an average score across a conversation. This turns a hidden gate into an observable operational signal.

Those risk values sit alongside broader commerce analytics, including:

• Conversation volume and engagement patterns
• Product mentions and time to first product mention
• Product-link clicks
• Cart and checkout events
• Tool calls and token usage
• Bounce rate

With those signals in one reporting model, a merchant can investigate meaningful questions: Are lower-but-accepted scores correlated with short, unproductive sessions? Does a threshold adjustment change time to first product mention? Are product-link clicks and checkout activity holding steady after security tuning? The platform does not assume that every low-ish score is malicious; instead, it gives operators evidence for making better policy choices.

Implementation lessons from the architecture

1. Enforce where the cost begins

The strongest practical savings come from rejecting unwanted traffic before model calls and catalog-tool execution. Late-stage filtering still leaves the business paying for work that should never have happened.

2. Keep secrets and final decisions on the server

Public identifiers can be used by the browser to obtain a token. Server-only credentials must remain outside browser bundles, and the server should be the source of truth for token validity, score interpretation, and acceptance.

3. Make configuration flexible without making it ambiguous

Local settings are useful for development, while database-backed configuration supports multi-tenant operations. Clear precedence rules, safe secret handling, explicit deployment steps, and verification checks keep that flexibility manageable.

4. Treat reporting as a feedback loop, not an afterthought

A fixed threshold may be a reasonable default, but it should be reviewable. Recording accepted scores with user behavior provides an evidence base for tuning without guessing.

5. Use defense in depth

The honeypot, invisible browser execution, server-side assessment, threshold enforcement, logging, and reporting each address a different part of the problem. No individual layer should carry the entire security burden.

A safer foundation for Shopify conversational commerce

Conversational catalog discovery is most useful when it is connected to current product, content, and policy data. It is also most sustainable when merchants can provide that capability without opening an unbounded path to expensive automated requests.

By verifying bot risk on the server before AI and MCP work, applying tenant-specific thresholds, rejecting failed submissions before analytics creation, and retaining accepted scores for reporting, this approach preserves both protection and visibility. It gives teams a way to make the AI shopping experience more resilient while still measuring what matters: whether real customers are finding products, engaging with recommendations, and moving toward purchase.

For retailers planning store-aware AI experiences, the broader foundation includes reliable storefront and integration work, thoughtful data access, and ongoing operations. Explore ecommerce development services, Shopify development, and application support services for the systems behind a production-ready conversational commerce program.