The SLED AI Search Procurement Checklist: 52 Questions to Ask Every Vendor

Where exactly is our data stored?

Accept only a specific answer: data centre region(s), cloud provider, and whether storage is US-only or shared globally. 'We use AWS' is not an answer. 'US-East-1 and US-West-2, AWS GovCloud, single-tenant' is an answer.

Is our indexed content stored separately from other customers' data?

Require explicit confirmation of data isolation architecture. Shared infrastructure is acceptable only if tenant isolation is cryptographically enforced at the storage layer.

Is our query log data (what users search for) stored, and where?

Query logs often contain sensitive information — staff names, citizen queries about personal situations, internal project names. Know exactly where this data goes, for how long, and who can access it.

Is our data used to train AI models — including foundation model fine-tuning?

This must be a contractual no, not just a policy statement. Require a specific contractual provision. Some vendors train models on customer data by default unless you opt out — often buried in terms of service.

What happens to our data if we cancel or don't renew?

Require a documented data deletion policy with a specific timeline (e.g., within 30 days of contract termination), written confirmation of deletion, and export options before termination.

Is a private/on-premise deployment option available?

Critical for agencies with strict data residency requirements or classified data environments. If yes, understand the performance and feature parity with the cloud offering.

What access controls govern who at the vendor can see our data?

Require details on vendor staff access: is it policy-restricted, technically restricted, or fully prevented? What audit log exists for vendor-side access? What background check requirements apply to staff with access?

Does your platform comply with our state's data privacy statutes?

Many states have specific data privacy requirements for technology vendors contracting with government agencies. Require the vendor to confirm compliance with the specific statutes applicable to your jurisdiction, in writing.

Provide your WCAG 2.1 AA conformance testing documentation

Ask for a third-party accessibility audit report, not just a vendor assertion. The report should identify specific success criteria tested, the testing methodology (automated + manual), and any known issues with remediation status.

How does your search widget handle keyboard navigation?

Request a live demonstration of full keyboard navigation: tab to search, type query, navigate results with arrow keys, activate a result with Enter, return focus to search with Escape. If any step requires a mouse, it fails 2.1.1.

Is the AI-generated answer output compatible with screen readers?

AI answer panels are a common accessibility failure point. The answer must be announced by screen readers when it appears (using aria-live or equivalent), with all citations navigable by keyboard.

What is your FedRAMP authorisation status?

Distinguish between 'FedRAMP Authorised', 'In Process', 'Ready', and 'not pursuing FedRAMP'. For state government use, FedRAMP is not always required — but the architecture that enables FedRAMP (encryption, audit logs, access controls) signals genuine security maturity.

Provide your most recent SOC 2 Type II report

A SOC 2 Type II report covers operational security controls over a 6–12 month period. A SOC 2 Type I report only covers a point-in-time assessment. Require Type II. The report should be available under NDA.

How do you enforce Section 508 compliance for federal procurement requirements?

For federal grant-funded SLED organisations, Section 508 of the Rehabilitation Act applies to technology purchased with federal funds. Require a Voluntary Product Accessibility Template (VPAT) or equivalent conformance documentation.

For K-12 deployments: confirm FERPA compliance architecture

FERPA compliance requires that student educational records are not accessible without appropriate authorisation. Ask specifically how the indexing and retrieval architecture prevents cross-role data exposure.

For K-12 deployments: confirm CIPA and COPPA applicability

If the AI search or chat widget is accessible to students under 13, COPPA applies to data collection. CIPA applies to filtering requirements for schools receiving E-Rate funding. Require specific written confirmation of how the platform addresses both.

Does your platform qualify under any state contract vehicles (NASPO, OMNIA Partners, etc.)?

Procurement via existing state contract vehicles significantly reduces procurement timeline and legal review burden. Carahsoft, OMNIA Partners, and NASPO ValuePoint are the most common channels for government technology. Confirm exact contract numbers.

What is your data breach notification process and timeline?

State breach notification laws typically require notification within 30–72 hours of discovery. Require the vendor's breach notification SLA in writing and confirm it meets your state's statutory requirement.

Explain your retrieval architecture — is it RAG-based? What retrieval model?

Retrieval-Augmented Generation (RAG) is the current standard for grounded AI search. Ask which embedding model is used for semantic retrieval, what the vector store is, and how answers are grounded — i.e., how the system prevents the LLM from generating answers beyond the retrieved documents.

What LLM does your platform use for answer synthesis?

Know whether the vendor uses OpenAI, Anthropic, Google, or a proprietary model. Ask whether they have a model-agnostic architecture or are locked to a single provider. LLM providers change their terms — vendor lock-in to a single LLM provider carries risk.

How does your platform handle multi-domain and multi-format content?

Government and education organisations typically have content spread across multiple websites, document formats (PDF, DOCX, XLSX), and systems. Ask for specific confirmation of which formats are supported and how cross-domain search is handled (shared index or federation).

What is the real-time indexing latency?

Ask: if we publish a new policy page at 10am, when will it appear in search results? The answer should be minutes, not hours or days. Overnight crawls are unacceptable for organisations that publish time-sensitive information.

How does role-based access control work in your search index?

For organisations with sensitive internal content (staff-only documents, student records, HR files), the search system must enforce existing access permissions. Ask specifically: does a user searching as a staff member see content that an anonymous visitor cannot? How is this enforced at the index level?

What is the maximum content scale your platform has been tested to?

Relevant for large agencies or universities with millions of documents. Ask for specific customer reference points — e.g., 'we currently index 2.4 million documents for [organisation type]'. Vendor benchmarks from isolated test environments are less meaningful.

What is your uptime track record for the past 12 months?

Request historical uptime data, not just the contractual SLA. A vendor claiming 99.9% uptime on their website should be able to show you a status page or incident history that confirms it.

How is your search widget integrated — JavaScript snippet, API, or CMS plugin?

Integration method affects your IT team's workload and the deployment timeline. JavaScript snippet embeds are the fastest to deploy. CMS plugins are easier to maintain long-term. API integration provides maximum flexibility but requires development resources.

Can the AI answer interface be fully white-labelled and branded?

Government websites have brand and accessibility standards. Confirm the search widget and answer interface can be customised to match your visual identity — colours, typography, logo — without custom development costs.

What analytics does the platform provide, and can we export the data?

At minimum, require: query volume over time, top queries, zero-result queries, click-through rates per result, and AI answer engagement rates. Confirm that analytics data is exportable (CSV at minimum) and that historical data is retained for at least 12 months.

What is the typical time from contract signature to public go-live?

The honest answer for a well-designed AI search platform is 2–10 days. If a vendor says 3–6 months, ask why — the complexity they are describing is likely either in their platform's architecture (a red flag) or in a custom implementation that you are paying for and that you could obtain from a simpler vendor faster.

What do we need to provide to start the implementation?

The answer should be: access to your CMS or website (read-only crawl credentials), a list of domains to index, and a kickoff call. If the vendor requires months of data export, system integration work, or a complex implementation project, that is not a 'plug-and-play' platform.

Who manages the ongoing indexing — us or you?

Some platforms require the customer to manually trigger re-indexing or manage crawl schedules. Best-in-class platforms index continuously and automatically. Clarify exactly what ongoing operational burden rests with your team.

What does your onboarding process include, and is it included in the contract price?

Scope what is and isn't included in the base contract: content audit, relevance tuning, WCAG testing, staff training, analytics setup. Vendors who charge separately for each of these can make a nominally cheap platform very expensive in practice.

Do you provide a dedicated customer success manager or onboarding specialist?

SLED organisations typically don't have large internal teams to manage a new platform deployment. Ask specifically whether you will have a named point of contact during implementation, and what their availability and response time commitment is.

What is your process for handling content that should not be indexed?

Every organisation has content that should not surface in search — draft documents, internal communications, restricted files. Ask how the platform handles exclusion rules, robots.txt compliance, and access-controlled content.

Have you deployed for an organisation similar to ours in scale and sector?

Ask for a relevant reference customer — not a general 'we work with government' claim, but a specific organisation of similar size and type. A vendor who cannot provide a relevant reference for a government or education deployment has limited SLED experience.

What does a typical 90-day post-launch engagement look like?

The first 90 days are when relevance tuning, analytics review, and user feedback integration happen. A vendor who goes quiet after go-live is a vendor who doesn't have a mature customer success function.

What is the pricing model — per document, per query, per user, or flat licence?

Per-query pricing can be unpredictable and expensive at scale. Per-user pricing is difficult to forecast for public-facing search (you cannot predict citizen volumes). Flat-licence or tiered-content-volume pricing is generally most favourable for government deployments.

Are there any usage caps, and what happens if we exceed them?

Some vendors cap query volume, document count, or API calls. Ask specifically what happens at cap: does the search continue at degraded performance, stop entirely, or incur overage charges?

Is implementation cost included, or is it a separate professional services fee?

A vendor who charges $5K/year for the platform but $20K for implementation has not disclosed their true cost of ownership. Ask for a fully loaded cost including all implementation, training, and onboarding fees.

What is the minimum contract term, and what are the exit provisions?

Multi-year lock-in is common. Understand the penalty for early exit, and whether you can exit without cause at annual renewal. For government organisations subject to budget cycles, one-year renewal options are often preferable.

Is the platform available on a state contract vehicle that avoids a standalone RFP?

Procurement via Carahsoft, NASPO ValuePoint, OMNIA Partners, or a state-specific contract vehicle eliminates most of the RFP burden. If the vendor is not on any contract vehicle, factor in the time and cost of a full procurement process.

What is included in the annual licence renewal — are feature updates included?

Confirm whether new features, AI model upgrades, and compliance updates are included in the annual fee or whether they require additional purchase. Some vendors charge upgrade fees for major platform releases.

Is there a pilot or proof-of-concept option before full contract commitment?

A 2–4 week paid or unpaid pilot on your actual content is the most reliable way to evaluate an AI search platform. Vendors who refuse to do a POC before contract commitment should be regarded with scepticism.

What financial remedies apply if SLA commitments are missed?

Uptime SLAs are only meaningful if there are financial consequences for missing them. Service credits should be automatic (not requiring manual claims) and material (typically 10–30% of monthly fee per SLA breach).

What are your support tiers and what is included at each level?

Understand the full support model: what is included in the base contract vs purchased separately. Email-only support with a 72-hour response time is not appropriate for a public-facing government service.

What is the documented response time for a Severity 1 (service outage) incident?

For a Severity 1 incident (search is completely unavailable), the response time SLA should be 30 minutes or less, and the resolution SLA should be 4 hours or less. Get these in writing in the contract.

Do you provide a public status page with incident history?

A status page (e.g., status.vendor.com) with real-time status and historical incident records is a basic trust signal. If a vendor doesn't have one, their uptime claims are unverifiable.

What is the process for reporting a potential WCAG compliance issue in the search interface?

Your organisation may receive accessibility complaints from users. Know the specific process for reporting these to the vendor and the committed response time for accessibility fixes.

Is support staffed in your time zone?

For US government organisations, support during US business hours is typically required. Offshore-only support with a significant time zone gap can mean multi-day delays for non-critical issues.

What is your process for communicating AI model changes or retraining events?

AI model updates can change answer quality, formatting, or behaviour. Ask how the vendor communicates planned model changes, what the notice period is, and whether you have a testing window before changes go live.

What documentation is provided for your admin portal and configuration tools?

Government IT teams need documentation that is complete, current, and searchable. Ask to see actual documentation — not the vendor's promise that documentation exists. Evaluate its quality and completeness.

What training is provided for admin users, and in what format?

Training should cover: relevance tuning, exclusion rule management, analytics interpretation, and widget configuration. Ask whether training is recorded (asynchronous) or live-only, and whether future staff can access training materials without additional cost.