RAG inside an existing Laravel product

RAG can be added to an existing Laravel product without rewriting the application or moving its domain model into an AI framework. Laravel should remain responsible for users, organisations, permissions, business records, and audit history. The retrieval and model layer should enter through explicit services with the same access rules as the rest of the product.

The first design question is not which vector database to install. It is which sources may answer which user, how freshness is determined, and what evidence must be shown with an answer. These decisions shape ingestion, metadata, permissions, citations, and evaluation more than the choice between pgvector and a separate vector service.

Keep the application as the authority

An authenticated request begins in Laravel. The application resolves the user, organisation, feature access, and relevant resource scope before retrieval. The AI service receives a constrained query and permission context, not a master key to the database or a complete export of internal documents.

Source ingestion can run through queues and workers already used by the product. A pipeline extracts text, divides it into useful units, attaches stable source identifiers and permission metadata, calculates embeddings, and records the version indexed. Deletion and permission changes must propagate as deliberately as creation; otherwise the search index becomes a stale copy of access the application has already revoked.

The answer should return source identifiers and passages alongside generated text. Laravel can then render links to records the user is allowed to open. A citation is useful only when it points to a recognisable source and the user can inspect the supporting context.

In one NDA-protected engagement, the important access rules already lived in the product’s relational domain model. Copying documents into a separate AI index without that context would have flattened organisation and record-level permissions. We kept the application authoritative and passed only scoped record identifiers into retrieval, so the new capability inherited the product’s existing access boundaries instead of inventing a second permission system.

Evaluation belongs next to product behaviour

Before launch, collect real questions from support, operations, or domain specialists. Each case should define acceptable sources, required facts, prohibited disclosure, and whether the system should refuse or ask for clarification. The same cases are rerun when chunking, retrieval, prompts, source content, or models change.

Evaluation should separate retrieval failure from generation failure. If the right passage never entered the context, rewriting the prompt will not solve the problem. If retrieval is correct but the answer misstates the source, the model or answer instructions need attention. This distinction prevents random tuning.

Introduce the feature as a bounded capability

The first version may answer questions over one controlled document set rather than every record in the product. It may draft a response for review instead of sending it. It may refuse when no sufficiently relevant source exists. These limits make the feature useful sooner and create evidence for expanding it.

The durable architecture is not “Laravel plus a chatbot.” It is an existing product with a new source-aware capability: application-owned identity, explicit retrieval permissions, inspectable citations, versioned evaluation, queues, monitoring, and a fallback when the model cannot produce a defensible answer.