An AI-assisted audit planning application designed to help auditors during the client pre-planning / initial planning stage.
The system analyzes uploaded client information, identifies business risks, retrieves relevant PCAOB knowledge from a Pinecone vector database, and generates a structured preliminary audit planning analysis.
Important: This project is an AI assistant and decision-support tool. It is not designed to replace the auditor's professional judgment, professional skepticism, responsibility, or accountability. AI-generated results require auditor review and approval.
The current workflow is:
Auditor
|
| Upload client information
v
Next.js API Route
|
| Extract document text
v
GPT-1 — Client Analysis
|
| Client understanding + business risks
v
RAG Query Construction
|
| Industry + identified business risks
v
Pinecone Vector Database
|
| Retrieve relevant PCAOB chunks
v
RAG Context Builder
|
| Format retrieved standards
v
GPT-2 — Audit Planning
|
| Structured Outputs / JSON Schema
v
Audit Planning Result
|
v
Web Application
The RAG knowledge base is maintained separately from the live client-analysis workflow.
The system is intended to reduce the time auditors spend starting an engagement from a blank page.
It provides an initial structure by:
- organizing client information;
- identifying business risks from the supplied client information;
- retrieving relevant PCAOB knowledge;
- connecting client risks with retrieved audit-standard context;
- generating preliminary planning areas;
- identifying areas requiring professional judgment; and
- presenting the result in a structured format.
The output is preliminary planning support, not a final audit conclusion.
The project uses the following technologies that are confirmed by the current implementation:
- Next.js
- TypeScript
- OpenAI API
- Pinecone
- PDF.js Extract
- Pinecone vector database / RAG
- Structured JSON Schema output from OpenAI
The current development environment shown during testing used:
Next.js 16.2.11
Turbopack
Node.js / npm
The RAG preprocessing workflow also uses Python tooling for PCAOB document preparation, including:
python-docxpandaslangchain_text_splitters
GPT-1 receives the extracted client document text.
Its job is to understand the client rather than perform the final audit planning.
It produces structured information including:
- company profile;
- industry;
- service sector;
- business model;
- headquarters;
- fiscal year end;
- employee count;
- company description;
- revenue streams;
- products and services;
- business processes;
- operational complexity;
- business risks; and
- supporting reasoning/evidence.
The current implementation uses:
gpt-4o-mini
GPT-1 is intentionally separated from GPT-2 so that client understanding and audit-planning generation are different stages.
The client analysis is used to create the retrieval query.
The current route builds the query from:
client industry
+
identified business risks
The application then calls:
retrieveChunks({
query,
module,
topK: 8
})
The module currently used by the audit-planning workflow is:
auditPlanning
The retriever returns the most relevant PCAOB knowledge chunks from Pinecone.
The retrieved chunks are passed into:
buildContext(chunks)
The context builder formats the retrieved knowledge into a GPT-ready string containing information such as:
- standard;
- paragraph;
- title when available; and
- retrieved content.
This formatted context is then provided to GPT-2.
GPT-2 receives:
1. Client analysis
2. Retrieved PCAOB RAG context
The current implementation uses:
gpt-4.1
The response is generated using OpenAI Structured Outputs with a strict JSON Schema.
The schema currently defines:
executiveSummaryclientBusinessUnderstandingplanningAreasstandardsAppliedoverallAuditPlanningAssessmentlimitations
Each planning area contains fields such as:
- planning area;
- client risk;
- applicable standard;
- explanation;
- audit implication;
- recommended auditor focus; and
- priority.
The priority values are restricted to:
Low
Medium
High
Critical
The overall risk level is restricted to:
Low
Moderate
High
The RAG system is separated into two major processes.
PCAOB documents are processed before the application uses them.
PCAOB source documents
|
v
Document preprocessing
|
v
Paragraph / section detection
|
v
Chunking
|
v
Metadata creation
|
v
Embedding
|
v
Pinecone
The preprocessing code shown in the project:
- loads PCAOB
.docxfiles; - cleans document text;
- detects numbered paragraphs;
- detects sections/headings;
- keeps footnotes atomic;
- validates paragraph grouping;
- chunks long text;
- creates deterministic chunk IDs;
- stores metadata;
- creates JSONL/CSV outputs.
The demonstrated chunking configuration is:
chunk size: 900
chunk overlap: 150
When an auditor uploads client information:
Client document
|
v
Text extraction
|
v
GPT-1 client analysis
|
v
Industry + business risks
|
v
Embedding / similarity retrieval
|
v
Pinecone
|
v
Top 8 retrieved chunks
|
v
Formatted RAG context
|
v
GPT-2 audit planning
The uploaded client document is not automatically added to the PCAOB knowledge base.
The runtime workflow retrieves from the existing PCAOB knowledge base.
Pinecone is the vector database used by this project.
It stores vector representations of the PCAOB knowledge chunks together with metadata.
The current index used during testing was:
pcaob-knowledge-base
The retrieved records contain fields such as:
id
content
metadata
Metadata includes information such as:
document
standard
paragraph
section
is_footnote
effective_date
chunk
Pinecone is therefore the retrieval layer of the RAG architecture.
It is not the LLM.
The following files have been discussed and/or provided during development of this project.
This is the main runtime API route for the current audit-planning workflow.
Responsibilities:
- Receive the uploaded file.
- Parse the multipart form data.
- Extract PDF text.
- Run GPT-1 client analysis.
- Build the RAG query.
- Retrieve PCAOB chunks from Pinecone.
- Build the RAG context.
- Run GPT-2 audit planning.
- Return the structured audit-planning result.
The runtime flow is:
POST /api/audit-planning
|
v
formData()
|
v
PDF extraction
|
v
runClientAnalysis()
|
v
retrieveChunks()
|
v
buildContext()
|
v
chatCompletion()
|
v
NextResponse.json()
The route shown during development reads:
files[0]
and extracts that first file as a PDF using pdf.js-extract.
Therefore, although the UI may display support for several file types, the currently shown backend route should be treated as PDF-first / PDF-specific unless additional parsers have been implemented elsewhere.
Contains a general text chunking utility.
It defines:
TextChunk
ChunkOptions
chunkText()
The default runtime utility configuration is:
chunkSize = 1000
overlap = 200
It:
- normalizes whitespace;
- handles empty documents;
- splits text into chunks;
- attempts to avoid cutting at sentence boundaries;
- adds overlap between chunks; and
- returns chunk IDs and text.
The PCAOB preprocessing script shown in the project uses a separate RecursiveCharacterTextSplitter configuration:
chunk_size = 900
chunk_overlap = 150
Therefore, the project contains a general TypeScript chunking utility and a separate PCAOB ingestion/preprocessing configuration.
Transforms retrieved Pinecone chunks into a single GPT-ready context string.
It:
- Checks whether any chunks were retrieved.
- Returns a fallback message when none are available.
- Formats each chunk.
- Includes standard and paragraph metadata.
- Includes the retrieved content.
- Separates the chunks for the model.
This is the bridge between:
Pinecone retrieval
and:
GPT-2 audit planning
This is the retrieval layer referenced by the audit-planning route.
The route calls:
retrieveChunks()
with:
query
module
topK
Its purpose is to retrieve relevant knowledge from the vector database.
The exact internal implementation of this file should be kept synchronized with the Pinecone index configuration.
Contains the OpenAI client initialization.
The current implementation uses lazy initialization through:
getOpenAI()
It checks for:
OPENAI_API_KEY
before creating the OpenAI client.
This prevents the client from being created until it is actually needed.
Contains the GPT-2 audit-planning call.
Responsibilities:
- Receive client analysis and RAG context.
- Build the audit-planning prompt.
- Call OpenAI Responses API.
- Apply the strict JSON Schema.
- Read
response.output_text. - Parse the JSON.
- Return
AuditPlanningResult.
This file is the main LLM-generation layer for the final audit-planning result.
Contains the system-level instructions used by GPT-2.
It defines the role and behavior expected from the audit-planning model.
Builds the runtime user prompt from:
clientAnalysis
+
context
This allows the GPT-2 prompt to receive the current client analysis together with the retrieved PCAOB knowledge.
This is the PCAOB ingestion/upload script referenced during development.
Its role is to:
- Load PCAOB knowledge.
- Generate embeddings.
- Prepare vectors.
- Upload vectors to Pinecone.
- Store metadata with the vectors.
During testing, the script reported:
Pinecone index: pcaob-knowledge-base
and processed multiple PCAOB standards.
Never commit API keys, Pinecone keys, or other credentials into this file or into GitHub.
The PCAOB preprocessing code shown during development performs several steps.
It loads a PCAOB Word document using:
Document(INPUT_DOC)It:
- normalizes Unicode;
- removes unwanted footnote superscript markers;
- normalizes whitespace;
- converts bullet characters into consistent text.
It identifies actual numbered paragraph starts using a regular expression rather than searching a flattened document for paragraph-like numbers.
This prevents references such as:
AS 2505.08
from being incorrectly treated as the start of paragraph .08.
The preprocessing code checks:
- Word heading styles;
- title styles;
- bold short paragraphs;
- list formatting;
- fallback text characteristics.
Footnotes are kept as atomic entries because they contain many paragraph references that could otherwise be mistaken for new paragraph boundaries.
The script checks for duplicate paragraph numbers before chunking.
The PCAOB preprocessing script uses:
RecursiveCharacterTextSplitter
chunk_size = 900
chunk_overlap = 150
Each chunk contains metadata such as:
document
standard
paragraph
section
is_footnote
effective_date
chunk
The preprocessing process creates:
cleaned_document.txt
rag_chunks.jsonl
rag_chunks.csv
metadata.json
These outputs can be inspected before uploading vectors to Pinecone.
Create:
.env.local
Do not commit it to GitHub.
The project requires an OpenAI API key and Pinecone configuration.
A typical configuration should contain placeholders similar to:
OPENAI_API_KEY=your_openai_api_key
PINECONE_API_KEY=your_pinecone_api_key
PINECONE_INDEX=pcaob-knowledge-baseUse the exact Pinecone environment-variable names expected by your current retriever.ts and upload-pcaob.ts.
Do not copy real credentials into this README.
git clone <YOUR_GITHUB_REPOSITORY_URL>
cd <YOUR_REPOSITORY_FOLDER>If the repository contains package-lock.json:
npm ciOtherwise:
npm installCreate:
.env.local
Add the required API credentials.
Example:
OPENAI_API_KEY=your_openai_api_key
PINECONE_API_KEY=your_pinecone_api_key
PINECONE_INDEX=pcaob-knowledge-baseUse your actual variable names if the implementation differs.
Make sure the Pinecone index exists and matches the index configured by the application.
Current demonstrated index:
pcaob-knowledge-base
Run the ingestion script using the project's current command:
npx tsx --env-file=.env.local scripts/upload-pcaob.tsThe script should report that the Pinecone client and required keys are loaded and should begin embedding/uploading PCAOB knowledge.
Do this when initializing or intentionally updating the knowledge base.
npm run devThe development server should start on:
http://localhost:3000
Open the web application.
Go to:
Client Understanding
→ Information Request
Upload the client information document.
The intended client-information document can contain:
- company profile;
- industry;
- service sector;
- headquarters;
- employees;
- company description;
- business model;
- products and services;
- major business processes;
- financial information;
- client-identified business risks;
- internal controls;
- management structure; and
- audit information request.
Submit the document for analysis.
The backend extracts the document text.
GPT-1 analyzes the client information.
The system constructs a RAG query from the client's industry and identified business risks.
Pinecone retrieves the most relevant PCAOB knowledge chunks.
The current audit-planning route requests:
topK = 8
The retrieved PCAOB chunks are formatted into the RAG context.
GPT-2 receives:
Client Analysis
+
PCAOB RAG Context
and generates the preliminary audit-planning result.
The result is returned to the web application.
During development, the pipeline was verified through logs similar to:
EXTRACTED TEXT:
...
GPT1 Client Analysis:
...
========== RETRIEVED CHUNKS ==========
Retrieved chunks: 8
========== RAG CONTEXT ==========
========== Chunk 1 ==========
Standard: AS 2110
Paragraph: .70
...
GPT2 Audit Planning:
...
POST /api/audit-planning 200
These logs are useful for verifying that:
document extraction
→ GPT-1
→ Pinecone retrieval
→ RAG context
→ GPT-2
are all executing.
Do not log confidential client information or secrets in production.
The knowledge base is maintained separately from client uploads.
When PCAOB source material is changed:
1. Update the source document.
2. Re-run preprocessing.
3. Validate the generated chunks.
4. Generate embeddings.
5. Upload/update the vectors in Pinecone.
6. Test retrieval.
7. Test the final audit-planning output.
Do not assume that simply uploading a new document automatically removes obsolete vectors.
If an old document must be replaced, the ingestion process should use deterministic IDs/namespaces or an explicit delete/upsert strategy so obsolete vectors do not remain in the index.
These are intentionally different data flows.
Used at runtime:
Client document
→ text extraction
→ GPT-1
→ client analysis
Maintained separately:
PCAOB documents
→ preprocessing
→ chunks
→ embeddings
→ Pinecone
GPT-1 client analysis
+
Pinecone PCAOB retrieval
→ RAG context
→ GPT-2
This separation is an important part of the architecture.
┌──────────────────────────┐
│ Auditor / UI │
└────────────┬─────────────┘
│
│ Client document
▼
┌──────────────────────────┐
│ audit-planning/route.ts │
└────────────┬─────────────┘
│
│ Extract text
▼
┌──────────────────────────┐
│ GPT-1 Analysis │
│ gpt-4o-mini │
└────────────┬─────────────┘
│
│ Industry + risks
▼
┌──────────────────────────┐
│ RAG Retriever │
│ retrieveChunks() │
└────────────┬─────────────┘
│
│ Similarity retrieval
▼
┌──────────────────────────┐
│ Pinecone │
│ PCAOB Knowledge Base │
└────────────┬─────────────┘
│
│ Top 8 chunks
▼
┌──────────────────────────┐
│ context_builder.ts │
└────────────┬─────────────┘
│
│ RAG context
▼
┌──────────────────────────┐
│ GPT-2 Planning │
│ gpt-4.1 │
│ Structured Outputs │
└────────────┬─────────────┘
│
▼
┌──────────────────────────┐
│ Preliminary Audit Plan │
└──────────────────────────┘
The system deliberately separates:
Client Understanding
from:
Audit Planning
GPT-1 focuses on extracting and organizing information about the client.
GPT-2 uses that analysis together with retrieved PCAOB knowledge to generate the preliminary planning output.
This makes the pipeline easier to inspect, test, and improve.
The final audit-planning response is not simply free-form text.
OpenAI Structured Outputs are used with:
strict: true
and a defined JSON Schema.
This helps keep the response consistent with the application's expected data structure.
The final response is parsed using:
JSON.parse(response.output_text)
and returned as:
AuditPlanningResult
This system should be treated as an AI-assisted planning tool.
It does not:
- replace an auditor;
- replace professional judgment;
- replace professional skepticism;
- establish audit conclusions by itself;
- perform substantive audit testing;
- independently verify client information;
- replace management inquiry;
- replace walkthroughs; or
- assume responsibility for the audit engagement.
The auditor remains responsible for evaluating whether AI-generated information is appropriate for the engagement.
Never commit:
.env.local
or API credentials.
Your .gitignore should include at minimum:
.env
.env.local
.env.*.localAlso avoid logging:
- API keys;
- client confidential information;
- personally identifiable information;
- confidential financial information; or
- sensitive audit documentation.
If a secret has accidentally been exposed in source control, rotate it immediately.
Before running the application:
[ ] Node.js installed
[ ] npm installed
[ ] Dependencies installed
[ ] .env.local created
[ ] OPENAI_API_KEY configured
[ ] Pinecone API key configured
[ ] Pinecone index exists
[ ] PCAOB knowledge uploaded
[ ] No secrets committed to Git
Before testing the RAG:
[ ] PCAOB vectors exist in Pinecone
[ ] Metadata is present
[ ] Retriever can return chunks
[ ] Retrieved chunks are relevant
[ ] RAG context contains standard references
[ ] GPT-2 receives the RAG context
Run:
npm install
npm run devCheck the terminal for the exact Next.js error.
Check:
OPENAI_API_KEY
and confirm the key is available to the server environment.
Check:
PINECONE_API_KEY
PINECONE_INDEX
and verify that the index contains vectors.
Also verify that the retriever's namespace/filter configuration matches the ingestion script.
Inspect:
========== RETRIEVED CHUNKS ==========
in the development logs.
Check:
- The GPT-1 client analysis.
- The generated query.
- The selected module.
- The retrieved chunk scores.
- The PCAOB metadata.
- The chunk quality.
- The embedding/index configuration.
Check:
auditPlanningResultSchema
and compare it with:
AuditPlanningResult
Also check the prompt files:
src/lib/prompts/audit-planning.ts
src/lib/prompts/build-audit-planning-prompt.ts
Test the system from left to right.
Verify that:
EXTRACTED TEXT
contains the expected client information.
Verify that the client analysis contains the expected:
industry
business risks
business model
revenue sources
processes
Verify the query is constructed from the client analysis.
Verify that:
Retrieved chunks: 8
or an appropriate number of relevant chunks is returned.
Verify that the context contains PCAOB standard and paragraph information.
Verify that the final output contains the required structured fields.
Verify that the final result is displayed correctly.
The current feature demonstrated in development is:
Client Pre-Planning / Initial Audit Planning Assistance
The system currently focuses on:
Client Information
→ Client Understanding
→ Business Risk Identification
→ PCAOB Retrieval
→ Preliminary Audit Planning
It is intended to provide an initial structure so the auditor does not have to start the planning process completely from scratch.
Potential future improvements include:
- broader document-format support;
- improved document ingestion;
- better citation/source display;
- richer Pinecone metadata filtering;
- audit-standard coverage expansion;
- stronger retrieval evaluation;
- document versioning;
- knowledge-base update workflows;
- audit-trail logging;
- user authentication;
- role-based access;
- persistent engagement storage;
- auditor approval workflows;
- human-in-the-loop review;
- additional audit-planning modules;
- evidence exploration;
- change detection;
- accounting-context analysis; and
- additional AI-assisted audit features.
These items are future development considerations and should not be interpreted as currently implemented functionality unless present in the repository.
The following structure reflects the files confirmed during development. Other UI files may exist in the repository but were not fully provided in the materials used to prepare this README.
client-understanding-ai-team-ui/
│
├── src/
│ ├── app/
│ │ └── api/
│ │ └── audit-planning/
│ │ └── route.ts
│ │
│ ├── lib/
│ │ ├── openai/
│ │ │ ├── client.ts
│ │ │ └── chat.ts
│ │ │
│ │ ├── prompts/
│ │ │ ├── audit-planning.ts
│ │ │ └── build-audit-planning-prompt.ts
│ │ │
│ │ └── rag/
│ │ ├── chunk.ts
│ │ ├── context_builder.ts
│ │ └── retriever.ts
│ │
│ └── components/
│ └── client-understanding/
│ └── client-intake/
│ └── types/
│ ├── analysis.ts
│ └── audit-planning.ts
│
├── scripts/
│ └── upload-pcaob.ts
│
├── .env.local
├── package.json
└── README.md
| File | Responsibility |
|---|---|
src/app/api/audit-planning/route.ts |
Main audit-planning API pipeline |
src/lib/rag/chunk.ts |
General TypeScript text chunking |
src/lib/rag/retriever.ts |
Retrieves relevant Pinecone knowledge |
src/lib/rag/context_builder.ts |
Converts retrieved chunks into GPT context |
src/lib/openai/client.ts |
Initializes OpenAI client |
src/lib/openai/chat.ts |
GPT-2 audit-planning generation |
src/lib/prompts/audit-planning.ts |
GPT-2 system instructions |
src/lib/prompts/build-audit-planning-prompt.ts |
Builds GPT-2 runtime prompt |
scripts/upload-pcaob.ts |
PCAOB embedding/vector ingestion |
src/components/.../analysis.ts |
Client-analysis TypeScript type |
src/components/.../audit-planning.ts |
Audit-planning TypeScript type |
For an already configured environment:
npm ciCreate/configure:
.env.local
Load/update PCAOB knowledge when needed:
npx tsx --env-file=.env.local scripts/upload-pcaob.tsStart the application:
npm run devOpen:
http://localhost:3000
Then:
Client Understanding
→ Information Request
→ Upload Client Information
→ Generate AI Recommendations
This project demonstrates how an AI assistant can combine:
LLM reasoning
+
structured outputs
+
retrieval-augmented generation
+
Pinecone vector search
+
PCAOB knowledge
+
human auditor review
The core design principle is:
AI assists the auditor; the auditor remains the decision-maker.
The objective is to make the initial audit-planning process more organized, productive, and efficient while keeping professional judgment at the center of the engagement.