Scaling Content Audits with GenAI

As part of an AI leadership capstone project, I applied Generative AI (GenAI) to a large-scale content audit workflow to reduce the time and effort required while maintaining accuracy and structure. I completed the project over 6 weeks within Stanford’s AI-Driven Leadership program using an enterprise-style documentation dataset.

The goal was to determine whether GenAI could reliably shift content audits from a manual, high-effort activity to a repeatable, scalable workflow for large documentation systems.

Prompt Engineering

Prompt design was critical to system performance. I iterated through a couple strategies and developed a set of effective prompts that improved the accuracy and reliability of the results:

  • Single vs combined - Combined prompts reduced context loss and improved consistency.

  • Direct vs conversational: Conversational prompts significantly improved accuracy by validating assumptions and refining the outputs.

Audit Prompts

Help me complete an audit of the content in the attached PDF file. 
Use this document to create the audit spreadsheet by completing these steps: 
> Place the title of each "topic" in a separate row in an Excel spreadsheet. 
> Indicate each topic's level in the hierarchy using breadcrumbs that show the path to the topic. 
> Summarize the first paragraph of text that appears below the title. When there isn't text following the title (some topics start with tables or notes or have no text at all) indicate that this is the case. 
> Place the output of the next 2 steps on the same tab in the spreadsheet as part 1. Generate key findings based on your analysis and place these on a separate tab in the spreadsheet. 
> Analyze the contents to determine whether the topic title and the first paragraph (meta description) accurately represents the contents of each topic. 
> Analyze whether the format of each title matches the DITA topic type (topic, concept, task, reference) and whether the topic contents is a mix of topic types. Include the analysis in the spreadsheet. Typically topics are overview or title-only topics. Concepts describe how something works and when or why to use it. Tasks include numbered steps. Reference is a list of facts or tables. 

Any suggestions or questions before we start?

AI-Augmented Audit Execution

I restructured the traditional audit process into an AI-supported system:

  • Spreadsheet generation (titles, hierarchy, meta descriptions)

  • Title-to-content alignment analysis

  • DITA topic type classification validation

  • Pattern detection (duplication, mixed-topic structures)

Audit Spreadsheet

Results

  • ~1,000 topics processed

  • ~395,000 words analyzed

  • Time reduction: ~90% (from 6–11 days to 2–3 hours)

  • Accuracy threshold achieved: ~80% across audit criteria

  • Successfully generated:

    • Audit spreadsheet structure

    • Duplicate content identification

    • Topic classification insights

    • Governance recommendations

Manual Audit

6-11 days
~971 topics reviewed manually

AI Audit

2-3 hours
Same dataset analyzed
10x faster