# auditor-quiz

Quiz an auditor on a codebase's mechanics and vulnerabilities, drawn from its own code and docs. Use to test understanding before or during a review.

- **Kind:** skill
- **Source:** https://github.com/forefy/.context
- **Page:** https://forefy.com/skills/4aca4395-b3d3-4ce6-a9bb-3efccf7f5c63
- **API (JSON + files):** https://forefy.com/api/asr/4aca4395-b3d3-4ce6-a9bb-3efccf7f5c63

---

## SKILL.md

---
name: auditor-quiz
description: Quiz an auditor on a codebase's mechanics and vulnerabilities, drawn from its own code and docs. Use to test understanding before or during a review.
---

# Auditor Quiz Skill

Generate focused, security-oriented quizzes to test auditor understanding of codebases, protocols, and documentation.

## Overview

This skill creates 8-10 question quizzes that focus on:
- Protocol/system understanding and core mechanisms
- Weakness points and potential vulnerabilities
- Security considerations and attack vectors
- Core functionality and critical code paths

Questions are generated from repository documentation (README, docs/, whitepapers, specifications, inline comments) and presented interactively with immediate feedback.

## Workflow

1. **Identify documentation sources**
   - Search for documentation files: `*.md`, `README*`, `docs/*`, `*.sol` (comments), `*.rs` (comments), etc.
   - Prioritize: security docs, architecture docs, README, specification files
   - Use grep/glob to find relevant files efficiently

2. **Analyze documentation**
   - Read and synthesize key information about:
     - Core protocol/system mechanics
     - Security assumptions and trust boundaries
     - Known edge cases or limitations
     - Critical functions and state transitions
     - Potential attack vectors or vulnerability areas
   
3. **Generate quiz questions**
   - Create 8-10 questions (mix of multiple choice, true/false)
   - Follow guidelines in `references/question-types.md`
   - Focus on security-critical aspects and deep understanding
   - Balance difficulty: 2-3 easy, 4-5 medium, 2-3 hard questions
   - Include specific references (line numbers, function names)
   - Store questions in memory (not in files)
   
4. **Run the quiz conversationally**
   - Present questions ONE AT A TIME in the conversation
   - Format clearly with question number, text, and answer options
   - WAIT for the user's answer in their next message
   - After receiving answer, provide immediate feedback:
     - ✅ CORRECT or ❌ INCORRECT
     - Show correct answer if wrong
     - Provide detailed explanation
     - Show current score (e.g., "Score: 3/5")
   - Continue to next question only after user responds
   - Track score throughout
   - Display final results at the end with percentage and feedback
   
5. **Important: Conversational Mode**
   - Do NOT use terminal scripts, bash sessions, or file-based quiz systems
   - Present each question directly in your response
   - Use the `ask_user` tool if helpful for getting answers
   - Keep the interaction natural and conversational

## Question Generation Guidelines

### Focus Areas

**Protocol Understanding** (2-3 questions):
- How core mechanisms work
- State transitions and workflows
- Design rationale

**Weakness Points** (2-3 questions):
- Known edge cases
- Potential attack vectors
- Boundary conditions

**Security Considerations** (2-3 questions):
- Access controls
- Trust assumptions
- Input validation
- Privilege boundaries

**Core Functionality** (1-2 questions):
- Main entry points
- Critical algorithms
- Key data structures

### Quality Standards

- **Specific**: Reference actual code (function names, line numbers)
- **Relevant**: Focus on audit-critical aspects
- **Clear**: Avoid ambiguity in questions and answers
- **Educational**: Explanations should teach, not just confirm
- **Deep**: Test understanding over memorization

Consult `references/question-types.md` for detailed examples and patterns.

## Resources

- **references/question-types.md** - Question format guidelines, examples, and best practices

## Tips

- When documentation is extensive (>10 files), prioritize security-relevant docs first
- Include code references in explanations (e.g., "line 142", "deposit() function")
- Present questions one at a time, waiting for user response between each
- Keep conversational flow natural - don't use scripts or terminal sessions
- Track score internally and display after each question
- Don't make the correct question obvious by it being always the longer answer, or always the same choice field

## references

```

```

## references/question-types.md

# Question Types and Guidelines

## Question Categories

When generating quiz questions, focus on these critical areas:

### 1. Protocol Understanding
- Core mechanisms and workflows
- State transitions and lifecycle
- Key invariants and assumptions
- Design rationale and tradeoffs

### 2. Weakness Points
- Known attack vectors
- Edge cases and boundary conditions
- Potential exploit scenarios
- Mitigation strategies

### 3. Security Considerations
- Access control mechanisms
- Input validation patterns
- Trust boundaries
- Cryptographic operations
- Privilege escalation risks

### 4. Core Functionality
- Main entry points
- Critical data structures
- Key algorithms
- Integration points
- Dependencies and assumptions

## Question Formats

### Multiple Choice (Preferred for complex concepts)
- 4 options (A, B, C, D)
- One clearly correct answer
- Plausible distractors that test understanding
- Focus on "why" and "how" rather than simple facts

**Example:**
```json
{
  "question": "What happens if a user calls withdraw() before the timelock period expires?",
  "type": "multiple_choice",
  "options": [
    "The transaction reverts with a TimelockNotExpired error",
    "The withdrawal succeeds with a reduced amount",
    "The transaction succeeds but emits a warning event",
    "The funds are sent to a penalty address"
  ],
  "correct_answer": "A",
  "explanation": "The require(block.timestamp >= unlockTime) check on line 142 ensures transactions revert if called before the timelock expires, preventing premature withdrawals."
}
```

### True/False (Best for testing assumptions)
- Clear statement that is unambiguously true or false
- Should test critical understanding or common misconceptions

**Example:**
```json
{
  "question": "The contract allows the owner to pause all user withdrawals indefinitely without any time limit.",
  "type": "true_false",
  "options": ["True", "False"],
  "correct_answer": "B",
  "explanation": "False. The pauseWithdrawals() function includes a MAX_PAUSE_DURATION constant (72 hours) to prevent indefinite pausing, protecting user funds from admin abuse."
}
```

### Short Answer (For specific technical details)
- Requires recalling specific function names, values, or mechanisms
- Tests precise knowledge of implementation

**Example:**
```json
{
  "question": "What modifier is used to restrict administrative functions to the owner?",
  "type": "short_answer",
  "correct_answer": "onlyOwner",
  "explanation": "The onlyOwner modifier checks that msg.sender == owner before allowing execution of admin functions."
}
```

## Difficulty Balance

Distribute questions across difficulty levels:
- **2-3 Easy**: Basic concepts, main functionality
- **4-5 Medium**: Interactions, edge cases, security patterns
- **2-3 Hard**: Subtle vulnerabilities, complex scenarios, attack vectors

## Quality Guidelines

1. **Specificity**: Questions should reference actual code, not generic concepts
2. **Relevance**: Focus on security-critical and audit-relevant aspects
3. **Clarity**: Avoid ambiguous wording
4. **Learning**: Explanations should add value and cite specific lines/functions
5. **Depth**: Test understanding, not memorization