← LearnProof

Research, privacy & pilot readiness

Prototype documentation. School authorization is required before real participation.

# LearnProof school pilot guide

## Before real participation

- Obtain written school/ethics authorization, an adult staff sponsor, and the school's required consent/assent process. Explain voluntary participation, withdrawal, ordinary teaching alternatives, and that participation does not determine grades.
- Confirm the hosting region, provider terms, access audience, encryption, retention, incident response, and any required school data agreement with the responsible school office. The prototype itself is not a compliance determination.
- Keep the mapping between participant codes and student names offline with the teacher. Do not put student names, emails, AI chat histories, or identifiable free text into this app.
- Approve the curriculum questions, explanations, answer keys, conceptual tags, and form length. Have a second subject teacher review where feasible. Do not describe generated explanations as verified until that review occurs.
- Decide the sampling frame, eligibility, study group allocation, outcome measures, review time, supervision, delayed interval, missing-data plan, and analysis plan before collecting outcomes. Document approval for random allocation if used.
- Teachers may self-register with email; researcher accounts require a staff invitation. only the owning teacher can share a session with a researcher. Store exports in restricted school-approved storage. Set a deletion date and a backup-retention policy.

## Assessment protocol

1. Teacher creates a class/session, chooses supervised verification or unsupervised self-check, delay (default 7 days), review time (default 5 minutes), and short/full form. Legacy research sessions require a school-authorization attestation. New classroom checks use the Create Check wizard without an organization gate; this does not substitute for required consent or research approval.
2. Teacher reviews questions and approves the set. Generate pseudonymous participant IDs and private access keys. Give each learner only their own key plus the session code. Never project the private roster.
3. Open baseline for 20 minutes. Students complete the 5-item form in a target of 3–5 minutes (10-item form may take 8–10). Confidence follows each answer, from 0–100. No hints or answer reveal during the window. Record actual classroom procedures; browser controls cannot prove independence.
4. Close baseline, releasing feedback. Give both groups comparable study time and approved study resources. Students may use the guided learning module or normal approved AI tools. No private AI chats are requested.
5. Open immediate verification with a different matched form. Students submit independently with confidence. Close the window to release feedback. Do not reopen released forms.
6. Start the common review period. Group A uses general review materials; Group B answers targeted retrieval items before receiving explanations. Require the same minimum review duration. Record practical deviations separately. Minimum timer duration is not proof of engagement.
7. At approximately seven days from each student's immediate submission, open delayed verification. Each learner's due time is enforced server-side. The teacher manages reminders and the window. Review and report each stage's setting separately if your implementation expands mixed modes.
8. Collect student satisfaction (1–5) and optional nonidentifying feedback. Teacher records setup minutes, interpretation minutes, usefulness (1–5), and comments. Export question-level CSV.

## Research interpretation

Primary descriptive outcome: delayed score. Report group sample sizes, missing checks, completed cycles, baseline and immediate scores, and delayed-minus-baseline paired change. The dashboard reports means, Brier calibration, and approximate normal intervals when at least two observations exist. Small, bounded samples often violate that approximation. Report uncertainty and avoid claims of effectiveness from the demonstration or a small convenience sample.

The two conditions should differ in the review intervention, with comparable study tools, time, forms and supervision. Alternating allocation is not randomized. Randomized allocation uses a shuffled balanced list within each roster batch; record batch size and assignment procedure. Allocation locks after a learner starts. Form comparability is designed, not validated. Consider testing difficulty and item discrimination before a formal study.

The dashboard also fits a complete-case OLS model delayed ~ condition + baseline and reports the condition coefficient and its unclustered standard error, with at least four observations, both groups, and a nonsingular design. This assumes a common linear baseline slope and independent residuals. It is descriptive, not proof of causality. For a preregistered analysis, export data to Python, with classroom clustering or a multilevel model when justified and enough clusters exist. Check overlap, model assumptions, attrition, and randomization integrity; do not label observational group differences causal. A complete-case analysis may be biased. Do not fill missing scores with zero. Report why an assessment is missing separately from teacher-marked dropout.

Confidence calibration: Brier = mean((confidence / 100 − correctness)^2), where correctness is 0/1. Lower is better. A wrong answer at high confidence can guide reflection; it is not a diagnosis of a student's ability or honesty. Unmatched wrong answers are flagged as a concept needing review, not a confirmed misconception.

## Teacher acceptance test

Use a synthetic session. Assign one participant to each condition. Submit correct roots in reverse order, fractional roots, a sign error, and 85% confidence. Check feedback is hidden until close. Verify only Group B gets targeted practice and that first retrieval is recorded before reveal. Test early delayed check rejection, then use the synthetic-only due-date override. Verify updated teacher scores, Brier and missingness; open the CSV in Python or a spreadsheet. Sign in as another teacher and verify the session is inaccessible. Invite a researcher and verify only granted sessions are available. Delete the test session and confirm participant access is gone.

## Withdrawal, retention, and privacy

Teachers are responsible for the code-to-identity mapping and authorized participation. The session owner can export and permanently delete the full session in Research & impact by typing its exact name. This removes participants, access tokens, responses, attempts, practice, surveys and researcher grants from the active database. A school-approved operator can perform a scoped participant deletion for withdrawal, using the same associated-table order as session deletion, without retaining identity in the app. Hosting backups must follow the separately approved retention policy. Do not promise instant removal from provider backups.

No webcam, keylogging, AI cheating classification, private chat collection, public individual scores, subscriptions, or marketing tracking is used. User-facing data are React-escaped; CSV formula-like strings are neutralized. The system uses pseudonyms, scoped server authorization and expiring sessions, but operational approval is still needed before inviting minors.
# Research CSV data dictionary

UTF-8 with BOM, RFC-style quoted fields, comma delimiters, CRLF rows. One row per independent response; one blank-question row for each participant/stage with no submitted response. Missing numeric values are blank, not zero. All exports remain restricted to the owning teacher or explicitly authorized researcher.

| Field | Meaning |
|---|---|
| data_type | `synthetic` or `pilot`; never combine without deliberate filtering |
| session_id, session_name, class_name | Pseudonymous session UUID and teacher-entered labels |
| participant_code | Teacher-assigned code, unique within session; not a name |
| study_group | A comparison/general review; B targeted retrieval |
| stage | baseline, immediate, delayed |
| mode | supervised or unsupervised; selected setting, not AI-use proof |
| question_id, concept | Stable item identifier and conceptual objective category |
| answer | Submitted answer text; formula-like leading characters neutralized in export |
| correct | 0 or 1; blank if missing |
| confidence | Integer 0–100; blank if missing |
| brier | Per-item `(confidence/100 − correct)^2` |
| time_ms_client | Client-reported time on item; mutable and not supervision evidence |
| assessment_elapsed_ms | Server timestamp difference from start to submission |
| misconception | Pattern-matched or concept-level follow-up category; blank for correct |
| classification | none, matched-pattern, or concept-needs-review |
| score | Percent correct for the assessment; equal item weights |
| baseline, immediate, delayed | Participant stage scores repeated for analysis convenience |
| submitted_at | UTC ISO timestamp; blank if not submitted |
| retention_due | UTC ISO timestamp set from immediate submission plus configured delay |
| missing_assessment | 1 if no submitted attempt for this stage, otherwise 0 |
| dropout | Teacher-marked dropout flag; distinct from missingness |
| intervention_type | targeted retrieval (B) or general review (A) |
| intervention_completed | 1 after required timer and retrieval checks; otherwise 0 |
| practice_count | Number of first retrieval responses recorded |
| student_rating, student_feedback | Optional 1–5 usefulness rating and nonidentifying text |
| teacher_setup_minutes, teacher_interpret_minutes | Self-reported nonnegative integer minutes |
| teacher_rating, teacher_feedback | Optional 1–5 usefulness and teacher comments |

Practice attempts are persisted in the `practice` table with item ID, first answer, correctness, confidence and timestamp. The research CSV currently includes the practice count and completion indicator rather than a separate practice-row export. Auth hashes, participant keys, password derivations and staff session cookies are never in research exports.

Avoid row-level pseudoreplication: each learner appears on multiple question rows. Deduplicate session/participant/stage before estimating participant-level outcomes. Use participant-level Brier means or prespecify question weighting. Pair baseline and delayed scores using session plus participant code, not participant code alone. The sample export was produced by automated synthetic tests and contains deliberately missing stages and formula-safe feedback.
# 60-second Diamond Challenge walkthrough

0–10s — Landing: “Great AI-assisted answers do not always show what a student can explain independently. LearnProof asks what students know now and what they retain a week later.”

10–25s — Explore the live demo. Point out **Synthetic demonstration**. On the teacher report, show stage averages, participation, and the concepts needing another look. “These are illustrative records, not impact evidence.”

25–40s — Open the empty demo session and choose **Try as student**. Start baseline and show an answer plus confidence slider. “We pair independently scored responses with confidence. Teachers control the assessment windows; home checks are explicitly unverified.” Complete the full flow before the presentation if you want a ready-to-view student report.

40–50s — Return to the teacher view. Show targeted retrieval and the seven-day reminder. “The intervention asks learners to retrieve before seeing an explanation, then checks retention with a different matched form.”

50–60s — Research & impact: comparison versus LearnProof, sample sizes, missingness, CSV. “Our pilot asks whether targeted retrieval improves retained understanding with comparable review time. We do not yet claim it does.”

Positioning: LearnProof is designed to measure independently retained learning across students' study tools. Short supervised checks, confidence calibration and delayed retention are hypotheses for differentiation from general AI tutoring, not validated competitive advantages. PACT remains a separate governance/disclosure project.

For a full working rehearsal: baseline → close baseline → guided study → open verification → submit → close verification → timed retrieval → make retention due now (synthetic only) → open retention → submit → close → inspect report and export. The live app does not bypass independent assessment gates merely to fit a 60-second pitch.