When a district approves a classroom app, what has actually been checked? In most districts the answer is a privacy evaluation — a structured review of the vendor's public policy, its data practices, and its legal agreements, conducted either in-house or through frameworks and services that publish ratings. These evaluations are genuinely useful: they catch apps that sell student data, lack encryption, or hide troubling clauses on page nine. But they are document reviews, not audits, and the gap between what an evaluation verifies and what a district imagines it verifies is where most privacy incidents actually live. Understanding both halves — the method and its ceiling — is what turns a checklist into a program.
What does a privacy evaluation actually examine?
A competent evaluation covers four layers. Policy: what the vendor's privacy policy claims to collect, share, and retain, and whether the language is vague enough to permit more. Disclosure and rights: whether the product discloses third-party trackers and honors parental rights under state student-privacy laws. Security posture: encryption in transit and at rest, single sign-on support, and breach-notification commitments. Contract: whether the district's data privacy agreement overrides policy language where they conflict, because the policy is the vendor's document and the contract is the district's. Frameworks like Common Sense Privacy Program publish evaluations along these lines, and several states operate their own approval processes — Illinois, under SOPPA, maintains a registry of vendors that have met state requirements, effectively delegating part of the review.
What can an evaluation tell you, reliably?
Evaluations reliably surface the paper problems, and paper problems are common. They flag products whose policies permit targeted advertising to students, disclose data to unnamed partners, or set no retention limit. They compare what the vendor says against what the law allows, which matters because a policy can be fully legal and still unacceptable to a district. They also create institutional memory: a district with a documented evaluation for every approved app knows, when a vendor changes its policy, exactly what changed and when — a fact that becomes evidence in a breach dispute. This is unglamorous work, and it is the part of privacy programs that demonstrably reduces risk.
Where do evaluations stop working?
The ceiling arrives quickly. An evaluation reads documents; it cannot observe behavior. If a vendor's app sends student data to an analytics intermediary the policy never mentions, no policy review catches it — that is the territory of network traffic analysis, which some districts run at the firewall and most do not. Evaluations also age: a clean rating reflects the policy as written on the review date, vendors update policies without notice, and product updates can change data flows underneath an unchanged policy. And evaluators differ: the same app can pass one framework and fail another, because the rubrics weigh advertising, encryption, and contract terms differently. A district that treats a published rating as a permanent verdict rather than a dated snapshot has bought confidence, not protection.
How should a district use evaluations well?
Three practices close most of the practical gap. First, date everything: record the evaluation date and re-review at renewal, not just at adoption — renewals are when vendors change terms, because they have leverage. Second, pair the policy review with a traffic check for high-use apps: watching what the app actually transmits, even once, catches the class of problems documents cannot. Third, put the contract above the policy: a district data privacy agreement that bars selling data, limits retention to the school relationship, and requires breach notice within a defined number of days converts vendor prose into enforceable terms. Districts that adopted state template agreements — several states publish them — report that vendors accept standardized terms far more readily than bespoke ones, which makes the clause stick in practice.
What about teacher-downloaded apps?
The unsolved problem is the long tail. Districts formally evaluate dozens of apps, while students encounter hundreds — signed up by individual teachers with free accounts, never passing through any review. Free teacher accounts are the dominant channel by which unreviewed products reach students, and no evaluation program can review its way around that. The districts making progress treat it as an access problem rather than a compliance problem: fast approval lanes so teachers are not tempted to bypass, an LMS integration that makes approved apps easier to use than unapproved ones, and periodic firewall reports that show which unreviewed domains are receiving student traffic. The honest summary of the whole field is that evaluations are a filter, not a wall. They remove the worst paperwork offenders cheaply and consistently, and then the district's real privacy posture is set by contracts, traffic monitoring, and how easy it is to do the right thing.
A final word on scale, because it decides program design. A large district can field a dedicated privacy officer; a small one cannot, and expecting every district to maintain current evaluations of hundreds of apps is a design flaw, not a staffing failure. The workable models share the load: state registries that let districts inherit vetting, regional service agencies that run evaluations for members, and consortium agreements negotiated once for many districts. Privacy practice in schools is consolidating toward exactly this structure, and a technology director's honest question in a small district is not how to build a review office but which collective to join. The evaluation itself was never the hard part; doing it alone was.
For more context, read State student-privacy laws are changing what apps may collect.
For more context, read What FERPA actually lets edtech vendors do with student data.
For more context, read How states are writing AI disclosure rules for classrooms.