| name | upgrade-integration |
| description | Integrate Carnegie Learning's UpGrade A/B testing platform into LMS and EdTech applications. Guides setup of decision points, experiment conditions, LTI/xAPI integration, and outcome logging. Use when asked to add A/B testing, experiments, or feature flags to educational software. |
| allowed-tools | Read, Write, Edit, Bash, WebFetch, Glob, Grep |
UpGrade A/B Testing Integration for EdTech
You are helping integrate UpGrade, Carnegie Learning's open-source A/B testing platform designed specifically for educational technology.
When This Skill Applies
- Adding A/B testing or experiments to educational apps
- Integrating feature flags in learning platforms
- Connecting EdTech apps to LMS via LTI with experiment support
- Setting up learning analytics with xAPI + experimentation
- Debugging experiment assignment issues
Critical Context
UpGrade is NOT a generic A/B testing tool. It's designed for education with features like:
- Group-level assignment (whole classrooms get same condition)
- Consistency rules (students don't get re-randomized)
- Educational context metadata (grade, subject, proficiency)
ALWAYS ask these questions before implementing:
- What's the assignment unit? (Individual student vs classroom vs school)
- What happens if a student changes classes mid-experiment?
- How will teachers access both conditions (or will they)?
- Is the app used in LMS iframes? (Cookie issues!)
Quick Reference
Installation
npm install upgrade_client_lib
<dependency>
<groupId>com.carnegielearning</groupId>
<artifactId>upgrade-client</artifactId>
</dependency>
Core Integration Pattern
import UpgradeClient from 'upgrade_client_lib/dist/browser';
import { MARKED_DECISION_POINT_STATUS } from 'upgrade_client_lib';
const client = new UpgradeClient(userId, hostUrl, context);
await client.init(groupData, workingGroupData);
const assignment = await client.getDecisionPointAssignment('feature_name', 'target');
const condition = assignment.getCondition();
const payload = assignment.getPayload();
client.markDecisionPoint('feature_name', MARKED_DECISION_POINT_STATUS.CONDITION_APPLIED);
client.log('score', 85);
client.log('completion_time_ms', 45000);
Three SDK Modes
| Mode | Use Case | Requires init()? |
|---|
| Standard | Pre-registered users with stored groups | Yes |
| Ephemeral | Runtime-provided groups, no DB lookup | No |
| Merged | Combine session + stored groups | Yes |
Integration Architecture
┌─────────────────────────────────────────────────────────────┐
│ LMS │
│ (Canvas, Moodle, etc.) │
└─────────────────────┬───────────────────────────────────────┘
│ LTI 1.3 Launch
│ (user_id, context_id, roles)
▼
┌─────────────────────────────────────────────────────────────┐
│ Your EdTech App │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Identity Mapper │ │
│ │ LTI user_id → UpGrade userId + altUserIds │ │
│ └─────────────────────┬───────────────────────────────┘ │
│ │ │
│ ┌─────────────────────▼───────────────────────────────┐ │
│ │ UpGrade SDK │ │
│ │ - init(groupData) │ │
│ │ - getDecisionPointAssignment() │ │
│ │ - markDecisionPoint() │ │
│ │ - log() │ │
│ └─────────────────────┬───────────────────────────────┘ │
└─────────────────────────┼───────────────────────────────────┘
│ HTTPS
▼
┌─────────────────────────────────────────────────────────────┐
│ UpGrade Backend │
│ (Self-hosted or Carnegie Learning) │
└─────────────────────────────────────────────────────────────┘
CRITICAL: Known Pitfalls
Before writing ANY integration code, read resources/pitfalls-and-edge-cases.md. The most dangerous issues:
1. Third-Party Cookie Blocking (CRITICAL)
If your app runs in an LMS iframe, Safari and Chrome WILL block your session cookies by default.
Symptoms:
- User gets re-randomized on every page load
init() seems to work but assignments are inconsistent
- Works in development, fails in production LMS
Solutions:
- Set
SameSite=None; Secure on all cookies
- Use localStorage with user-provided identifier (not session-based)
- Consider opening in new window instead of iframe
2. Race Condition: init() vs getDecisionPointAssignment()
const client = new UpgradeClient(userId, hostUrl, context);
client.init(groupData);
const assignment = await client.getDecisionPointAssignment('feature');
const client = new UpgradeClient(userId, hostUrl, context);
await client.init(groupData);
const assignment = await client.getDecisionPointAssignment('feature');
3. LTI Identity Mismatch
LTI provides user_id which is opaque and LMS-specific. Your app may use email or internal IDs.
await client.init(groupData);
client.setAltUserIds([
ltiUserId,
userEmail,
internalDatabaseId
]);
4. Group Assignment Drift
Student transfers from Mrs. Smith's class (Treatment A) to Mr. Jones' class (Treatment B).
Default behavior: Student KEEPS original assignment (consistency rule)
Problem: Student now sees different UI than classmates, causing confusion.
Solution: Decide upfront:
- Accept drift (experimental purity)
- Re-assign on class change (classroom consistency)
- Exclude transferred students from analysis
5. Shared Device / Chromebook Cart Problem
const client = new UpgradeClient(getUserIdFromLocalStorage(), ...);
const client = new UpgradeClient(getCurrentAuthenticatedUserId(), ...);
Decision Point Patterns for Education
See resources/decision-point-patterns.md for detailed examples:
| Pattern | Example | Assignment Unit |
|---|
| UI Variant | New vs old problem interface | Individual |
| Pedagogical | Worked examples vs practice problems | Classroom |
| Adaptive Algorithm | Mastery threshold 80% vs 90% | Individual |
| Content Ordering | Teach fractions before decimals vs after | School |
| Feedback Timing | Immediate vs delayed feedback | Classroom |
Code Templates
See templates/ directory for:
react-hook.tsx - React hook with proper error handling
nextjs-middleware.ts - Next.js API route pattern
vanilla-js.js - Framework-agnostic implementation
lti-launch-handler.ts - LTI 1.3 launch with UpGrade init
Logging Best Practices
client.log('score', 85);
client.log('completed', true);
client.log('response', 'correct');
client.log('score', 85);
client.log('score', '85');
client.log('score', 85);
client.markDecisionPoint('quiz', MARKED_DECISION_POINT_STATUS.CONDITION_APPLIED);
client.markDecisionPoint('quiz', MARKED_DECISION_POINT_STATUS.CONDITION_APPLIED);
client.log('score', 85);
Troubleshooting Decision Tree
Assignment returns NO_CONDITION_ASSIGNED
├── Did you await init()?
│ └── No → Add await before init()
├── Is experiment in ENROLLING state?
│ └── No → Check UpGrade UI, start experiment
├── Does user match segment criteria?
│ └── No → Check inclusion/exclusion rules
├── Is context correct?
│ └── No → Verify context string matches experiment config
└── Check network tab for failed requests
User gets different condition than classmates
├── Is assignment unit set to GROUP?
│ └── No → Change to GROUP in experiment settings
├── Was groupData provided to init()?
│ └── No → Pass { classId: 'xxx' } to init()
├── Did student join after experiment started?
│ └── Yes → Check POST_EXPERIMENT_RULE
└── Is student in multiple groups?
└── Yes → Check group priority/override logic
External Resources
When to Escalate
Recommend the developer contact UpGrade support or file a GitHub issue if:
- Assignment algorithm seems incorrect (statistical anomaly)
- Data export shows impossible values
- Backend returns 5xx errors consistently
- Need custom consistency rules not supported by SDK