The Problem With "Smart" Recommenders in Education

Recommendation systems are everywhere. On the internet, social media, shopping platforms. Netflix knows what you want to watch next. Spotify curates your weekend playlist. Amazon predicts your next purchase before you've think of it.

But for education, this is different.

In entertainment, a wrong recommendation may waste a few minutes. But for learning, a wrong recommendation could waste weeks --- or worse, permanently frustrate a student who was never ready for what they were given.

The core failure mode of standard recommenders in education is this: they optimize for engagement or predicted performance, but they don't know that you can't learn calculus before you understand algebra.

Collaborative Filtering (CF) and Matrix Factorization --- the workhorses of modern recommendation --- learn patterns from what students have done before. But they have no concept of prerequisite structure. They don't know that polynomial equations come before differentiation. They don't know that loops must precede recursion.

I wanted to know how bad this problem actually was. So I tested it.

When I ran standard CF and Matrix Factorization baselines against a real curriculum graph, over 81% of their recommendations violated at least one prerequisite relationship. More than 4 in 5 suggestions were, in a meaningful educational sense, wrong --- not because the model predicted poor performance, but because the model had no idea how knowledge is structured.

To close that gap, I built the Logic Engine.

What Is the Logic Engine?

The Logic Engine is a domain-agnostic, constraint-aware personalized learning recommendation framework. It combines three components:

1. A Deep Knowledge Tracing model (SAKT) --- to estimate a student's current mastery state across all knowledge concepts

2. A prerequisite knowledge graph (DAG) --- encoding the structural dependencies between concepts in a given curriculum

3. A multi-objective ranking function --- that scores candidate recommendations by balancing predicted performance, knowledge gain, and prerequisite compliance

The system is designed to be domain-agnostic: the same architecture works for any curriculum, as long as you can define the knowledge concepts and their dependencies. I built and tested it on two domains drawn from the Nigerian NERDC secondary school curriculum: Secondary School Mathematics (JSS3--SS2) and CS Fundamentals.

GitHub: clementina-tom/PLRS

HuggingFace: Clementio/PLRS

Live Demo: Streamlit App

System Architecture: End-to-End

Figure 1: The Logic Engine's five-layer pipeline. Each layer is independently swappable. Diagram by the clementina-tom.

The pipeline has five distinct layers. Each layer has one job, and each can be swapped or upgraded independently:

Let me walk through each layer --- with the real code that drives it.

Component 1: Data Pipeline

Dataset: Open University Learning Analytics Dataset (OULAD)

OULAD contains anonymized interaction logs from ~32,000 students across multiple OU modules. It records student activity types, dates, click counts, and assessment results --- a rich source of sequential learning interaction data.

The critical preprocessing step is skill encoding: mapping raw OULAD activity labels to curriculum knowledge concepts, so the model operates on educationally meaningful units rather than arbitrary content IDs.

import pandas as pd
import numpy as np

def build_interaction_sequences(student_logs: pd.DataFrame,
                                skill_encoder: pd.DataFrame,
                                max_seq_len: int = 100) -> dict:
    """
    Transform raw OULAD student logs into SAKT-ready interaction sequences.
    
    Args:
        student_logs: DataFrame with columns [student_id, activity_type, date, score]
        skill_encoder: Mapping from activity_type -> concept_id
        max_seq_len: Maximum sequence length (pad/truncate to this)
    
    Returns:
        Dictionary of {student_id: {"concepts": [...], "responses": [...],
                                    "timestamps": [...]}}
    """
    # Merge activity logs with skill encoder
    merged = student_logs.merge(skill_encoder, on="activity_type", how="inner")
    merged = merged.sort_values(["student_id", "date"])
    
    sequences = {}
    
    for student_id, group in merged.groupby("student_id"):
        concepts = group["concept_id"].tolist()
        # Binarize: score >= 50 -> correct (1), else incorrect (0)
        responses = (group["score"] >= 50).astype(int).tolist()
        timestamps = group["date"].tolist()
        
        # Truncate to max_seq_len from the right (most recent interactions)
        if len(concepts) > max_seq_len:
            concepts = concepts[-max_seq_len:]
            responses = responses[-max_seq_len:]
            timestamps = timestamps[-max_seq_len:]
        
        sequences[student_id] = {
            "concepts": concepts,
            "responses": responses,
            "timestamps": timestamps,
            "seq_len": len(concepts)
        }
    
    return sequences


def initialize_cold_start_mastery(n_concepts: int) -> np.ndarray:
    """
    For students with no interaction history, initialize mastery to 0.
    Cold-start students will be recommended foundational concepts first.
    """
    return np.zeros(n_concepts, dtype=np.float32)

The skill encoding step is domain-specific --- this is where you map your content to your knowledge graph. The rest of the pipeline is domain-agnostic.

Component 2: SAKT --- The Knowledge Tracing Model

Self-Attentive Knowledge Tracing (SAKT) is a transformer-based model designed to track a student's knowledge state over time as they interact with learning content.

Unlike traditional Bayesian Knowledge Tracing (BKT), which models one skill at a time, SAKT uses a self-attention mechanism to capture cross-concept dependencies --- recognizing, for example, that correctly answering a question on quadratic equations is evidence of algebraic competency more broadly.

import torch
import torch.nn as nn
import torch.nn.functional as F

class SAKTModel(nn.Module):
    """
    Self-Attentive Knowledge Tracing model.
    Based on: "A Self-Attentive model for Knowledge Tracing" (Pandey & Karypis, 2019)
    
    The key idea: use self-attention over a student's interaction history
    to predict the probability of correctly answering each concept next.
    This probability distribution becomes the student's mastery vector.
    """
    
    def __init__(self, n_concepts: int, embed_dim: int = 64,
                 n_heads: int = 4, dropout: float = 0.2, max_seq: int = 100):
        super().__init__()
        self.n_concepts = n_concepts
        self.embed_dim = embed_dim
        
        # Embed (concept, response) pairs into a shared space
        # We use 2*n_concepts because each concept has a correct/incorrect embedding
        self.interaction_embed = nn.Embedding(2 * n_concepts + 1, embed_dim,
                                               padding_idx=0)
        self.concept_embed = nn.Embedding(n_concepts + 1, embed_dim,
                                           padding_idx=0)
        self.pos_embed = nn.Embedding(max_seq, embed_dim)
        
        # Multi-head self-attention: the core of the model
        self.attention = nn.MultiheadAttention(embed_dim, n_heads,
                                                dropout=dropout, batch_first=True)
        self.layer_norm = nn.LayerNorm(embed_dim)
        self.dropout = nn.Dropout(dropout)
        
        # Output head: predict P(correct | concept, history)
        self.output_head = nn.Sequential(
            nn.Linear(embed_dim, embed_dim // 2),
            nn.ReLU(),
            nn.Dropout(dropout),
            nn.Linear(embed_dim // 2, 1),
            nn.Sigmoid()
        )
    
    def forward(self, concepts: torch.Tensor,
                responses: torch.Tensor) -> torch.Tensor:
        """
        Args:
            concepts: (batch, seq_len) --- concept IDs in interaction order
            responses: (batch, seq_len) --- 0/1 correctness for each interaction
        
        Returns:
            mastery_vector: (batch, n_concepts) --- P(correct) for each concept
        """
        batch_size, seq_len = concepts.shape
        device = concepts.device
        
        # Encode past interactions as (concept + response) embeddings
        interaction_ids = concepts + responses * self.n_concepts  # shift for response
        interaction_ids = interaction_ids.clamp(0, 2 * self.n_concepts)
        
        x = self.interaction_embed(interaction_ids)
        
        # Add positional encoding
        positions = torch.arange(seq_len, device=device).unsqueeze(0).expand(batch_size, -1)
        x = x + self.pos_embed(positions)
        x = self.dropout(x)
        
        # Causal mask: student can only attend to past interactions
        causal_mask = torch.triu(torch.ones(seq_len, seq_len, device=device),
                                  diagonal=1).bool()
        
        # Self-attention over interaction history
        attn_out, _ = self.attention(x, x, x, attn_mask=causal_mask)
        x = self.layer_norm(x + attn_out)
        
        # Predict mastery for ALL concepts using the final hidden state
        final_state = x[:, -1, :]  # (batch, embed_dim) --- most recent state
        
        all_concepts = torch.arange(1, self.n_concepts + 1, device=device)
        concept_embeds = self.concept_embed(all_concepts)  # (n_concepts, embed_dim)
        
        # Score each concept against the student's current state
        final_expanded = final_state.unsqueeze(1).expand(-1, self.n_concepts, -1)
        combined = final_expanded * concept_embeds.unsqueeze(0)
        
        mastery_vector = self.output_head(combined).squeeze(-1)  # (batch, n_concepts)
        return mastery_vector


def get_student_mastery(model: SAKTModel,
                        student_sequence: dict,
                        device: str = "cpu") -> np.ndarray:
    """
    Infer a student's current mastery vector from their interaction history.
    Returns an array of shape (n_concepts,) with values in [0, 1].
    """
    model.eval()
    with torch.no_grad():
        concepts = torch.tensor([student_sequence["concepts"]],
                                 dtype=torch.long).to(device)
        responses = torch.tensor([student_sequence["responses"]],
                                  dtype=torch.long).to(device)
        mastery = model(concepts, responses)
        return mastery.squeeze(0).cpu().numpy()

Training results:

• Validation AUC: 0.7692 (26 epochs with early stopping on val AUC)

• Model hosted on HuggingFace: Clementio/PLRS

The mastery vector output --- a probability distribution over all knowledge concepts --- is the snapshot of what the student knows right now. This feeds directly into the ranking function.

Component 3: The Prerequisite Knowledge Graph

This is the most educationally interesting component, and the one that most clearly differentiates the Logic Engine from standard approaches.

A Directed Acyclic Graph (DAG) encodes prerequisite relationships between knowledge concepts. An edge from concept A -> concept B means: a student must have sufficient mastery of A before being recommended B.

import networkx as nx
from typing import List, Set

def build_curriculum_dag(edges: List[tuple]) -> nx.DiGraph:
    """
    Construct a prerequisite knowledge graph from a list of (prerequisite, concept) edges.
    
    Example edge: ("basic_algebra", "linear_equations") means
    "a student must understand basic algebra before linear equations."
    """
    G = nx.DiGraph()
    G.add_edges_from(edges)
    
    # Validate: a curriculum graph must be acyclic
    if not nx.is_directed_acyclic_graph(G):
        cycles = list(nx.find_cycle(G))
        raise ValueError(f"Curriculum graph contains cycles: {cycles}. "
                         f"Check your prerequisite definitions.")
    return G


# Secondary School Mathematics knowledge map (NERDC JSS3--SS2)
MATHS_EDGES = [
    ("number_theory", "fractions"),
    ("fractions", "ratio_proportion"),
    ("ratio_proportion", "percentages"),
    ("percentages", "financial_maths"),
    ("number_theory", "basic_algebra"),
    ("basic_algebra", "linear_equations"),
    ("linear_equations", "simultaneous_equations"),
    ("basic_algebra", "indices_logarithms"),
    ("linear_equations", "quadratic_equations"),
    ("quadratic_equations", "polynomials"),
    ("polynomials", "differentiation"),
    ("differentiation", "integration"),
    ("basic_algebra", "sequences_series"),
    ("fractions", "statistics_basics"),
    ("statistics_basics", "probability"),
    # ... 45 edges total
]

# CS Fundamentals knowledge map (NERDC)
CS_EDGES = [
    ("variables", "data_types"),
    ("data_types", "operators"),
    ("operators", "control_flow"),
    ("control_flow", "loops"),
    ("loops", "functions"),
    ("functions", "recursion"),
    ("data_types", "arrays"),
    ("arrays", "sorting_algorithms"),
    ("functions", "oop_basics"),
    ("oop_basics", "inheritance"),
    # ... 39 edges total
]

maths_dag = build_curriculum_dag(MATHS_EDGES)
cs_dag = build_curriculum_dag(CS_EDGES)


def get_prerequisites(dag: nx.DiGraph, concept: str) -> Set[str]:
    """
    Return all concepts that must be mastered before `concept`.
    Uses ancestor traversal: returns the full transitive prerequisite closure.
    """
    return nx.ancestors(dag, concept)


def get_next_learnable(dag: nx.DiGraph,
                       mastered: Set[str],
                       threshold: float = 0.7,
                       mastery_vector: dict = None) -> List[str]:
    """
    Return concepts whose full prerequisite chain is satisfied.
    A concept is learnable if all its prerequisites are in the mastered set
    (i.e., mastery score >= threshold).
    """
    learnable = []
    for concept in dag.nodes():
        if concept in mastered:
            continue  # Already mastered
        prereqs = get_prerequisites(dag, concept)
        if all(mastery_vector.get(p, 0.0) >= threshold for p in prereqs):
            learnable.append(concept)
    return learnable

The DAG construction is the most domain-specific part of the system. In production, this could be sourced from a curriculum authority, generated semi-automatically from textbook structure, or learned from expert annotation.

Component 4: Multi-Objective Ranking Function

Given a student's mastery vector and a set of learnable candidates, the ranking function scores each concept on three dimensions and surfaces the best next steps:

import numpy as np
from dataclasses import dataclass
from typing import Dict, List

@dataclass
class RankingWeights:
    """
    Tunable weights for the multi-objective scoring function.
    Different pedagogical contexts call for different priorities.
    
    Examples:
    - Remediation mode: upweight knowledge_gain (fill gaps first)
    - Exam prep mode: upweight predicted_performance (reinforce strengths)
    - Exploration mode: increase novelty_bonus (broaden exposure)
    """
    predicted_performance: float = 0.40   # alpha
    knowledge_gain: float = 0.40          # beta
    prerequisite_satisfaction: float = 0.20  # gamma


def score_candidate(concept: str,
                    mastery_vector: Dict[str, float],
                    dag: nx.DiGraph,
                    weights: RankingWeights,
                    mastery_threshold: float = 0.7) -> float:
    """
    Compute a multi-objective score for a candidate concept.
    
    Score = alpha * predicted_performance
          + beta * knowledge_gain
          + gamma * prerequisite_satisfaction
    
    Args:
        concept: Concept ID to score
        mastery_vector: {concept_id: mastery_probability} from SAKT
        dag: Prerequisite knowledge graph
        weights: Tunable objective weights
        mastery_threshold: Minimum mastery to consider a prerequisite satisfied
    
    Returns:
        Scalar score in [0, 1] --- higher is a better recommendation
    """
    # alpha --- Predicted performance: P(student succeeds) from SAKT
    predicted_performance = mastery_vector.get(concept, 0.0)
    
    # beta --- Knowledge gain: how much is this concept at the frontier of mastery?
    # Concepts just beyond current mastery score highest (zone of proximal development)
    current_mastery = mastery_vector.get(concept, 0.0)
    knowledge_gain = 1.0 - current_mastery  # High gain if low mastery
    
    # gamma --- Prerequisite satisfaction: are all prerequisites sufficiently mastered?
    prereqs = get_prerequisites(dag, concept)
    if not prereqs:
        prereq_score = 1.0  # No prerequisites -> fully eligible
    else:
        prereq_scores = [mastery_vector.get(p, 0.0) for p in prereqs]
        prereq_score = np.mean([s / mastery_threshold for s in prereq_scores])
        prereq_score = min(prereq_score, 1.0)  # Cap at 1.0
    
    score = (weights.predicted_performance * predicted_performance
             + weights.knowledge_gain * knowledge_gain
             + weights.prerequisite_satisfaction * prereq_score)
    
    return float(score)


def rank_recommendations(mastery_vector: Dict[str, float],
                         dag: nx.DiGraph,
                         top_k: int = 5,
                         weights: RankingWeights = None,
                         mastery_threshold: float = 0.7) -> List[dict]:
    """
    Full recommendation pipeline: filter -> score -> rank.
    
    1. Filter: remove already-mastered concepts and prerequisite violations
    2. Score: apply multi-objective scoring to each candidate
    3. Rank: return top-k by score
    
    Returns:
        List of dicts: [{"concept": ..., "score": ..., "reason": ...}, ...]
    """
    if weights is None:
        weights = RankingWeights()
    
    mastered = {c for c, m in mastery_vector.items() if m >= mastery_threshold}
    
    # Step 1: Constraint filtering --- only learnable concepts pass through
    candidates = get_next_learnable(dag, mastered,
                                    threshold=mastery_threshold,
                                    mastery_vector=mastery_vector)
    
    if not candidates:
        return []  # Student has mastered everything in the graph
    
    # Step 2: Score each candidate
    scored = []
    for concept in candidates:
        score = score_candidate(concept, mastery_vector, dag, weights,
                                mastery_threshold)
        scored.append({"concept": concept, "score": round(score, 4)})
    
    # Step 3: Rank and return top-k
    ranked = sorted(scored, key=lambda x: x["score"], reverse=True)
    return ranked[:top_k]

The constraint layer is where the 0.0% violation rate comes from: no concept passes through unless its full prerequisite chain is satisfied. This is a hard filter, not a soft penalty --- the ranking function never even sees a violating concept.

Results: The Number That Matters

This is the headline result. Standard approaches recommend concepts in educationally invalid sequences more than 80% of the time. The Logic Engine eliminates prerequisite violations entirely --- while maintaining competitive knowledge tracing accuracy.

Beyond the numbers, the system produces pedagogically sensible recommendation sequences --- always moving from foundational to advanced concepts, respecting the actual structure of the curriculum.

Live Demo

The Logic Engine is deployed as a Streamlit application with two modes:

Assessment Mode --- Input real student interaction data. The SAKT model computes a mastery vector, the constraint layer filters candidates, and the ranking function returns a prioritized recommendation list.

Simulate Mode --- Explore recommendations for a hypothetical student profile. Select a knowledge domain and see how the system responds as you adjust the student's mastery state.

Try the live demo here: Streamlit App

What I'd Build Next

Here's my honest engineering backlog:

1. Soft prerequisite constraints: The current system treats prerequisites as hard binary gates. In reality, learning is fuzzy --- a student with 60% mastery of algebra can still benefit from early exposure to quadratics. A soft constraint system would apply graduated penalties rather than hard cutoffs, enabling more nuanced recommendations.

2. SHAP-style attention visualization: The SAKT model uses attention weights internally to relate past interactions to current predictions. Surfacing these as explainability outputs --- "this recommendation is based heavily on your performance in Algebra last week" --- would make the system far more interpretable to both students and educators.

3. What-If prerequisite simulator: An interactive tool that lets a student (or teacher) ask: "What would I need to master before being ready for Calculus?" The prerequisite DAG already encodes this --- it just needs a traversal interface.

4. AUC improvement: Val AUC of 0.7692 is solid but not the ceiling. Better sequence length handling, hyperparameter tuning, and curriculum-aligned data augmentation are all on the roadmap.

Closing Thoughts

The Logic Engine is an argument that educational AI needs domain knowledge, not just data.

A model that optimizes purely for predicted performance will recommend whatever the student is most likely to score well on --- not what they most need to learn next. A model that knows the structure of a curriculum can do something more powerful: chart a coherent path through knowledge.

The 0.0% prerequisite violation metric of our logic engine is the difference between a system that recommends, and a system that teaches.

The full project is open-source. Code, model weights, and datasets are available below:

GitHub: clementina-tom/PLRS

HuggingFace: Clementio/PLRS

Live Demo: Streamlit App

Clementina | ML Engineer --- RAG pipelines, MLOps, and predictive modeling | GitHub