OBE & Curriculum Mapping — Project Notes

Programme: BE Chemical Engineering · NUST SCME
Last updated: 2026-07-22

Site:

• https://haiderejaz6.github.io/ChE_Department/
• OBE_Dashboard_architecture — site structure reference, read before touching index.html/css/js
• related work and ontology — full literature comparison table + research-gap analysis
• Curriculum Analysis Literature Review — running index of all papers read

Overview

Two related but distinct tools are under active development at the intersection of chemical engineering education and educational technology. Both serve the SCME ChE programme but have different primary goals — one is a departmental dashboard for curriculum administration, the other is a research instrument for studying how CLOs are constructed and mapped to knowledge.

I recently joined the department curriculum review committee, which gives both tools a direct institutional use case beyond the research angle — particularly overlap detection, PLO coverage analysis, and Bloom's progression visualisation for committee review sessions.

The literature review (18 papers read, see below) confirmed a genuine gap: no existing work combines (1) formal OBE/accreditation structure, (2) semantic topic-level extraction validated via Cohen's κ, (3) a specific engineering discipline, and (4) programme-level analytic findings (overlap, Bloom's progression) as the output. This is now the explicit positioning for the paper.

Tool 1 — OBE Dashboard

What it does

An interactive curriculum mapping dashboard that reads from Qalam_CLOs.xlsx and renders visualisations across all courses and semesters of the BE ChE programme. Deployed as a static site on GitHub Pages (haiderejaz6/ChE_Department) — single-page app, no build step, no bundler.

Site structure (post-split)

The single index.html file was split into organised css/ and js/ source files for maintainability — this is still one single-page app, not multiple routes; tabs toggle visibility via switchTab(), they don't navigate. Full rationale and file-by-file breakdown in OBE_Dashboard_architecture. Key points to remember:

• Load order: data.js → shared.js → dashboard.js → graph.js → ontology.js, then init()
• responsive.css must load last (single @media(max-width:720px) block touching all other stylesheets)
• data.json and courses/ paths are unchanged, relative to index.html
• Course outline .md files (57 of them) live in courses/, referenced by path pointer from data.json rather than embedded

Tabs / Views

Tech Stack

• Frontend: index.html (shell only) · css/ (base, dashboard, graph, ontology, responsive) · js/ (data, shared, dashboard, graph, ontology) · Chart.js · D3.js — no bundler, plain <script>/<link> tags
• Data pipeline: excel_to_json.py (Python) → data.json
  • Mode A: Rule-based extraction (~140-pattern regex pass)
  • Mode B: Ollama (local LLM) extraction
  • Merge strategy + deterministic post-normalisation dictionary as final pass
• Data source: Qalam_CLOs.xlsx (Qalam system export)
• Course outlines: 57 markdown files, cleaned from OCR/PDF state, consistent template (metadata header, weekly plan table, references)

Completed Work

• Full dashboard, split into OBE Dashboard / Curriculum Graph / ChE Ontology tabs
• Mobile optimisation — hamburger nav, responsive layouts, touch-friendly interactions
• Terminology pass: display labels updated to "Knowledge Area / Knowledge Unit" (see Key Terminology below) — underlying JSON schema keys deliberately left unchanged
• Python conversion script supporting rule-based + LLM-based (Ollama) extraction modes
• Chart.js and D3.js visualisations wired to JSON output
• Course explorer + PLO-first Coverage Matrix redesign (PLO rows, course pills)
• Bloom's Distribution split into Cognitive/Affective/Psychomotor domain charts
• Curriculum Graph prerequisite arrow highlighting
• Knowledge Area Overlap: persistent D3 force-directed zoomable topic space, physics clustering, organic blob hulls, dropdown course selector with checkboxes (capped at ~6–10 active blobs at a time to stay legible with 51 courses), animated topic pill sizing
• Codebase split into css/+js/ modules (see architecture doc), one dead shadowed showTip definition removed
• 57 course outline files cleaned and structured, referenced from data.json

Known Data Quality Issues (unresolved)

• PHY-102 has a duplicate CLO array
• Filename/course-code mismatches: HI-114, CHE-555/CHE-455 mapping, MGT-442/422
• CHE-103 and CHE-222 near-duplicate content
• MSE-226 reference list anomaly

Open Items

• Fix the data quality issues above before next pipeline run
• Add prerequisites column to source Excel (currently missing — PEC 2023 Obsidian vault may be a faster source than re-gathering from Qalam)
• Validate Ollama extraction quality against rule-based baseline
• Consider exposing an extraction-mode toggle in the UI

Tool 2 — CLO Extraction & Refinement Tool

What it does

A research-oriented web tool for extracting Knowledge Units (topics) from CLO descriptions, designed for teacher participation. Inspired by the UCSD curriculum graph project (2024 BLopez ASEE / 2025 BLopez CompChemEngg). Dual purpose: practical CLO refinement aid and research data collection instrument — teacher votes on extracted topics are the actual research data.

Functional Phases

Phase 1  →  CLO + course outline input
Phase 2  →  Three parallel extraction methods
              ├─ LLM (Anthropic API + local Ollama pass)
              ├─ Rule-based (~140-pattern regex / 200-term ChE taxonomy, 10 domains)
              └─ TF-IDF statistical  — documented, NOT YET IMPLEMENTED (see Open Items)
Phase 3  →  Teacher voting on extracted Knowledge Units
Phase 4  →  Literature augmentation
Phase 5  →  Bloom's-aligned CLO revision output

Extraction pipeline architecture (current)

Hybrid pipeline, now with course outlines as a second input alongside CLOs:

1. Rule-based regex pass (~140 patterns)
2. Optional LLM pass via local Ollama instance — prompt caps course-outline content at 4,000 characters
3. Merge strategy combining both passes
4. CLO-level attribution (which CLO each extracted unit maps back to)
5. Global LLM normalisation pass
6. Deterministic post-normalisation dictionary (~120 entries) — final pass regardless of which mode produced the term, catches nonsense outputs (e.g. early runs produced roi in Physics, stokes' theorem in a Quran course)

Known Pipeline Issues (unresolved)

• 2–4 word length filter on LLM-extracted concepts creates an asymmetry with rule-based extraction, which allows single-word concepts
• CLO-level attribution logic is overly loose
• Normalisation currently runs on chunked batches, not full-corpus — not yet implemented as full-corpus pass

Tech Stack

• Frontend: React (single-page prototype)
• AI backend: Anthropic Claude API + local Ollama instance (LLaMA 3 / Mistral / Phi-3)
• Visualisation: Draggable concept map (D3.js)
• Taxonomy: 200-term ChE domain taxonomy (10 domains)
• Research metric: Cohen's κ for inter-rater agreement (target benchmark κ = 0.75, per 2025 RKumar Arxiv and general human–human literature benchmark)

Completed Work

• All five functional phases implemented (Phase 2 currently rule + LLM only — see below)
• Rule-based + LLM extraction with merge, CLO-level attribution, global normalisation, and post-normalisation dictionary
• Course outline files integrated as additional extraction input
• Draggable concept/knowledge map
• Session saving
• 200-term ChE taxonomy across 10 domains embedded

Open Items / Design Decisions Pending

• Implement TF-IDF extraction mode — documented in architecture but not yet coded; only rule, LLM, and combined modes currently exist
• Build extract_topics_compare.py — runs the pipeline twice per course (CLOs-only vs. CLOs + outline), outputs topics_comparison.json with Jaccard similarity, topic-count deltas, added/lost topics, and network-edge change implications. This is the core ablation study for the paper.
• Loosen or replace the 2–4 word LLM concept filter with vocabulary-based scoring
• Fix CLO-level attribution logic
• Teacher voting interface
• Cohen's κ computation for multi-teacher comparison
• Literature citations — formalise sources for augmentation phase (AIChE concept inventories, ABET outcomes, standard ChE textbooks)
• Data persistence backend — needed before running the actual study (options: Supabase, Firebase, simple Flask/FastAPI endpoint)
• Usability questionnaire — decide whether to add SUS or TAM-2 as a dedicated Phase 6
• Prerequisite edges on concept map — manually defined vs. LLM-suggested; impacts research validity; PEC 2023 Obsidian vault (55 markdown files, already converted) may be the faster source than re-deriving from Qalam

Shared Research Context

Key Terminology

Reference Frameworks

• PEC 2023 Curriculum Guidelines — the programme's accreditation framework; full curriculum converted to 55 Obsidian-compatible markdown files
• Washington Accord — OBE standards context
• HEC framework
• Bloom's Taxonomy — CLO verb alignment and cognitive level classification
• ABET Student Outcomes — accreditation alignment reference point
• AIChE Concept Inventories — benchmark for ChE knowledge domain coverage
• Standard ChE textbooks — literature source baseline for augmentation
• UCSD Curriculum Graph — design inspiration for the extraction tool's interface
• ACM/IEEE-CS Knowledge Area / Knowledge Unit / Learning Outcome model — terminology backbone (see literature review)

Research Metrics Planned

Related Work / Literature Review

Full comparison table and citation-ready gap analysis live in related work and ontology. 18 papers read to date, indexed in Curriculum Analysis Literature Review. Summary below for quick reference when drafting the paper.

Where the gap sits

No existing work combines all four of:

1. Formal OBE/accreditation structure (CLO → PLO, Bloom's, Washington Accord / PEC / ABET)
2. Semantic, topic-level extraction validated against human raters (κ)
3. A specific engineering discipline (not generic, not CS-only)
4. Programme-level analytic findings (overlap/redundancy, Bloom's progression) as the output — not personalisation/recommendation, not matrix arithmetic

Sharpest unclaimed angles for the paper:

• Extraction input ablation — CLOs-only vs. CLOs + full course outlines (Jaccard similarity, topic-count deltas). Nobody has studied this explicitly.
• Human validation of automated topic extraction — existing curriculum-KG papers validate graph structure (modularity, centrality), not agreement with human raters; existing CLO-PLO papers don't extract topics at all. The κ = 0.75 benchmark against teacher voting is unclaimed.
• Redundancy/overlap as a derived finding — existing OBE tools only answer "is coverage numerically balanced." Nobody visualises or quantifies where content actually repeats across courses, or how Bloom's cognitive demand shifts semester-to-semester as an emergent, visualised result.

Papers by theme

OBE Framework

• 2022 MMSyeed IEEEAccess — four-stage OBE implementation process
• 2025 MDerouich DiscEdu — CLO–PLO alignment feedback-loop model; supports the semester-based labelling scheme in the dashboard

Bloom's & NLP

• 2021 ASImran IEEEAccess — LSTM Bloom classification of CLOs, 87% accuracy benchmark; rule-based degrades on higher Bloom levels, justifying the LLM layer
• 2025 RKumar Arxiv — model comparison (incl. Ollama, Anthropic) on 600 Bloom-labelled sentences; validates the three-method comparison design; also uses Cohen's κ
• 2021 AWaheed Arxiv — BloomNet; TF-IDF outperforms transformer baselines out-of-distribution, direct justification for the TF-IDF branch

Curriculum mapping / topic extraction

• 2021 FZablith IEEETransLearnTech — closest published precedent to the Knowledge Area Overlap view; validates concept-level (not course-level) granularity as the right unit of analysis
• 2024 MJacquinet AppSysInnov — grounds the "Knowledge Concepts" terminology; network graph + heatmap visualisation precedent
• 2024 BLopez ASEE / 2025 BLopez CompChemEngg — ChE-specific curriculum graphs (topics as nodes, courses as clusters); UCSD-adjacent design inspiration
• 2015 AJouhri IEEE — unsupervised topic modelling over course descriptions, web-based overlap comparison tool
• 2007 SLToral IntJofTechDesEdu — Concept Mapping Technique methodology (14-participant brainstorming + Likert clustering) for curriculum design
• 2021 CSimon PLOSONE — network analysis for course sequencing/selection (community detection, centrality)

Ontology-based approaches

• 2017 MBussemaker CompChemEngg — ChEEdO, the closest prior formal ChE domain ontology (Module/Learning Outcome/Topic, Bloom's verb properties)
• 2021 MZouri IEEE — general case for ontology-based curriculum mapping under accreditation
• 2024 NDitcharoen SmartLearnEnv — alternative ontology-building approach from actual curriculum documents
• 2025 SAminah IEEE — OBC-ONTO, OWL ontology applying OBE principles directly (clarity of focus, backward design)

To-read (ontology grounding, not yet read in full)

• OntoCAPE (Marquardt, Morbach, Wiesner) — ChE process-engineering ontology, most likely direct citation for "prior formal ChE domain ontology"
• OntoKin — combustion/kinetics ontology
• OntoRXN (Garay-Ruiz & Bo, 2022) — reaction network KG built on OntoCAPE
• KG framework for digital twins of chemical processes (Nature Chem. Eng., 2026) — LLM + ontology integration, directly relevant to the Ollama + normalisation-dictionary approach
• "An Ontology for Representing Curriculum and Learning Material" (arXiv 2506.05751) — closest precedent for the Knowledge Area/Unit terminology decision
• LLM-Assisted KG Completion for Curriculum Modelling (arXiv 2501.12300) — contrast material for the κ-validation gap argument
• PALM: Panoramic Learning Map (arXiv 2507.18393) — check references for older curriculum-mapping foundational sources (Harden 2001, Wager 1978)
• CyBOKClaw (arXiv 2605.24663) — human-in-the-loop syllabus-to-body-of-knowledge alignment, structurally close to a possible "ChE Body of Knowledge" extension
• ACM/IEEE-CS/AAAI Computer Science Curricula — primary source of the Knowledge Area/Unit/Learning Outcome terminology, worth citing directly rather than second-hand

Research Perspective — Still To Be Completed

Near-Term

• Pilot the CLO Extraction Tool with 2–3 teachers at SCME to test usability before formal study
• Decide on and implement a data persistence backend
• Formalise literature sources used in Phase 4 (augmentation) with proper citations
• Resolve prerequisite edge methodology (manual vs. LLM) — affects inter-rater reliability design
• Implement TF-IDF extraction mode to close the gap between documented and actual architecture

Study Design

• Define participant recruitment criteria (how many teachers, which courses)
• Draft consent/ethics documentation if required by NUST IRB
• Finalise whether usability (SUS/TAM-2) is included as a study outcome
• Determine whether teachers complete voting independently (blind) or collaboratively

Analysis Plan

• Cohen's κ analysis across teacher pairs per CLO
• Comparison of extraction method accuracy (LLM vs. rule-based vs. TF-IDF) against a gold-standard annotation
• Extraction input ablation: extract_topics_compare.py output (CLOs-only vs. CLOs + outlines) — Jaccard similarity, topic-count deltas, network-edge change implications
• Qualitative analysis of CLO revision quality post-Bloom's alignment

Publication / Dissemination

• Target venue: ASEE conference or IEEE Transactions on Education
• Structure paper around: tool design → related work / gap → study methodology → inter-rater results → CLO quality improvement evidence
• Prepare supplementary materials: taxonomy list, tool screenshots, κ tables, topics_comparison.json ablation results

File & Asset Reference

Key Technical Learnings

• D3 force simulation caching: forceX/forceY/forceCollide/forceManyBody cache per-node values at init and don't re-read accessors on later ticks. Fix: call simulation.nodes(nodes) to force re-init, then sim.alpha(0.7).restart().
• Blob visualisation scaling: with 51 courses, rendering all blobs at once produces an indiscernible fused mass. Cap active blobs to ~6–10 via a course selector; fill-opacity 0.08–0.12 with mix-blend-mode: multiply.
• create_file collision behaviour: creating a file at a path that already exists silently collides rather than overwriting — rm -f [path] first.
• TF-IDF vs. within-course frequency: TF-IDF signals cross-course distinctiveness; within-course CLO frequency signals salience. Both matter for the extraction-input ablation study, for different reasons.

Tags: #OBE #CurriculumMapping #ChemicalEngineering #NUST #SCME #EdTech #CLO #KnowledgeUnits #Research

Notes

• Ontological approach is already published in the field — possible scope to cover: OBE has objectives, and a knowledge graph of chemical engineering can be compared against the learning outcomes extracted from the CLOs
• https://protege.stanford.edu/ — OWL ontology editor
  • https://github.com/aminahcsui0195/OBC-ONTO/tree/main — git for OBC-ONTO
• OntoCAPE scope vs. this project's scope — where does subject-matter ontology end and curriculum-topic ontology begin? Is there a clean line, or should the ChE Ontology view eventually link to OntoCAPE classes rather than reinvent them?
• Check whether arXiv 2506.05751's KA/KU/LO ontology has a public schema/OWL file worth structurally comparing against data.json
• Check if any of the ontology papers report inter-rater agreement numbers at all (for comparison against the κ = 0.75 benchmark) — none found yet

Activity Log

• July 30, 2026
  • https://service.tib.eu/webvowl/#file=che_curriculum.owl
  • used to visualize the owl file of ontology
  • competency_questions_summary
    • PLO coverage (Q1/Q7) — heavily skewed. PLO-1 (Engineering Knowledge, 23.2%) and PLO-2 (Problem Analysis, 19.2%) dominate, while PLO-7 (Environment & Sustainability) and PLO-10 (Communication) each sit at only 2% (4 CLOs across all 51 courses) — flagged automatically as "LOW COVERAGE." That's a genuine, citable accreditation-relevant finding straight out of the box.
    • Domain split (Q2) — 71.7% Cognitive, 16.2% Psychomotor, 12.1% Affective. Worth reporting as evidence your curriculum isn't purely cognitive-focused, which is a nice OBE-completeness point.
    • Data integrity (Q6) — flagged a real gap: several CS-117 CLOs have no knowledge areas/concepts extracted at all (empty concepts list in your source data). That's worth checking against your extraction pipeline — either those CLOs genuinely weren't run through extraction, or it's a data quality issue like the ones you've already been tracking (CHE-103/CHE-222, PHY-102).
• July 31, 2026
  • Created a new version of exel_to_json.py which extracts both the topics and the edges of the topics connections.
  • Pushing code to GitHub
• August 17, 2026
  • Updated the payment details on the website