Dimensionality Reduction Techniques in Big Data and Their Impact on E-Learning

Abstract

With the increasing use of e-learning in various fields, there is a growing need to analyse and process big data generated from student interactions with digital learning systems. This data includes test results, content interactions, and learner behavioural data. High dimensionality in data can hinder analysis using AI and machine learning, necessitating dimensionality reduction to enhance model efficiency and reduce computational complexity.

The study examines dimensionality reduction techniques like PCA, LDA, autoencoders, and t-SNE in e-learning. It finds traditional methods effective, but advanced methods like deep autoencoders and hybrid AI models offer superior performance. UMAP outperforms t-SNE for clustering and visualisation tasks.