Assessing Authentic Generative AI Learning Experiences

The objective of this research is to focus specifically on assessment of authentic learning experiences, explored as part of Using Generative AI to Create Authentic Learning Experiences, through use of Generative AI (preferably open source and in house) and innovative methodologies. Key areas that will be focused include leveraging rubrics for AI-enabled grading and measuring simulation-based assessments.

Rubric-Guided AI grading

• Explore, develop and implement rubrics (adaptable and scalable across diverse assessment types) tailored to AI based assessments.
• Integrate rubrics into existing (or new) algorithms
• Address challenges such as developing objective criteria for subjective tasks (e.g. creativity), effective and impartial data (subject to availability), adaptability to different types of assessments, capturing nuanced aspects, e.g. critical thinking.
• Real time rubrics, i.e. refine rubrics on the go as well as having experts in the loop.
  Evaluate the reliability and validity of these adaptable rubrics.

Measuring Simulation-based assessments

• Explore existing and develop a framework to measure and assess data generated from simulation-based learning experiences.
• Implement tools and methodologies for capturing, analysing and interpreting simulation-driven assessment data.

As a by-product of assessment data, state of the art AI/ML techniques will be employed to suggest optimised personalised learning pathways. The data and the related research outcomes will feed into Generative AI Personalised Learning Planner, for creation of learning pathways. To conclude, the outputs of this research stream will in turn, help to answer the following research question:

Definition of the research question being addressed

How can Generative AI and innovative (learning) methodologies enhance authentic learning assessment through rubric-guided AI grading and simulation-based evaluations, while addressing challenges such as subjective task criteria, adaptability across assessment types, and leveraging assessment data for personalised learning pathways?”