Back to Observatory
Open-ended questionnaires assisted analysis for science education.
Description of the service
Text mining applied to analyzing scientific texts allows the identification of the text's main topic, the concepts in these documents, and trends in the stance on a topic, among others. For this project, text mining is used to support the analysis of the answers given by high school students to a questionnaire on genetics and thus determine the conceptual understanding achieved by the students. The web app uses a supervised learning algorithm and adapts the general stages of text mining. Specifically, it supports researchers in analyzing concepts and relationships between frequent concepts and in the automatic scoring of answers.
Problem that it solves
In science education, analyzing answers to these types of questions helps to determine students' understanding. The traditional method is manual and time-consuming, as researchers review answers individually and set out categories and levels to achieve a uniform evaluation. The automatic classification will drastically reduce the time required to analyze this information.
Type of AI app used
Reasoning with knowledge structures, Prediction
Main results to June 30, 2021
The proposed web app was tested with Question No. 1, modifying the training sample size, and the model's best performance was 74% in classification accuracy. The results of a set of automatically-scored responses were compared to those scored by experts to analyze the automatic scoring process validity.
This analysis identified that concepts and relationships identified by AI and experts are very similar, indicating that automatic scoring could assist in quickly visualizing students' responses and relationships without experts needing to score all of them.
Three main bottlenecks faced during implementation
- The amount of data: Questionnaires are applied to relatively small populations for data mining (tens to hundreds).
- The system evaluation required the entire sample to be evaluated by experts, which was very time-consuming.
- Making the system available online so that other researchers can access it has been complex because of computer security problems. Given that the project focused on learning algorithms development and integration available on the Internet, it has been affected by computer attacks, so the system only runs on a local network which affects the visibility of the project.
Lessons learned in the design or use of AI for social impact
- For Machine Learning and data science, we address the need for large amounts of data to obtain accurate models. The project demonstrated that good models can result from smaller amounts of data, which is very important for the case of teaching.
- The available tools help apply AI to solve community problems, such as the teaching community.
- AI can become a tool to assist researchers in speeding up the analysis and decision-making processes on teaching problems, leading to their improvement.
Country of origin
Geographic scope of operations
Mexico City
Type of executing entity
UniversitySector/industry
Sustainable Development Goal(s) to which your AI solution contributes
4 (quality education)
IA app developed internally or by a third party
Internally
Name of implementing entity
Institute of Applied Sciences and Technology, National Autonomous University of Mexico.
Stakeholders involved
Researchers in the area of science education and team specialized in data science and web apps development
Percentage of the development team that are women
40%
Year they started using AI-based models
2010It may interest you
Esta guía fue diseñada para el personal directivo y docentes que buscan fortalecer la protección de los datos de los estudiantes en las plataformas en línea de sus instituciones educativas.
The Regional Landscape and 12 Country Snapshots
Este documento fue diseñado como insumo de una hoja de ruta que permite crear un marco para el uso ético, responsable y seguro de la IA en Costa Rica.