Abstract
Background
Expanded Food and Nutrition Education Program (EFNEP) curricula are mandated to follow evidence-based guidelines such as the Dietary Guidelines for Americans (DGA). Content analysis methods can be used to assess curriculum alignment with established guidelines and corresponding evaluation measures. However, this is a time-intensive process that typically relies on multiple human reviewers. Artificial intelligence (AI) tools present opportunities to compare human and AI content analyses.
Objective
Apply a sequential human- then AI-based content analysis to assess alignment of commonly used EFNEP adult curriculum to the 2020–2025 DGA.
Study Design, Setting, Participants
A 20-item instrument assessing the coverage of DGA messages was developed by EFNEP and nutrition experts; each item was rated using a 3-point scale (0 = not mentioned; 1 = briefly mentioned; 2 = explicitly taught with supporting discussion/activities). Two nutrition experts independently rated the coverage in the 8-lesson curriculum; any disagreements were resolved through consensus. Parallel analyses were conducted using Google Gemini AI.
Measurable Outcome/Analysis
Coverage frequencies of 20 DGA items (i.e., how often they were coded as 0, 1, or 2) were identified and descriptively compared with assess agreement between human and AI ratings by using Excel.
Results
Most frequently mentioned items in the curriculum were increasing fruit and vegetables, consuming a variety of vegetables, varying protein in nutrient-dense forms, consuming fat-free dairy/fortified soy alternatives, and choosing whole grains. Agreement between human and AI reviews was 35%, 20%, and 20% for explicitly taught, briefly mentioned, and not mentioned content, respectively. Discrepancies in agreement occurred mostly in the first and last lessons, where concepts were mentioned indirectly or embedded within another topic. Artificial intelligence scored items based on semantic comparisons, while human reviewers also considered the intent of the DGA messages.
Conclusions
Artificial intelligence demonstrated stronger agreement with human-based analyses for clearly explicit content, but was less consistent when nuanced interpretation was required. These findings support AI use as a complementary tool, while emphasizing human presence to ensure comprehensive curriculum evaluation.