Back to the program

AI Literacy for Family physicians (ALF): development and evaluation of artificial intelligence training for family physicians

Ronen BAREKand, Shani AFEK and Sophia EILAT-TSANANI

Despite rapid artificial intelligence integration in healthcare, family physicians receive inadequate training to critically evaluate and safely apply AI-based clinical tools. Studies show strong physician interest but limited confidence and skills, leaving primary care providers unable to assess AI tool validity, understand performance metrics, or identify potential biases.To evaluate the effectiveness of a structured continuing professional development course in improving family physicians' knowledge, attitudes, and self-efficacy regarding artificial intelligence and machine learning applications in primary care.Pre-post intervention study of Israeli family physicians enrolled in a two-day course (November 27-28, 2025). The curriculum covered: machine learning fundamentals (supervised/unsupervised learning, overfitting, AUC, calibration, neural networks), large language models in medicine, hands-on predictive modeling demonstrations, and critical appraisal of AI clinical prediction studies. Three questionnaires assessed: (1) ML knowledge (7 multiple-choice questions on overfitting, calibration, supervised learning, LLM mechanisms); (2) attitudes toward AI across 11 clinical scenarios (4-point Likert); (3) self-efficacy in AI-assisted decision-making (adapted validated scale, 4-point Likert). Paired t-tests and Wilcoxon signed-rank tests were used (p<0.05).Of 62 enrolled physicians, 40 completed pre-intervention and 27 completed both assessments (67.5% completion rate). Significant knowledge improvement was observed (pre: 0.51, post: 0.71; 39% increase; p<0.001). No significant changes in attitudes (pre: 2.40, post: 2.39; p=0.93) or self-efficacy (pre: 3.30, post: 3.36; p=0.14). High baseline attitudes and self-efficacy scores suggest potential ceiling effects.This study demonstrates that a multi-modal AI literacy course significantly improves family physicians' foundational machine learning knowledge. The 39% improvement shows complex technical concepts can be effectively taught to practicing physicians without computational backgrounds. Unchanged attitudes and self-efficacy, despite high baseline scores, may reflect existing positive AI perceptions among self-selected participants rather than intervention ineffectiveness. The limited sample size (n=27) warrants validation in larger cohorts.A focused continuing professional development intervention effectively enhances family physicians' AI literacy, providing essential skills to critically evaluate AI-based clinical tools. This evidence-based approach demonstrates feasibility for integration into primary care professional development programs, addressing a critical gap in physician preparedness for AI-augmented practice.