Assessing the Performance of Artificial Intelligence Models: Insights from the American Society of Functional Neuroradiology Artificial Intelligence Competition.
Bin JiangBurak B OzkaraGuangming ZhuDerek BoothroydJason W AllenDaniel P BarboriakPeter ChangCynthia ChanRuchir ChaudhariHui ChenAnjeza ChukusVictoria DingDavid B DouglasChristopher G FilippiAdam E FlandersRyan GodwinSyed S HashmiChristopher P HessKevin HsuYvonne W LuiJoseph A MaldjianPatrik MichelSahil S NalawadeVishal PatelPrashant RaghavanHaris I SairJody TanabeKirk WelkerChristopher T WhitlowGreg ZaharchukMax WintermarkPublished in: AJNR. American journal of neuroradiology (2024)
To assess the performance of artificial intelligence models in real-world clinical scenarios, we analyzed their performance in the ASFNR Artificial Intelligence Competition. The first ASFNR Competition underscored the gap between expectation and reality; and the models largely fell short in their assessments. As the integration of artificial intelligence tools into clinical workflows increases, neuroradiologists must carefully recognize the capabilities, constraints, and consistency of these technologies. Before institutions adopt these algorithms, thorough validation is essential to ensure acceptable levels of performance in clinical settings.