A Review on Human-AI Interaction in Machine Learning and Insights for Medical Applications.

We design four questions to investigate and describe human-AI interaction in ML applications. These questions are "Why should humans be in the loop?", "Where does human-AI interaction occur in the ML processes?", "Who are the humans in the loop?", and "How do humans interact with ML in Human-In-the-Loop ML (HILML)?". To answer the first question, we describe three main reasons regarding the importance of human involvement in ML applications. To address the second question, human-AI interaction is investigated in three main algorithmic stages: 1. data producing and pre-processing; 2. ML modelling; and 3. ML evaluation and refinement. The importance of the expertise level of the humans in human-AI interaction is described to answer the third question. The number of human interactions in HILML is grouped into three categories to address the fourth question. We conclude the paper by offering a discussion on open opportunities for future research in HILML.