Towards building a comprehensive evaluation framework for Human Computer Interaction with a specific focus on AutoML

The rapid advancement of Automated Machine Learning (AutoML) has revolutionized the field of Artificial Intelligence (AI), enabling the automation of complex machine learning workflows. However, the increasing autonomy of AutoML systems has highlighted the critical importance of Human-Computer Interaction (HCI) principles in ensuring their usability, transparency, and trustworthiness. This paper proposes a comprehensive evaluation framework for HCI in AutoML, focusing on five key dimensions: Contracts and User Development, User Interface, Interaction and Experience Design, Information Architecture, Human Augmentation Factors Design, and Care and Responsibility. The framework emphasizes the need for transparent disclosures, intuitive interfaces, structured information presentation, user empowerment, and ethical considerations, including fairness and accountability. By prioritizing user-centered design, the framework aims to bridge the gap between AutoML’s technical capabilities and diverse user needs, fostering trust and collaboration between humans and AI systems. The integration of HCI principles is crucial for realizing the full potential of AutoML, ensuring that these powerful tools are developed and deployed in a manner that benefits all users and society. As AutoML continues to evolve towards greater autonomy, the nature of human interaction is expected to transform from direct operation to strategic supervision and collaborative partnership, leveraging the complementary strengths of humans and machines. The proposed framework serves as a foundation for creating transparent, user-friendly, and trustworthy AutoML systems that balance automation with user understanding and control, paving the way for a future where AI systems are not merely efficient but also equitable, safe, and truly beneficial for all. CCS CONCEPTS • Human-centered computing ~ Human computer interaction (HCI) ~ HCI design and evaluation methods ~ Walkthrough evaluations Additional