Reliable Data Labeling Methods for Developing Mobile ML Systems

Mobile ML systems heavily rely on high-quality well-labeled data to make accurate predictions. However, labeling data in mobile environments has its own set of challenges, including heterogeneity of data, real-time constraints, and limited resources. This paper presents robust data labeling methods in particular for mobile ML system development, focused on crowdsourcing, semi-supervised labeling, and active learning. We consider crowdsourcing to be an economically viable solution for data labeling but also discuss label quality and label accuracy problems. Semi-supervised learning and active learning are presented as effective methods to minimize the need for large, labeled training sets without sacrificing model performance. We also briefly mention techniques to ensure label quality, avoid bias, and maintain efficiency in the context of mobile resource constraint. The work highlights prominent problems such as data privacy, cost, scalability, and bias within labeled sets of data, as well as promising directions such as hybrid labeling solutions and coordination with edge computing and IoT devices. Our research is aimed to present a thorough snapshot of reliable labeling methods which play a central role in developing and thriving mobile ML applications.