Low Query Budget Active Learning for Classification and Regression

The labeling process for supervised learning is costly and time-consuming, and is often impractical to scale due to real-world constraints. Active learning (AL) addresses this challenge by strategically selecting representative and informative data points to reduce labeling efforts. This paper focuses on an AL scenario in which only a very limited number of labels can be acquired. We propose an algorithm operating in two phases: (1) an exploration phase that prioritizes representative and diverse data points using density-driven criteria, and (2) an exploitation phase that combines predictive uncertainty with density weighting to select informative samples from densely populated regions. This enhances both representativeness and informativeness. Our results demonstrate significant improvements in model quality compared to other algorithms typically employed for this scenario, across various scenarios involving imbalanced data in classification tasks and skewness in regression tasks. Through this work, we aim to provide a new algorithm for this scenario and investigate general principles for AL. While most AL studies focus on either classification or regression, our work applies the algorithms to both. Therefore, we can analyze the differences between classification and regression problems and their effects on AL strategies. Furthermore, we explore different categories of AL criteria and their effectiveness in the low-budget regime. These results also provide insight into the cold-start problem, which involves selecting an initial labeled set and is faced by many model-based AL methods.