Abstract: To address the difficulty of recognizing tail classes in Web service classification caused by long-tailed data distributions, a reinforcement learning-enhanced framework is proposed, which integrates multi-round data augmentation with adaptive loss optimization. A large language model (LLM) is employed as the core semantic generator, and a reinforcement learning agent adaptively adjusts class-specific augmentation ratios and filtering thresholds at each iteration based on the observed environment state, driving a closed-loop multi-round process of “generation-evaluation-filtering-feedback.” In parallel, a Chain-of-Thought-based reasoning mechanism is introduced to evaluate generated samples from multiple dimensions－including novelty, semantic consistency, and reasoning quality－thereby filtering out template-like and semantically drifting instances and progressively improving the training data distribution. During classifier training, class weights are dynamically computed from the frequency statistics of the augmented dataset at each iteration. A Top-k Near-Miss Focal Loss is further designed to jointly emphasize long-tailed classes and near-miss boundary samples, penalizing ambiguous semantic regions and enabling adaptive loss optimization tailored to long-tailed and hard examples. Experiments conducted on a real-world long-tailed Web service dataset and the PMTD (Productive Math Tutoring Dialogue) instructional dialogue dataset demonstrate that the proposed method outperforms mainstream baselines such as NCAL (Neural-Collapse-Advanced personalized Learning), RGPT, SRaSLR (Social Relation Aware Service Label Recommendation Model) and LLMEmbed across multiple evaluation metrics. In particular, substantial improvements are observed for tail-class recognition: on several lightweight models, Macro- \mathrmF_1 improves by up to 5 percentage points on the Web service dataset, and Weighted- \mathrmF_1 increases by approximately 2-3 percentage points. These results verify the effectiveness of the proposed approach in mitigating the bias introduced by long-tailed distributions and provide a practical solution for intelligent recognition and classification of semantically sparse services in open environments.