Speaking anxiety is a persistent barrier for Pakistani adult ESL learners. Classroom conditions make it worse. Teachers punish errors. Students fear low grades. These patterns are not coincidental. This qualitative descriptive study examines two things. First, it documents the speaking anxiety experiences of Pakistani adult ESL learners. Second, it explores how AI-mediated Informal Digital Learning of English (AI-IDLE) shapes those experiences. AI-IDLE refers to using AI tools such as ChatGPT, Copilot, and Grok to practice English outside class. Ten undergraduate students at a government university in Pakistan participated. Data came from semi-structured Zoom interviews, 44 screenshots of AI interactions, and 10 voice recordings. Thematic analysis followed the six-step framework by Naeem et al. (2023), with digital artifact triangulation grounded in referential adequacy. Three anxiety dimensions emerged: fear of negative evaluation, teacher evaluation anxiety, and performance-event anxiety. AI tools acted as affective buffers. They offered patient, non-punitive feedback that classrooms rarely provide. Learners built self-efficacy through repetition, competence accumulation, and psychological inoculation. Students benefited from AI practice outside class. No equivalent low-stakes interaction existed in the classroom. This is the classroom IDLE gap. Curriculum integration of IDLE activities could close it. Findings are grounded in Bandura’s (1997) self-efficacy theory and Lee’s (2021) IDLE framework. The study recommends compulsory pedagogical training for university teachers in Pakistan. It also calls for formal curriculum space for AI-mediated speaking practice. Mobile AI tools make this achievable even where institutional resources are limited.