Job Description
<p>A project dedicated to assessing and benchmarking advanced agentic audio models against leading systems. The program’s mission is to evaluate and optimize model performance for real-world customer support use cases.</p>
<p>Responsibilities</p>
<ul>
<li>Create and execute <strong>role-play–based evaluation scenarios</strong> that simulate realistic customer service interactions across multiple domains, including:</li>
<ul>
<li>Flight bookings and travel support</li>
<li>Financial services</li>
<li>Telecommunications and technical support</li>
</ul>
<li>Contribute to the development of <strong>diverse and representative datasets</strong> used to assess conversational audio agents.</li>
<li>Evaluate model performance across a standardized set of qualitative and quantitative metrics.</li>
<li>Ensure evaluations reflect real customer expectations for clarity, efficiency, and natural conversational flow.</li>
</ul>
<p>Evaluation Metrics</p>
<p>Model performance is assessed using a combination of conversational, technical, and audio-specific criteria, including but not limited to:</p>
<ul>
<li><strong>Task completion accuracy</strong> and efficiency</li>
<li><strong>Conversational naturalness</strong> (tone, flow, and coherence)</li>
<li><strong>Audio comprehension and response quality</strong></li>
<li><strong>Instruction adherence and contextual understanding</strong></li>
<li><strong>Basic computer programming literacy</strong>, including:</li>
<ul>
<li>Understanding of <strong>JSON structures</strong></li>
<li>Familiarity with <strong>functions and methods</strong></li>
<li>Ability to reason about structured data and simple logic</li>
</ul>
<li><strong>Technical communication clarity</strong> when handling support-style problem-solving</li>
</ul>
<p>Technical Equipment Requirements</p>
<ul>
<li>Strong verbal communication skills in a simulated customer support context</li>
<li>Access to a <strong>high-quality microphone</strong> to ensure clean, reliable audio input during evaluations</li>
<li>Comfort working with structured prompts, evaluation rubrics, and technical guidelines</li>
</ul>
<p>We offer a pay range of $8-to- $25 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you’ll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.</p>
<p>Job title: Spanish Audio Specialist – AI Trainer<br>Employment type: Contract<br>Workplace type: Remote<br>Seniority level: Mid‑Senior Level</p>