Job Description
Project Overview
We are seeking experts and researchers to provide their expertise for an AI evaluation project focused on Product Deep Research. As AI systems increasingly act as advanced shopping assistants and market analysts, their utility depends on their ability to move beyond simple fact lookups. The objective of this project is to train and evaluate an AI model’s capacity to perform structured, high-nuance analysis on specific products, rigorously assessing market position, feature trade-offs, and long-term durability.
Project Deliverables Scope
Prompt Engineering Generation: Designing and writing complex, multidimensional prompts that compel the AI model to execute deep, nuanced product analysis rather than surface-level summaries.
Atomic Rubric Creation: Generating auditable, and atomic rubric items per task. These rubrics must objectively evaluate the model’s response across mandatory analytical dimensions.
Multidimensional Analysis Auditing: Evaluate the model's generated outputs to ensure it accurately weighs product trade-offs, evaluates competitive market positioning, and assesses long-term value.
Fact-Checking Data Validation: Delivering feedback verifying the factual precision of the model's deep research capabilities across various product categories.
Required Expertise Background
Deep Research Fact-Checking: Demonstrable ability to source, verify, and synthesize complex product information from highly reliable sources.
Multidimensional Analysis: The ability to break down complex consumer or enterprise products into critical evaluation vectors (e.g., build quality, ecosystem lock-in, price-to-performance ratio).
Prompt Writing Engineering: Familiarity with structuring prompts to elicit highly specific, constrained, and analytical outputs from Large Language Models (LLMs).
Baseline Product Knowledge: Broad familiarity with consumer or enterprise product landscapes and how to properly assess them.
We offer a pay range of $6-to- $65 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you’ll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.
Contract Type: Freelance / Contract
Workplace Type: Remote
Seniority level: Entry Level