Conversational AI Data Scientist

Job description

**Role & responsibilities** - Design, execute, and analyze A/B tests and experimentation frameworks to measure the impact of conversational AI features and product enhancements. - Develop robust statistical models and perform hypothesis testing, causal inference, power analysis, and significance testing to generate reliable business insights. - Build analytical frameworks to evaluate conversation quality, customer engagement, sentiment, warmth scores, intent, and business outcomes. - Define, monitor, and optimize product KPIs using Python, SQL, and dashboarding tools such as Tableau or Power BI. - Perform deep-dive root cause analysis on product performance, user behavior, and customer interaction metrics to identify optimization opportunities. - Collaborate with AI Engineers to evaluate NLP models, conversational AI performance, training datasets, and model quality using analytical and statistical techniques. - Design evaluation methodologies including holdout strategies, sampling techniques, model drift monitoring, fairness evaluation, and annotation quality assessment. - Translate complex analytical findings into actionable business recommendations and executive-level presentations. - Develop scalable analytical pipelines, automate reporting workflows, and support data-driven decision making across Product, Engineering, and Business teams. - Partner with cross-functional stakeholders to quantify business impact through revenue attribution, customer satisfaction, retention, and conversion metrics. **Preferred candidate profile** - 36 years of experience as a **Data Scientist**, **Decision Scientist**, **Product Data Scientist**, or **Applied Data Scientist** in a product or customer analytics environment. - Strong hands-on experience with **Python**, **Advanced SQL**, **Pandas**, **NumPy**, **Scikit-learn**, and statistical analysis libraries. - Proven expertise in **A/B Testing**, **Hypothesis Testing**, **Causal Inference**, **Power Analysis**, and experimentation methodologies on production systems. - Strong understanding of **Product Analytics**, **Customer Analytics**, **Behavioral Analytics**, **Cohort Analysis**, **Funnel Analysis**, and KPI development. - Experience evaluating NLP or Conversational AI solutions, including **Sentiment Analysis**, **Intent Classification**, **Embeddings**, annotation strategies, and model performance evaluation. - Hands-on experience building dashboards using **Tableau**, **Power BI**, or equivalent BI tools for business and executive stakeholders. - Strong understanding of Machine Learning evaluation techniques, model validation, confusion matrices, model drift detection, and fairness metrics. - Experience working with large-scale datasets using SQL and modern data platforms; exposure to orchestration tools such as **Airflow**, **dbt**, or similar is preferred. - Excellent stakeholder management, business communication, and storytelling skills with the ability to translate statistical findings into commercial impact. - Experience working closely with Product Managers, AI Engineers, and cross-functional teams in an Agile environment is highly desirable.