TikTok LIVE - AI/ ML Data Operations Manager

TikTok LIVE - AI/ ML Data Operations Manager

Location:

Singapore

Employment Type:

Regular

Job Code:

A109486A

Share this listing:

Responsibilities

About the Team The livestream industry has seen tremendous growth in recent years and has become the next growth driver for TikTok global business. It also brings great joy to users globally and it creates a new channel for our creators to show themselves. We are seeking a AI/ ML Data Operations Manager who owns the end-to-end quality and operating system of prompts, policy skills, and evaluation data that power multimodal LLM and Agent moderation. The role translates policies and expert judgment into model-executable strategies, runs closed-loop iteration from bad cases through release, and establishes standards so that delivery across multiple queues remains consistent, measurable, and scalable. This role sits at the intersection of Prompt Engineering, Data Operations, Evaluation Design and Cross functional Program Delivery, and works closely with Algorithm, Product, Engineering, Data Science, Operations and Labelling team. Responsibilities: - Lead Prompt Engineering and Data Operations. Own the full lifecycle of prompts, skills, and policy provisions for multimodal LLM Agents across video, image, for TTLive scenarios, including Prompt and skill generation, Version management, Release management , Online maintenance and Quality governance. - Design Training and Evaluation Data Strategies. Define data and iteration strategies for model training and evaluation, including: - Sampling strategies, Golden-set and benchmark construction, Blind-label protocols, Offline and online evaluation packages, A/B measurement , Substitute evaluation designs when instrumentation is incomplete - Align Models with Policies. Convert policy provisions and expert judgment into executable prompts and Agent skills, establishing: Prompt-writing standards, Writer principles, Admission criteria, Standard operating procedures, Quality and consistency requirements , Explainability requirements , Chain-of-thought or title and sub-title-level decision structures - Partner with Labeling Teams. Drive the delivery of high-quality training and evaluation data through: Labeling specifications , Quality-control SOPs , Ground-truth alignment , Edge-case mining, Throughput and quality SLAs, Error analysis, Dataset refresh, Model and strategy iteration - Define Tooling Requirements. Work with Product and Engineering to define requirements for: Case retrieval, Batch testing, Prompt debugging, Online-result backfill, Guidance and policy tools, Self-service RCA tools, Analysis assistants, Data-iteration workflows - Own Evaluation, Iteration, and Rollout. Diagnose false positives, false negatives, and metric movements across indicators such as: Accuracy , Risk, False Rejection, Coverage - Prioritize remediation actions, prepare Launch Review evidence, and support: Canary testing, A/B testing, Rollout decisions, Cross-queue launches - Build and Scale the PE Operating System. Standardize the full operating workflow: Intake → Sampling → Labeling → QC → Evaluation → Release - Build program-tracking and multi-queue delivery mechanisms so that new prompts and maintenance cycles are delivered predictably and at scale. - Drive Continuous Improvement from Production Signals. Convert online bad cases, RCA findings, and hard-policy constraints into: Prompt updates, Skill updates, Data updates, Reusable playbooks, Clear separation between confirmed conclusions and hypotheses

Qualifications

Minimum Qualification(s): - Bachelor’s degree or above. - 3+ years of experience in AI/ML data operations, prompt engineering, content-moderation strategy, annotation operations, technical program management, strategy, or related fields. - Hands-on experience designing or operating prompts and LLM workflows for production systems, including training, evaluation, or online inference. - Proven experience building evaluation frameworks, including golden sets, benchmarks, labeling SOPs, quality control, offline and online metrics, and experiment-ready evaluation packages; strong skills with a proactive approach to identifying and driving process improvements. - Strong experience working with labeling vendors or internal annotation teams. - Experience in taxonomy and specification design. - Experience in quality control and delivery management. Preferred Qualification(s): - Experience in content governance, Trust & Safety, or other high-stakes decision systems. - Exposure to multimodal model evaluation across video, image, text, URL, or webpage content. - Exposure to Agent systems, including planners, skills, tool use, and chain-of-thought or structured decision-making. - Familiarity with A/B experiments, metric decomposition, numerator and denominator analysis, mix-shift analysis, and case-level RCA. - Experience productizing internal tools for prompt debugging, case replay, and data-iteration platforms. - Experience owning zero-to-one operating systems, including SOPs, workflow tools such as Meego or Jira, dashboards, and scalable delivery processes. - Basic SQL and data-analysis capability. - Python scripting experience for evaluation pipelines is a plus. - PMP or equivalent program-management experience is a plus.

Job Information

About TikTok

TikTok is the leading destination for short-form mobile video. At TikTok, our mission is to inspire creativity and bring joy. TikTok's global headquarters are in Los Angeles and Singapore, and we also have offices in New York City, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.​

Why Join Us

Inspiring creativity is at the core of TikTok's mission. Our innovative product is built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and bring joy - a mission we work towards every day.​

We strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. Every challenge is an opportunity to learn and innovate as one team. We're resilient and embrace challenges as they come. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our company, and our users. When we create and grow together, the possibilities are limitless. Join us.​

Diversity & Inclusion​

TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.​

Ready for a career

at TikTok?

Discover a career that energizes and excites you every day.