Our client is seeking an experienced AI Observability Engineer with strong expertise in data analysis, visualization, monitoring, and dashboard development. The ideal candidate will have hands-on experience with Grafana, Tableau, Prometheus, PromQL, and time-series data.
The candidate will design, implement, and maintain observability solutions for AI/ML and Large Language Model (LLM) applications, ensuring visibility into model performance, reliability, latency, cost, data quality, and user experience.
Required Technical Skills
- Minimum 5 years of experience in data visualization and dashboard development.
- Strong hands-on experience with Grafana and Tableau.
- Experience with Prometheus, PromQL, and time-series monitoring.
- Strong knowledge of Linux and SQL.
- Expertise in creating interactive dashboards, visualizations, graphs, and performance reports.
- Experience with application monitoring, telemetry, and observability solutions.
- Understanding of AI/ML application monitoring and performance metrics.
Roles & Responsibilities
- Design, develop, and maintain Grafana and Tableau dashboards for application and AI/ML observability.
- Implement monitoring solutions using Prometheus and PromQL.
- Build telemetry dashboards to track model latency, token usage, throughput, error rates, and service reliability.
- Monitor AI/LLM application performance, data quality, model drift, safety, and operational costs.
- Define and monitor Service Level Objectives (SLOs) and performance indicators.
- Analyze time-series data and identify performance bottlenecks and operational issues.
- Evaluate and integrate observability tools for metrics, logs, traces, and model telemetry.
- Collaborate with ML engineers and platform teams to improve production observability.
- Develop self-service dashboards and reporting solutions for engineering teams.
Preferred Skills
- Thanos monitoring stack.
- Python programming.
- Bash scripting.
- Agile methodologies and Jira.
- Experience with AI/ML and LLM monitoring frameworks.
Ideal Candidate Profile
- Strong analytical and problem-solving skills.
- Excellent expertise in data visualization and dashboard development.
- Hands-on experience with Grafana, Prometheus, Tableau, and observability tools.
- Understanding of AI/ML application performance monitoring.
- Ability to translate complex telemetry data into actionable insights.