Staff AI Engineer - QA
Harris Computer is hiring a Staff AI Engineer to join their, expanding team! We’re seeking a seasoned expert to pioneer the quality and reliability of our cutting-edge AI and machine learning solutions for healthcare.
As a Staff AI Engineer focused on Quality Assurance, you will be the technical authority responsible for ensuring our AI systems are robust, fair, accurate, and safe.
This is a unique opportunity to define the future of AI quality in a domain where it matters most, mentoring a talented team and establishing the gold standard for testing intelligent systems that impact patient care.
This is a full-time, remote position open to candidates in the U.S.
or Canada.
What your impact will be:
* Build the core AI system end-to-end, from the service layer and data pipelines to the evaluation infrastructure, and own its architecture.
* Design and build the evaluation infrastructure, including dataset management, automated test harnesses, and replay of AI systems against historical workflows.
* Build and maintain evaluation datasets with clinician-reviewed correct answers, ensuring they evolve as our systems change.
* Create error metrics weighted by severity, so mistakes with clinical or financial impact are prioritized accordingly.
* Build production monitoring for accuracy, overrides, and safety issues, with actionable alerts for the engineering team.
* Write adversarial tests for AI systems, including hidden or malicious instructions in documents and invented content.
* Work with product and clinical leads to define accuracy thresholds and the criteria for expanding or pulling back an AI capability.
* The candidate must possess a curiosity and willingness to actively adopt and leverage emerging AI tools to improve workflows, solve problems, and drive efficiency along with being comfortable using a range of AI-enabled tools (such as copilots, chat-based AI, and automation solutions) as part of everyday work.
What we’re looking for:
* 7 or more years in software or machine learning engineering, with at least 3 years focused on evaluation, testing, or quality for AI systems.
* A track record of designing and building complete systems from scratch, taking them from prototype to production.
* A track record of building and running evaluation systems that teams use in production, not just for one-off analyses.
* At least 2 years of experience evaluating large language model (LLM) systems in a production environment.
* Strong statistical skills for measuring accuracy, including sampling, confidence intervals, and error analysis.
* 1 year + of experience designing evaluation sets for document extraction, classification, or structured output.
* Deep knowledge of explainable AI (XAI) techniques and AI-powered tools (e.g., SHAP, LIME, Claude).
* Clear written communication, including design documents, and the ability to explain complex trade-offs to clinicians, e...
- Rate: 160000
- Location: Raleigh, US-NC
- Type: Permanent
- Industry: IT
- Recruiter: N. Harris Computer Corporation - USA
- Contact: Not Specified
- Email: to view click here
- Reference: R0475114
- Posted: 2026-11-18 07:55:59 -
- View all Jobs from N. Harris Computer Corporation - USA
More Jobs from N. Harris Computer Corporation - USA
- Process and Continuous Improvement Engineer
- Multi Craft Maintenance Technician
- Carpenter - West Texas
- Material Handler - West Texas
- Staff Electro-Optical System Lead
- Accounting Assistant
- Manufacturing Engineer
- Associate Global Category Manager
- P&ID Supervisor
- Project Controls Scheduler
- Technical Sales Engineer
- IT Project Manager
- Project Controls Scheduler
- Production Operator
- Fixed Equipment Leader
- Quality Engineer
- Assistant Construction Manager - El Paso
- Sr Director, IT Anchor Packaging
- Program Manager- NPI
- Civil Foreman