Postdoctoral Scholar, AI Evaluation & Standards

At Johnson & Johnson,we believe health is everything.

Our strength in healthcare innovation empowers us to build aworld where complex diseases are prevented, treated, and cured,where treatments are smarter and less invasive, andsolutions are personal.Through our expertise in Innovative Medicine and MedTech, we are uniquely positioned to innovate across the full spectrum of healthcare solutions today to deliver the breakthroughs of tomorrow, and profoundly impact health for humanity.Learn more at jnj.com .

As guided by Our Credo, Johnson & Johnson is responsible to our employees who work with us throughout the world.

We provide an inclusive work environment where each person is considered as an individual.

At Johnson & Johnson, we respect the diversity and dignity of our employees and recognize their merit.

Job Function:
Career Programs

Job Sub Function:
Post Doc - Data Analytics & Computational Sciences

Job Category:
Career Program

All Job Posting Locations:
Barcelona, Spain, Beerse, Antwerp, Belgium, Madrid, Spain, Raritan, New Jersey, United States of America, Titusville, New Jersey, United States of America

Job Description:
Job description

At J&J we are developing Generative AI solutions to support pharmaceutical R&D, including literature review, evidence synthesis, document Q&A, therapeutic area knowledge search, translational science workflows, and R&D decision support.

These systems need to be tested before teams use them in scientific workflows.

In pharmaceutical R&D, a useful AI response depends on the question, user, source material, therapeutic area, and risk of error.

We are looking for a postdoctoral researcher to help design methods that test whether GenAI tools produce answers that are accurate, evidence-grounded, traceable, usable, and appropriate for the intended task.

The role reports to the Associate Director, Generative AI Evaluation & Quality Standards.

The team defines how J&J Innovative Medicine evaluates GenAI systems before use and helps determine when they are ready for release, expansion, or improvement.
Key responsibilities


* Design evaluation frameworks, rubrics, and criteria for GenAI tools used across pharmaceutical R&D.


* Develop therapeutic-area-specific criteria with business and scientific teams to reflect domain and use-case quality needs.


* Build benchmark datasets, reference answer sets, annotation guides, and evaluation datasets.


* Run expert reviews with scientific, clinical, regulatory, medical, data science, and engineering teams.


* Test LLM, RAG, and agent performance, including accuracy, source grounding, retrieval quality, citation fidelity, task completion, robustness, safety, and usability.


* Analyze failure patterns such as unsupported claims, incorrect reasoning, poor evidence use, missing uncertainty, weak traceability, or failure to follow instructions.


* Translate evaluation findings into improvements in prompts, retrieval methods, agent workflows, ...