Research Engineer Position in Uncertainty Quantification of LLMs

Overview of the Role

By scaling up data, compute and model size, large language models (LLMs) have gained an impressive and ever growing array of capabilities. The next phase of development will be dominated by the use of LLMs in real world scenarios, and improving LLM reliability will be a crucial and exciting component of this next phase.

A postdoctoral research fellow position is available in a joint project across Oxford (Prof Yee Whye TEH), Nanyang Technological University (Prof Luke Ong) and National University of Singapore (Prof Wee Sun Lee) studying the reliability of LLMs through the lens of uncertainty quantification (UQ), Bayesian inference, conformal prediction, and world models.

Along with industrial and government partners, the postholder and project team will use these methods to address current issues with LLMs, such as reward hacking, active learning and testing of LLMs.

Reporting to Prof Luke Ong, the postholder will be a member of his research group in the College of Computing and Data Science, NTU, with responsibility for managing the computational infrastructure, enabling large scale research implementation and experimentation for the project team, and collaborating on research pertinent to the project.

Responsibilities and Duties

• Maintaining the project's internal communication infrastructure to ensure that researchers across all three universities are able to work effectively together.
• Collaborate with the project team on research pertinent to the project, by
  • contributing to and developing ideas for novel methodologies and understanding of the research questions,
  • acting as a source of information and advice to other members of the research team, especially on computational matters,
  • enabling large scale research implementations and managing research experiments,
  • contributing to the writing of research papers and submissions to high profile venues,
  • leading on efforts at open sourcing research code, and
  • presenting outcomes at international conferences and workshops.

Selection Criteria

Candidate must be close to completion or holds a MSc or above in a relevant technical subject (e.g. computer science, statistics, engineering).

The ideal candidate for these positions should

• possess sufficient specialist knowledge, research skills and interests in deep learning and large language models to work within established research programmes and contribute ideas and solutions;
• possess strong technical knowledge and abilities of deep learning and large language modelling frameworks (pytorch, jax, huggingface) as well as popular open-source codebases and LLM APIs;
• possess strong motivation and interests in developing computational infrastructure (including both software and cloud solutions) to enable large scale implementations and experimentations in large language models;
• possess strong collaboration abilities in person, on paper and over electronic channels; excellent communication skills, including the ability to write for publication, present research proposals and results, and represent the research group at meetings.

Applications and Enquiries

If you are interested, please send your CV and a brief cover letter to Prof Luke Ong (Email: luke.ong @ ntu . edu . sg).

To apply, click the red APPLY button on the webpage Research Engineer II (Computer Science/Statistics/Engineering). Please also email a copy of your application to the PI, Luke Ong (Email: luke.ong @ ntu . edu . sg).

This call was posted in January 2025. It will remain open until the position is filled. Only shortlisted candidates will be notified.