Back to list

Vacancy description

Senior Software Engineer - LLM Evaluation in Australia

Turing

  • Full-timeStandard weekly hours
  • Part-timeFewer hours each week
  • ContractFixed-term or freelance

Australia

Evaluate and refine AI-generated code for efficiency and reliability.

About Turing

Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L

Role Overview

As a Software Engineering evaluator, you will create cutting-edge datasets for training, benchmarking, and advancing large language models, collaborating closely with researchers. This includes curating code examples, providing precise solutions, and making corrections in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go; evaluating and refining AI-generated code for efficiency, scalability, and reliability; and working with cross-functional teams to enhance enterprise-level AI-driven coding solutions.

What does day-to-day life look like?

  • Working on AI model training initiatives by curating code examples, building solutions, and correcting code in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go.
  • Evaluate and refine AI-generated code to ensure that it is efficient, scalable, and reliable.
  • Collaborate with cross-functional teams to enhance AI-driven coding solutions against industry performance benchmarks.
  • Build agents that can verify the quality of the code and identify error patterns.
  • Hypothetize on steps in the software engineering cycle (prototyping, architecture design, API design, production implementation, launch, experiments, monitoring, operation maintenance) and evaluate model capabilities on them
  • Design verification mechanisms that can automatically verify a solution to a software engineering task.

Requirements

  • Several years of software engineering experience, including 2+ years of continuous full-time experience at a top-tier product company (e.g., Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal, IBM Research).
  • Strong expertise in building full-stack applications and deploying scalable, production-grade software using modern languages and tools.
  • Deep understanding of software architecture, design, development, debugging, and code quality/review assessment.
  • Excellent oral and written communication skills for clear, structured evaluation rationales.

Offer Details

  • Commitment : flexible engagement, minimum 10 hrs/week, up to 40 hrs/week (partial PST overlap required)
  • Type : Contractor (no medical/paid leave)
  • Duration :1 month (starting next week; potential extensions based on performance and fit)

Skills: JavaScript, JavaScript, Python

Similar vacancies

More like this role

  • Remote
  • Contract
  • IT
Analyze GitHub issues and evaluate LLM performance in software projects. Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, ...

Australia

  • Remote
  • Contract
  • IT

$100 - $150 per hour

... LLMs or AI coding tools Experience with LLM evaluation, coding agents, RLHF, preference data, rubric design, or post-training is a plus, but not required. Evaluation Process - AI interview (~25 minutes) - Practical code/AI evaluation exercise (~30 minutes) - Hiring manager interview (~20 minutes) The practical exercise ...

Australia

  • Remote
  • Part-time
  • Contract
  • IT

$80 - $120 per hour

... data is the hardest to produce and most valuable for advanced model performance. Our pipeline performance system ensures both quality and velocity. Job Title: Senior Software Engineer Job Type: Contractor (~15 hrs a week) Location: Remote (Globally) Job Summary: In this role, you'll apply your expertise to help train next-generation ...

Australia

Insignia Financial

Senior Software Engineer

  • Full-time
  • IT
... supports better financial advice experiences - Work across the full software development lifecycle in an Agile environment - Join a senior role focused on modern engineering, automation and continuous improvement The opportunity to join our team Join Insignia Financial as a Senior Software Engineer full-stack Java, within our ...

Perth, Australia

  • Hybrid
  • Full-time
  • IT

A$137.6 AUD

... both to enhance our existing MongoDB cloud storage architecture and to power more of our customers' use cases more efficiently. We are looking for talented Senior Engineers to join the team and be founding members of the team, where you will play a crucial role in our multi-year roadmap. Our team champions a strong culture ...

Syd, Australia

Topcon Positioning

Senior Software Engineer

  • Full-time
  • IT
... agriculture solutions that help farmers increase productivity, improve sustainability, and make data-driven decisions every day. We're looking for an experienced Senior Software Engineer to join our Adelaide-based engineering team and help shape the next generation of innovative agricultural technology used around the world. ...

Adelaide, Australia

  • Hybrid
  • Full-time
  • IT
  • Legal
About CORTO We are CORTO, a cutting-edge software company dedicated to revolutionising the legal industry. Our mission is to empower legal practitioners with AI-driven solutions that streamline their workflow, boost productivity, and provide more efficient client service. Our team of AI experts and engineers collaborate ...

Syd, Australia

  • Remote
  • Hybrid
  • On-site
  • IT
... Communications System (DHFCS), a critical capability connecting Australian Defence Force command centres with deployed aircraft, ships and land forces. As a Senior Software Engineer, you will lead and deliver software maintenance activities and engineering changes, developing and maintaining software components and associated ...

Keswick, Australia

  • Hybrid
  • Full-time
  • IT
  • Legal
About CORTO We are CORTO, a cutting-edge software company dedicated to revolutionising the legal industry. Our mission is to empower legal practitioners with AI-driven solutions that streamline their workflow, boost productivity, and provide more efficient client service. Our team of AI experts and engineers collaborate ...

Syd, Australia

  • Full-time
  • IT
About the Role Our Senior Software Engineers work closely with other members of the engineering team to scope, design, document, develop, integrate and test software to address end-user needs for a large web-based application. Responsibilities - Designing, implementing and testing software components in Java - Design and ...

Adelaide, Australia