Vacancy description

Data Platform Engineer in Brisbane, Australia

NCS Australia

  • ContractFixed-term or freelance

Brisbane, Australia



Make extraordinary happen

At NCS Australia, we’re the challenger alternative in technology services and consulting. With more than 47 years of Australian heritage, we partner with governments and leading organisations to solve complex challenges, embrace new opportunities and deliver outcomes that make a real difference.

Our people bring together expertise across technology, consulting, delivery, operations, sales and corporate functions. We work collaboratively, challenge the status quo and give our people the opportunity to make an impact — for our clients, our communities and each other.

Job Description



We are seeking a Data Platform Engineer with strong quality engineering and automation expertise to support the development, validation, and reliability of large-scale data platforms. This role will focus on Databricks Lakehouse, PySpark, SQL, distributed data pipelines, event-driven architectures, and cloud-based data ecosystems, while also supporting automated testing, data validation, CI/CD quality gates, and platform resilience.

The successful candidate will play a key role in ensuring end-to-end data integrity across complex systems, validating data flows between applications and data platforms, and supporting performance, scalability, and reliability initiatives. Working closely with cross-functional teams, the individual will leverage strong Databricks, Python, and SQL expertise to investigate data quality issues, validate platform behaviour, and contribute to the continuous improvement of automation and engineering practices.

This is a hands-on technical role suited to a data engineering professional who can operate independently, collaborate across multiple teams, and apply quality engineering principles within a modern data platform environment. While testing and automation experience are important, the primary focus is deep expertise in Databricks, Python, SQL, and data platform engineering.

Responsibilities

  • Design, develop, and maintain automated test frameworks and test suites.
  • Build and enhance quality gates within GitLab CI/CD pipelines.
  • Perform end-to-end testing of complex distributed data flows.
  • Validate data integrity across RabbitMQ, Databricks Lakehouse, Lakebase, and Elasticsearch platforms.
  • Conduct functional, integration, regression, and data validation testing.
  • Collaborate with developers, platform engineers, and fellow SDETs to ensure seamless system integration.
  • Coordinate cross-team testing activities across multiple application teams.
  • Design and execute load, performance, and scalability testing strategies.
  • Perform chaos testing and resilience validation of distributed systems.
  • Investigate and troubleshoot data discrepancies across multiple storage systems.
  • Contribute to continuous improvement of test automation practices and quality engineering standards.

Qualifications

  • Proven experience as a Senior Data Engineer, Data Platform Engineer, SDET, Data Quality Engineer, or similar technical role.
  • Strong hands-on experience with Python, PySpark, and SQL in enterprise-scale environments.
  • Demonstrated experience working with Databricks Lakehouse and large-scale distributed data processing platforms.
  • Experience validating and testing complex data pipelines, data integrations, and distributed systems.
  • Strong understanding of data quality, data validation, and automated testing methodologies.
  • Experience with GitLab CI/CD and implementing automated quality gates within deployment pipelines.
  • Experience working with Elasticsearch, event-driven architectures, and message brokers such as RabbitMQ.
  • Knowledge of Infrastructure as Code (IaC) tools, preferably Terraform.
  • Strong analytical, problem-solving, and troubleshooting skills.
  • Excellent communication and stakeholder management skills with the ability to collaborate across multiple teams.
  • Desirable

    Qualifications

    & Experience
  • Experience with performance, load, scalability, resilience, and chaos testing.
  • Experience working within cloud-native and modern data platform environments.
  • Exposure to DevOps and Site Reliability Engineering (SRE) practices.
  • Knowledge of C# and .NET technologies.
  • Experience working in Agile delivery environments.
  • Relevant certifications in Databricks, Cloud Technologies, or Data Engineering would be advantageous.

Additional Information



Why NCS?

This is a place for people who like to get stuck in, bring ideas to life and make things happen. You'll work alongside experienced people who care about what they do, contribute to meaningful work and have the support to keep learning and grow your career. Whether you want to deepen your expertise, explore something new or take your career in a different direction, there's room to make it your own. We value Adventure, Excellence, Integrity, Ownership and Unity - bringing curiosity, collaboration and accountability to the way we work with our clients and each other.

Ready to make extraordinary happen?

We'd love to hear from you.

We celebrate diversity and inclusion

We value different perspectives, experiences and strengths our people bring. We're committed to an inclusive workplace where everyone has the opportunity to contribute, grow and succeed, and to providing equal employment opportunities and reasonable adjustments throughout the recruitment process.

Important information

Applicants will need valid Australian work rights and may be required to undergo relevant background, probity and police checks.

Agencies: NCS accepts candidate submissions only from agencies on our preferred supplier panel through the NCS Agency Portal.

Similar vacancies

More like this role

  • Full-time
  • DevOps
... a Data Platform Engineer to report to the Engineering Manager for the Internal Data Platform team. This role suits candidates with experience building data platforms at scale and a passion for solving complex analytical problems with modern technologies. The successful candidate will have experience with big data systems ...

Syd, Australia

  • Full-time
  • Part-time
  • DevOps
  • Operations

A$130046 AUD - A$134956 AUD

- Department for Health and Wellbeing – Digital Health SA – Data and Analytics Platform Operations - Salary: $130,046 to $134,956 – ASO8 – Full Time / Ongoing – Permanent Overview: The Senior Data Platform Engineer – Data and Analytics Platform (DAP) Operations is accountable to the Specialist Operations Manager and is ...

Adelaide, Australia

  • Remote
  • Contract
  • DevOps
  • Consulting
... clearance About the Opportunity Powerdata Group is seeking an experienced Senior Azure Data Platform Engineer / Architect to support a major enterprise data platform uplift for a leading Australian financial services organisation. The engagement involves working within an established Azure data platform and aligned data ...

Syd, Australia

XPT Software Australia Pty Ltd

Data Engineer-Unified data platform

  • Contract
... cloud-native data engineering patterns and object storage-based data offload architectures (e.g. AWS S3, Azure Data Lake). Understanding of real-time decisioning platforms such as Pega, and how they integrate with data engineering layers. Exposure to MarTech or Transformation data programs in a large enterprise or telco context. ...

Syd, Australia

  • Remote
  • Hybrid
  • Full-time
We are seeking a highly skilled Data Engineer / Databricks Platform Administrator to join our Data & AI team. This hybrid role combines Databricks platform engineering and administration with hands-on data engineering delivery responsibilities. You will play a key role in establishing and operating a secure, scalable, enterprise-grade ...

Canberra, Australia

... across data platform, data product and governance initiatives. Required background: - A data-focused technical BA, engagement analyst or consultant, or an engineer / analyst who moved into the analyst seat, or wants to - Deep data understanding: how data moves, is structured, where it - comes from and what it means in ...

Syd, Australia

  • Full-time
  • Contract
  • DevOps
Looking for Platform Engineer - Gold Coast based - Coolangatta About the Role: Project Description: Data Platform / Snowflake Transformation - Requirement - The organisation is currently undertaking a large-scale data platform transformation, working alongside a third-party implementation/vendor partner. - They are looking ...

City of Cockburn, Australia

  • Remote
  • Full-time
... you to Go Bold in how you approach these problems, and to Go Fast without cutting corners. The Team And How They Connect Our Infrastructure team builds the platform foundations that Xero engineers rely on every day. Our engineers come from software, platform and infrastructure backgrounds, and we work closely with teams ...

Brisbane, Australia

  • Full-time

$100,000

... data engineering, data warehousing, ETL/ELT development, cloud data platforms, and technical delivery leadership. - Microsoft Certified: DP-900 Microsoft Azure Data Fundamentals. - Microsoft Certified: AZ-900 Azure Fundamentals. - Databricks Data Engineer Associate Certification. - Led modern data platform development using ...

Australia

  • Hybrid
  • Full-time
  • Contract
  • DevOps

$130,000 - $155,000

- SAP S/4HANA migration - Can be based in Sydney / Melbourne / Adelaide - Attractive salary / 12m FTC Data Platform Engineer – Snowflake Location: Sydney/Hybrid Contract: 12-month fixed-term Salary: $130,000 - $155,000 + Super An established Australian organisation is looking for a hands-on Data Platform Engineer to support ...

Syd, Australia