Transport for NSW is the lead agency of the NSW Transport cluster. Our role is to lead the development of a safe, efficient, integrated transport system that keeps people and goods moving, connects communities and shapes the future of our cities, centres and regions. We work with several government agencies to coordinate road, rail, bus and maritime services, always keeping the safety and comfort of our customers at the heart of what we do.

 

About IT at TfNSW

TfNSW’s IT group provides the tools, information and services that enable 30,000 employees to deliver a smart and efficient transport network. We bring the future to our people by delivering innovative solutions and minimising business disruptions, building an engaged and connected workforce through tech-enablement. This is an exciting time to join the team, with a committed annual investment of $700m for progressive technology projects including bots, apps, AI solutions, autonomous 3D mapping drones, cybersecurity and transforming cameras across our network.

 

The opportunity

We have a new opportunity for a Senior Data Engineer, permanent full time role within IT division. This role has the responsibility for the provision of all maintenance tasks, modifications, improvements, cleansing, and manipulation of data from divisional and other databases.

 

Your role will include:

• Create and maintain the implementation of an optimal data pipelines from a wide variety of internal and external sources into a large cloud environment.
• Work with genuinely large datasets and implement against reference designs to meet the functional requirements of the cluster, branch and consumers.
• Identify, design, and implement internal process improvements: automating manual processes, streamlining data delivery and re-designing infrastructure for greater scalability and elegance.
• Build and support the infrastructure as code required for the ETL / ELT (extract, load, transform), streaming of data into a data warehouse and data lake including SQL and a range of native ‘big data’ cloud technologies.
• Keep the virtualised separation of cloud environments and play a strong role in contributing to protecting the data inside a secured perimeter.
• Create of data pipeline, data tools and scripts for Analytics, Business Intelligence, Data Science and AI teams to assist in building highly optimised products ready for a production environment.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Actively apply expertise in SQL and provide support in RDBMS design, data flow and analysis activities, and troubleshoot data issues and present / implement the solution.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• Ensure that Data Governance is embedded at each point of the data management lifecycle and is a key consideration of all parts of any implementation or data service.

  

About you

We are looking for a hands-on Senior Data Engineer with extensive experience in ‘Big Data’ suite, data modelling, data enginnering, orchestrtaing ETL pipeline techniques, a guru in SQL database design and has worked with large, complex data sets.

 

To be successful in the role you will

• Extensive experience with the ‘Big Data’ suite of tools preferably Azure.
• Previous experience as a data engineer or similar role.
• Experience with Cloud native ETL tools.
• Technical expertise with data modelling, data engineering, data transformation and orchestrtaing ETL pipeline techniques.
• Knowledge of programming scripting languages (e.g. SQL, Python, Spark etc).
• Demonstrable expertise of SQL database design

 

Salary and Benefits

The salary for this position is TfNSW Grade 9 ($138,670.00 - $155,309.00) per annum plus superannuation and leave loading.