EMPLOYMENT

Senior Data Engineer: Firewall Automation Program Contract: 6-12 months (Pty Ltd only) $1,150-$1,200/day GST Sydney based (Hybrid) Urgent start - interviewing immediately About the role Build MS SQL ETL pipelines that consolidate 5 million network flows down to 5,000-10,000 automated firewall rules for a tier-1 bank's CoreMod migration. You'll enrich NetFlow data with CMDB/DNS/VMware metadata, produce JSON/CSV deployment feeds for APIC/CSM, and optimise performance at scale. This role suits a data engineer who thrives on genuinely interesting technical problems in collaborative mob programming environments. About the engagement Major infrastructure transformation migrating 400 applications across ~3,000 servers. Three-person specialist team (you, Python dev, network engineer) automating firewall implementation across Legacy and CoreMod infrastructure. Fast-tracked delivery with high autonomy and creative problem-solving. What you'll own Data Pipeline: Consolidate 5M network flows stored in MS SQL into contextualised, validated firewall rules using high-performance T-SQL ETL jobs with deduplication, enrichment and sliding-window aggregations. Deployment Feeds: Produce controller-ready JSON/CSV exports conforming to CSM import formats and APIC object model payloads that enable the Python dev to automate rule implementation via Cisco APIs. Orchestration: Create Tidal job definitions, retries, alerts and exception queues. Implement lineage tracking from aggregated tuples back to raw flows for audit purposes. This is collaborative data engineering - mob programming setup where all three specialists work in lockstep. You'll optimise partitioning, indexing and query plans for very large NetFlow tables while navigating access problems and data challenges with a proactive mindset. What we need Must have: Expert MS SQL Server (T-SQL, indexing, partitioning, window functions, query tuning), high-performance ETL pipelines for large-scale datasets (5M rows), controller-ready export formats (JSON/CSV) for network automation platforms, Tidal or similar orchestration tools, data engineering mindset (NOT DBA), collaborative team player, Pty Ltd contractor. Nice to have: Dask/Spark experience, Power BI, understanding of network flows/NetFlow data, migration project experience. Won't work: Pure DBA backgrounds, expecting cloud migration work, needing extensive banking experience. The opportunity Real infrastructure impact enabling automated firewall deployment at enterprise scale. Strong likelihood of 12-month engagement.