Related skills
aws sql python scala airflow๐ Description
- Design and build batch/streaming data pipelines for ingestion and analytics.
- Own end-to-end delivery of complex data initiatives: architecture, deployment, testing, monitoring, and docs.
- Evolve the data platform for large-scale processing using cloud-native tech.
- Automate data operations: validation, quality checks, backfills, recovery workflows.
- Optimize cost, performance, and reliability of data workloads.
- Partner with Data Science, Product, and Engineering to translate requirements into solutions.
๐ฏ Requirements
- 5+ years hands-on data engineering building production-grade data platforms.
- Python or Scala for data processing; SQL for analytics.
- Apache Spark with performance tuning.
- AWS data services (EMR, Lambda, S3) and data warehousing concepts.
- Production pipelines operations: monitoring, alerting, incident response.
- Strong software practices: version control, CI/CD, testing, code reviews.
Meet JobCopilot: Your Personal AI Job Hunter
Automatically Apply to Data Jobs. Just set your
preferences and Job Copilot will do the rest โ finding, filtering, and applying while you focus on what matters.
Help us maintain the quality of jobs posted on Empllo!
Is this position not a remote job?
Let us know!