Related skills
azure aws python airflow kafkaπ Description
- Data Ingestion & Acquisition from APIs, RDBMS, files, real-time streams.
- Pipeline Development: Design scalable ETL/ELT pipelines.
- Cloud Deployment: Build data pipelines on AWS/Azure with Lambda, Glue, Step Functions.
- Database Architecture: Optimize storage engines (PostgreSQL/Redshift).
- Streaming Data Processing: Work with Kafka for high-volume streams.
- Workflow Orchestration: Use Apache Airflow to schedule/monitor workflows.
π― Requirements
- Active TS/SCI security clearance.
- Bachelor's or master's in CS, engineering, or related field.
- 10+ years of experience in data engineering or software development roles.
- Strong Python proficiency, including pandas, PySpark, FastAPI.
- Experience with AWS or Azure and cloud-native data tools.
- Proven experience building data pipelines using Kafka, Airflow.
π Benefits
- M9 Benefits: https://m9solutions.com/why-join-m9/#our-benefits
Meet JobCopilot: Your Personal AI Job Hunter
Automatically Apply to Data Jobs. Just set your
preferences and Job Copilot will do the rest β finding, filtering, and applying while you focus on what matters.
Help us maintain the quality of jobs posted on Empllo!
Is this position not a remote job?
Let us know!