Related skills
selenium postgresql mysql python airflow📋 Description
- Design, develop, and maintain scalable web scraping solutions for diverse sites
- Build robust data pipelines for collection, cleaning, validation, and transformation
- Prepare scraped data into MDR-ready formats meeting quality
- Monitor and troubleshoot scraping jobs; handle anti-bot, CAPTCHAs, rate limits
- Collaborate with teams to define data sources and scraping specs
- Document scraping processes, data schemas, and decisions for continuity
🎯 Requirements
- 8+ years of experience in web scraping, data extraction, or data engineering
- Strong Python skills with Scrapy, BeautifulSoup, Selenium, Playwright
- Experience building and scheduling automated data pipelines (cron, Airflow)
- HTML, CSS, DOM understanding and browser dev tools
- Familiarity with REST APIs and JSON for data extraction
- Experience with PostgreSQL/MySQL and SQL
🎁 Benefits
- Cloud platforms (AWS/GCP/Azure) for deploying and scaling scraping infra
- Docker and CI/CD pipelines
- Data transformation tools or ETL frameworks
- NLP or AI-assisted data extraction techniques
- Experience with education or institutional data
- NoSQL databases (MongoDB/Elasticsearch) for semi-structured data
Meet JobCopilot: Your Personal AI Job Hunter
Automatically Apply to Data Jobs. Just set your
preferences and Job Copilot will do the rest — finding, filtering, and applying while you focus on what matters.
Help us maintain the quality of jobs posted on Empllo!
Is this position not a remote job?
Let us know!