Senior Software Engineer

Added
13 days ago
Type
Full time
Salary
Salary not provided

Related skills

selenium python airflow playwright scrapy

๐Ÿ“‹ Description

  • Design, develop, maintain scalable web scraping solutions
  • Build data pipelines and automation for collection, cleaning, validation, transformation
  • Prepare scraped data into MDR-ready formats meeting quality and timelines
  • Monitor scraping jobs; handle anti-bot measures, CAPTCHAs, rate limits
  • Collaborate with teams to define data requirements and sources
  • Document data schemas and scraping processes

๐ŸŽฏ Requirements

  • 8+ years of professional experience in web scraping, data extraction, or data engineering
  • Strong proficiency in Python with scraping libraries (Scrapy, BeautifulSoup, Selenium, Playwright, or similar)
  • Experience building and scheduling automated data pipelines (cron, Airflow, or equivalent)
  • Solid understanding of HTML, CSS, DOM, and browser dev tools
  • Familiarity with REST APIs, JSON, and data extraction from API endpoints
  • Experience with relational databases (PostgreSQL, MySQL) and SQL
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Data Jobs. Just set your preferences and Job Copilot will do the rest โ€” finding, filtering, and applying while you focus on what matters.

Related Data Jobs

See more Data jobs โ†’