Software Development Engineer II - Bot Detection

Company Overview

CommerceIQ’s AI-powered digital commerce platform is revolutionizing the way brands sell online. Our unified ecommerce management solutions empower brands to make smarter, faster decisions through insights that optimize the digital shelf, increase retail media ROI and fuel incremental sales across the world’s largest marketplaces. With a global network of more than 900 retailers, our end-to-end platform helps 2,200+ of the world’s leading brands streamline marketing, supply chain, and sales operations to profitably grow market share in more than 50 countries. Learn more at commerceiq.ai.

Role: 

A key component of our Data Acquisition platform is the fleet of crawlers that we use to gather millions of web pages from hundreds of retailers every day. In order to gather data in a predictable and reliable manner, it’s vital that each of our crawlers is able to navigate a retailer’s website without being detected as being a bot.

The mission of our Bot Detection / evasion platform team is to provide tools and services that allow our crawlers to avoid detection and to gather data with high-levels of success. To achieve this, the team provides two key systems,  Firstly, a service to provide ‘good’ browsers, tailored to the needs and challenges of each retailer. Secondly, a library of captcha-solvers that help handle scenarios where detection is harder to avoid. We are looking for a highly skilled and technically curious software engineer to lead the team as we embark on the development of these vital systems.

What You will do: 

• Lead a team of specialist backend software engineers in the development of new services to help avoid bot detection systems and to solve captchas
• Analyse bot-detection systems, establishing how they work and how to avoid them
• Reverse engineer highly-obfuscated bot-detection JavaScript libraries, and determine exactly what actions they perform.
• Collaborate with other senior engineers in the Data Acquisition tribe to determine an architectural approach to incorporate detection-avoidance capabilities into our wider platform.
• Take responsibility for the health of the team; fostering a positive and collaborative culture
• Be responsible for all aspects of the team’s delivery and development; leading and championing agile processes, and driving continuous improvement.
• Balance hands-on development alongside team leadership responsibilities.
• Drive the growth and development of the team and its processes; exploring and championing new techniques, technologies and tools.
• Participate in engineering and architecture forums to help build a culture of excellence within the engineering organisation.

What You'll Bring: 

• 7-9 years of experience in software engineering.
• 4+ years of experience in leading small to medium-sized teams.
• Expertise in scalable, distributed systems with strong problem-solving skills.
• Very strong understanding of web technologies: HTML, DOM, HTTP, and CSS
• Extensive experience and a deep understanding of JavaScript
• Significant experience working with Linux/Unix
• Strong understanding and hands-on experience with observability, system networking, performance optimizations.
• Experience working with cloud-based technologies and development processes
• Familiarity with SDLC and agile methodologies
• Strong understanding of modern coding practices and standards
• A passion for delivering quality software
• Experience with web-scraping and its challenges
• Strong ability to navigate ambiguity, break down complex problems, and drive solutions.

Nice to Haves: 

• Experience with TypeScript
• Experience of building services in the cloud.
• Experience of containerisation using Docker and Kubernetes.
• Experience of building infrastructure as code, especially with Terraform.

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, disability status or any other category prohibited by applicable law.