26th Campus Recruitment - AI Crawler Engineer

This position comes from Liepin. Joining SiTech, you will:

1. Be responsible for the network collection, parsing, and cleaning of various data required for AI model training;
2. Design and implement efficient and scalable distributed crawler frameworks and data collection systems;
3. Develop customized scraping and anti-scraping strategies for different websites/platforms;
4. Optimize the speed and stability of data scraping to ensure the quality and completeness of data collection;
5. Collaborate closely with teams responsible for data cleaning, labeling, and modeling to ensure data formats and requirements are consistent;
6. Pay attention to data compliance and copyright issues to ensure collection activities comply with relevant laws and regulations.

If you are:

1. A graduate student from a recognized national institution, majoring in Computer Science and Technology, Software Engineering, Data Science, or related fields;
2. Proficient in Python, familiar with scraping frameworks such as Scrapy, Selenium/Playwright, PySpider;
3. Familiar with parsing technologies such as HTTP/HTTPS protocols, HTML, XPath, CSS Selector, regular expressions; have experience dealing with anti-scraping mechanisms (such as CAPTCHAs, IP blocking, dynamic loading);
4. Familiar with common databases (MySQL, MongoDB, Redis) and optimization of storage and queries;
5. Preferably with experience in large-scale data collection and design of distributed crawler systems;
6. Possessing good learning ability, communication and collaboration skills, and data security awareness;
7. Having AI thinking and practical ability, able to proficiently use AI tools to improve R&D efficiency, and understand the boundaries of AI technology, participating in the implementation and iteration of AI-related requirements.

Welcome to apply to SiTech. Please ensure to refer to official channels (official website and public account information) for recruitment information, and do not download paid apps casually to avoid scams.