Revolutionizing Recruitment: The Story Behind Building an Automated Headhunting System with AI & LinkedIn Scraping

Finding the ideal candidate for a critical role is no easy task. The long and tedious manual process remains a classic challenge for every HR team. But what if you could automate the entire process — from reading job descriptions to matching them directly with LinkedIn profiles, complete with a compatibility score and explanation?

Introducing an intelligent, end-to-end headhunting system — and in this article, we’ll dive into how it was built.

Workflow Overview of the Automated Headhunting System with AI & LinkedIn Scraping

The main goal of this system is simple: to help HR professionals find the best candidates in an efficient, consistent, and data-driven way. It starts with a PDF job description and transforms it into an automated talent hunt mission.

How Does It Work? A 4-Step Journey

Rather than relying on a manual process, the system follows a carefully designed workflow:

1. Job Description Analysis
   It all starts when a user uploads a job description PDF via the web interface. Behind the scenes, Google Gemini reads and analyzes the document to extract core requirements such as skills, experience, and qualifications.

2. Scraping LinkedIn
   Armed with this understanding, the system automatically generates search queries. The key here is a “polite” scraping technique. Instead of accessing LinkedIn directly, it uses Yahoo Search with the filter site:linkedin.com/in to safely find public profiles — a trick to avoid being blocked.

3. Profile Parsing
   Each relevant LinkedIn profile is then “captured” or screenshotted. These images are passed back to Gemini, which uses OCR (Optical Character Recognition) to extract crucial information from the text — such as work experience and skills.

4. Scoring & Ranking
   The extracted profile data is compared against the job requirements. Gemini acts as the judge, scoring candidates from 0 to 10 and — most importantly — providing a narrative explanation of why a candidate is a good (or poor) fit.
   
   

The System Is Powered by Key Technologies:

• Google Gemini

• LinkedIn web scraping via Playwright

• Resume parsing from images using OCR & AI

• Flask for the web interface

• LangGraph for modular workflow orchestration

Once the process is complete, the user is presented with a clean, interactive web interface containing:

• A summary of the job description

• A list of key candidate requirements

• URLs of the LinkedIn profiles found

• Top candidates with their AI-generated scores and explanations

This project is living proof of how modern technologies — generative AI, web scraping, and workflow orchestration — can come together to build powerful solutions. It’s not just about automating repetitive tasks, but about empowering HR teams to focus on what matters most: engaging with the best talent.

 

 

Interested in building a similar solution using AI, cloud, or custom applications? Contact Radya Labs at contact page to explore how we can collaborate. If you’re an SME looking to automate your customer service quickly and efficiently, check out our omnichannel platform at jangkau.ai.