EZZY JOB
Problem
Job hunting? It’s like trying to find a needle in a haystack, only the haystack is on fire, and the needle is also seeking a job in engineering. So, I put on my hacker hat, whipped up a script to yoink job ads from Indeed.com every now and then, and now I’ve got my own digital sidekick spoon-feeding me the scoop on who’s hiring. It’s like having a cheat code for the job market game!
How I did that
Just holler at ChatGPT, and it’ll walk you through it step-by-step. But hold your horses, it’s not exactly a live feed, is it? You’ve got to roll up your sleeves and dive into the HTML jungle from time to time, since these sites are always upping their game to keep scrapers at bay.
Stack
- Selenium
- Pandas
- HTML&CSS
- Grafana
- PostgreSQL
If my script ever hits the jackpot and lands me a job, I’ll slap it up on GitHub for the world to see – the link will be shining bright on my homepage! I’m not saying it’ll shake the earth, but it sure adds a sprinkle of Zen to my life, giving me that eagle-eye view over the job market. Let’s wait and see what unfolds…
Dashboard Preview
Belows are the screenshots of the dashboard I created for this project. As you can see, this would be a great tool for job hunters to keep track of the job market.
Don’t you think it’s a great idea to figure out which state is hiring the most? I’m sure you do! This would help you to make a better decision on where to apply for a job and given this market is so competitive, you need to be smart about it.
Then how do we figure out what companies are hiring the most? Here we will just add a word cloud to the dashboard and you can see the most frequent words in the hiring companies’ names. Along with the company names, I also have a word cloud for the job titles. This would help you to understand what kind of jobs are in demand in the market.
Cool, now we can finally skip the boring part of job hunting sitting in front of the computer and browse every page of a job posting. Instead, we can just focus on the jobs that are in demand and apply for them.
How does this automate the process?
In order to automate the process, I have set up a cron job that runs every day at 8 am and scrapes the job postings from Indeed.com. Then it stores the data in a PostgreSQL database. After that, I have set up a Grafana dashboard that connects to the database and visualizes the data. This way, I can see the job market trends every day without having to do anything manually. But here comes the problem, even though I have set everything up to scrape the job process, what can we do to make the job application process easier and faster? Well, I have a solution for that too. We can talk about that in the near future. But still I will attach a video of how fast the process is and how easy it is to apply for a job after you have set up some magic!
Automated Job Scrapper as of 2024-04-22
Note: This video is just a demo and it doesn’t actually apply for a job. It just shows how fast the process can be if you have set up the magic.
Epilogue
This is just one of many cases that Grafana can bring value to your life. I’m sure you can come up with a better idea than this. So, why don’t you give it a try and see what you can come up with and let me know?
