First web scraper ✔️

Call me 2014 while everyone is into crypto, AI, ML, ... but holy moly, it is actually done - I can't believe it.

Well, not really done, but what the heck, I built my first scraper. Time to give myself a clap on the back.


I can't recall when I started learning about coding the first time. Potentially 8 years ago maybe. Oh, actually 17 years ago, when I was 14 or so in high school. The high school was actually - in retrospect and in this aspect - progressive and we did study R back then. Not that I remember a single command of R, but that's not the point.

After that I guess 8 years ago with Codecademy back when it was I think completely free. Every since I did the classic HTML and CSS classes, started Python ~4x and "finished 2x. SQL, also done! Think I have 22% of JavaScript. So basically, hire me as your new senior dev!

I can't build shit. And also, for my job specifically I would not need to. I think product manager or growth folks don't need to be able to code. Yes, an understanding is good and you should at least "get" it to some extent. Not only to be able to tell when devs are bullshitting, but rather to truly understand their perspective and why your teammates think something takes a longer time.

However, especially over the last two years, the more and more also the no code movement grew, I felt I want to be able to do more myself. Zapier is awesome, Airtable is awesome, others are awesome. Also, dev resources are limited. But not only from a professional perspective, also for personal projects (that might become something, maybe not), I just personally wanted to be able to get data from a website. And today, well, I did it!

For what you ask? Some examples:
1. When I was trying to find the cheapest connection from Germany (3 airports) to different countries in Central America (bc hadn't yet decided where I would go) and with flexible traveling dates => find me the cheapest option
2. calorie trackers out there are hideous. give me the data and let me build a nicer one => would still need a developer for this, don't want to get that much into it;)
3. i am looking for a new apartment => send me a notification (email whatev) when there is a new apartment online that matches my criteria
4. ...

...


...well, maybe that's it. Idk, but feels good to be able to automate stuff. Not only scraping but over time entering/pushing information and also when data is being pulled how to analyse it and build dashboards. Because that is actually something I missed being able to do. Extract data from different APIs/data sources, bring it together into a database and analyse it.


One step at a time.