Last month, our community news aggregator site BINUS Today faced several intermittent issues. While our site still works and accessible during the period, we haven’t noticed our issues with our article fetcher which is meant to run automatically four times a day!
Since Shift and I still have many things to do, we ran out of time to investigate and fix this ongoing issue. Sure, while our article fetching service is entirely based on GitHub Actions, we heavily rely on our private, self-hosted runners to avoid site-blocking and rate-limiting issues. And unfortunately, our self-hosted runners broke down.
That’s why I decided to look and help Shift to clean up the roothouse, which was expanding into six(!) individual houses. It’s the place where our private things are stored and automated, including this one.
And of course, the good news is that we’ve finally fixed the issue by ensuring every required PHP package is installed on every house, re-registering our self-hosted runners, and some upgrades to web hosting service. However, there might be some lost articles, especially BINUS TV’s JURNAL 19 archive, from our expanding database of news.
To commemorate this fix, we also decided to improve our BINUS TV video scraping process, which now also includes lesser-known BINUS TV programs including Zoom In, How To, and Movie Freak. This means that BINUS Today will eventually boot the infamous BINUS TV for Android app for an integrated “Knowledge” experience!
We also decided to fix how canonical URLs do appear on article pages, which we assumed is the root of this specific issue on Google Search Console:
Hopefully GitHub, Google, and the roothouse won’t be *that* hostile again to our service, so we can still continue to deliver the best “Knowledge” experience for BINUSIAN and everyone – and still even better than “Knowledge” on BINUS Mobile!
Leave a Reply