This month, Fortune.com reported that TikTok’s web scraper — known as Bytespider — is aggressively sucking up content to fuel generative AI models. We noticed the same thing when looking at bot management analytics produced by HAProxy Edge — our global network that we ourselves use to serve traffic for haproxy.com. Some of the numbers we are seeing are fairly shocking, so let’s review the traffic sources and where they originate.

HAProxy Fusion 1.3 is now available—making it even simpler to adopt modern, scalable application delivery. New custom dashboards, high-performance Kubernetes service discovery, and optimized workflows bolster HAProxy Fusion's observability and flexibility.

Standardized logging formats are important for teams that rely on logging for observability, troubleshooting, and workflow integration. Using structured formats simplifies parsing and eliminates the need to interpret fields manually, ensuring consistency across logging formats. This reduces manual work, prevents brittleness from unstructured logs, and simplifies integration between teams that feed logs into a shared aggregation system.