We service approximately 1.2 million users daily, which between them produce 120 to 220 million events. Our cluster is currently a bit over 870TB. Our biggest event log generates around 120GB of data daily (without replication).

In the ad serving industry, URL targeting is an effective mechanism that enables advertisers to specify URLs (web pages) on which their ad can be served. Subject of targeting can be specific URL or group of URLs.

Malicious URLs are a common and serious threat to cybersecurity. There are many ways for malicious attackers to try to cheat end user such as hacking attempts, drive-by-download, denial of service, phishing, social engineering, and many others.

Historically, our teams were organized as separate products, each with its own technology stack, although the stacks were very similar for some teams. Now we are unifying our platforms, tools, and languages so that we can share knowledge and reuse services between teams in a better way.