Confronting Security: A Post-Mortem
Confronting Security: A Post-Mortem
What growth leaders can learn from the Safary exploit (November 17, 2023)
tldr:
Safary’s platform experienced a sophisticated cyber attack, exploiting our tracking script to infiltrate Trader Joe’s frontend.
Frontend attacks are becoming increasingly common. In the 2 weeks since this attack, several other DeFi platforms have fallen victim to frontend attacks incl. SpookySwap, Aerodrome, and Velodrome (none of which are Safary users).
Summary of Exploit
Safary is a marketing attribution platform enabling web3 teams to measure which of their marketing campaigns are driving revenue on-chain. Our platform uses a tracking script placed in the frontend of our customers’ websites to attribute web3 conversions to marketing channels.
Around 9:30 pm UTC, November 17, 2023, Trader Joe announced they had a potential frontend vulnerability. 20 minutes later, Safary’s founding engineer noticed our app displaying weird behavior & redirecting to Trader Joe (Safary platform user)’s website.
To protect users, Trader Joe quickly took down their frontend, neutralizing the attack; and Safary informed all platform users to remove Safary’s script to be 100% safe while we investigated. The attack on Trader Joe lasted roughly 3.5 hours, during which $100K+ was stolen by the hacker.
After a thorough investigation by external security experts, our team alongside Resonance Security has identified that our domain (tag.safary.club) was likely the target of a DNS takeover, where DNS records are manipulated to redirect users toward a malicious server.
In this case, Safary’s tracking tag was temporarily redirected to a server hosting a malicious JavaScript script. The malicious script was purportedly designed to mimic Trader Joe’s website, altering the smart contract addresses to intercept token swaps from users.
In an effort to be as transparent as possible, we want to openly share our findings from the attack with the web3 growth community so that we can all learn and protect against future security threats for the companies and users we serve.
Timeline of Events
2023-11-16 23:22 UTC: The attacker created two carbon copies of Trader Joe’s website (trader-joe.pages.dev and dex-joe.com), then these fake websites began testing Safary’s tracking script using Trader Joe’s product ID.
2023-11-17 18:31 UTC: The attacker deployed 0x887ed0fd4dd331d56c38bade06040838f07045af, the smart contract that was used in the traderjoexyz.com frontend instead of the legitimate router addresses, signaling the start of the attack.
2023-11-17 18:47 UTC: The first evidence of a malicious transaction
2023-11-17 21:34 UTC: Trader Joe posts that they have a potential security alert
2023-11-17 21:56 UTC: Safary’s founding engineer noticed that our web app was acting in a weird manner, and the engineering team began investigating. The app was trying to redirect somewhere and ended up rendering our 404 page. He left the problematic page open, enabling us to identify the malicious script.
2023-11-17 22:04 UTC: Trader Joe announces they took down their frontend and that investigations were underway. Since that time, no further signs of the attack were observed.
2023-11-17 23:57 UTC: Trader Joe sent Safary a message forwarded from Sphere Finance stating that their website had redirected to Trader Joe, suggesting Safary may’ve been exploited.
2023-11-18 02:17 UTC: Safary cofounder messages all Safary customers, urging them to remove Safary’s script to be 100% safe while Resonance Security investigated the attack.
Financial Damage
The attacker stole $100K+ from around 100 Trader Joe users during the 3.5 hour attack.
Root Cause
We found evidence that the attacker built two carbon copies of Trader Joe’s website and tested their attack before launching it via these two malicious sites. This was a sophisticated, premeditated attack on Trader Joe.
Nginx backend logs revealed a suspicious variable coming from one of the malicious sites (dex-joe.com) calling the legitimate Safary tag service script of Trader Joe, indicating potential automated actions or misconfigured payloads for taking over DNS or for testing the DNS attack.
Subsequent logs indicated changes of the variable to an unknown direct Cloudflare UUID, suggesting an evolution in the attacker’s methods.
The correlation between the malicious site’s Cloudflare hosting and the anomalies in nginx logs implies that the attacker likely exploited DNS services. The attackers could have manipulated DNS records, leading to the redirection of legitimate traffic to the falsified Trader Joe frontend.
Actions Taken & Next Steps
Since the exploit, we’ve taken numerous actions to strengthen Safary’s security:
We partnered with Resonance Security who conducted an end-to-end forensic analysis of Safary’s supply chain (AWS, containers, Github, tag service, DNS, etc). The investigation found no evidence that Safary’s infra was compromised and that the likely culprit was DNS manipulation.
We transferred our domain out of the exploited DNS registrar, and our team is working on Resonance’s action plan which includes advanced security work on our DNS records.
We took a deep look into every part of our system, and we’re implementing strategic defensive enhancements to bolster security practices across our entire infrastructure. We’ll also undergo regular security audits by external firms to ensure state of the art security.
This week, we’ll release a new version of our tracking script that users can self-host. It contains several security upgrades and grants our customers full control over their own security, thus minimizing the attack surface on their web3 frontends.
While comprehensive smart contract and protocol level security audits have become the norm, this incident serves as an important reminder that frontend exploits can be equally dangerous. In the two weeks since this attack, several other DeFi platforms have been targeted in frontend attacks including SpookySwap, Aerodrome, and Velodrome (none of which are Safary users).
As frontend attacks become more common in web3, it’s clear that attribution providers and other frontend SDKs must undergo regular security audits to stay up-to-date when implementing best practices across their entire infrastructure. In the coming weeks, we’ll share more on the steps we’ve taken to increase our operational security and how we’re helping others do the same.