Yesterday I received a breach notification stating a SaaS provider I use “potentially” had a breach and I found myself asking “has it been a week already?”. Still burning from the DoorDash breach which leaked my email, phone number, physical address and shameful late night ordering habits along with 4.9 million others, it turns out that Yet Another SaaS Breach has effected me. This time, it is a company called ConferenceBadge.com, a small SaaS out of Quebec that many conferences, including my own BSides Vancouver, use to print conference badges. I will humbly brag, I have typically come out fairly unscathed on many of the recent SaaS breaches. That is because though I am a geek at heart, I practice certain luddite-isms and are usually pretty hesitant to sign up for the latest and greatest SaaS or mobile app. That practice has kept my data less exposed than the average netizen, but clearly even luddites like myself are not immune.
Quick Debrief
So why are SaaSes getting breached so frequently? I think the most obvious answer is: Cloud. Now, if you have kept up with my writings you know I am actually a big proponent of Cloud adoption and argue “Cloud is more secure… If you let it”. However, my thesis is that Cloud has all but eliminated the barriers to entry in becoming a software company. Year’s ago a brilliant software idea required major capital and infrastructure to get into the market. Now anyone with a credit card can spin up an enterprise grade infrastructure in a matter of seconds and a nifty minimum viable product (MVP) in a matter of weeks. This has had a great net benefit to enabling countless startups to exist, however it has also enabled startups with a decent looking brand to convince consumers they are as mature, trustworthy and regimented as a traditional software company.
However, upwards of 80% of SaaSes and mobile apps are run by Ma and Pa dev shops. These shops typically run lean, with short funding runways and are focused on building an MVP as soon as possible. This development velocity means privacy and security must take a back seat during the stage of development when they are most critical, leading to flawed software and the inevitable security breach.
What Happened?
ConferenceBadge.com has been operating a fairly convenient service since 2013, allowing event organizers who use EventBrite to seamlessly get professional looking badges printed and delivered within two days. On October 18th a security researcher discovered a publicly accessible AWS S3 bucket filled with ConferenceBadge.com customer data. The security researcher promptly notified ConferenceBadge.com who disabled the public access that same day. On October 21st, ConferenceBadge.com notified their clients that a “potential” breach of client data had occurred and that they were making efforts to not let this happen again.
What Went Right?
First things first is to give credit where credit is due. ConferenceBadge.com received the complaint, corrected the issue and notified their customers in a very timely manner. From a timing and a communications perspective, it was well within the expectations of PIPEDA and GDPR, though I am not sure if they had notified any privacy commissioners.
What Went Wrong?
Now, for the bad.
Not Following Cloud Security Best Practices
The obvious elephant in the room is that this breach was entirely preventable. The “potential” theft of our data required next to no hacking skills and preventing this flaw from being implemented also required next to no Cloud skills. That is because the #facepalm of leaving an S3 bucket full of goodies publicly accessible is becoming the overly cliche reason for SaaS breaches. In fact, I am surprised why we still see these preventable breaches continuing to make the news considering the ample news articles, the literal warning AWS provides when you try to deploy a public S3 bucket as well as a plethora of available tools to retroactively detect them. It is clear that ConferenceBadge.com’s developers and/or Cloud Engineers were not aware or following Cloud Security best practices.
They Don’t Really Know If Our Data Was Stolen
You have probably noticed I keep referring to the breach as a “potential” data theft. That is because in the breach notification from ConferenceBadge.com they proudly stated that no usernames or passwords were compromised, but:
“it is impossible to determine if someone has downloaded these files with malicious intent”.
-ConferenceBadge.com
What this means is that they did not turn on access logs for the S3 bucket and thus have no record of who what where when how our sensitive data was access. While there is a cost to collecting these logs, in pennies, it is indicative again that whoever set up ConferenceBadge.com’s SaaS failed to follow the most rudimentary of best practice.
No Data Retention Policy
When I am involved with an incident response for a client, it is fairly common to not have all the logs I need and the usual reasoning is log retention is too expensive. In a data centre I would agree that log management/SIEM is not cheap, however in Cloud its pennies, and it is clear storage costs were never a concern for ConferenceBadge.com because they kept all client data since 2013. Now while there may be reasons to keep client data for extended periods, ConferenceBadge.com’s business model is a printing service for spot in time events, so why retain the data? Furthermore, with the introduction and maturing of privacy laws, retaining unneeded personal data is literally as Jim Balsillie stated it is like keeping plutonium.
“Data is not the new oil, it’s the new plutonium. Amazingly powerful, dangerous when it spreads, difficult to clean up and with serious consequences when improperly used”
-Jim Balsillie
You are probably starting to notice a bit of a theme here: ConferenceBadge.com was not following best practices and sadly, they failed to enable a free AWS S3 feature which would have automatically destroyed data after a certain period. Turning on this free feature would have greatly reduced the blast radius and minimized effected customers.
Notification Was Good But Not Great
ConferenceBadge.com did notify their clients fairly quickly about the “potential” breach. I truly do appreciate the timeliness, but considering the nature of their business, collecting Personal Identifiable Information (PII) from another SaaS (EventBrite), they failed to notify the real victims. As of this writing I am under the impression that the data subjects, which is essentially the true victims of this breach, have not been notified that their data may have been compromised. I am not sure if ConferenceBadge.com is expecting every one of their client’s to essentially notify every one of our clients, but this “Breach Inheritance” issue will be an excellent blog for the future.
Third Party Disclosure
Last but not least, due to what I can only perceive as a SaaS that needs some Cloud Security Expertise stat, ConferenceBadge.com would never have known that their data was vulnerable to and potentially exposed by “someone with malicious intent”. Lucky for them a noble security researcher who found the vulnerability and responsibly disclosed it to ConferenceBadge.com is the true hero. However, it is a sad reality that most breaches, especially in the SaaS space, will be discovered not by the SaaS company itself, but rather from a third party such as a user, researcher, law enforcement, the media or the hacker themselves.
What Can You Do to Avoid Showing Up in the News?
Cyber security is not a simple problem to solve and admittedly is challenging to measure success. However, just like car insurance you want it to be there when you need it. ConferenceBadge.com is a prime example of the problem with most SaaS and mobile apps today: privacy and security were not build into the design.
Below are a few tips that you can follow to better prevent a cyber security incident from impacting your business:
Increase Awareness of YOUR Legal Obligations
If you are collecting Personal Identifiable Information, you definitely need to up your privacy and security game. Current and maturing privacy laws are becoming a fact of life and violating them can result in brand reputational damage, massive punitive fines or if the US government has its way, jail time!
I would recommend at a minimum:
-
Understanding what data you are collecting
-
Understand and documenting how are you protecting said data
-
Developing a policy around retention and destroying said data when possible
-
Getting familiar with the various privacy laws and jurisdictions you may or may not be aware are relevant to your company
Following (Cloud) Security Best Practices
I get it, cyber security can be pretty complicated to implement in a standardized, consistent and measurable manner. However as I have argued in my previous writings, “Cloud is more secure… If you let it”. All this means is that Cloud provides better tooling and capabilities to do security better than traditional IT infrastructures, but you have to want to do it. It is obvious with SaaS breach after SaaS breach that developers and Cloud engineers need help.
So I would recommend thinking about the following:
-
Design systems with privacy and security as a foundational requirement
-
Acknowledge that developers by nature are not security professionals and need to be incentivized to slow down and do security better
-
Governance is boring to geeks but is a necessary evil to ensure standardized, consistent and measurable risk reductions
-
Leverage Cloud Security Experts as “swim buddies” for your engineering team and implement Security Operations tooling to increase situational awareness in your Cloud
-
Perform third party security and risk assessments to provide a check and balance to internal controls and processes
Prepare For Breach
Even the most mature Cloud adopters such as Capital One, have bad days… However, their incident response processes likely minimize the pain and duration of their recent breach. It is important to acknowledge that there is no 100% secure system and regardless of the security controls in place, expertise on-call and training your team has had, there is still the potential for a breach. Therefore it is important to develop an incident response plan. While NIST 800-61r2 is the gold standard for incident handling, it is a bit dated when it comes to Cloud and is over-kill for small and medium enterprise. That said, we should all have a plan and here are some helpful tips:
-
Start small by documenting key information such as environmental architecture, build books and contact information
-
Understand what you can detect and where there are visibility gaps that need to be remediated
-
Develop the culture of writing playbooks for both investigative/detective activities as well as containment and eradication activities
-
Define roles and document a process the team will follow upon detection of a potential incident
-
Practice Practice Practice! Hold semi-annual table-top exercise drills to test your team’s ability to respond to an incident and leverage failures to improve the process…Way better than learning in a real incident
By no means is the above a definitive list of what you need to do to have a solid incident response plan, but we need to start somewhere. Organizations that start small, run table-top exercises and/or leverage third parties to better prepare them for breach, stand a much better chance at detecting a small problem and preventing it from becoming a large one.
Conclusion
Software development never really cared about security, but they didn’t really need to as most of their software was protected behind the walled-garden (firewall) of yesteryear’s IT environments. However, we are heading steadfast for the future where firewalls and walled-gardens are fading into the rear-view mirror and our apps and data are being directly connected to the Internet. Hackers and cyber criminals are hungry and SaaSes and mobile apps that fail to take security seriously before they need to, will inevitably be the next headline.
If you use Cloud and want to take security more seriously, give me a shout.
COMMENTS