On Measuring the Unmeasurable

Intelligence and Security Culture: The Challenge of Defining the Theoretical

I've seen a lot of posts lately about what security culture really means, and I've seen many criticising its use as just another buzzword in a vendor-driven industry, or leveraging it to imply a lack of understanding or knowledge in others. I was left with an overwhelming feeling that as an industry, we’re getting lost in details that don’t really matter.

Actually, there are plenty of things we understand, measure, and see making an impact, even if we don’t have a perfect universal definition for them, and I believe we can (and should) look to these examples when we discuss security culture. 

With this in mind, let me ask you a question: What is intelligence?

Intelligence: measuring a theoretical construct

For as long as humans have been able to think critically about our own abilities, we've struggled with the concept of intelligence. Despite centuries of debate, no single definition of intelligence satisfies everyone. Is intelligence a matter of raw cognitive power? Or is it about the ability to adapt and learn from experience? Or is it the ability to solve problems? And, just as important, how do we measure something so inherently abstract?

Theoretical concepts are not new to us. Intelligence, in all its complexity, has been debated by philosophers, psychologists, and educators for centuries. It encompasses a range of attributes—problem-solving skills, creativity, adaptability, emotional awareness, and more. Yet, despite all the scholarly attention, intelligence remains somewhat elusive. We understand its importance, but defining it in concrete, universally accepted terms remains nearly impossible. It is ultimately a theoretical construct.

The struggle to define security culture

Security culture faces a similar dilemma. Much like intelligence, security culture is essential, widely discussed, but frustratingly difficult to define in a way that satisfies everyone. We know it's crucial for resilience, but articulating exactly what it is—and how to measure it—remains an ongoing challenge that’s frustrating a lot of us in the industry.

In reality,  I believe we are fighting a similar battle as with intelligence. We all agree it's crucial, lots of people discuss it, but if you asked 100 cybersecurity professionals what it means, you could well end up with 100 different answers. Is it about behaviour? Attitudes? Beliefs? The understanding of risks? Or is it a combination of this and others?

In the practical world we've found ways to work around this lack of a universal definition for intelligence. We’ve created measurements for intelligence that help us to scaffold and talk about it—IQ tests being one of the most popular (although don’t get me started on those) —based on certain agreed-upon aspects. While no test perfectly captures what it means to be "intelligent," they give us a benchmark, a way to quantify and track something inherently subjective that we know exists, we feel its impact, but we can’t quite grab it.

Measurement as the first step to definition

Security culture impacts the overall success and resilience of an organization. Most can agree that an organization can have a "good"  or “bad” security culture, as well as a “strong” or “weak” one (if you’re wondering what this is, keep an eye for our next blog). But how do we measure that? And importantly, how do we improve it if we can’t even define it clearly?

Much like the IQ test provided a framework for evaluating intelligence, we need a similar approach for understanding security culture. Without clear metrics or baselines, organizations can’t effectively gauge their current state or identify areas for improvement. So, how do we start tackling this?

A good place to start with any theoretical concept is to start defining through acknowledging its tangible, measurable components. The good news for the industry is that this step is well underway, with The Security Culture Report (2017) (link) offering an early framework to build on. While not perfect, it has laid the groundwork for further development. We might not yet have all the tools to measure every aspect of security culture, but we can leverage existing data points—behavioural insights, incident reporting trends, and training outcomes—to shape this framework. The framework is not static; it evolves and adapts over time as our understanding deepens and new methodologies emerge.

Importantly, just as intelligence cannot be summed up by a single test, measuring only one or two elements of security culture misses the bigger picture. For example, assessing my intelligence based solely on spatial intelligence assessment would paint an incomplete, and might I say, misleading picture. The same applies to security culture: we need to combine varied data sources to form a more comprehensive view. The goal isn’t to achieve a perfect definition but to build a reliable framework that guides better understanding and actions, just as we do with intelligence.

Why it matters

The importance of both intelligence and security culture lies in their ability to drive real-world outcomes. An intelligent individual can navigate complex problems, adapt to new challenges, and innovate solutions. A strong security culture ensures that an organization is resilient, prepared for threats, and capable of minimizing human risk.

In both cases, without understanding the theoretical underpinning, we risk falling into the trap of valuing surface-level behaviours or easy metrics that don't tell the whole story. And much like with intelligence, improving security culture requires understanding not just where you are now, but where you need to go and how to get there. There are approx. 200 IQ tests in existence, measuring various forms of intelligence. And even if you took all of them, you’re still only left with an estimate of something that, if we relied on definition alone, doesn’t exist.

Moving Forward

At Praxis Security Labs, we understand that defining and measuring security culture isn't a one-size-fits-all solution. It requires a thoughtful approach that considers the nuances of each organization. Just as intelligence can't be boiled down to a single test, security culture can't be reduced to a simple checklist. But with the right tools and metrics, we can start to quantify and improve it in meaningful ways.

Let’s stop thinking about security culture as something too vague or abstract to manage. If we can measure intelligence—an ever changing, theoretical concept—then we can absolutely measure and refine security culture, which is the same. We just need the right framework to get started. Let’s start measuring.