Foundations

Let’s start with some discussion about mission and work.

Mission: Reducing risks, increasing trust, and complying with governance.
Work: Balancing effort across business needs, operations, engineering, and incidents.

This section should help a new manager tackle existential questions about what security is trying to accomplish.

Let’s start by sharpening our sense of how we are contributing to the overall mission an organization is pursuing.

A mission for the security team

The overall organization’s mission is often simple. Often, this mission is succinct: just a sentence or two.

Security teams may define how they work in support of this mission with a mission of their own.

A mission is unlikely to spells out all of the laws, risks, ethics, and other constraints it operates within; To do so would weaken the mission’s purpose of simplicity and memorability. A simple, agreeable mission has the power to align a group of individuals towards a north star.

A security team typically aims to:

📉 Reduce risks.
📈 Increase trust.
✅ Comply with governance.

And best for last:

4. 🚀 Empower the organization!

I assume your overall organization has an ambitious mission. Hopefully, to change the world in a positive way. The decisions you make with risk, trust, and governance should have a positive effect toward that mission.

📉 Risk based missions hope to reduce the frequency and impact of undesirable scenarios.

These scenarios may injure the company or impose harm on others (e.g., customers). Our organizations, customers, or society can experience harm in a variety of forms.

📈 Trust missions aim to improve the perception of risk that others have of your organization.

Trust may improve the efficient pursuit of goals (revenue, customers, renewals, etc). Managing risk does not necessarily build trust. Customers can distrust an organization that manages its risks. Customers can trust a high risk organization on reputation alone. These strategies in isolation are not honest, desirable, or even efficient.

✅ Governance missions seek to adhere to laws, contracts, and regulations that may impose some level of requirements on how an organization operates.

These missions intermingle. All pursuit of these missions produces work for an organization. The concept of work is crucial to these essays. We’ll discuss how work occurs within each of these motivations by examining each idea individually.

🚀 Empowering missions create opportunities in new waters. A leap in security capability allows whole new prospects. For example, consider how empowered Amazon needed to be before they considered selling unused compute and storage (EC2 and S3) to anyone on the internet? Or, advances in cryptography allowing payment transactions over the internet?

Empowerment is when you eliminate a risk so thoroughly that a new idea can take it’s place.

📉 Risk

A risk is an undesirable future scenario with consequences. Here is an example:

An insider accesses our customer service tools and steals customer data.

Security teams are chiefly concerned with mitigating a complex constellation of risk scenarios like these.

A security team wouldn’t exist without risk. If a security team cannot describe the risks they are concerned about… we might question where they’ve gotten their original direction. Direction without any defined risks can lead to directionless organizations. So let’s start with mitigating risks.

We can build a foundational view of what an organization does and what motivates it. The reduction of risk should be a core principle for the mission of a security team. Risks generate the work a security team undertakes, and gives direction to trust and governance activity that we’ll describe soon.

Let’s work on our previous insider scenario. We might create particular work tasks to mitigate that risk. Here are some common examples:

Reduce the number of employees with access to production.
Centralize logging from production administrative tools.
Delete customer data after 30 days.

Assuming these are good ideas, the security team is now off and running doing security work. That specific work was derived from the risk scenario we used in our example. Those work tasks are based on risk.

📈 Trust

Trust is the perception of risk associated with our organization. All organizations hope to build trust in others: Customers, partners, consumers, etc.

Unique organizations are exposed to trust issues more than others.

A social website will see reduced engagement and adoption if there’s a belief that their privacy guarantees are not reliable.
A SaaS platform might not pass a security review if a customer believes the response to a security questionnaire is not representative of reality.
A hardware manufacturer might have reduced sales if customers believe that a backdoor exists in the product (whether there is one or not).

A security team can choose work that has the potential to build trust. Trust is a delicate subject that is often considered an intangible. KPI selection and precision measurement of trust is a difficult task.

Trust still appears in second-order measurements. Here are some example questions:

Will transactions increase if we switch our payment processor?
How often do users quit if they see spam on our platform?
How many sales conversations churn because we lack at-rest encryption?

A laser focus on trust can result in narrow, unauthentic tasks aimed at influencing these metrics. For example, a growth team may be interested in adding a “certification” badge on a sign-up or payment form. Or a lock icon.

Did this build trust while mitigating a risk?

Narrow trust tasks are less efficient as ones that can also reduce prioritized risks. For instance, a blog post describing a risk they’ve mitigated may also build trust amongst readers. Security teams blogging about their regular work is example of this efficiency at work.

Trust is often a byproduct of transparency from an organization’s risk reductions. An honest focus on risk reduction will often have opportunities for making these improvements transparent.

A security team will burn out unless they are working efficiently to improve risks and trust.

Imposed Risk

Additionally, trust increases by mitigating the risks imposed on others. While an organization may only be concerned with risks to itself… a customer may experience risks being imposed on themselves.

Let’s examine an example SaaS platform. The company may have a robust security program intent on securing its customer’s data at rest. The customer may still be concerned with what risks it has to manage by using the platform.

Here are some example reasons:

Customer is limited to a single, shared user account for use across many employees.
The customer has no logs available to investigate an incident with the platform.
Customers cannot restrict confidential information to their employees.

These sorts of product decisions limit a customer’s ability to manage their own risks. These risks are imposed and left for the customer to handle.

Markets that require a higher level of trust (larger enterprise) can be more easily accessed when customers can manage their risks. Listen to customers for opportunities to self-manage risk.

✅ Governance

Our institutions impose risk reductions on one another, especially when trust begins to weaken en masse.

Laws influence us to behave within constraints.
Contracts address requirements before a party enters a relationship.
Regulations impose restrictions on the industry.

Any organization is often subject to a patchwork of these forms of governance. Compliance frameworks exist to gather up a complex set of requirements and standardize them. Frameworks may simplify a governance situation while also introducing complexities of their own.

Rather than eliminating the work, security teams often own a function called “Governance, Risk Management, and Compliance” or GRC. Their goal is compliance to this constellation of governance sources.

In theory, a compliant organization:

Has reduced risks. A compliant organization is a secure one.
Has increased trust. A compliant organization is a trustworthy one.

In practice, compliance frameworks, laws, and regulations may not reduce the most critical risks that concern an organization. Additionally, compliance with some standard or framework may also build negligible trust among a market you operate it.

An approach to governance should be aware of how it can also manage risks to the business and build trust amongst a customer base while performing activities it already has to govern itself.

This level of efficiency isn’t always possible. Some compliance needs are outlandish and cannot be reasonably bypassed. This doesn’t excuse good prioritization: always make sure that risk and trust is forefront in compliance work.

Note: Negative views of governance-driven security teams are widespread. These views describe governance as authoritarian, friction based, and prescriptive. Leadership should be on the lookout for these patterns, and should screen candidates that may contribute to this perception.

🚀 Empowerment

A security team could accomplish their goals by simply turning everything off. Why don’t they?

Security teams want to find leverage that advances the organization towards it’s goals.

The example of AWS selling unused storage, compute, and bandwidth was used previously. How about the advent of CI/CD patterns that allowed developers to deploy code frequently? Or SSl/TLS that allowed eCommerce? Hard work got us there.

An example of empowerment can be as minor as a developer framework that eliminates entire classes of vulnerability, or automating evidence gathering for audits.

We should aspire to find leverage in mitigating risk, increasing trust, and being compliant. The motions are not enough, we want to be efficient. The rewards of that leverage advances the organizations we live in.

What does a security team do?

You could say that reducing risk, increasing trust, staying compliant, and empowering the business is advice that offers us very little in deciding what to actually work on.

You’d be right! Objectives like these are cheap advice, like a sports coach saying let’s go win! 🙈

So, let’s have a talk about how we’re going to make decisions and apply security patterns.

Why does a security organization have a product security team? Why did they segment their network? Why is there a dedicated red team? Why do they do these things? Who told them to? Where did they get these ideas? Why didn’t they do different things? Who’s actually in charge?

The security teams you see in the wild aren’t coming from a playbook. No step-by-step order is fully agreed on.

Rather, security organizations form from decisions under uncertainty about what risks to mitigate, and how they’re going to do it. This means paint is hitting a canvas. Through collaboration and some consensus, yourself and a team will be making unique decisions. These decisions consider the following constraints:

What are our risks?
What mitigates these risks?
What mitigations are best for our customers?
How difficult are all of these mitigations?
Who do we have available to implement these mitigations?
What mitigations are we required to have to be compliant?

and most importantly:

Do these mitigations further the mission of the overall organization?

This is why security organizations are so different from one another. This is not to say that every organization works from scratch. Rather, there are popular patterns you’ll find across many security teams.

Patterns include strategic concepts (defense in depth), best practices (SSO + MFA), and leadership philosophies (Never let a crisis go to waste). Some patterns show up as programs and team structures: Product security, detection and response, the red team.

Some patterns are incredibly nuanced. Patterns that describe early hiring, building or buying solutions, and prioritization can become murky.

Patterns are subjective, opinionated, and emerge from our accrued knowledge. They could also be described as the meta by gamers, doctrine by a military, or strategy by a coach. These phrases are related… they are competing approaches towards success. The right answers are only confirmed after success or failure. These patterns change when our adversaries do.

Here’s an example essay I wrote in support of detection engineering as a pattern. It’s my opinion on how the whole topic should be pursued, and others were influenced by it. The community is flooded with these sorts of ideas, some more influential than others.

However, whether or not you should apply detection engineering patterns in your organization is not a given. These decisions are made when they seem to satisfy the most constraints.

Patterns are not pokemon. We are accountable for the patterns we choose to adopt. To achieve our goals, we adopt many different patterns, but we can make sure the ones we do adopt make sense.

There are two starting patterns that are safe to consider for early programs.

Risk Assessments: Study and prioritize risks, then consider mitigations.
Best Practices: Study and prioritize gaps, then fill them.

These are equally viable philosophies to start a security team with. Notice, they’re in conflict with each other. One wants to assume that best practices will be the most efficient mitigators of risk, and the other wants you to study those risks before deciding on a mitigation.

The answer is that whatever suits your team is the right approach, and these first decisions start you down a path of having a unique security organization.

Take comfort in knowing that no organization knows for sure if they’ve found the most optimal path. None of them can be totally optimal without being able to predict the future.

To review:

Security objectives are a settled concept. Not much invention going on.
Decision making is in flux from one organization to the next.

Next, we’ll discuss work, which also lies on solid ground like objectives do.

How does a security team work?

Let’s discuss how a security team engages in work day to day. What is work, anyway?

Work applies to anything where our team spends time and effort.

Work is loosely defined. You know it when you’re doing it. Work may be a project, a commitment, an opportunity, or a process.

Work makes us busy.

We can categorize work into four buckets. Three are planned, and one is unplanned.

This model is based on lessons popularized by The Phoenix Project.

Business Projects: How does security help with new business efforts?
Operations: What ongoing tasks are security engineers committed to accomplishing?
Engineering: How does security make investments for a better future?
Incidents (Unplanned): What takes attention from planned work?

To simplify, we’ll use the example of a farm:

Business project: Build a new barn, buy animals, and hire a farmhand.
Operations: Feed the animals daily.
Engineering: Pave a concrete path to more easily move equipment.
Incidents and unplanned: Wolves ate the chickens. Repair fence.

We’ll move to discuss these as security team concepts.

Business Projects

A security team often finds itself supporting a business’s efforts towards a mission or objective. Work piles up for security at the business’ request.

We can approximate what a security team will have tasks for by exploring how the business grows.

For example, leadership may plan a new office. The plans immediately cause security work. Security may need to install appliances or configure networks at the location before employees arrive.

Another example might be a vendor relationship. The business might intend to purchase a service from a vendor. Security might tag along to manage risks associated with it with contracts, access control, or how data transits.

A final example: A business might be launching a new software product. A security team will get involved to reduce vulnerabilities to the company or consumers using it. Security requires collaborative work on product decisions or various types of assessments to find and fix bugs.

A defining factor of business projects is that security can anticipate involvement and plan their involvement.

Business projects are planned work. Planned work can be staffed and resourced effectively, unplanned work is thrown together in a panic.

Operations

A decision to mitigate a risk might commit a security team to own operational work. Operational work comes in many forms. We can better predict how much work a security team holds by exploring what it is regularly committed to performing.

Examples:

An infrastructure environment might be too complex to understand its risks fully. A security team might rely on detection to gain certainty that no adversaries are creeping around. Though, what happens when something is detected? A detection pipeline might require an on-call employee to respond to alerts. Detection commitments create operational work. Now, the rotation and scheduling of an on-call to respond to alerts is the resulting operational work.
The follow-up work from vulnerability scanning. It makes little sense to scan for vulnerabilities in any environment and never return to scan again. Things change! The regularity a security team commits to managing vulnerabilities becomes operational work predicated on that regularity.
A final example might be a security awareness program. A typical output of these programs is training. We hope employees are suspicious of certain behaviors and escalate them to security. These are rarely one-and-done efforts and categorize easily into operational work, especially as employees are onboarded.

Operational work, while necessary, is often a target to be minimized or eliminated through automation. Operational work can become a difficult management and performance subject as a result. A realistic view of KPI’s and performance management can help deter operational work from being used as an unrealistic proxy for risk reduction.

Operational work must relate to the reduction of risk. Work with a low impact on risks may contribute to the feeling of toil in operational work, as does repetitive work.

Additionally, opportunities to eliminate operational toil will surface with engineering practices that we’ll discuss next.

Engineering

No organization wants to repeat tasks for too long. A principle of any good organization is to become more efficient. A security team often invests in itself to expand its ability to support the business and eliminate toil. The result should be more efficient business projects, operations, and reduced incidents or unplanned work.

I will use examples to highlight engineering’s effect on other types of work.

Engineering Projects:

Integrating static analysis into a build pipeline may discover issues in business projects more quickly.
Automating the analysis of suspicious emails to reduce the amount of operations time spent investigating benign links.
Segmenting an untrustworthy part of the network exposed to a vendor may reduce unplanned incidents.

Effective security engineering builds towards reducing the entire organization’s risks, as opposed to solely making their own lives easier. Additionally, project work is often the most desirable for a talented engineer.

It’s for this reason that strong, well-directed engineering organizations will eventually shift environments away from toil (repetitive, low impact, or unplanned work) and into engineering projects.

A security team should push for a positive project workload that increases efficiency and keeps heads above water.

Lastly, we have incidents and other unplanned work.

Incidents and Unplanned

Unplanned work is a broad term that counts all distractions from planned work—included as a special case is incident response. Incidents are straightforward to plan for, but difficult to predict the work they’ll create when they occur. As a result, there is often some amount of toil involved.

It’s through unplanned work that we can develop an organization. Unplanned work guides the resources and tasks to shift work into planned work we can anticipate. A case for headcount or budget is easiest with a demonstrated need to manage unplanned work that regularly rears its head.

Organizations make predictions about where unplanned work may rear its head. Predictions generate work in all three areas: business projects, operations, and engineering. These predictions would have to be accurate to successfully shift left from unplanned work into planned work.

This ties types of work together—a clear, upward relationship forms from transitioning unplanned work into planned work.

Business projects hope to protect an expanding business.
Operations runs the commitments made in protecting the business.
Engineering makes it more efficient to support business projects, perform operational work, and avoid risks.
Should any of these efforts fail to anticipate problems; they become unplanned work.

The resulting unplanned work then is where learning begins. Those lessons bring us back into improving business projects, operations, and security engineering.

To summarize:

Unplanned work inspires business projects, operations, and security engineering.
Business projects and security operations feed work into engineering.
Engineering projects march toward a better state of being.

Foundations in practice

Every security team has problems. A perfect strategy is unobtainable.

The risks at every organization are different. Missions are different. Technology choices are different. The management principles that leadership may subscribe to will be different.

As organizations grow, the security team may come together slowly, first from brave souls stepping up. Then, expanded headcount by headcount. A team may come to realize that a security team was never really founded at a precise time or direction. Rather, the team was more of a culmination of ad-hoc tasks and self-directed projects.

Early incidents often influence team-building decisions as well. Frequent re-organizations or mergers / acquisitions are also highly disruptive to the trajectory of a team. Large customers can also be an outsized voice in what a security team looks like to have their needs addressed in a contract.

All of these dynamics pull us away from idealist views of a security team. Security teams all work towards a larger mission by tackling risk, trust, and governance issues. These concepts are only a north star. A north star is helpful with distractions but does not eliminate them.

As a result, security teams are never as clean-cut as this writing might suggest they could be.

Next, imbalances in work are also very common. Teams drown themselves with operational work (alerts, vulnerabilities, tickets) or overly focus on pet project work. There is no prescribed balance for these, other than making sure engineering projects help diminish the efforts involved with business projects, operations, or incidents. Having the terminology to discuss friction points in terms of work allows an organization to find this balance. We’ll discuss some of the efforts to find this balance next with management.