In Defense of Friction

Why Humans in the Loop still matter?

Introduction

AI reduces friction, while humans increase it; but each has their own place in even the most AI forward company. Friction provides a way to reduce mistakes, and to let the company think about the consequences of a decision, but too much of it can drastically slow down productivity, so it’s important to selectively apply friction in the right places to enable your company to make the right decisions quickly and firstly.

In this blog post, we’ll talk about how friction can be used as a tool for a successful security program, while still enabling high velocity engineering teams with AI.

Examining the Role of AppSec: Friction

Application Security has traditionally been the human touchpoint for developers during the Software Development Lifecycle (SDLC). AppSec charters have been built on human interaction, such as introducing the security team during developer security training, partnering with security champions, or even conducting threat modeling sessions. However, as companies try to maximize AI usage and minimize humans-in-the-loop in their engineering processes, how do AppSec teams respond?

Working on a human-centric team can become a bottleneck for AI driven companies; humans work much slower than agents. If velocity is the main thing that your organization values, friction can be viewed as a bad thing. However, some friction has its own use, as it forces reflection by adding speedbumps. These speedbumps can be critical to allow your business to understand the risks involved with any major decisions:

• Identity and Access Approvals: Stops employees from getting access they shouldn’t have

• Security Education: Stops developers until they learn best practices

• Security Tooling: Stops developers from pushing code until security issues are corrected

• Threat Modeling: Stops employees from implementing until they think about the security of the current architecture

Case Study: Threat Modeling

Threat Modeling has historically been a highly manual process in which a security engineer partners with an engineering team to understand the design, identify threats, and partner on remediation. Threat Modeling creates friction by design, introducing a checkpoint for engineers to reflect on their design and iterate with security in mind.

In reality, threat modeling can add much friction and toil. Threat modeling can be time consuming, and security teams may not be able to scale threat modeling programs with rapidly expanding engineering orgs. With long wait times, tight release timelines, and diminishing value of threat models after launch, developers can perceive threat modeling as a blocker that delays launch timelines.

AI offers the promise of significantly reduced friction. Agents can process code, design docs, and even Slack conversations to understand systems and identify threats. Security teams are already experimenting with agentic threat modeling, showing us a glimpse into a future where agents could be deployed to threat model entire systems and ship fixes without any human involvement.

With context-driven and rapid agents, are human-led threat models still valuable? While one value of threat modeling is the output (threat modeling artifacts), another important value is in the collaboration between humans. Threat modeling is an important exercise for engineers to think about security and determine its weight for their product and the business. Security and engineering teams debate and ultimately agree on an acceptable risk level, and continuously work together to reduce risk. The value of threat modeling is in the pause that this collaboration inherently introduces.

To scale responsibly, security teams must use agents and human friction deliberately. Agentic tooling can eliminate toil, while humans own strategic decision making. Lengthy intake forms, information gathering, and self-service recommendations can all be replaced by agents. Then, humans are unblocked to do the security work that really matters: collaborating on risk tradeoffs in the context of business priorities.

Introducing friction in agentic threat modeling can look like this:

• Integrate feedback loops with agentic threat models: Agents can draft a threat model but the value comes from the discussion. Engineers should be encouraged to challenge recommendations and weigh tradeoffs through a chat-based threat modeling agent.

• Keep humans in the loop for complex projects: For complex and risky products, or historically insecure products, maintain the partnership model. The partnership is important to sustain context and guide engineering teams to own the security of the product and ultimately reduce risk.

Reconciling Friction with AI

Applying friction deliberately requires defining how humans and AI work together. There are three ways to enable AI:

• Human-in-the-loop (HITL): AI enablement, where humans are consulted before any state changing decisions. This allows AI to provide quicker guidance, but is still limited by the speed of a human. Example: Cursor requires that you approve every code change.

• Human-on-the-loop (HOTL): AI powered, where humans can inspect AI, and potentially correct mistakes, but still allowing AI to conduct operations on its own. Example: AI Observability Tools that let you monitor all AI outputs.

• Human-out-of-the-loop (HOOTL): AI is the decision maker, and humans are not able to correct mistakes. Example: Clawdbot running fully autonomously on your Mac Mini.

Each of these has its own place in an AppSec program. Human-in-the-Loop tools are useful nearly everywhere; AI assistants make humans faster, but do not reduce friction significantly. A human is always required to “push the button” when something risky is happening. As such, you can enable these tools without having to think about legal or compliance issues.

Human-on-the-loop and Human-out-of-the-loop, on the other hand, reduce friction to near zero, but require much more thought to safely enable. When AI is allowed to make decisions autonomously, this can lead to questions from auditors and your legal team. Who is responsible for a mistake? How can mistakes be minimized, and how can the impact of any mistake be reduced?

For example, using an agent to threat model can reduce friction significantly when it performs the first pass (this is Humans-in-the-Loop). However, relying solely on this auto generated threat model (which constitutes Humans-on-the-Loop) could miss critical threats or hallucinate recommendations. Engineers might acknowledge the threats but ship anyway to meet tight launch timelines.

In a future blog post, we will discuss a risk framework to determine where Humans-on-the-Loop or Humans-out-of-the-loop agents are appropriate in your business.

Conclusion

With every new AI headline, engineers face more pressure to maximize AI usage and eliminate all friction. With all of the excitement and increasing momentum, it’s important to pause and evaluate the role of friction. Not all friction is waste. Some friction creates the conversations and partnerships that build security ownership in engineering teams. As you evolve your AppSec Program, be intentional about the role of friction. Eliminate the toil but preserve the deliberate pause that adds value.