Stakeholder views on independent CDR protocol review

Over the past two years, there has been a marked increase in the number of carbon dioxide removal (CDR) protocols. Similarly, there are more entities involved in developing, providing input, and using those protocols. More protocols means more choice when deciding what rules should be used for crediting CDR. It’s possible new infrastructure could help stakeholders navigate this complexity – but do they agree?

To explore this question, CarbonPlan, Lawrence Livermore National Laboratory (LLNL), and Lawrence Berkeley National Laboratory (LBNL) recently conducted a survey of stakeholder opinions on the role of independent protocol review. This survey was partly motivated by an open letter published in early 2023. In the letter, 35 organizations working on CDR called for the creation of a new standards initiative to review protocols and harmonize MRV approaches across and within CDR pathways. Although some protocol review processes are currently operating, none that we know of produce public outputs focused on long-duration CDR.

The survey found that there is still broad agreement about the value of independent CDR protocol review — but there seem to be some important differences in what stakeholders want from these reviews and how they think the process should function. Since a single protocol review process is unlikely to serve all these needs at once, anyone designing a protocol review must establish very clear goals upfront and set stakeholder expectations about what outcomes to expect.

Survey approach

We designed this survey to capture views about both the importance of independent CDR protocol review and what outputs and focus areas respondents would want such a review to prioritize. The survey consisted of 12 structured and open-ended questions, a copy of which can be found here.

Throughout the survey, we defined “independent” as indicating a review conducted by parties substantially free of conflicts of interest, e.g., parties who are not involved in writing protocols, issuing credits, or receiving credits under a CDR protocol. We defined “protocol” as the document or set of documents specifying the detailed rules by which projects demonstrate what they have achieved and generate CDR credits that can be sold to buyers. We asked survey respondents to assume that the results of any independent protocol review would be made publicly available.

We invited 36 organizations to respond to the survey, including CDR providers, registries, brokers, buyers, investors, and non-profits. Survey participation was voluntary and participants were informed that all responses would be anonymized in the public reporting. In total, representatives from 26 organizations participated. No brokers or investors provided a response.

Survey results

Stakeholders widely agreed that novel independent protocol review efforts could support the development of a robust CDR ecosystem. A majority also expressed that such an effort should prioritize reviewing the accounting and quantification rules underpinning crediting calculations, rather than dimensions like protocol governance.

However, there was some substantive disagreement around the form a novel protocol review effort should take. Stakeholders had diverging opinions about whether protocol review would be most helpful if it focused on pathway-specific versus cross-cutting issues, and about what kind of outputs would be most useful.

01 — Stakeholders generally agreed that novel independent protocol review efforts could be helpful.

Stakeholders expressed broad consensus that protocol review has a near-term role to play (Table 2). As one Standard-Setter put it, “When you really look into it, there are shockingly few checks and balances that exist to ensure that protocols are rigorous and remain rigorous over the long term.”

Some protocol review efforts do already exist. Organizations like the Integrity Council for the Voluntary Carbon Markets (ICVCM) conduct protocol reviews, but to-date, they have not focused on long-duration CDR. Sophisticated CDR buyers run internal protocol review processes, but the resources and effort that requires are unlikely to scale. Credit rating agencies also review protocols, but the results are not made fully public. In this context, the majority of stakeholders expressed the opinion that there is a place for novel, independent protocol review efforts. There was some divergence, however, around the perceived urgency of filling those gaps (Table 3).

Interestingly, stakeholders were also divided in their opinions about whether the importance of independent protocol review was likely to increase, decrease, or remain stable over time. This seemed to be partially explained by expectations about whether or not the content of protocols would converge or harmonize over time, perhaps by the entrance of definitive standards set by policy. For example, one Standard-Setter who thought independent review would become less important over time said, “The freedom for a project to choose a protocol will be limited by emerging regulations by EU, UN and other institutions.” In contrast, one Buyer argued that protocol review is likely to “become more important as the landscape of registries becomes more competitive.”

02 — Most stakeholders wanted novel efforts to prioritize reviewing quantification and accounting rules.

Protocols include many elements, including eligibility rules, rules for quantifying how much net CDR a project has achieved, and procedural rules that determine how a project interacts with the protocol to receive credits. Stakeholders indicated that of the potential targets for review, they would want a novel protocol review process to focus on the rules for quantifying net CDR impacts — namely the accounting and quantification approach laid out by different protocols (Table 5).

The degree of consensus around prioritizing accounting and quantification approach review was striking. Taking a different view on the data presented above, all but four stakeholders (92%) ranked "Accounting approach" in the top two, and all but nine stakeholders (65%) listed "Quantification approach" in the top two. As one Buyer put it, “I tend to find a lot of past efforts (e.g., ICVCM and the like) have tended to focus on governance and procedure more than scientific accuracy, which I think is the more important issue for an emerging field like durable CDR.” And as one supplier noted, “The most urgent issue is that different protocols and different technology pathways are subject to different boundary conditions which can result in very different removals -subsequently creating a convoluted climate impact.”

Despite this high-level consensus, we observed some underlying nuance around what kinds of questions should be prioritized within these categories. Some stakeholders indicated they wanted protocol review in these categories to focus on elements that are already broadly well-understood. For example, one Standard-Setter said, “I bias towards the categories where consistency is needed, rather than innovation.” Other stakeholders emphasized the importance of reviewing dimensions of a protocol that involve open scientific questions or subjective decisions. As one Buyer put it, “The most critical components that need review are those that are prone to subjective design or sampling decisions…”

03 — Stakeholders expressed diverging views about the preferred outputs of a protocol and what those outputs could enable.

When asked what the primary utility of a protocol review would be, stakeholders referenced a number of potential functions. Many stakeholders expressed the desire for a watchdog function, establishing a “quality floor” to ensure that every protocol meets a minimum standard, and preventing a race to the bottom. Several stakeholders who were interested in this role expressed an expectation that protocols will continue to proliferate. They raised the concern that there will be an increasing danger of protocol shopping, making it hard for suppliers who are doing the right thing to compete. For example, one CDR supplier said, “I fear CDR pathways (particularly open system ones) following the race to the bottom we have seen in REDD+ and soil carbon if we are not very careful.”

Another commonly mentioned role for protocol review was enhancing trust in the industry and enabling new demand. In this role, stakeholders emphasized the possibility to provide a general assurance of quality control and a shortcut for decision-makers, such as buyers or policy-makers, to make better decisions. However, multiple stakeholders expressed concerns that unless deliberately executed, a novel protocol review effort could cause more confusion than it resolves. As one ecosystem actor explained, “[T]here are a number of efforts to improve the credibility of the CDR market at the moment and buyers/policymakers are often responding to dozens of new "validators". I think any near-term protocol review would need to be done by a credible source that ensures it rises above the rest.”

Additional roles mentioned included helping CDR suppliers identify what protocols to use, facilitating learning and consensus building, facilitating protocol updates, standardizing language and concepts, and providing a model for similar efforts in other fields like methane removal.

While these roles are not all necessarily mutually exclusive, optimizing a protocol review process for any one of them is likely to involve trade-offs for the others. This was, perhaps, reflected in the divergence around the ideal scope of a protocol review and the preferred structure of its outputs.

Stakeholders were split when asked if they would prefer novel protocol review efforts to prioritize pathway-specific issues (e.g., modeling standards for enhanced rock weathering) versus cross-cutting issues that apply across multiple pathways (e.g., lifecycle assessment boundaries). Stakeholders also diverged on the preferred output of a protocol review – structured information versus unambiguous pass/fail judgements – though the majority indicated a preference for structured information that enables individual actors to make their own judgement (Table 6).

We also observed tension between how various stakeholders imagined a review process interacting with established protocols. Many respondents emphasized the importance of reviewing existing protocols in an ongoing manner to incorporate new science and assess the real-world outcomes of the protocol. For example, one Ecosystem Actor said, “[W]e need regular cycles that also assess what worked and what didn’t work in the implementation of the protocol so far.” But other stakeholders raised concerns that reviewing protocols that are already in use could call into question the viability of the protocol itself and credits that have been previously issued under it. As one Buyer said, “I’d prefer protocols get reviewed and improved before a supplier starts issuing credits against it to us, vs be[sic] getting credits and then changes being made that casts doubt on prior issued credits.”

Takeaways

In sum, these survey results highlight that there remains significant potential for an independent protocol review process to support the development of the CDR ecosystem. However, it seems unlikely that a single review process will satisfy all stakeholders’ preferences.

Based on the perspectives shared in the survey, we could imagine many versions of near-term protocol review. For example, one version could be a small group of NGO actors reviewing cross-cutting accounting issues, like lifecycle assessment boundaries, on an ongoing basis when protocols are released or updated. Another version could involve convening pathway-specific review committees composed of academic, NGO, and government experts to periodically assess protocol rules around critical quantification topics. Yet another version could rely on a survey-based mechanism to trigger a process of synthesizing pathway-specific learnings and publishing protocol update recommendations proactively.

Each of these versions of protocol review would be responsive to a different set of expressed stakeholder preferences. The question at hand is which function or functions should be prioritized and who is best positioned to lead them.