No Gold Standard: Measuring Success in Medical Affairs

Concept graphic - colored blocks

To understand true publication impact and influence patient outcomes, Medical Affairs teams must have their own benchmarks

Compass Points: The Future of Medical Affairs is a series exploring the strategic challenges facing Medical Affairs teams in today’s communication landscape—and the tools that will help them get it right.

The best publication strategy is like a treatment plan: bespoke

In the past, journal citations served as the primary metric for measuring publication impact. Citations all but guaranteed a share of voice and influence amongst key opinion leaders and healthcare practitioners; they were also a simple, clear metric to share upward, proving research impact and justifying the allocation of resources.

Today, journals have come to occupy a different place in the Medical Affairs community. They still confer legitimacy, but they’re not the only way to have an impact; they aren’t even necessarily the most appropriate channel through which teams can or should disseminate information. 

How scientific information travels can be measured in both scientific impact and real world impact. Scientific impact comprises long-tail, more static measures such as citations and subsequent policy changes tracked over the course of months or years. But real-world impact—how a publication influences thought, conversation and even behavior—can be observed in how information ripples through other more immediate channels, like social or broadcast media and forums. 

Capturing an accurate picture of how a publication has performed requires a view of both. This holistic view allows teams to accurately benchmark performance, measure impact, and demonstrate value to stakeholders, supporting the overarching goal of improving patient outcomes.

The role of the journal has changed

While journals still heavily inform the provision of healthcare alongside clinical guidelines and regulatory bodies, they are not the only place members of the life sciences community can encounter and learn about new research.

As scientific information has come to travel on more horizontal, peer-to-peer channels, such as social media or podcasts hosted by trusted key opinion leaders, practitioners are able to learn about and interact with new research outside of the journal publication and conference cycle. This makes it easier for HCPs to stay on top of relevant research, and to quickly sift through the studies that are relevant to their clinical practice. This is why, depending on the therapeutic area and the goals of a given publication launch, Medical Affairs teams may find they gain more traction by diversifying to non-traditional channels. 

But in order for publication planners to take advantage of this reality—to optimize distribution across geographies, channels and a variety of timescales—requires dynamic, granular data that is consistently tracked through time. And as journals have come to form only part of the life sciences research diet, Medical Affairs teams have been left without a single, strategic reference point both for forward-planning and post-publication performance reporting. 

As a result, teams find themselves in a familiar position: unable to reliably demonstrate impact, defend decisions to stakeholders, or to quickly iterate for later distribution plans. This can undermine a team’s efficacy, and ultimately delay or limit influence on patient outcomes. 

What teams lose without a consistent benchmark

Benchmarking plays a critical role in publication planning. It allows teams to reference both the performance of earlier publications and the work of competitors, and to learn, in real time, what is working and what is not. Without benchmarks, it is impossible to know what is reasonable for research to achieve, and therefore, impossible to contextualize impact and prove a return on education. 

But benchmarking is also one of the most laborious parts of the publication planning cycle. The process of consistently benchmarking, tracking, reconciling and cleaning point-in-time data—from social media, journals, podcasts, conferences, magazine articles, and more—can take teams weeks of work. And because of the fragmentation inherent to the process, all of this work, ultimately, still may not be able to capture the nuance of a publication’s impact.

The knock-on effect is that teams are unable to design strategies which are optimized for a given therapeutic area and to meet specific performance goals, such as social media engagement. This undermines the team’s ability to demonstrate that strategic objectives are met and can limit the diffusion of information into communities that could benefit from it. 

This cycle repeats; without live, ongoing benchmarking, it’s impossible to see what’s changing in the competitive landscape and react to it. 

In recent years, the Medical Affairs community has matured dramatically with regards to its use of data. Teams rely heavily on analytics and data-driven decision-making. They know what data is available to them and how they can use it to inform publication strategies. 

But the tools available for assembling and parsing this data haven’t kept pace. Even as many teams embrace the use of general-purpose AI, the output is often unstandardized and non-reproducible—what AI surfaces today may be different from what it surfaces tomorrow, so teams can’t be sure they’re comparing like with like. In other words, teams gain speed, but not certainty.

This is what Compass by Dimensions was created to address. It brings together both traditional and alternative metrics so teams can easily benchmark against internal and competitor data, track publication performance through time and across channels in a standard and simple way, allowing to better demonstrate value, influence therapeutic behavior, meet education objectives, and trace real-world research impact. As a result, teams can move from publication strategies which are fundamentally reactive to those which are proactive, and as a result, better able to meet strategic objectives.

Figure 1: Workspace performance. Total number of attention events tracked across supported sources.

Tapping into the discussions that matter

Improving patient outcomes is the result of a confluence of events: research must be carried out, written up, disseminated, and then found by the relevant policy makers or healthcare practitioners to stand a chance of driving real-world impact. That means Medical Affairs teams need to look at both formal and peer-to-peer channels to measure influence. 

Compass is driven by data sources from two leading services in the scientific and research community, Dimensions and Altmetric. 

Dimensions hosts one of the largest collections of interconnected global research data, re-imagining research discovery with access to grants, publications, clinical trials, patents and policy documents all in one place. This data source provides a robust view of traditional channels.

Altmetric is a leading provider of alternative research metrics, helping everyone involved in research gauge the impact of their work. Altmetric searches thousands of online sources including social media, revealing where research is being shared and discussed—and the sentiment of that discussion. This is where real-world impact manifests first. 

In bringing these two data sources together, Compass allows teams to track the whole publication attention lifecycle, from social media posts minutes and hours after publication, all the way to citations and guideline mentions years after it was published, all reliably benchmarked through time. 

Teams need only set up a benchmarking dashboard to define which internal and competitor publications they want to track. Then the dashboard can be referenced any time for a simple view of a publication’s performance. Instead of the heavily manual work that teams had to endure before, Compass brings precise and consistent performance measurement and real-world benchmarking across assets or disease areas, giving teams the evidence needed to understand what’s working and where science is influencing practice.

Figure 2: Total mentions across domains.

Democratizing data-driven publication strategy

Medical Affairs teams are too-often forced to rely on costly and slow agency relationships to understand how their publications are performing. Teams who can take these tasks in-house with tools such as Compass will amplify the efficiency and agility with which they can work. 

Compass performs three functions which are critical to creating a truly bespoke, data-driven publication strategy

  • Unified, easy-to-reference metrics: Compass allows teams to quickly and consistently track performance through time to understand if a publication has reached the right people.
  • Custom benchmarking: Teams can benchmark publications against their own portfolio’s historical performance, establishing what a realistic result looks like. This exercise can also encompass competitors and specific therapeutic areas, helping teams identify opportunities and learnings. The task of surveying channels and creating internal and external benchmarks would have taken weeks before—now, once a dashboard is created, it takes just a few minutes to check.
  • Self-service interface: Compass creates shareable, stakeholder-ready visuals to clearly demonstrate the direct impact of work to decision-makers and support resource discussions with concrete data points. Teams can cut and re-cut data in a few minutes, saving the time, cost and hassle of looping in an agency every time a new report is needed.

Instead of waiting for an agency to return a report, or spending hours parsing data only for it to immediately stale, Compass places the power of real-time insights in the hands of the people best placed to wield it, streamlining the distribution process and supporting decision-makers with clear targets and performance measurement. 

Use case: Moving from lagging indicators to real-time feedback

Take a mid-size oncology-focused biopharma preparing to launch a publication for a second-line therapy. The Medical Affairs team’s usual process might take three to four weeks per reporting cycle: they would need to pull citation counts from one system, social and news mentions from another, then manually reconcile both in spreadsheets. If socials spiked after review had concluded on a given platform, that data would be missed. This means by the time a report reached leadership, the data would already be stale, and there would be no consistent way to benchmark the publication against prior launches in the same therapeutic area, because citations move too slowly and social media moves too quickly. 

For this team, Compass is designed to remove the manual burden of the benchmarking and tracking process. The team would first set up their benchmarking dashboard, tracking both the performance of previous portfolio publications and those of competitors working in specific therapeutic areas. Then, it would take just a few minutes to monitor performance: the team would easily be able to track journal citation activity against historical norms for the launch stage alongside engagement on clinician-focused social platforms and podcasts. 

Teams would be able to see where the research was finding the most resonance and quickly take action on that basis, for example, redirecting a portion of dissemination budget towards channels demonstrating traction. The consistency of this data also makes it easy to keep leadership informed; teams can slice and share reports from their Compass dashboard to illustrate what’s working and what isn’t. 

For teams that have previously relied on a single lagging metric, having a live, multi-channel benchmark can turn publication planning from a retrospective, best-guess exercise into a real-time, data-driven strategy.

The future of Medical Affairs is here—it’s time your strategy caught up

The proliferation of scientific information through popular channels is a good thing. 

That new scientific and medical information can reach much wider audiences through a variety of channels undoubtedly has a net positive impact on patient outcomes. Patient and rare disease advocates, healthcare practitioners working in remote or underfunded areas, and even patients themselves all benefit from having access to the cutting edge science that will shape the future of medical care. 

But the result of this proliferation has been a significant challenge for publication planners—no one could’ve predicted the rise of peer-reviewed podcasts. Now, teams need to take all of these data points into account when measuring performance and planning future publication launches. 

Manually compiling point-in-time data is a time-intensive process that puts publications at a disadvantage and undermines strategic and patient outcome objectives.

The right strategy is one entirely specific to a given publication—it must be tailored to relevant journals, audiences and channels, and these may all change through time. To date, with this level of nuance, it wasn’t possible for teams to keep up—not at the level of granular detail that could shape truly powerful publication strategies. 

Compass turns that complexity into an opportunity.