Topic 43 – Use of Net Promoter Score Measures to Evaluate MSLs

A couple of my clients have discussed the use of the Net Promoter Score lately so I thought I would address it in my blog.

Quick background:

The concept of the Net Promoter Score was introduced in a Harvard Business Review article in 2003 by Fred Reichheld of Bain & Company. The net promoter score is measured by asking a single question: “How likely are you to recommend the company/product/service to a friend or colleague?” and is usually measured on a 0 to 10 basis. Scores of 9 and 10 are called Promoters, scores of 0 to 6 are Detractors and scores of 7 and 8 are called Passives. The Net Promoter Score is calculated by subtracting the percentage of customers that are Detractors from the percentage of customers that are Promoters.

People like the net promoter score because it is a simple measure of loyalty and when it is paired with an open ended question that asks why the particular score was given, it provides insight into what is important to the customer.

So, does the Net Promoter Score (NPS) provide value to MA? My research has not been able to find a single academic or metric-driven study on the use of NPS in MA specifically related to MSL activity. Common sense says this approach should be helpful but for now anyone using this approach is in experimental mode.

In MA the NPS question is often modified to be:

• "How likely are you to recommend engaging with [COMPANY X] Medical Science Liaisons to your colleagues or peers?” OR
• “How likely are you to recommend working with [MSL NAME] from [COMPANY X] to your colleagues?

PRO’s of Using NPS in MA:

• Brief nature of survey makes it suitable for rapid deployment immediately following MSL interaction to avoid the “blending” affect that occurs when HCPs are asked about MSL performance on a standard survey often weeks after their last interaction
• Relatively inexpensive to conduct compared to other market research
• NPS can help gather insights into what an HCP value in an MSL interaction, if open ended questions are employed as well

CONs of Using NPS in MA:

• Message vs Messenger: When an HCP recommends working or engaging with an MSL is that recommendation based on the quality of the content of the interaction or the interpersonal qualities of the MSL herself or himself?
• Not comparative: NPS does not give insight into whether HCPs recommend your MSLs any differently than they recommend competitor MSLs. Perhaps HCPs in a particular therapeutic area simply recommend all MSLs the similarly regardless of company.
• Not clearly actionable: If your NPS drops from one month to the next, what action should be taken? Some insight might be provided by the open ended questions but those responses are often only provided by the most dissatisfied

Given the inherent challenge, it is my opinion that the NPS is still a worthwhile measure, but it needs to be gathered as a part of a broader market research effort to give it the context that can help tease apart the reason for the scores.

The most effective NPS is gathered as soon as possible after the last interaction. In the case of MSLs, a system should be established to seek this guidance directly after a contact has been noted in the company’s contact management system. And, like all market research with HCPs, participation is highly impacted by compensation, so sufficient compensation must be offered to ensure enough participation to make the measure meaningful.

What is your experience with NPS? How do you frame the question? Share your experiences by clicking here.

Topic 41: The Implications of Amarin. Is It “Good” for MA?

A number of my friends in MA leadership have been debating the implications of the Amarin case for MA overall. Spoiler alert – I think it will prove to be another avenue for MA to add value and I will share my rationale below.

In case you have not been reading up on this case, you can see my brief discussion and some links to other sources of information on this blog post. When discussing the Amarin case it is critical to understand the context – this case was decided by a federal district judge in Manhattan for the Southern District of New York – so obviously this ruling does not set national standards.  However, the Southern District court is one of the most influential and active courts in the US and it has a history of leading the nation.

Assuming that this ruling becomes the precedent for either other cases in other districts or even national cases, the question is if/when pharmaceutical companies have the flexibility to promote off-label data (with all the fair/balanced caveats) – what are the implications for MA?

So let’s go on a trip to a speculative future in which the ability to share off-label data with HCPs on proactive basis becomes accepted in the US.  In this future, I would fully expect that the FDA decides to put guardrails around this freedom.  Their rationale will be simple – there is high risk to patients if HCPs make decisions based on off-label information which have not run the full risk/benefit analysis of a product with an approved NDA.  What would these regulations look like?  We can get a good view from FDA’s response to Amarin (which we covered indetail here) the summary of which is that this education would need to be fair and balanced and, among many other things:

• Discussions should be conducted by persons with the appropriate background or training to accurately communicate scientific information

That would clearly call for a role similar to the MSL or field force role of today’s MA. But would that role need to be in MA, or more provocatively, would MA need to be a separate entity from commercial.  

Those you that have been around the industry long enough remember when MA was sometimes a function of commercial, often called Scientific Sales. Driven by increasing FDA and EMA scrutiny primarily concerning off-label promotion and a desire to be seen as a voice of science instead of promotion, MA as an independent, non-promoting entity became the standard. 

In our speculative future the US has loosened the off-label promotion rules, but the EMA has not and the need for an independent voice for science has not decreased so I do not foresee an effort to move MA back under commercial. Frankly that ship has already sailed since so many other functions that MA serves also benefit from it being independent from commercial.

So if MA is likely to remain independent would the MSL role remain in MA? I think that they would, not only due to inertia (although you can never over-estimate the power of inertia in pharma) but also because keeping the MSL role in MA would improve the case that the information presented was fair/balanced and not tainted by promotional messages.

Following my logic stream, MSLs would gain a new capability (ability to pro-actively share off-label data) while remaining in an MA function independent from commercial – so a net improvement to the value that MSLs and MA overall brings to the company. That can only be a good thing for MA leaders.

That was quite a bit of speculation – what is your opinion?  Please click here to leave a comment.

Topic 34: New FDA Consumer Advertisement Guidance and Potential MA Impact

Have you seen the new FDA guidance about disclosing risk in consumer-directed print advertising that came out in February? (You can see it here) Unless you are a gluten for FDA guidance-reading punishment, my guess is that you skipped this one since it seems to be commercially focused.

BUT, there is actually something that MA should be aware of and perhaps an opportunity to add some value to our commercial brethren. The focus of the guidance is straight forward – under current law print advertising has to also disclose risks, and the safest approach for disclosing that risk is to publish the full package insert (PI) along with the print ad. As we know PIs are a tough read normally, but when shrunk down to fit in a magazine they are almost unreadable and certainly mostly incomprehensible to the very audience they are supposed to be protecting – consumers.

This has not been lost on the FDA and the guidance linked to above was entirely focused on resolving this issue.

In an FDA survey, few respondents reported reading half or more of the brief summary presented in the traditional format. Of those who read at least some of the brief summary, 55 percent described it as hard to read. Over 40 percent of respondents in the survey reported they do not usually read any of the brief summary in direct-to-consumer prescription drug print advertisements.

The FDA realizes that the full PI is aimed at medical professionals and full of details that the vast majority of consumers don’t care about like clinical pharmacology or chemistry. So the FDA is suggesting that manufactures should have the flexibility to replace the PI with something they are now calling “consumer brief summary.”
What is a consumer brief summary? Per the guidance it is an explanation written in consumer-friendly language (ie. drowsiness not somnolence) that includes:

• Boxed warnings
• All contraindications
• Certain information regarding Warnings and Precautions:
  • The most clinically significant information from the Warnings and Precautions section(s) of the PI;
  • Information that would affect a decision to prescribe or take a drug;
  • Monitoring or laboratory tests that may be needed;
  • Special precautions not set forth in other parts of the PI;
  • Measures that can be taken to prevent or mitigate harm
• Most frequently occurring Adverse Reaction, and those ARs that are serious or that lead to discontinuation of use, and the severity of the risk
• Indications for use
• Significant drug interactions

And this is where Medical Affairs comes into play. Now our commercial colleagues and their agencies are going to be needing to develop information that includes medical judgement, like:

• What is the most clinical significant information?
• Why is that information considered most clinically significant?
• How do practitioners view what is most clinically significant?
• What information should affect the decision to take or prescribe the drug?

These questions are great ones for Medical Affairs to either provide guidance, answer directly or gather information from practitioners during their interactions to answer these questions and others. Given the proper but strong firewalls between MA and commercial, this new guidance provides a value-added opportunity for MA to provide some guidance to commercial.

MA leaders may wish to discuss this topic with their commercial colleagues.

Do you have any thoughts on the new guidance? Leave them in the comments by clicking HERE and scrolling down to the comment box.

Slightly OffTopic: NEJM Article on BioPharma Research Credibility

Have you seen this special article in the NEJM entitled: A Randomized Study of How Physicians Interpret Research Funding Disclosures.  It is disturbing reading for all of us in the industry, including MA.

In the study, researchers presented 503 board-certified internists with three random studies with a high, medium and low level of methodological rigor.  And then randomly assigned each study one of three funding sources: NIH, Biopharma Industry, Not Disclosed.

The good news is that respondents generally assigned stronger credibility to studies with stronger methodologies.

The difficult news is that the internists believed the results of industry-sponsored were less credible than those that were NIH funded and with no funding disclosed.  And that results remained the same whether the study was of high rigor or low rigor.  Since the funding source was randomly assigned its clear that simply associating industry funding reduced credibility.

And the kicker – over 75% of the respondents accept support from industry in some manner, so these are not a bunch of ivory tower purists throwing stones.  These are exactly the people that MA needs to educate and these are exactly the kinds of studies that we use for that education.  Clearly we have a problem.

In an editorial that accompanies the study entitled Believe the Data, Dr. Jeffery Drazen suggests that physicians need to focus on the data and the rigor of developing that data.  As an industry, we need to echo that exact point.

We know where this skepticism comes from - too many newspaper stories of incomplete study disclosures, too many studies that seemed to lack rigor, too much willingness to spin results.

We cannot ignore this issue.  We need to hit it head on – discussing this now proven bias and why it should not impact the interpretation of results for a given study due to its rigor, etc.

Something that many of us have suspected for awhile, that industry science was being discounted simply because of funding, is now shown to be true.  We must address it openly because like any bias when it is pointed out to people they are less likely to be prey to it.

What do you think we can do to address anti-industry bias?  Leave your thoughts in the comments section.

Topic9: HEOR Studies – Who Should Take the Lead

I was talking with a longtime friend and client the other day and he was describing a situation in his organization.  A tug-of-war of sorts was going on with HEOR studies.  In his business, the amount of funding going into HEOR studies was going up, and although they had typically be run out of the MA shop, there was increasing pressure to move all HEOR studies over to the Access and Reimbursement group.  We had a good discussion on this which I will try to recap here.

Traditionally all post-marketing research has been driven out of MA.  I use the term driven, because in many organizations the actual studies may be managed by a CRO or a clinical development resource.  But the group responsible for identifying the need for the study, budgeting for it, getting approvals to conduct the study and driving the protocol development was almost always MA.  There is a good reason for this, the Medical Director, who typically owns the scientific lifecycle of the compound is in MA and thus should be driving this process.

Now, post-marketing studies is a very large bucket.  It can include additional safety studies, both mandated and not mandated, reformulation studies, label expansion studies, HEOR studies, just to name a few.  And, given the huge range of post-marketing research, it is fairly standard that the Medical Director and MA in general partners with other parts of the organization that have expertise in these areas.

Fast forward to today’s world.  HEOR studies are increasingly critical to the medical and business case that Access and Reimbursement needs to make to payers in order to achieve a beneficial position in their formularies.  Investment in HEOR studies are growing by 0ver 40% in the past few years, as discussed here.  Given their knowledge of the audience, the data that they need and the competitive data that they are likely to have available, no one is in a better position to define the requirements for an HEOR study than the Access and Reimbursement team.

But, does that mean that Access and Reimbursement should be driving the HEOR studies themselves.  My answer remains been no.  Access and Reimbursement should be a defining voice in the development of the protocol, but once that protocol concept has been finalized it becomes an issue of study execution.  This is frankly how it works with most post-marketing studies.  While I can grant that an HEOR study has unique issues, so do many of the targeted post-marketing studies.

Having said all of this, it is also true that I, myself, have set up just such a function for an Access and Reimbursement client.  They had circumstances that made it clear that the MA function was not valuing the need for HEOR research and thus it was not getting done on a timely basis.  This was essentially a political issue but the only way to resolve it in the short-term was the create an dedicated function within Access and Reimbursement.

I am curious to know your experiences in terms of HEOR studies.  Leave me a comment below.  And if you have any question that you would like me to address, please click on the email link to the right.

Topic 6: Measuring the Value of Medical Affairs

Introduction

A reader requested a discussion of different approaches for measuring the value of medical affairs.  This is a critical question and one that I touched on briefly when I discussed strategic planning in MA as it relates to launch.  But it is equally important overall for MA.

Like any function in pharma, MA must be able to communicate its value.  Value can be a tricky concept, however, for a non-commercial function like MA.  Value is ultimately a company-specific question.  It stems from the company achieving its strategic goals, which is why defining goals is so important.  Having said this, there are some general categories of value measures that we can discuss as well as some best practices around measuring value.

Value measures in MA typically fall into two types:

1. Objective measures


2. Subjective measures

Objective Measures

In MA, objective measures tend to focus around activity-based measurements.  Since MA cannot be seen as promotional, the other available objective measures like prescriptions written, sales numbers, profitability, etc. are not relevant.

Activity-based measures have both strengths and weaknesses.  On the strength side activity based measures are:

• Easy and cost efficient to gather


• Easy to explain to non-MA colleagues


• Non-controversial

Unfortunately, activity based measures also are:

• Not outcomes focused


• Not usually tied to the direct concerns of the organization


• Potentially misleading

While it may be interesting to know that the MSL group conducted 150 meetings with KOLs in the last quarter, it tends to beg the question – So What?  Activity-based measures tend to focus on the tactics (e.g. reach out to KOLs) rather than the goals (e.g. ensure broad awareness of the latest disease state information).

Activity-based measurement is very appropriate for certain MA functions.  For example, on CME funding, which by definition must be a hands off process, the only important measure is activity-based: Did the organization fund the number of CME programs it had intended to fund?

Or, in the Medical Information function, the key measures relate to answering inquiries and responding.  Activity-based measures (e.g. total number of inquiries managed, average turn-around on inquiries, number of inquiries requiring a second contact) are very appropriate for this type of service.

Finally, activities with multi-year goals like post-approval clinical trials, use activity-based measures to ensure that the overall program is on track.  So activity-based measures like patient recruitment and data collection provides valuable insight.

Activity-based measures are a necessary for MA’s ability to track progress against its goals, but they not sufficient to account for all of MA’s goals.

Subjective Measures

Most MA organizations have goals that require a more subjective type of measure.  Some of these goals relate to the degree of awareness or understanding in the healthcare community.  Measuring understanding or awareness is not as simple as measuring the number of discussions about a certain topic.  Instead, to measure progress on these types of goals, MA needs to gather more subjective data.  Simply put – it needs to ask.

For goals related to subjective measures, the only practical way to measure success is through research.  Unless a company is very lucky and finds that one of its objectives just happens to align with a topic that is already the subject of someone else’s research, this almost always means conduct primary research.

Research-based measures have their own strengths and weaknesses.  The strength of research-based measures include:

• Direct connection to goals related to healthcare community awareness and understanding


• Insight into the knowledge and beliefs of the healthcare community


• May provide insight into other, unmet needs

Research-based measures have some significant drawbacks, including:

• Expensive to conduct, leading to limited number of data points


• Subject to research bias


• May be taxing to the community that the organization is trying to serve

For research-based measures to be understandable, they usually need to be measured against a baseline.  For example, the measure of awareness of new disease-state information in the healthcare community after the efforts of the MA organization is only relevant if the measure of awareness before the efforts were known.  This requires the development of a baseline which both adds to the cost and requires good up-front planning.

But by far the biggest challenge of research-based metrics is the lack of experience and budget to conduct such research.  Many MA organizations do not have experience conducting this type of “market research”.  Without such experience, it can be daunting to initiate the research and structure it in the best manner.  Additionally, many MA organizations do not have money in the budget to conduct this research in a high quality manner.  This is both a product of the lack of experience and the view that this type of research is outside of the mission of medical affairs.

Best Practices

Clearly most MA organizations can benefit from a set of mixed objective and subjective measures.  Developing such a set of measures starts analyzing the strategy and goals outlined in the MA strategic plan.   From there a set of potential measures can be defined and prioritized.  Overall, MA should have between 5 and 10 metrics.  Specific functions within MA may have from 3 to 5 additional detailed metrics.  Once these metrics are defined, the approach for gathering each metric can be identified.  The activity-based metrics are typically gathered from existing tracking systems while the research-based metrics require more active management.

Once the research-based metrics are defined, a draft of a research outline can be developed, specifying the key questions, the research targets, the number of responses expected and the number of research samples to be taken.  This outline can be used as the basis for developing a request for proposal from a research company.

One caution – many organizations are tempted to combine this research with research already being conducted by the commercial market research function.  This may be a problem if the marketing questions are, in themselves, seen as promotional.  Most of my MA clients have been unwilling to risk this crossing of the line but some organizations do follow this practice.  Please work with your compliance function to discuss this approach.  Many organizations are not even comfortable with using research companies that are primarily commercially oriented but, frankly, there is little reason to avoid these companies.

After the approach to both activity- and research-based metrics is in place the next step is to determine how the results will be communicated and to whom.  This should become an ongoing process so it should be automated and standardized as much as possible.

Closing Thoughts

Explaining the value of MA remains one of the primary responsibilities of MA management.  For MA to be perceived as valuable to the organization, its progress against its strategies and goals has to be proven.

What has your experience been with value and measures in MA?  Leave your comments below.

Click on the email link to the right to suggest a topic for future posts.