Topic 43 – Use of Net Promoter Score Measures to Evaluate MSLs

A couple of my clients have discussed the use of the Net Promoter Score lately so I thought I would address it in my blog.

Quick background:

The concept of the Net Promoter Score was introduced in a Harvard Business Review article in 2003 by Fred Reichheld of Bain & Company. The net promoter score is measured by asking a single question: “How likely are you to recommend the company/product/service to a friend or colleague?” and is usually measured on a 0 to 10 basis. Scores of 9 and 10 are called Promoters, scores of 0 to 6 are Detractors and scores of 7 and 8 are called Passives. The Net Promoter Score is calculated by subtracting the percentage of customers that are Detractors from the percentage of customers that are Promoters.

People like the net promoter score because it is a simple measure of loyalty and when it is paired with an open ended question that asks why the particular score was given, it provides insight into what is important to the customer.

So, does the Net Promoter Score (NPS) provide value to MA? My research has not been able to find a single academic or metric-driven study on the use of NPS in MA specifically related to MSL activity. Common sense says this approach should be helpful but for now anyone using this approach is in experimental mode.

In MA the NPS question is often modified to be:

• "How likely are you to recommend engaging with [COMPANY X] Medical Science Liaisons to your colleagues or peers?” OR
• “How likely are you to recommend working with [MSL NAME] from [COMPANY X] to your colleagues?

PRO’s of Using NPS in MA:

• Brief nature of survey makes it suitable for rapid deployment immediately following MSL interaction to avoid the “blending” affect that occurs when HCPs are asked about MSL performance on a standard survey often weeks after their last interaction
• Relatively inexpensive to conduct compared to other market research
• NPS can help gather insights into what an HCP value in an MSL interaction, if open ended questions are employed as well

CONs of Using NPS in MA:

• Message vs Messenger: When an HCP recommends working or engaging with an MSL is that recommendation based on the quality of the content of the interaction or the interpersonal qualities of the MSL herself or himself?
• Not comparative: NPS does not give insight into whether HCPs recommend your MSLs any differently than they recommend competitor MSLs. Perhaps HCPs in a particular therapeutic area simply recommend all MSLs the similarly regardless of company.
• Not clearly actionable: If your NPS drops from one month to the next, what action should be taken? Some insight might be provided by the open ended questions but those responses are often only provided by the most dissatisfied

Given the inherent challenge, it is my opinion that the NPS is still a worthwhile measure, but it needs to be gathered as a part of a broader market research effort to give it the context that can help tease apart the reason for the scores.

The most effective NPS is gathered as soon as possible after the last interaction. In the case of MSLs, a system should be established to seek this guidance directly after a contact has been noted in the company’s contact management system. And, like all market research with HCPs, participation is highly impacted by compensation, so sufficient compensation must be offered to ensure enough participation to make the measure meaningful.

What is your experience with NPS? How do you frame the question? Share your experiences by clicking here.

Topic 34: New FDA Consumer Advertisement Guidance and Potential MA Impact

Have you seen the new FDA guidance about disclosing risk in consumer-directed print advertising that came out in February? (You can see it here) Unless you are a gluten for FDA guidance-reading punishment, my guess is that you skipped this one since it seems to be commercially focused.

BUT, there is actually something that MA should be aware of and perhaps an opportunity to add some value to our commercial brethren. The focus of the guidance is straight forward – under current law print advertising has to also disclose risks, and the safest approach for disclosing that risk is to publish the full package insert (PI) along with the print ad. As we know PIs are a tough read normally, but when shrunk down to fit in a magazine they are almost unreadable and certainly mostly incomprehensible to the very audience they are supposed to be protecting – consumers.

This has not been lost on the FDA and the guidance linked to above was entirely focused on resolving this issue.

In an FDA survey, few respondents reported reading half or more of the brief summary presented in the traditional format. Of those who read at least some of the brief summary, 55 percent described it as hard to read. Over 40 percent of respondents in the survey reported they do not usually read any of the brief summary in direct-to-consumer prescription drug print advertisements.

The FDA realizes that the full PI is aimed at medical professionals and full of details that the vast majority of consumers don’t care about like clinical pharmacology or chemistry. So the FDA is suggesting that manufactures should have the flexibility to replace the PI with something they are now calling “consumer brief summary.”
What is a consumer brief summary? Per the guidance it is an explanation written in consumer-friendly language (ie. drowsiness not somnolence) that includes:

• Boxed warnings
• All contraindications
• Certain information regarding Warnings and Precautions:
  • The most clinically significant information from the Warnings and Precautions section(s) of the PI;
  • Information that would affect a decision to prescribe or take a drug;
  • Monitoring or laboratory tests that may be needed;
  • Special precautions not set forth in other parts of the PI;
  • Measures that can be taken to prevent or mitigate harm
• Most frequently occurring Adverse Reaction, and those ARs that are serious or that lead to discontinuation of use, and the severity of the risk
• Indications for use
• Significant drug interactions

And this is where Medical Affairs comes into play. Now our commercial colleagues and their agencies are going to be needing to develop information that includes medical judgement, like:

• What is the most clinical significant information?
• Why is that information considered most clinically significant?
• How do practitioners view what is most clinically significant?
• What information should affect the decision to take or prescribe the drug?

These questions are great ones for Medical Affairs to either provide guidance, answer directly or gather information from practitioners during their interactions to answer these questions and others. Given the proper but strong firewalls between MA and commercial, this new guidance provides a value-added opportunity for MA to provide some guidance to commercial.

MA leaders may wish to discuss this topic with their commercial colleagues.

Do you have any thoughts on the new guidance? Leave them in the comments by clicking HERE and scrolling down to the comment box.

Topic 31: Changing Relationship between Medical Affairs and Access and Reimbursement

Note: This is a revised and extended version of a post I first published two years ago. This issue continues to evolve and be an area of focus for many MA Leaders.

Introduction

I received a question about what an optimal relationship should be between MA and Access & Reimbursement in the US. Access & Reimbursement (AR) is the function in pharma that is primarily responsible for negotiating the relationship between the company and the major payers and/or providers. In some organizations this group is known as Managed Markets, Market Access, Payer Relationship, or Contracting. Their primary goal is ensuring that the company’s drugs are listed as advantageously as possible on the formulary of the payer.

AR has to make the case for reimbursement of their drug to a payer/providers Pharmacy and Technology Committee (P&T Committee) which is the body that ultimately makes the decision for the payer/provider. In the US, these P&T Committees consider the efficacy and safety of the treatment but they also consider the cost effectiveness of the treatment and its impact on total cost of care for a patient when deciding where to place the treatment on their formularies. The AR function has had to deal with a range of both government and private payers/providers, each with their own formularies.
The AR function in the US has grown in importance as the payers/providers have worked to limit their exposure to treatments they viewed as not cost effective through formulary placements that drive limitations like prior authorizations and co-payments for the patients. Even specialty areas like oncology, which used to have very few restrictions, are now seeing greater control exerted by the payers.

Defining a New Relationship

Just as the pressures on AR are forcing changes in the way they work with payers, those same pressures are changing the relationship between MA and AR. In the past, MA had a limited role to play in AR. For example, MA may have had a responsibility to train AR Account Managers on the scientific underpinnings of a new treatment, not dissimilarly to how MA may train sales staff. And AR might have occasionally asked an MA resource, typically Field Medical, to provide some scientific support for a formulary presentation. But, in general, these situations were ad hoc and limited.

However, now that AR’s success more directly drives the success of the pharma company and thus their importance has grown, the relationship between AR and MA is changing.

For a P&T Committee to control costs, they must be able to differentiate between treatments. This drives two major scientific needs:

1. The formulary committee needs a more robust scientific understanding of the drug’s properties, its known efficacy, its known risks and its place in the overall therapeutic area’s treatment options


2. The formulary committee is demanding more specialized data, specifically health economics and outcomes research (HEOR) data like cost effectiveness and total cost of care, and comparator data to allow them to understand the full impact of the drug’s use

Both of these ramped up requirements have direct impact on MA’s relationship with AR.

More Robust Scientific Understanding

MA’s role in terms of providing scientific support for P&T Committee presentations is growing from a part of the presentation to the core of the presentation. And with that growth comes the need for greater specialization by the presenters.
MA which develops and delivers the scientific elements of those presentations need to have a much more robust understanding of their P&T Committee audiences and how to effectively meet their scientific needs. This is leading to two trends in MA:

1. Much greater degree of training for Field Medical on the role of AR and P&T Committees
   OR


2. The identification and hiring of full time Field Medical-type roles specifically targeted at supporting AR

Given the importance of AR, supporting their needs can no longer be seen by MA as a side responsibility. Instead, it needs to be a core responsibility and an investment in training or personnel is needed to ensure that that Field Medical is prepared to adequately support this need. In addition to training, this will require new measures to be put in place to track Field Medical effectiveness, which I will discuss in a future blog post.

More Specialized Data

In many organizations, MA has taken the lead in developing data post-approval. And while HEOR has always been a part of generating that post-approval data, its importance has grown significantly. The increased demand for HEOR data has a number of implications:

• HEOR data should start being gathered in Phase 3B at least, and thus MA HEOR leaders need to engage with clinical development to ensure endpoints are included to begin the generation of HEOR data sets


• Post-Approval Data Generation Plans, which should be developed by MA to help drive the post-approval study efforts, must give greater consideration to the HEOR needs


• The priority given to Investigator Initiated Studies that cover HEO subjects may need to increase


• The need for specialized MA resources dedicated to developing and managing HEOR may need to increase, with new dedicated positions developed


• Processes for ensuring that the input of AR is gathered in the development of HEOR protocols should be re-examined to ensure that the results will meet the demands of the key formulary committees

In some companies have decided that HEOR is so important to AR that they have shifted the leadership of this research to the AR function itself. Whether the HEOR function reports to AR or is developed within MA, the need to ensure that the changing needs of P&T Committees are addressed has become a major priority for post-approval research.

Closing Thoughts

MA’s role as the owner of scientific education and communication for post-approval drugs is a critical element in today’s formulary-driven environment. MA needs to be an active partner to AR as it works to ensure patient access to the company’s drugs.

In your experience what has been the key to effective MA / AR partnership? Leave your comments below.

If you have a topic you would like me to cover, please email me from the link to the right.

Topic 28: Big Data and Medical Affairs

“The era of Big Data is here!”  That may be true but what does that mean for Medical Affairs?  As in all of biopharma, MA is comfortable working with data.  So much of our work revolves around discussing data and the implications of data that many people may think that we were already living in the era of Big Data.

But for most MA organizations, the data sets we have focused on are purpose generated - either our own data or data from similarly-scaled studies conducted by others.  Big Data refers to something different.  I like the differentiation that SAS uses when comparing Big Data to the past data sets.  They break it down to four “V”s and a C:

• Volume: Hugely increased data volume from the past


• Variety: Since the data is produced in many different ways, it has many different formats and structures


• Velocity: Both how fast the data is being produced and how fast it must be processed


• Variability: Inconsistent data flows, with peaks and valleys


• Complexity: Driving value out of these data sets is highly complex and difficult

This is not your grandfather’s data sets.  What are some examples of Big Data as relevant to biopharma and MA:

• Electronic Health Records data from a variety of sources


• Search engine data (see an example of analyzing search data to find safety signals here)


• Sunshine Act Physician Spend Data (when it becomes available)


• Social media data


• Competitors clinical trial data as it is released

Contained within these and many Big Data sources are key tools for MA:

• Valuable therapeutic information


• Unique customer insights


• KOL identification and information


• Visibility of competitors drug development and support efforts


• Important drug safety signals

But, none of these benefits can be achieved unless the question is asked and the data is analyzed.  I would suggest that effective MA organizations of the future will need to have the capacity to ask and answer these types of questions.

In order to do so, MA organizations will either need to build or have access to increased levels of biostatistical and epidemiological resources.  And these resources need to have skills directly related to Big Data.  The characteristics that differentiate Big Data from existing data sets also means that many existing biostats and epi staff do not have the expertise or confidence working with these large, external data sets.  MA organizations need to ensure that people with exactly these skills sets are available within their organizations or from outside vendors and that these resources have the capacity to support MA.

Then, MA needs to improve its overall level of confidence defining Big Data questions, conducting Big Data analysis, and discussing the results with others.  Given the difference in the source of this type of data, the way this data is presented and discussed must be different too.  Everyone in MA, but especially the MSLs, must become more comfortable understanding the nuance of this type of data analysis and discussing both the strengths and weaknesses of working with Big Data.

The era of Big Data is here.  MA has a long history of effectively using data and explaining data in support of its organization.  MA leaders must investigate and embrace Big Data to take advantage of all the tools available today.  The questions unasked are always the questions unanswered.

What is your experience with Big Data?  Please leave a comment.

Topic19: MA Impact of the Caronia Ruling

As most of you know, the Caronia Ruling by the Second Circuit Court of Appeals found that a pharma rep was within his 1^st Amendment rights to discuss off label use of his company’s drugs, assuming that those discussions were truthful.  This went off like a rifle shot within the pharma world with all sorts of discussions, you can read some of them here, here and here.

It is very likely that this ruling will be appealed to the full Second Circuit, and/or directly to the Supreme Court.  But there are reasons to believe that the Supremes may be open this interpretation given their past rulings on similar subjects.  Regardless, until this is settled it only applies to the Second Circuit so unless you are a pharma company only doing business in the Second Circuit in and around New York, you can’t make much change.

BUT, what if this becomes the new law of the land?  What does it mean for Medical Affairs?  That’s what I want to explore in this blog post.

The most obvious impact it has is on all the current focus we place on “proactivity”.  I have discussed this topic in detail here, here, and here.  I bemoaned the unclear state of the current regs here. Now we are imagining a world where the issue of proactivity has to be seen in a completely different light.  MA avoided proactively discussing off-label data on our products because proactivity implied promotion and it is (or in our scenario was) illegal to promote off label.  Under this scenario that thinking would be wrong.  Replaced, potentially, with a focus on “truthfulness”.

Now, for MA at least, this is an expectation we are more than willing to meet.  In the past MA has typically thought of any peer reviewed study as truthful but some of the commentators are suggesting that the definition might become the same one that they use for the FTC.  The FTC definition id focused on “…competent, reliable scientific evidence supporting the claims you are making…” so it may not require a peer reviewed journal publication.  The one caveat, however, is that if you conduct a test and find that the content being shared is misleading to 20% or more of the targeted consumers, than it is not considered truthful.  In general, then, although peer reviewed journals would not be a requirement, it would represent a fairly safe harbor to avoid the risk of sounding misleading.

So, in the future world we are imagining, our field force of MSLs would be free to go out to HCPs armed with peer reviewed journal articles, and introduce the HCP to that article and then proceed to have a scientific exchange about the results, assuming they stick to findings documented in that or other peer reviewed articles.

Scientifically speaking, this frees MSLs to have very wide ranging discussions with HCPs at their initiation and allows for much greater control over the type of discussions that we have with the HCPs.  It will allow MSLs to show much greater value to the organization by allowing the targeting of discussions that are the most meaningful to our products.  And if you think is hard to hire MSLs today, watch out.  With that increase in value will come greater investment in both MSLs and in Investigator Initiated Studies and P4 studies that will now be seen as more valuable as well.

What do you think?  Do you think it we will see the end of proactivity restrictions in the next three years?  Leave your comments above.

Topic13: Medical Information Groups

I was talking with a friend the other day who is charged with running the medical information (MI) function for their small pharma company.  Currently they outsource the MI function and she was wondering if it made sense to bring the function in-house of keep the current arrangement.  We discussed how effectively the outsourced MI function works with the current MA group, including the MSL field force, and the answer is that they have some formal links but rarely talk.  We talked some more and reached some conclusions, but I thought this might be a good topic for a post.

MI is both a regulatory mandated function, and a critical component in achieving MA’s educational and public outreach goals.  It is possible to structure and run the MI function merely to meet the regulatory requirements but that would represent a significant waste of an opportunity.

From a regulatory perspective, the MI function needs to be structured to answer calls from both patients and healthcare professionals (HCPs), differentiate between inquires and pharmacovigilance calls and route the calls appropriately, answer questions directly when appropriate, distribute accurate and approved materials to appropriate audiences, and develop outgoing medical communications on important issues (Dear Doctor letters).  It needs to be staffed by people with appropriate qualifications to answer medical and technical questions about the use of company’s products.

But, some organizations gain much more benefit from their investment in MI.  For one thing, MI is often the first sign that a problem is occurring relating to our products or issues are being raised concerning the scientific underpinnings of our products.  Spikes in the type and content of questions help provide a Zeitgeist view of the needs for education in the HCP community.  But, this only works if MA listens.  There must be processes and systems in place to ensure that MI is coding their contacts in a manner that allows for analysis and reporting in place to ensure that the information is summarized on a timely basis.  Finally, clear responsibilities need to be established for who will review and respond to this information.

Other organizations utilize the MI resources to more effectively partner with the MSL field force.  MI and MSL groups tend to talk with related but mutually exclusive audiences.  Given the limitations on MSL time, they need to focus that time on those HCPs that can offer the greatest impact on the medical community, the so-called Key Opinion Leaders (KOLs).  Based on research I have seen in the past, it is very rare for a KOL to call MI.  They tend to expect the answers to come to them through MSLs or research on the internet.  Non-KOL HCPs tend to call MI.  Given that, the opportunities for MSLs and MI to interact is not in terms of serving the same HCPs but instead in terms of “having each other’s backs.”  Often when MSLs are interacting with HCPs, they need to provide those HCPs with additional information and support.  MI is well suited to provide that information and support.  And, when MI is answering questions for HCPs they encounter situations that require more in-depth support, and MSLs are well positioned to provide it.

Establishing a strong partnership between MI and MSL groups brings value to both sides, yet it is not as common as you might expect.  Sometimes it’s due to organization structure – the two groups have unrelated reporting relationships and thus lack incentive to interact.  Other times it’s due to a difference in philosophy, where one of the two groups does not see value in interacting with the other.  Regardless, there is value in establishing this relationship, but it needs to be clearly structured and actively managed to avoid either group “dumping” on the other.

This leads back to the discussion of outsourcing MI.  To really gain value from MI, it needs to tie in easily with the rest of the MA organization.  It may be possible to pull that off with an outsourcer, but its unlikely.  For that reason, I suggest that insourcing MI brings more value to the organization.

My final thought on MI relates to technology.  A call does not have to be a phone call.  Web chats are significantly more efficient to manage and allow for careful scripting of responses.  Providing this service on an after-hours basis may allow more of those KOLs discussed earlier to actually connect.  And, I have yet to see a company develop an MI app, but I think we can’t be too far from having one – just tap the icon and link directly to an MI chat or other dialog.

 

What has been your experience with MI and how it relates to other function in MA?  Please leave your comments below.  And if this is your first time on the blog, please sign up on the right to receive email updates of the latest posts – don’t worry its spam free.

Topic9: HEOR Studies – Who Should Take the Lead

I was talking with a longtime friend and client the other day and he was describing a situation in his organization.  A tug-of-war of sorts was going on with HEOR studies.  In his business, the amount of funding going into HEOR studies was going up, and although they had typically be run out of the MA shop, there was increasing pressure to move all HEOR studies over to the Access and Reimbursement group.  We had a good discussion on this which I will try to recap here.

Traditionally all post-marketing research has been driven out of MA.  I use the term driven, because in many organizations the actual studies may be managed by a CRO or a clinical development resource.  But the group responsible for identifying the need for the study, budgeting for it, getting approvals to conduct the study and driving the protocol development was almost always MA.  There is a good reason for this, the Medical Director, who typically owns the scientific lifecycle of the compound is in MA and thus should be driving this process.

Now, post-marketing studies is a very large bucket.  It can include additional safety studies, both mandated and not mandated, reformulation studies, label expansion studies, HEOR studies, just to name a few.  And, given the huge range of post-marketing research, it is fairly standard that the Medical Director and MA in general partners with other parts of the organization that have expertise in these areas.

Fast forward to today’s world.  HEOR studies are increasingly critical to the medical and business case that Access and Reimbursement needs to make to payers in order to achieve a beneficial position in their formularies.  Investment in HEOR studies are growing by 0ver 40% in the past few years, as discussed here.  Given their knowledge of the audience, the data that they need and the competitive data that they are likely to have available, no one is in a better position to define the requirements for an HEOR study than the Access and Reimbursement team.

But, does that mean that Access and Reimbursement should be driving the HEOR studies themselves.  My answer remains been no.  Access and Reimbursement should be a defining voice in the development of the protocol, but once that protocol concept has been finalized it becomes an issue of study execution.  This is frankly how it works with most post-marketing studies.  While I can grant that an HEOR study has unique issues, so do many of the targeted post-marketing studies.

Having said all of this, it is also true that I, myself, have set up just such a function for an Access and Reimbursement client.  They had circumstances that made it clear that the MA function was not valuing the need for HEOR research and thus it was not getting done on a timely basis.  This was essentially a political issue but the only way to resolve it in the short-term was the create an dedicated function within Access and Reimbursement.

I am curious to know your experiences in terms of HEOR studies.  Leave me a comment below.  And if you have any question that you would like me to address, please click on the email link to the right.

Topic7: Coordinating MA and Commercial

A reader asks:  “Given the importance of consistent communication to the healthcare community, my company is seeking to better coordinate commercial and MA messaging.  What is the best approach for doing this?”

Thanks for the question.   The answer is: Very Carefully!  My sense is that your commercial function is driving this requirement.  I infer this from your question because you use the term “messaging” which is generally a marketing term.  Let’s remember that the purpose of marketing is promotional – promoting the sale and safe use of our products.  Since it is promotional, their communications are limited to on-label information.  If their communications go beyond the label, they are promoting off-label.  We all know this is against the law.

MA is allowed to discuss off label topics only under very specific circumstances – when they receive an unsolicited request.  MA is allowed to have some proactive discussions which are educational and non-promotional about related non-label topics depending on the rules of each organization.  See a detailed discussion of this here.

So, marketing is promotional and MA is non-promotional.  What “messaging” can a promotional and non-promotional functions share?  Conservative organizations would argue that any messaging topics that support the promotional needs of commercial is, by definition, not an acceptable topic for non-promotional MA.  But, many organizations would agree that promotional topics can focus on the product and its competitive positioning; while there can be related non-promotional topics such as the underlying disease state.

Most organizations that allow some coordination between promotional and non-promotional topics understand that this is a risk.  To avoid the risk of MA appearing as a promotional entity, most companies expect the coordination will go one way only:  from MA to commercial.  The theory is that MA is going to be educating the market regardless and the commercial function is just becoming aware of what MA is doing.

Going the other way, from commercial to MA, runs the risk of appearing as though MA is part of the promotional machinery.



So, reader, be careful.  Processes and systems must be put in place to ensure that the coordination is going the right way only.  Otherwise, you run the risk of a future whistleblower pulling out a document showing that MA has become a promotional resource.

For example, there need to be clear firewalls between the way MA is measured and the way commercial is measured on the effectiveness of their communication efforts.  We have discussed MA measurements here.  Commercial measures of effectiveness should be much more related to sales and penetration.

Thanks for the question!

If you have a question, please email me or leave it in the comments!

Topic 6: Measuring the Value of Medical Affairs

Introduction

A reader requested a discussion of different approaches for measuring the value of medical affairs.  This is a critical question and one that I touched on briefly when I discussed strategic planning in MA as it relates to launch.  But it is equally important overall for MA.

Like any function in pharma, MA must be able to communicate its value.  Value can be a tricky concept, however, for a non-commercial function like MA.  Value is ultimately a company-specific question.  It stems from the company achieving its strategic goals, which is why defining goals is so important.  Having said this, there are some general categories of value measures that we can discuss as well as some best practices around measuring value.

Value measures in MA typically fall into two types:

1. Objective measures


2. Subjective measures

Objective Measures

In MA, objective measures tend to focus around activity-based measurements.  Since MA cannot be seen as promotional, the other available objective measures like prescriptions written, sales numbers, profitability, etc. are not relevant.

Activity-based measures have both strengths and weaknesses.  On the strength side activity based measures are:

• Easy and cost efficient to gather


• Easy to explain to non-MA colleagues


• Non-controversial

Unfortunately, activity based measures also are:

• Not outcomes focused


• Not usually tied to the direct concerns of the organization


• Potentially misleading

While it may be interesting to know that the MSL group conducted 150 meetings with KOLs in the last quarter, it tends to beg the question – So What?  Activity-based measures tend to focus on the tactics (e.g. reach out to KOLs) rather than the goals (e.g. ensure broad awareness of the latest disease state information).

Activity-based measurement is very appropriate for certain MA functions.  For example, on CME funding, which by definition must be a hands off process, the only important measure is activity-based: Did the organization fund the number of CME programs it had intended to fund?

Or, in the Medical Information function, the key measures relate to answering inquiries and responding.  Activity-based measures (e.g. total number of inquiries managed, average turn-around on inquiries, number of inquiries requiring a second contact) are very appropriate for this type of service.

Finally, activities with multi-year goals like post-approval clinical trials, use activity-based measures to ensure that the overall program is on track.  So activity-based measures like patient recruitment and data collection provides valuable insight.

Activity-based measures are a necessary for MA’s ability to track progress against its goals, but they not sufficient to account for all of MA’s goals.

Subjective Measures

Most MA organizations have goals that require a more subjective type of measure.  Some of these goals relate to the degree of awareness or understanding in the healthcare community.  Measuring understanding or awareness is not as simple as measuring the number of discussions about a certain topic.  Instead, to measure progress on these types of goals, MA needs to gather more subjective data.  Simply put – it needs to ask.

For goals related to subjective measures, the only practical way to measure success is through research.  Unless a company is very lucky and finds that one of its objectives just happens to align with a topic that is already the subject of someone else’s research, this almost always means conduct primary research.

Research-based measures have their own strengths and weaknesses.  The strength of research-based measures include:

• Direct connection to goals related to healthcare community awareness and understanding


• Insight into the knowledge and beliefs of the healthcare community


• May provide insight into other, unmet needs

Research-based measures have some significant drawbacks, including:

• Expensive to conduct, leading to limited number of data points


• Subject to research bias


• May be taxing to the community that the organization is trying to serve

For research-based measures to be understandable, they usually need to be measured against a baseline.  For example, the measure of awareness of new disease-state information in the healthcare community after the efforts of the MA organization is only relevant if the measure of awareness before the efforts were known.  This requires the development of a baseline which both adds to the cost and requires good up-front planning.

But by far the biggest challenge of research-based metrics is the lack of experience and budget to conduct such research.  Many MA organizations do not have experience conducting this type of “market research”.  Without such experience, it can be daunting to initiate the research and structure it in the best manner.  Additionally, many MA organizations do not have money in the budget to conduct this research in a high quality manner.  This is both a product of the lack of experience and the view that this type of research is outside of the mission of medical affairs.

Best Practices

Clearly most MA organizations can benefit from a set of mixed objective and subjective measures.  Developing such a set of measures starts analyzing the strategy and goals outlined in the MA strategic plan.   From there a set of potential measures can be defined and prioritized.  Overall, MA should have between 5 and 10 metrics.  Specific functions within MA may have from 3 to 5 additional detailed metrics.  Once these metrics are defined, the approach for gathering each metric can be identified.  The activity-based metrics are typically gathered from existing tracking systems while the research-based metrics require more active management.

Once the research-based metrics are defined, a draft of a research outline can be developed, specifying the key questions, the research targets, the number of responses expected and the number of research samples to be taken.  This outline can be used as the basis for developing a request for proposal from a research company.

One caution – many organizations are tempted to combine this research with research already being conducted by the commercial market research function.  This may be a problem if the marketing questions are, in themselves, seen as promotional.  Most of my MA clients have been unwilling to risk this crossing of the line but some organizations do follow this practice.  Please work with your compliance function to discuss this approach.  Many organizations are not even comfortable with using research companies that are primarily commercially oriented but, frankly, there is little reason to avoid these companies.

After the approach to both activity- and research-based metrics is in place the next step is to determine how the results will be communicated and to whom.  This should become an ongoing process so it should be automated and standardized as much as possible.

Closing Thoughts

Explaining the value of MA remains one of the primary responsibilities of MA management.  For MA to be perceived as valuable to the organization, its progress against its strategies and goals has to be proven.

What has your experience been with value and measures in MA?  Leave your comments below.

Click on the email link to the right to suggest a topic for future posts.