Introduction
In the life science industry, there are fair possibilities to solve manually intensive work with the right usage of Generative AI. Popular examples include: automatic content summarization, asking follow-up questions based on already shared data / notes to bridge gaps, generation of marketing materials for review or generating relevant content from specific websites engaged in publishing relevant research. In all of these cases, the common issues are how to test (specifically “user acceptance test”) and certify the product before going live in production.
Common obstacles include:
- Non deterministic nature of output:
If the same input is given at various points of time to the application built with a certain Large Language Model (LLM), output is not always the same. It becomes tricky to validate every time with humans in the loop, due to the subjective nature of output validation. An example of this is content validation. - Hallucination:
Sometimes output may be out of context due to the nature of interaction. To address it, various strategies can be adopted at the architecture level with additional components. Examples include establishing a custom knowledge base and create embedding, defining prompts, and testing / refining prompts at lower environments to make sure that issues related to hallucinations are well documented. - Regulatory & Compliance:
In the life science industry, there are guidelines related to GxP (i.e., good ‘x’ processes where x covers Research, Development, Manufacturing, Supply Chain and Commercialization). Now for GenAI or AI in general, some of these processes may be perceived as posing an obstacle at the end before production goes live, and therefore need a rethink for how best to achieve compliance and meet regulatory requirements, especially as regulatory frameworks in the AI and ML (machine learning) space continue to evolve.
How to take care of these Obstacles?
There is well documented literature available from various consulting companies to address common obstacles, including the above examples, as suggested in the reference section below. Now the context and contacts will vary from one client to another based on the internal approach adopted by them in their software engineering process. The goal is to have solution to these obstacles around 3 areas:Content Generation:There will be a combination of generally available content in the internet (as LLMs are trained with a snapshot of data freely available in the internet) and enterprise data fed through knowledge base. As 100% accuracy achievement is not possible, the client team needs to agree on a certain % of accuracy (say more than 90%) and put humans in the loop to correct the remaining 10% to gradually learn over a period of time.Content Summarization:The client can follow the same approach as described above for content generation. The technical team can play with various parameters of the LLM model (i.e. top p, top k and temperature) to make sure that output is not missing the threshold target adopted as part of the user acceptance test.Hallucination:There is no practical way to solve issues originating from hallucinations with technology, as it encompasses risks associated with regulatory compliance, governance model and difficulty in managing these new types of risks out of new technology adoption. So cross-functional collaboration and timely resolution of risks are among the best practices of risk management that need to be adopted by project managers.
Conclusion
New technology adoption can create disruption in the marketplace. Enterprises in life sciences create new products that can cure disease or develop new therapies to improve the well-being of humans. Our project managers can be a great pair of helping hands to remove those obstacles by adopting the above in day-to-day project management as a basic minimum practice to enable the product’s delivery in production for general availability by our users (patients, healthcare providers, caregivers, etc.).
References:
- Deloitte AI Institute - AI Insights | Deloitte US
- The Gen AI Revolution | Cognizant
- Despite Increased Investment and Early Enthusiasm, Data and Risk Remain Key Challenges to Scaling Generative AI, Reveals New Deloitte Survey - Press release | Deloitte US
