March 27, 2012

Know Your IT Systems Vendors

Posted in Cloud, Computer Validation, IT, Vendor Audits tagged , , , , , , , , , , , at 9:38 AM by Solutions2Projects, LLC

IT systems and infrastructure are critical to any organization.  This is especially true for life sciences companies selecting and implementing IT systems critical to the business functions supporting compliance functions. Regulatory bodies expect life sciences companies to demonstrate control over these elements regardless of whether they are the ones developing or maintaining the IT systems (infrastructure, software, etc.).

Companies cannot simply toss the responsibility over the fence to the vendors. Life sciences companies are still responsible for the integrity of the data and control over the systems.  They may delegate but only after verifying the vendor can meet the compliance and control requirements. 

This is where vendor audits come in to play. 

Vendor audits for software are not new.  Over the past decade I’ve seen the importance of vendor audits for software wax and wane and wax again.  In light of the increase in cloud and hosted solutions chosen by companies to decrease overall spend, the need for vendor audits is critical.

And, as biotechs become more virtual and more services are outsourced (CRO, CMO, data management, complaint handling, etc.), it is imperative that companies verify their vendors meet compliance requirements as well as their own procedural and process requirements.   The vendor’s IT systems and controls must meet the requirements as if they were hosted by your own company.  Not all vendors perceive the need to meet compliance requirements at the same level and you need to know before you enter any agreements.  Once you’ve signed the contracts, you’ve lost your leverage for process improvements and controls. 

Why conduct the audits? 

  • Gain high level of confidence that the computerized system will meet technical, commercial and regulatory requirements (GAMP 5)
  • Confirm the supplier builds quality and integrity into the software product during development
  • Leverage knowledge, experience and documentation of supplier (GAMP 5) to potentially reduce validation effort
  • Confirm processes and controls when  outsourcing IT / software functions (SaaS, PaaS, IaaS, hosted solutions, co-locations)

When should audits be performed?

  • For high risk systems / outsourced services
  • Before any contracts are signed!
  • Scheduled follow up audits based on
    • Audit results
    • External audit program
    • Risk assessment
    • Significant vendor business changes
    • When there are issues with the vendor

How are audits performed?

  • Similar to other vendor audits for CMOs or other critical suppliers
  • Plan for the audit and communicate expectations to the vendor
  • Conduct the on-site audit (for IT systems, Quality and IT representatives should participate)
  • Summarize findings with the vendor at the end of the audit
  • Document findings in an audit report and provide to the vendor for a response
  • Follow up on observations and document

The financial cost of, and risk associated with, software solutions has increased exponentially which means that it is imperative for organizations to understand what they are getting into before they sign on the dotted line.  The cost of a software or IT system blunder can be expensive in terms of resources, time and can make or break a life sciences company.  If you cannot demonstrate control, and therefore the integrity of your data, for systems supporting drug product administered to patients, a regulatory body may not grant approval for your product or could shut down manufacturing operations.  Your company owns the data and the responsibility even if it service is outsourced. 

Knowing your IT vendors gives you the knowledge to reduce the risks associated with the IT solutions in your life sciences company.  Without this knowledge, you are powerless to defend your risk assessment and risk mitigation strategy to regulatory agencies.


June 22, 2011

Begin with the end in mind

Posted in Computer Validation at 5:39 PM by Solutions2Projects, LLC

I may not agree with everything Stephen Covey says but I have to admit it truly is critical to begin with the end in mind when developing or implementing a custom application or system in the life sciences industry. One of my clients is learning the hard way about the extra time and money associated with remediating a system to meet compliance requirements and it is turning into a truly painful process for all of us.

What could they have done differently? Planned ahead. This would have meant asking questions not just about what they wanted the system or application to do but how they expected to use the system and the data in the short and long-term.   I like to imagine they would have made some different and better decisions had they spent some time up front thinking about these things. 

Taking into consideration whether a system or application might be used to capture data for a submission or to support manufacturing processes is critical. If the system or application will do either, then it is critical to build controls into the system, define the electronic records in the system, define procedures to control the system, and seriously consider 21 CFR Part 11 compliance in advance of releasing it into production use. In 2011, it’s rather hard to defend a non-compliant system to the FDA or other regulatory bodies especially when it is critical to the manufacturing or quality testing process. 

Remediation is expensive and risky.  It’s risky in that you may not be able to get the system or application into compliance and even if you do, how can you adequately rely on the electronic records generated prior to implementing controls?  Companies have received 483s for not establishing control over their systems subject to 21 CFR Part 11. 

I understand that early stage companies are short on resources and money (generally) and look for ways to defer some activities until there is more money and more time (which there never is).   But unless you do the work up front to define the intended use and overall requirements including compliance requirements, in addition to planned life expectancy for the system,  how can you adequately assess the potential risk to your organization if you defer activities?   Or, if you put in a stopgap solution, do you really thing there will be time and resources to put in another solution once it’s in place and relieving most of the pain points but not satisfying compliance requirements?    As a colleague of mine states often, there is nothing as permanent as a temporary solution.

May 6, 2011

More on Computer Validation

Posted in Computer Validation at 7:54 AM by Solutions2Projects, LLC

Every day provides us with an opportunity to learn and grow.  Each time there is an issue on a project, we reflect on what we could have done differently to avoid the issue in the future or mitigate the impact.  A recent client project was no exception.

We were asked to participate in a project as individual contributors rather than driving the project for the client.  Other vendors with more experience with the software/system were assigned tasks that we typically handle but we (the client and I) assumed they would be more efficient and better suited to handle the tasks.  We were wrong. 

What we learned is that not everyone can write a good validation test script even if they claim to have been doing it and other clients have been satisfied with the scripts.  Apparently, other clients don’t actually read the scripts and verify the tests confirm requirements.  Test script development is fairly basic from our perspective.   

  • Define the objective
  • Establish acceptance criteria (usually tied to requirements)
  • Create specific action steps for the tester
  • Document expected results based on what should happen as a result of the action
  • Add assumptions and pre-requisites to ensure smooth set up and execution

Most importantly, actually verify the requirements are verified in the tests.  Screen shots of available functions do not actually confirm functionality. 

It seems that some vendors have been more focused on checking the proverbial boxes  than producing value-added validation deliverables.  No wonder most people groan, cringe, and look for ways to avoid computer validation.  Vendors and some internal computer validation staff are not looking at computer validation as an opportunity to increase the likelihood of success of a system but as a time consuming, paper generating, waste of resources.

Computer validation is designed to provide organizations with a high degree of confidence that the system they implement meets the requirements and is maintained in a controlled fashion.  This is done by

  • Defining the requirements
  • Configuring/developing to the requirements
  • Testing to verify the requirements are satisfied
  • Controlling changes to the system

While there are regulatory requirements that mandate computer validation, the investment decreases overall cost over time and increases the likelihood that your organization will get what it wants and needs in a system.  I continue to argue that if done well, computer validation makes good business sense by driving good selection, implementation, and maintenance practices for IT related systems. 

On this particular project, we tried to work with the client to reduce costs and time by utilizing the vendors canned requirements and test scripts (it’s a standard in the industry with very few variations across organizations) only to learn that the requirements were not fully tested in the standard test scripts and were not well written.  Solutions2Projects stepped in to rework the test scripts and create new ones to satisfy the requirements.  The client is now satisfied and has a defendable validation package.  Next time, we won’t make the same mistake.

April 27, 2011

Installation Qualifications

Posted in Computer Validation at 10:16 AM by Solutions2Projects, LLC

It’s always interesting working with vendors on installing applications and databases in a qualified manner.  Most of the installation technicians do not understand what is really required from an installation qualification perspective and they generally don’t follow the installation guides. 

When generating IQ documents, we generally refer to installation guides and work with the technicians on what should actually happen, assuming we have access to them.  More often than not we get it wrong and there are lots of edits made during the execution.  Since the objective of the IQ execution is to document the baseline installation, from a compliance perspective we don’t really have an issue with the number of changes but it is still frustrating in that vendors in our space just don’t ‘get it’ when it comes to validation documentation. This is especially frustrating when these same vendors sell validation packages as part of their offering.  Somehow the knowledge in the validation services group is not communicated to the installation group.  Validation is not a separate activity…it’s part of the overall process!

Recently I worked on a client project where we didn’t have access to the technician until the day of the installation.  We had the installation guide and hardware/software recommendations from the vendor so we generated the IQ from this documentation.  The technician arrived and joked that he hadn’t seen the installation guide before and then when he saw the horror stricken look on my face he said he had seen it before but doesn’t follow it.  For an application that is used in life sciences and is often validated, this is unacceptable.  

How do we get these software vendors to understand that validation by its very definition requires defining what you need/want before executing/performing and if you are going to be in this space that ‘winging it’ is not included in the definitions in our validation guidance documents?  I suppose I could send them the GAMP guidance documents on validating systems as a parting gift in the event that we are brought in after the purchase. 

I wonder if ISPE will give us a discounted rate on GAMP guides to share with these vendors to start spreading the word.  As a bonus, I’ll highlight and flag the critical sections pertinent to them! 

…Just in case you are curious about the recent client project, we were able to successfully execute the IQ but had to provide a lot of extra explanations as to the protocol generation errors.  It wasn’t pretty but the documentation meets the requirements.  And, we now know how to work with this same vendor at other client sites.