The “Patient Percentiler”: Ghent University embarks on ambitious, innovative laboratory quality monitoring project
Create a platform that enables labs and manufacturers to continuously document and monitor quality.
GLIMS provides functionality enabling the lab to have an overview of the assay stability in the mid to long term – all based on real patient samples.
Analytical Chemistry Lab
“Quality in the lab is the underlying theme of our work,” begins Prof. Dr. Linda Thienpont from the Laboratory for Analytical Chemistry, Faculty of Pharmaceutical Sciences, Ghent University. It’s a notion that is probably quite familiar to most laboratories. But while external quality assessment (EQA) is a requirement for ISO 15189 accreditation, and internal quality control (IQC) is a key responsibility, labs can find that they don’t always have the tools they need to perform the quality checks they would like. So Ghent University got together with STT Consulting to find a solution.
“Together, we act as a scientific, vendor-neutral third party, aiming to create a platform that enables labs and manufacturers to continuously document and monitor quality, and that provides them with an overview of the assay stability in the mid to long term – all based on real patient samples,” explains Dr. Dietmar Stöckl of STT.
Prof. Dr. Thienpont and Dr. Stöckl highlight some key gaps in the EQA process. To begin with, these external controls work with artificial samples, not real patient samples “Theoretically, the artificial samples should behave in the same way as the real samples, but this simply isn’t always the case,” comments Dr. Stöckl. In addition, external controls are carried out only a few times every year, whereas labs need to check their quality on a daily basis.
And while EQA organisations offer peer group comparisons, there is a lack of transparency regarding the stability of the peer groups. This means they cannot provide quality specifications or make recommendations regarding quality, so the lab must interpret the data on its own.
Ghent University and STT Consulting decided to work together to address these gaps and launched the Patient Percentiler project. The goal of the first phase of this project was to provide labs with a tool for continuous monitoring of outpatient percentiles. “We have started with the 50th percentile, or median,” explains Dr. Stöckl, “but later we may extend it to the 10th and 90th percentiles.”
The project scope covers the 20 most used analytes, including electrolytes such as sodium, substrates, lipids, enzymes such as aspartate transaminase (AST) and alanine transaminase (ALT), glucose and creatinine. Recently the project was expanded to include thyroid hormone – free thyroxin (FT4) and thyroid stimulating hormone (TSH).
The project also provides quality specifications. “Right now, there are no generally accepted specifications on stability in quality assurance, and the differences can be huge, especially across countries. We give labs an idea of how stable, comparable assays should be. We have established scientifically based norms for theoretic stability that have been verified in the ‘real world’. These norms are strict, but we believe they are realistic: after all, if one lab can provide minimum stability for eight years, why shouldn’t another be able to accomplish the same?” says Dr. Stöckl.
MIPS helps project get off the ground
To collect the data – i.e. real, patient data on a daily basis – the project team needed a source: the labs’ own laboratory information systems (LIS). One LIS in particular offered key advantages for this project. “We first contacted MIPS, because their GLIMS LIS has a market share of 60% in Belgium, and has a pan-European reach. So from the start it seemed to be an excellent starting point for extracting the data,” Dr. Stöckl recalls.
“We thought that MIPS would need to develop new functionalities in order to participate, but we were delighted to discover that GLIMS already included what we needed: specifically a function for automatically calculating the daily median for a number of test parameters (including instrument, method, reagent, percentile, etc.), as well as a way to flexibly export this data!”
This meant that any lab with GLIMS could take part in the project free of charge, with just a simple configuration to start the automatic background generation of daily outpatient percentiles – nothing else was needed. GLIMS anonymises the data and automatically transfers it daily by email to the central server.
“What’s more, we wanted to include only outpatients, and GLIMS already contained a code that distinguishes outpatients from inpatients,” continues Dr. Stöckl. And while other LIS providers are now taking part in the project, they had to adapt their software to generate the data needed, slowing the process. In some cases, the labs themselves have developed, in-house, the functionality to export the data.
As Prof. Dr. Thienpont explains: “GLIMS offers a free source to collect the data. We were really fortunate to be able to collaborate with MIPS: if it weren’t for them, we might not even have embarked on this project.”
“The data we collect from the labs for the project is unique in the world: firstly, because we collect data on a daily basis and, secondly, because it is real patient data,” highlights Dr. Stöckl. To handle the data, the University of Ghent and STT Consulting (together with Bruno Neckebroek) developed web-based data visualisation and interpretation software allowing the project team to provide regular, individual laboratory reports as well as general reports about assay quality and stability.
Each lab can access its own data via a secure connection to the software, while the project team can see the complete set, allowing data to be grouped per manufacturer or per lab instrument.
‘Flagging’ abnormal results
With the first stage of the project well established – monitoring patient medians within the normal range – the project team is ready to move on to the second phase. “We call this the ‘flagger’ application: monitoring results that are not considered as normal,” explains Prof. Dr. Thienpont. “With this promising application, labs can see if flagged rates are stable or not. If a system is not stable, the number of values beyond the limits will rise; some labs may have more flagged results than others.”
To get phase 2 off the ground, the project team again worked directly with MIPS, and once again the existing functions of the solution provided an important shortcut for the project. “The GLIMS statistics module already offered the ability to automatically calculate flagger rates and forward them, so that our own IT system can read them,” says Dr. Stöckl.
“This project is aimed, like GLIMS itself, at helping labs improve quality and service, so we were very pleased to be able to support the University of Ghent and STT Consulting,” says Filip Migom, GLIMS product and development manager at MIPS. “GLIMS was designed to be comprehensive and meet the needs of all lab users, which meant that it already had the functions necessary for the Patient Percentiler. It’s a great example of how an industry player can support academic development projects using expertise gained through experience supporting many, diverse laboratories.”
Aligned with quality
With 130 labs around the world – representing 250 instruments – participating in the project, the advantages are already becoming clear. The labs now have a user interface that allows them to view the stability per instrument for each analyte at all times.
Uniquely, continues Prof. Dr. Thienpont, “if we observe an instability in a lab, or a sudden deviation from peer labs, we can contact the affected lab and give them guidance on interpretation. And in the near future, the flagger will permit us to invest even more in advanced interpretations and comparisons. This will allow the labs to intervene more quickly when needed, giving them peace of mind.”
The larger labs appreciate the ability to see whether all their instruments are well aligned – which is not always the case, even with instruments from the same supplier. A lab can quickly address any deviant values, while the alignment is also a key advantage when the lab is participating in a clinical study. Labs are even using the results to strengthen their negotiations with suppliers: they can clearly demonstrate that an assay is unstable, especially as the data comes from real patient samples.
The manufacturers, as well, welcome the data from the Patient Percentiler. They receive peer group reports that indicate stability for their own systems and let them benchmark their performance to the competition.
Prof. Dr. Thienpont notes, “One important result for the industry overall is that, while we were surprised in certain cases with the variability in stability for certain manufacturers and analytes – for example in liver enzymes and even compounds like chloride and sodium, which are supposed to be very stable – in general we find that many labs are performing very well: they are doing everything possible to control stability.”
Collaboration creates potential
“We are very pleased that this project is receiving international recognition, including the Westgard QC award,” says Dr. Stöckl. “We are already making a good contribution to quality, with an overview of stability for almost 95% of all in vitro diagnostics performed in developed countries. In the longer run, we would like to provide a kind of biological ‘clearing house’: a tool for big data research that could be used for e.g. governmental healthcare decisions. We hope we will be able to set up a collaboration with other organisations, such as EQA providers, sharing the same scientific vision.”
“It may seem ambitious, but we already have a concrete example: together with the manufacturers I’m currently working on an IFCC project involving standardisation of thyroid function tests. Before implementing the new calibration status, the FDA wants the manufacturers to demonstrate the alignment of assays as a group and to monitor in the mid to long term the sustainability of that calibration status. I was able to propose doing the documenting and follow-up within the Patient Percentiler, which provides us with a straightforward reporting tool. So you see,” concludes Prof. Dr. Thienpont, “the potential of this project, which allows us to collect daily and real patient data and then make use of it, is very far-reaching.”
Last but not least, Prof. Dr. Thienpont and Dr. Stöckl want to give special thanks to Kenneth Goossens, PhD student, who manages the project with great dedication.
Contribution of MIPS and the GLIMS LIS
- MIPS worked directly with the project team to determine how participating labs using GLIMS could easily and without cost automate the daily collection and communication of the real, anonymised patient data.
- GLIMS includes a very wide range of applications and functionalities, making it an ideal source for supplying data for projects of this kind.
- GLIMS directly interfaces with lab instruments, so it can centralise data from all those devices.
GLIMS QC module:
The GLIMS quality control (QC) module supports laboratories to ensure the laboratory quality control practices they need to achieve satisfactory analytical quality:
- All stakeholders within the organisation who are reviewing their quality have access to all qualitative and quantitative QC data centralised and available in GLIMS.
- The GLIMS QC module offers support for a broad range of result verifications, including Westgard rules.
- GLIMS can calculate Day Median and an average of normals (AoN).
- Statistical values can be calculated using the standard mathematical formulas or Tukey.
- As a European solution, GLIMS is compliant with international recommendations, including the German Rilibäk requirements.