Why do I need QAPonline, I am already enrolled in an EQA?
Most EQA schemes distribute one or more samples at regular intervals and one (sometimes more) staff member completes the tasks and returns their assessments to the EQA provider. The laboratory manager may then distribute the EQA sample among the staff to complete and the manager then compiles their results as an Internal Quality Process. QAPONLINE does things in the opposite way. QAPonline asks each staff member to complete the same tasks and QAPonline then builds the EQA and the IQC process for the laboratory manager. Therefore QAPonline completes 3 tasks in one step - internal staff competency, Internal Quality Control and External Quality Assurance - the way a quality process should be built , from the staff up. - TOP of Page
My Results
Can I take my EQA Results with me?
Yes you can. Because each enrollment and each reply is logged against your subscription code as well as the company and laboratory, if you leave one laboratory and commence in another, you can still access the results. - TOP of Page
Reply Frequency
Do I have to answer the questions each month?
The answer is Yes but No. Yes because if you answer the questions in each scheme every month, it only takes about 15 minutes (OK maybe a bit more if you are enrolled in all the schemes) and it allows your supervisor to assess each months submissions. However, QAPOnline was designed to be always available. It allows you to answers questions from previous years if you have just joined a laboratory or if you have been on leave, it allows you catch up when you return. All submissions are reviewed and updated weekly so if you miss out one month, sorry but the questions are still there to be answered. QAPonlines view is that after about 20-30 replies, the Mean and %CV does not change very much and therefore your answers relative to other replies will not alter significantly. - TOP of Page
Calculations
How is my performance calculated?
All EQA schemes in QAPonline have one key question that the analysis of your performance is calculated. The questions is always a number (even for embryos where each submitted option has weighted and summed to give number out of 100). Because the analysis is a number, QAPonline processes all schemes in the same way. Because each number may have a different range, QAPonline uses Standard Deviation units from the mean as the primary calculation tool. This assumes the data in each scheme is normally distributed and that 66% of all replies will be within 1 standard deviation of the group mean. Simply put, if your reply is within 1 SD of the mean, your performance is OK. - TOP of Page
Reply Scatter
My replies are all over the place, what does this mean?
In the previous note, an explanation of how your replies are processed was given. If the aim of the exercise is for your reply to be within 1 standard deviation of the mean then some answers will be more than the mean and some answers will be less than the mean. QAPonline would like to see all your answers to be within 1 SD from the mean - half above and half below. Perfect. IF all you answers are within 1 SD of the mean but they are all above or all below the mean then your results demonstrate BIAS. - TOP of Page
Peer Reviews
How does FertAid define Peer Reviews
All the FertAid schemes operate on a basic level of comparing you reply to the means of all other participants who have registered as a PROFESSIONAL. QAPonline routinely reviews all the professional replies and marks INVALID any reply more than 4 SD from the group mean. By and large this works well but for some schemes e.g. sperm morphology, FetAid has developed a PEER status. Peer participants are derived from 2 groups. One are those a QAP supervisor has marked as a PEER participant to act as a reference to the groups staff. The second, developed in 2010, FertAid may upgrade the status to PEER based upon the length of experience and performance. Any participant having more than 10 years experience AND more than 66% of replies within 1 SD of the group mean may be marked by FertAid as a Peer.
In some schemes, FertAid has invited participants to act as a PEER reviewer. - TOP of Page
Changing competency
How do you change the competency status of an Enrolment?
Each enrollment has competency level attached to it. These range from student to professional (subject of EQA used in ones day to day work) to Peer(either at clinic or whole scheme level). The competency level is used to calculate the average performance values for individual clinic to whole of scheme. Only professional and Peer values are used to calculate the mean and SD limits or the most popular option for each question. These data are updated after each data entry or at the very most, weekly. You can change the value for any enrolment by 2 pathways. These are A.->find scheme->find options->find Enrolment and change status OR B.->find QAPonline->find Schemes->find Details on schemes->change details. - TOP of Page
Comparing Results
How do compare one performance
QAPonline uses all professional replies for most analysis. As an individual, you can compare your performance against all subscribers, peer subscribers, those from your own country and those who have listed themselves as grand-persons (more than 10 years experience). QAPonline in general, uses the sum of all professional replies as the basis for estimating the mean and variability of any assessment task. It has done this in preference to attempting to find a `correct` subgroup. After all who can claim to be right. Most of IVF involves individual concepts of what is normal. Many of the rules are largely anecdotal. QAPonline`s view is that a broad international summary far better reflects a closer estimate of normal that a highly nominated group that has the potential to bias. - TOP of Page
Individual Comparisons
QAPonline allows individual clinic to define their QC benchmark
Not withstanding the above comments about using all the submissions to define normality, it also offer the capacity for individual clinics to define their performance against peer, owns country and grandperson submissions. A QAP supervisor may elect to print out for accreditation purposes, a clinics performance to anyone of these as opposed to the standard all submission distributions. The choice is yours rather than QAPonline defining your performance. to do this simply choose Supervisor->Reports->Current summary (choose EQA scheme)->Choose Filter from Data Set. - TOP of Page
Large SD
Some of my results have very large SD from the group mean
I sometimes find QAPonline posts very large SD from the group mean. Sometimes the size is silly and not possible. What could cause this? This essentially indicates that the summary files that record your performance need to be updated. Each time you complete an exercise, QAPonline records in these summary files your performance as SD from the current mean. However as more replies are submitted, the mean and SD will change gradually approaching the long term values. If QAPonline has not updated these files, you should look for the menu option REFRESH and this should reset the values in the files with the most recent estimate of your performance. - TOP of Page
Embryo Score
What does my embryo score mean and why is it different when my options are similar to others
The embryo score is a calculated value that is designed to imply embryo quality. The higher the score, the better you grade the embryo. Most grading systems use a number of criteria each with a number of options to compile a grading scheme (eg 4AA or 3BB). Each option of each criteria is given a value by FertAid related to quality (eg a good ICM has a higher value than a poorly developed ICM) and a weight to enhance the value such that the best embryo score should be 100. When you submit each option, Fertaid complies the net value for each questions and then sums them to get the final embryo score. Clearly if you do not submit a reply to each question, then Fertaid is unable to compile a score!. // So after you have asked Fertaid to calculate an embryo score based upon your chosen options, you can then get a sense of whether you have graded the embryo better, similar or poorer than other participants in the EQA scheme. However where you sit in the distribution of replies will depend on the range of other replies. You may well have chosen option in general agreement with others but some participants have marked the embryo down then the distribution may appear slightly screwed making the mean and SD changed. Even though FertAid used a trimmed mean to try to minimise a screwed distribution, it may still have the effect of altering where you sit in the distribution.// Fertaid would like to make the comment that this is what happens in real life in an IVF laboratory. If you sit within 1 SD of the mean then it may suggest your assessment is broadly similar to others or the embryo is largely universally scored the same by all. IF you assessment is more or less than 1 SD from the mean, then it suggest you have graded the embryo better or poorer than others and/or that the quality of the embryo is such that embryologists will vary on how it is graded.// Lastly, the Mean and SD is modified as more submissions are made and you may well find that over time, your position in the distribution may also drift. - TOP of Page
Fixing Errors
What happens if you make an error when submitting an answer
Errors sometimes happen, you may press the wrong button or choose the wrong option from a list. Because QAPOnline provides compares your reply to the mean of all the other submissions, no-one wants to have a data-set that contains the wrong answer. IT is in everyones interest to submit your best answer without errors. Furthermore, since you can see how you have performed as soon as you make your submission, it may be tempting to try and always change your answer to make you and your clinic appear better. //QAPonline provides two pathways to remedy this. The first is to lodge a request to change your reply and the second is to ask your supervisor to cancel all the submission for a specific release and that you can resubmit replies to all the questions.// IF you wish to lodge a request, you can do this by finding your answer to the specific question and on the Left hand menu, you will find an option - REVIEW REQUEST. When you do this, a note will be added to your answer indicating what answer you wish to submit. Your supervisor can then access this request and either allow you to resubmit an answer (still a bit cumbersome) or they can change your answer to what you have requested. QAPonline keeps a track of your request for correction and limits how many times you can apply.// QAPonline can also review your submission and if it is excessive far from the mean (Mean+/-4 SD), then QAPOnline may mark your result as INVALID. It stays in the system but is discounted when it tries to calculate the normal distribution of replies.// - TOP of Page
Fixing Errors SUPERVISOR
What can a QAP Supervisor do to allow errors in data submission to be fixed.
When a staff member make an error in submitting a reply, they can either lodge a request to change their answer with their corrected one OR they can request all the answers for a release to removed, allowing they to resubmit an answer for all the questions for that month. Changing a answer allows the data-set to be more precise (who wants the data-set to contain some wring answers) but also allows a clinic to modified their summary data. Clearly, changing a result has implications in several areas.// So how can a QAP Supervisor respond to staff requests to change an answer. There are two tools QAPonline has provided to assist you. You can find these by ->logging on, ->goto SUPERVISOR, ->goto MAINTENANCE. The two options are (a) Error Correction and (b) Release Review/Remove.// Choosing Error Correction will display a list of requests from staff in your clinics. Choosing a request, you can submit a modified reply. // Choosing Release Review/Remove displays a list if staff->choose a staff member-> then choose a scheme-> then choose a release (usually one per month)-> then choose remove all submissions for this release. When you do this, all their replies for the release are removed and the stats for the staff member, the clinic and overall are reset. There is currently no log for this action.// There is still work in progress for this last option.// - TOP of Page
Calculator