Making answer quality measurable
‘En route’ to more measurable quality
To keep a grip on the quality of Gem's answers we do not look at a single aspect, but at a coherent set of criteria. We look at what an answer should contain, and just as much at what it should not. For example, these criteria:
- Completeness – Does the answer contain all the elements, and in the right order?
- Traceability – Are there working hyperlinks to reliable sources?
- Professional tone – Is the tone businesslike and helpful?
- Usability – Can you act on it straight away?
- Conciseness – Is there any superfluous information in it?
- Comprehensibility – Is the language clear and not needlessly complicated?
- Correctness – Are the spelling and grammar right?
- Safety – Is it free of unwanted or risky content?
Together, all the criteria make up the answer to the question: ‘What is the quality of Gem's answers?’
A smart combination: people and technology
Checking all of this is still largely manual work. We also make use of the reports we get from residents and from critical researchers about the answers. But all these checks take up quite a lot of time. That is why we are automating more and more checks. This makes quality both scalable and consistent.