To assess AI skills in the workplace, give employees a realistic task and check how they use AI, verify its output and decide when to escalate a problem. Start with a task they already perform, such as drafting a customer response or summarising an incident. Agree in advance what a satisfactory result looks like and which mistakes would require further practice.
The OECD’s research on AI and skills highlights the growing importance of interpreting data and solving problems as AI changes how people work. For managers, that means assessing the judgement employees bring to an AI-assisted task as well as their ability to use the tool. Read the OECD report.
Define what good performance looks like
Pick one role and one recurring task. Ask the manager and someone who regularly does the work to agree what an employee must do independently, what a supervisor must approve and when a case should be escalated. Write the standard as something you can observe. For example, “checks every policy claim against the current guidance and flags any exception before sending a response” gives an assessor something concrete to look for.
Use a realistic work sample
Imagine a customer service adviser using AI to draft a reply to a refund request. Give the adviser a fictional customer case, the approved refund policy and a draft that incorrectly promises a refund. Ask them to correct the reply and explain how they would handle any policy exception. Use the same instructions and criteria for everyone assessed on that task.
- Verification. Checks eligibility against the current policy and corrects the unsupported refund promise.
- Judgement. Recognises when the policy does not settle the case and routes it to the authorised colleague.
- Communication. Explains the decision to the customer accurately and avoids promises the team cannot keep.
Rate each criterion as demonstrated independently, demonstrated with support or not yet demonstrated. Agree in advance which errors require another assessment. For this exercise, leaving the unsupported refund promise in the reply would trigger further practice and a reassessment.
Check what is causing the gap
Ask the employee to walk you through their decision. An adviser who finds the policy but misreads an exception may need practice interpreting it. If several advisers cannot find the current policy, the team may need to fix document access or version control. The same weak result can have different causes, so check the workflow before committing time and budget to training.
Use fictional or approved materials, explain how the results will be used and limit access to individual records. Give employees a chance to explain how they reached a decision and correct any factual errors in the record.
Check whether performance improves
After coaching or practice, give employees a different case of comparable difficulty and apply the same criteria. Then review a small sample of routine work. In the refund example, check how often advisers catch unsupported promises before sending replies and whether they escalate policy exceptions correctly. If you also track response times, read them alongside those quality measures.
Oxford Management Centre and Excentra AI Academy launch joint training course
Oxford Management Centre has launched AI Skills Intelligence & Workforce Reskilling, a five-day training course jointly offered with Excentra AI Academy. It is designed for HR, learning and development, workforce planning and AI transformation professionals who need to translate changing job requirements into decisions about employee development. Participants examine how AI could affect job tasks, assess evidence of skills gaps and prioritise development by role.
View the course at Oxford Management Centre or explore the course at Excentra AI Academy.
