Reviewing eLearning Questions for Decision Quality in Bahrain

A front-office eLearning module at a hotel in Manama ends with ten multiple-choice questions, and almost every agent passes on the first attempt. One question asks for the standard check-out time; another asks which term describes a reservation guaranteed by card. Two weeks later, the front office manager is still handling the same escalations: late check-outs granted on fully booked nights, and arriving guests waiting in the lobby for rooms that are not ready.

The questions were accurate. They simply asked agents to remember, while the job asks them to decide. An eLearning question that checks a definition can confirm that someone read the policy. It cannot show whether they will apply it when a guest is standing at the desk, and the system shows something the policy summary did not mention.

This guide is for HR and digital learning teams in Bahrain who already have eLearning question banks and want to know which questions measure decisions and which only measure recall. It sets out a review method: compare definitional questions with situation questions, decide what feedback each answer should trigger, and state clearly what the results can and cannot tell you.

Why So Many eLearning Questions Test Recall

Recall questions are the quickest eLearning questions to write. A subject matter expert highlights the key terms in a policy, and each term becomes a question: what does this mean, which form is used, what is the limit? The result covers every page of the eLearning content and is easy to score. It also rewards the employees who are best at remembering text, not necessarily those who make good decisions at work.

Assessment researchers have long pointed to this gap. In a peer-reviewed paper on constructing multiple-choice items to measure higher-order thinking, Darina Scully notes repeated calls for assessments that target higher-order thinking as opposed to mere recall of facts, and argues that multiple-choice items have the capacity to assess certain higher-order skills. The format is not the problem. The way the question is written is.

Will Thalheimer’s Learning-Transfer Evaluation Model (LTEM) gives the review a useful lens, because it differentiates between knowledge, decision-making, and task performance. Most corporate eLearning question banks sit almost entirely in the first category. Reviewing for decision quality means moving the important items into the second.

Definition Question or Situation Question? Compare Them Side by Side

The quickest way to see the difference is to take one piece of eLearning content and write the question both ways. Suppose your policy says that late check-out may be offered free of charge until 14:00 only when the room is not needed for an arriving guest, and must otherwise be referred to the duty manager.

The Definitional Version

“Until what time can a free late check-out be offered?” The options are four times of day. A correct answer proves that the agent remembers a number. It says nothing about whether they check availability first, or what they say to a guest who cannot have the room.

The Situation Version

“At 10:30, a guest asks for a 15:00 check-out because of an evening flight. The system shows the room type is fully booked tonight and an early arrival is expected. What do you do?” The options might be: agree to keep the guest happy; refuse politely and offer luggage storage; offer 14:00 without checking; or refer the request to the duty manager with the alternatives ready. Now the agent has to read the situation, notice the detail that changes the answer, and choose an action.

What the Comparison Shows

Put the two versions side by side for each important topic in the eLearning module and ask three questions:

• What does a correct answer prove: a remembered fact, or a choice that matches the policy in a realistic case?

• Does the right answer depend on a detail in the situation, or could someone answer without reading the stem?

• Would a weak employee and a strong employee answer differently?

If only the definitional version exists in your eLearning, the question bank can tell you who read the policy, not who can use it.

Six Checks for Every eLearning Question in the Bank

Once the team sees the difference, review the whole bank item by item. For each eLearning question, record a yes or no against six checks:

• Situation: does the stem describe a moment the employee will actually meet, with the people, timing, and information they would have?

• Decision: does the employee have to choose an action, rather than identify a term?

• Deciding detail: does one detail in the situation change the correct answer, so that careful reading matters?

• Plausible options: is each wrong option something real employees do under pressure, rather than an obviously silly answer?

• No giveaways: is it impossible to find the answer by picking the longest option, matching words from the stem, or spotting a grammar mismatch?

• Importance: would a wrong decision here cost money, time, safety, or customer trust?

Then sort the items into three groups. Keep a small number of knowledge checks where a term or limit genuinely must be remembered exactly, such as a safety threshold. Rewrite the important recall items as situation questions. Remove items that test details nobody uses at work. A smaller bank of strong eLearning questions is worth more than a long one that only confirms reading.

Run the eLearning review as a joint session between someone from the learning team who knows question writing and an experienced supervisor who knows what really happens on shift. The supervisor supplies realistic wrong options; the learning specialist keeps the wording clear and fair. Together they turn a recall quiz into interactive learning.

Write Feedback That Explains the Decision

An eLearning situation question without useful feedback only measures. With good feedback, it also teaches. In a peer-reviewed review of feedback research, Valerie Shute defines formative feedback as information communicated to the learner that is intended to modify thinking or behavior to improve learning, and summarizes the view that it should be non-evaluative, supportive, timely, and specific. The review also describes its common forms, from simple verification of an answer to explanations, hints, and worked examples.

For eLearning questions that test decisions, this suggests a simple pattern for every option:

• Name the detail in the situation that mattered, such as the fully booked room type.

• Explain the likely consequence of the choice, for the guest and for the next shift.

• State the rule in one sentence, in the words your procedure uses.

• Give feedback on correct answers too, so that employees who guessed learn why the answer was right.

Avoid feedback that judges the person, and avoid simply revealing the correct letter. If the eLearning allows a second attempt, the feedback on the first attempt should offer a hint rather than the answer.

Set the Limits of Your eLearning Assessment Before You Report It

Situation questions are a clear improvement, but they still have limits, and the review should state them before any results reach managers. A decision question shows which option an employee chooses in a simplified, written situation. It does not show how they speak to an upset guest, how quickly they work at a busy desk, or whether they follow through. In LTEM terms, decision-making and task performance are different things.

For each eLearning assessment, write down what the score will and will not be used for:

• Use it to find decisions that many employees get wrong, and to improve the eLearning and the procedure.

• Use it to decide who would benefit from coaching or a short practice session with a supervisor.

• Do not use a single quiz score, on its own, for high-stakes decisions about an individual employee.

• Confirm performance through observation or work review for tasks that involve conversation, physical steps, or judgment over time.

Check the eLearning questions themselves after launch, too. An item that almost everyone answers correctly may be too easy or contain a giveaway; one that almost everyone gets wrong may be unclear, or may reveal a real gap in how the procedure is taught. If your eLearning platform reports results by item and option, as discussed in our article on choosing reports managers can use in an Arabic LMS in Bahrain, review those patterns every quarter.

Language is a final limit. If your teams in Bahrain work in Arabic, English, or both, and some employees read neither as their first language, a long eLearning situation question may measure reading speed rather than judgment. Keep stems short, use the terms employees see on their screens, and test the wording with a few employees before launch.

A Hypothetical Example: Reviewing a Front-Office Question Bank in Manama

The following example is hypothetical and is included only to show how the review works.

A hotel in Manama has a 40-item eLearning question bank for front-office agents, built from its standard operating procedures. The learning coordinator and the front office manager review it in two short sessions. Most items ask for times, terms, or form names; only a handful describe a guest.

They keep six knowledge checks, such as the documents required at check-in, because agents must know them exactly. They rewrite twelve recall items as situation questions around the decisions that cause the most escalations: late check-outs on full nights, room moves, early arrivals and billing disputes. Each new item has one deciding detail, three realistic wrong options supplied by the front office manager, and feedback that names the detail, the consequence, and the rule. The remaining items are removed.

Before launch, four agents from different shifts try the new eLearning questions and flag two stems that are too long to read quickly. After launch, the team uses the results only to adjust coaching and the procedure, and the front office manager observes a sample of real late check-out requests to see whether decisions at the desk match the answers given in the eLearning.

Summary

Many eLearning questions confirm that employees read a policy, not that they can apply it. Comparing definitional questions with situation questions makes the difference visible, and six simple checks turn that comparison into a review of the whole question bank.

Feedback that explains the deciding detail turns each eLearning question into practice, and written limits keep the results in proportion:

Useful for improving design and coaching, not a verdict on any one employee. The same review applies to roles with demanding procedures, as discussed in our article on eLearning for financial services teams in Bahrain.

Click here. If you would like a second view on your current eLearning question bank, you can talk through your requirements with the SkillUp MENA team.