Evaluation JSON (shape)
{
"q_1_1": {
"evaluator_selection_option": "Did the agent greet the customer according to the protocol? Evaluate only the detected agent's opening turn. Return Yes if the opening includes the required meaning of all three components: greeting/salam, agent name or identity, and offer of assistance. Minor wording variation is acceptable. Return No if any major component is completely missing, such as no greeting/salam, no agent identity, or no assistance offer. Return only: Yes/No",
"mark_distribution": "If all three greeting components are present in meaning, give full marks: 1. If any major component is missing, give 0. No partial marks. Return only score:",
"detail": "Check only the first agent turn. Required meaning: salam/greeting, agent identity/name, and offer of assistance. Accept natural variations. Fail if the opening starts abruptly or misses a major component."
},
"q_1_2": {
"evaluator_selection_option": "Did the agent greet with a friendly and welcoming tone? Since transcript may be the only available evidence, evaluate wording-based warmth only unless audio metadata is provided. Return Yes if the opening wording is respectful, welcoming, and service-oriented. Return No if the opening is abrupt, rude, commanding, careless, or not welcoming. Return only: Yes/No",
"mark_distribution": "If the opening wording is respectful and welcoming, give full marks: 2. If the opening is abrupt, rude, careless, or not welcoming, give 0. No partial marks. Return only score:",
"detail": "Evaluate wording only unless audio metrics are provided. Indicators include polite greeting, respectful address, and assistance-oriented language."
},
"q_2_1": {
"evaluator_selection_option": "Did the agent use verbal probing questions to understand and validate the caller's need? Return Yes if the agent asked relevant clarifying questions about the customer's specific issue, such as what happened, when it happened, which number, amount, method, location, ownership status, deregistration date, or other issue-specific details. Return No if the agent jumped to a solution without understanding, asked mostly irrelevant questions, repeatedly misunderstood the customer, or made the customer repeat the same core concern multiple times without progress. Return only: Yes/No",
"mark_distribution": "If relevant verbal probing was used to identify the customer's core problem, give full marks: 5. If probing was absent, irrelevant, repetitive without progress, or showed repeated misunderstanding, give 0. No partial marks. Return only score:",
"detail": "Judge whether the agent asked enough relevant questions to understand the real issue. One clarification is acceptable. Repeated misunderstanding of the same issue should fail."
},
"q_2_2": {
"evaluator_selection_option": "Did the agent use system or security probing to understand and validate the caller's need? Return Yes if the transcript or provided metadata shows relevant verification, system check, or security probing such as customer number, DOB, NID, biometric, OTP, balance, recharge, usage history, transaction history, registration status, or portal/CRM check. Do not infer system activity from silence. If metadata confirms the required system/security verification, return Yes even if the transcript does not explicitly mention the check. Return No if verification/checking was required but neither transcript nor metadata confirms it. Return only: Yes/No",
"mark_distribution": "If required system/security probing is confirmed by transcript or metadata, give full marks: 5. If required verification/checking is absent or cannot be confirmed, give 0. No partial marks. Return only score:",
"detail": "Use transcript and metadata. Do not guess. For balance/recharge issues, look for balance/history/transaction checks. For SIM/ownership issues, look for NID, biometric, OTP, or registration-related verification."
},
"q_2_3": {
"evaluator_selection_option": "Did the agent provide complete and correct information according to the customer's actual use case and the provided SOP/knowledge base? Return Yes if the agent communicated the required answer in the correct essence, even if not word-for-word. The answer must cover the mandatory information needed for this customer's specific query, including correct process, eligibility, charge, document, timeline, channel, code/app/SMS instruction, OTP/biometric instruction, or service-center instruction when those are mandatory or triggered by the customer's question. Return No if any mandatory information is missing, wrong, contradictory, misleading, or if the agent gave incorrect information and the correction did not clearly resolve the issue. Do not apply full-call fatal scoring. Return only: Yes/No",
"mark_distribution": "If the agent provided complete, correct, and SOP-aligned information in essence for the customer's actual query, give full marks: 15. If mandatory information was missing, wrong, contradictory, misleading, or unresolved, give 0. No partial marks. Do not set all other questions to 0. Return only score:",
"detail": "Before scoring, check the matching Knowledge Base section. Judge by correct service essence, mandatory information, and customer context, not exact wording."
},
"q_2_4": {
"evaluator_selection_option": "Did the agent close or handle the call with expertise in a timely and organized manner? Return Yes if the agent broadly followed the right service flow, understood the issue, addressed the customer's core questions, explained in a customer-friendly way, and handled the call competently overall. Minor imperfections should not fail this question if the issue was addressed. Return No only if the handling was clearly disorganized, scattered, contradictory, unresolved, or the customer remained confused because of the agent's handling. Return only: Yes/No",
"mark_distribution": "If the call was handled competently, logically, and the customer's core issue was addressed, give full marks: 6. If handling was clearly disorganized, confusing, contradictory, unresolved, or ineffective, give 0. No partial marks. Return only score:",
"detail": "Judge overall call-handling expertise. Minor repetition is acceptable. Fail when poor handling materially affects resolution or customer understanding."
},
"q_2_5": {
"evaluator_selection_option": "Did the agent ensure all necessary additional information required by the use case? Return Yes if the agent provided the next step or supporting information the customer needed to proceed, such as My Robi app guidance, USSD code, SMS format, website/app/self-care option, Robi Sheba/service center guidance, required documents, charge reminder, time limit, OTP/biometric instruction, or follow-up instruction. Return No only if a specific additional piece of information was clearly required by the use case context but was absent. Do not fail for missing generic promotional/self-care information unless it was clearly required. Return only: Yes/No",
"mark_distribution": "If necessary additional information or next-step guidance was provided, give full marks: 4. If a clearly required additional detail was missing, give 0. No partial marks. Return only score:",
"detail": "Before scoring, check the matching Knowledge Base section. Judge by correct service essence, mandatory information, and customer context, not exact wording."
},
"q_3_1": {
"evaluator_selection_option": "Did the agent actively listen to the caller? Return Yes if the agent understood the customer's actual concern, responded to the right issue, and did not force unnecessary repetition. A single polite clarification request is acceptable. Return No if the customer had to repeat the same core concern multiple times because the agent misunderstood, or if the agent answered a different issue than the customer raised. Return only: Yes/No",
"mark_distribution": "If the transcript shows active listening and correct understanding of the customer's concern, give full marks: 3. If repeated misunderstanding or unnecessary repetition occurred due to the agent, give 0. No partial marks. Return only score:",
"detail": "One clarification is fine. Fail when the same core issue is repeated because the agent keeps missing the point."
},
"q_3_2": {
"evaluator_selection_option": "Did the agent avoid interrupting the caller? In transcript-only evaluation, return Yes if no clear interruption is visible. Return No only if the transcript clearly shows the agent cutting off the customer, speaking over the customer, pushing an answer before the customer finished, or if the customer reacts with objection language such as শুনুন, আরে, আগে শুনেন, or similar. Do not assume audio-level interruption without evidence. Return only: Yes/No",
"mark_distribution": "If no clear interruption is visible, give full marks: 3. If clear interruption or customer objection to interruption is visible, give 0. No partial marks. Return only score:",
"detail": "Judge only visible evidence. Conversation turns alone are not enough to prove interruption."
},
"q_3_3": {
"evaluator_selection_option": "Did the agent show friendliness, positive tonality, empathy, and courtesy during the call? Evaluate wording-based courtesy unless audio metadata is available. Return Yes if the agent used polite, respectful, patient, and empathetic wording appropriate to the situation. Return No if the agent was rude, dismissive, impatient, dictating, arrogant, or if the customer faced inconvenience/service failure and the agent showed no empathy or apology. Return only: Yes/No",
"mark_distribution": "If the transcript shows appropriate friendliness, empathy, and courtesy, give full marks: 13. If the agent was rude, dismissive, impatient, or failed to show empathy when clearly needed, give 0. No partial marks. Return only score:",
"detail": "Look for polite and empathetic behavior across the call, not just one polite word. Service failure or confusion should usually be handled with apology/empathy."
},
"q_3_4": {
"evaluator_selection_option": "Did the agent use magic words during the call? Return Yes if the agent used at least one real courteous or magic expression anywhere in the transcript, such as ধন্যবাদ, আপনাকে ধন্যবাদ, সহযোগিতার জন্য ধন্যবাদ, দুঃখিত, আন্তরিকভাবে দুঃখিত, কাইন্ডলি, মোস্ট ওয়েলকাম, ভালো থাকবেন, sorry, or a similar courtesy phrase. Do not count only স্যার or ম্যাডাম as magic words; those are mainly personalization/respectful address indicators. Return No only if no real courtesy/magic expression appears. Return only: Yes/No",
"mark_distribution": "If at least one real magic/courtesy expression is present, give full marks: 2. If no real magic/courtesy expression is present, give 0. No partial marks. Return only score:",
"detail": "Search the full transcript for actual magic/courtesy expressions. Sir/madam alone is not enough for this criterion."
},
"q_3_5": {
"evaluator_selection_option": "Did the agent have appropriate energy, liveliness, and confidence throughout the call? This is primarily audio-based. If only transcript is available, apply benefit of doubt. Return Yes if there is no clear visible evidence of low confidence or confusion. Return No only if transcript clearly shows confusion, hesitation, self-contradiction, repeated correction, uncertain response, or disorganized handling that affected customer understanding. Return only: Yes/No",
"mark_distribution": "If audio/metadata or transcript shows adequate confidence, or if transcript-only evidence does not clearly prove failure, give full marks: 6. If clear evidence of confusion, contradiction, uncertainty, or disorganized handling exists, give 0. No partial marks. Return only score:",
"detail": "Do not penalize because audio is missing. Penalize only when visible transcript evidence shows lack of confidence or confused handling."
},
"q_3_6": {
"evaluator_selection_option": "Did the agent demonstrate good sentence construction and correct pronunciation? Pronunciation should be judged only if audio/audio metadata is available. If transcript only, evaluate sentence construction. Return Yes if the agent's sentences are generally understandable, organized, and customer-friendly. Return No only if sentences are visibly broken, incoherent, confusing, excessively disorganized, or likely caused customer misunderstanding. Return only: Yes/No",
"mark_distribution": "If sentence construction is understandable and organized, or if transcript-only evidence does not clearly prove failure, give full marks: 10. If visible sentence construction problems caused or could reasonably cause misunderstanding, give 0. No partial marks. Return only score:",
"detail": "Transcript-only rule: do not judge pronunciation. Evaluate whether the agent's explanations were clear enough from the text."
},
"q_3_7": {
"evaluator_selection_option": "Did the agent provide information that was completely understood by the customer in terms of pace and pitch? This is primarily audio-based. If only transcript is available, apply benefit of doubt. Return Yes if there is no visible evidence that the customer failed to understand. Return No only if the customer clearly says they did not understand, asks the same question repeatedly after the explanation, or the agent's explanation is visibly incoherent, contradictory, or confusing. Return only: Yes/No",
"mark_distribution": "If there is no clear evidence of customer comprehension failure, give full marks: 10. If the transcript clearly shows the customer did not understand due to the agent's explanation, give 0. No partial marks. Return only score:",
"detail": "Do not judge actual pace/pitch without audio. Use visible comprehension evidence from customer responses and clarity of agent explanation."
},
"q_3_8": {
"evaluator_selection_option": "Did the agent avoid using jargon or internal technical terms? Return Yes if the agent used customer-friendly language or explained any necessary technical term simply. Return No if the agent used unexplained internal terms such as dCCRM, ICMS, CQC, MCA, BTRC, DND, network dashboard, internal code names, or similar directly in customer-facing explanation. Return only: Yes/No",
"mark_distribution": "If no unexplained internal jargon was used, give full marks: 2. If unexplained internal jargon was used, give 0. No partial marks. Return only score:",
"detail": "Check whether the customer-facing language was simple and understandable. Fail only for unexplained internal jargon."
},
"q_3_9": {
"evaluator_selection_option": "Did the agent personalize the call? Return Yes if the agent made the interaction feel specific to the customer by using respectful address naturally, referring to the customer's number/situation/location/timeline/request, or tailoring the answer to the customer's exact context. Return No if the conversation remained generic, mechanical, or impersonal. Do not give Yes only because স্যার or ম্যাডাম appeared once mechanically if there is no other personalization or context-based response. Return only: Yes/No",
"mark_distribution": "If the agent meaningfully personalized the call, give full marks: 2. If the agent remained generic or mechanical with no meaningful personalization, give 0. No partial marks. Return only score:",
"detail": "Look for context-based personalization, not only one respectful address. Strong evidence includes referring to the customer's specific issue, number, date, location, or next step."
},
"q_3_10": {
"evaluator_selection_option": "Did the agent follow the right call transfer or hold protocol? If no hold or transfer occurred, return Yes automatically. If hold occurred, return Yes only if the agent asked permission before holding and thanked or apologized after returning when appropriate. If transfer occurred, return Yes only if the agent informed the customer properly before transfer. Return No if hold/transfer happened and the required protocol was clearly violated. Return only: Yes/No",
"mark_distribution": "If no hold/transfer occurred, give full marks: 4. If hold/transfer occurred and protocol was followed, give full marks: 4. If hold/transfer occurred and protocol was violated, give 0. No partial marks. Return only score:",
"detail": "Default Yes when no hold/transfer exists. For hold, look for permission before hold and thanks/apology after return. For transfer, look for proper notice."
},
"q_4_1": {
"evaluator_selection_option": "Did the agent check comprehension or offer further assistance at the end of the call? Return Yes if near the end the agent used a meaningful comprehension check or further-help offer, such as আমি কি আপনাকে বুঝাতে পেরেছি, আর কোনো তথ্য দিয়ে সহযোগিতা করতে পারি, আর কিছু জানতে চান, or a natural equivalent. Return No if the agent closed directly without checking understanding or offering further assistance. Return only: Yes/No",
"mark_distribution": "If a comprehension check or further-help offer is present near the end, give full marks: 1. If absent, give 0. No partial marks. Return only score:",
"detail": "Look near the end of the call. A natural variation is acceptable. Direct closing without checking understanding/further help should fail."
},
"q_4_2": {
"evaluator_selection_option": "Did the agent close the call according to the prescribed script or a professional close variation? Return Yes if the closing includes thanks, good wishes, and salam or a close equivalent. Accept natural variations such as আপনার সাথে কথা বলে ভালো লাগলো, ধন্যবাদ, ভালো থাকবেন, আসসালামু আলাইকুম. Return No if the closing is missing, abrupt, incomplete, or unprofessional. Return only: Yes/No",
"mark_distribution": "If the agent closed professionally with thanks, good wishes, and salam/equivalent, give full marks: 1. If closing is missing, abrupt, incomplete, or unprofessional, give 0. No partial marks. Return only score:",
"detail": "Required meaning: polite thanks, good wish, and closing salam/equivalent. Fail if the call ends with only okay/bye or no proper closing."
},
"q_5_1": {
"evaluator_selection_option": "Did the agent capture all information correctly and completely in CRM/CQC/wrap-up? Return Yes only if CRM, CQC, wrap-up code, or related metadata is explicitly provided and it correctly matches the customer's actual query. Return No if no CRM/CQC/wrap-up metadata is provided, because CRM capture cannot be verified from transcript alone. Return No if metadata is provided but mismatched, incomplete, or wrong for the detected use case. Return only: Yes/No",
"mark_distribution": "If CRM/CQC/wrap-up metadata is provided and correctly matches the customer's actual query, give full marks: 5. If metadata is missing, mismatched, incomplete, or wrong, give 0. No partial marks. Return only score:",
"detail": "Do not infer CRM capture from transcript. Verify only from provided CRM/CQC/wrap-up metadata and compare it with the customer's actual issue."
}
}
Evaluation JSON (result)
{
"q_1_1": {
"evaluator_selection_option": "Yes",
"mark_distribution": 1,
"detail": "Opening included salam, agent identity, and assistance offer: 'আসসালামু আলাইকুম, নুসরাত বলছি... কীভাবে সহযোগিতা করতে পারি?'"
},
"q_1_2": {
"evaluator_selection_option": "Yes",
"mark_distribution": 2,
"detail": "Greeting wording was polite, welcoming, and service-oriented."
},
"q_2_1": {
"evaluator_selection_option": "No",
"mark_distribution": 0,
"detail": "Agent asked for repetition but then misunderstood the issue as charge deduction: 'আপনি টাকা চার্জ হচ্ছে কেন...' causing the customer to repeat 'ট্রান্সফার' multiple times."
},
"q_2_2": {
"evaluator_selection_option": "N/A",
"mark_distribution": "-",
"detail": "No verifiable system/security probing metadata or transcript evidence for the balance transfer query."
},
"q_2_3": {
"evaluator_selection_option": "No",
"mark_distribution": 0,
"detail": "Use case is Balance Transfer Q. Agent mentioned channels and short code 25064, but gave incomplete/wrong process: did not provide SMS format, did not mention eligible prepaid receiver, amount limit Tk 5-300, sender charge, or daily/monthly limits. Also said My Robi app can transfer Robi to Airtel, which conflicts with KB cross-brand app limitation."
},
"q_2_4": {
"evaluator_selection_option": "No",
"mark_distribution": 0,
"detail": "Handling was not fully organized; agent initially misunderstood the issue and the final explanation remained incomplete, so the core query was not handled with full expertise."
},
"q_2_5": {
"evaluator_selection_option": "Yes",
"mark_distribution": 4,
"detail": "Agent provided an additional next step by offering to send the code/instruction via SMS, which was relevant to help the customer proceed."
},
"q_3_1": {
"evaluator_selection_option": "No",
"mark_distribution": 0,
"detail": "Customer had to restate the same balance transfer issue because the agent first interpreted it as a charge issue."
},
"q_3_2": {
"evaluator_selection_option": "Yes",
"mark_distribution": 3,
"detail": "No clear transcript evidence of interruption or talking over the customer."
},
"q_3_3": {
"evaluator_selection_option": "Yes",
"mark_distribution": 13,
"detail": "Agent remained polite and courteous throughout, using respectful address and apologies such as 'দুঃখিত স্যার' and 'ধন্যবাদ স্যার'."
},
"q_3_4": {
"evaluator_selection_option": "Yes",
"mark_distribution": 2,
"detail": "Used magic words including 'দুঃখিত', 'ধন্যবাদ', and 'কাইন্ডলি'."
},
"q_3_5": {
"evaluator_selection_option": "Yes",
"mark_distribution": 6,
"detail": "No clear transcript evidence of low energy or lack of confidence; benefit of doubt applied."
},
"q_3_6": {
"evaluator_selection_option": "Yes",
"mark_distribution": 10,
"detail": "Sentence construction was generally understandable despite some repetition of 'স্যার'."
},
"q_3_7": {
"evaluator_selection_option": "No",
"mark_distribution": 0,
"detail": "Customer asked again for SMS instruction after the explanation, showing the information was not fully clear/completely understood."
},
"q_3_8": {
"evaluator_selection_option": "Yes",
"mark_distribution": 2,
"detail": "Agent avoided internal technical jargon and used customer-facing language."
},
"q_3_9": {
"evaluator_selection_option": "Yes",
"mark_distribution": 2,
"detail": "Agent personalized the interaction by repeatedly addressing the customer respectfully and tailoring responses to the customer's balance transfer request and later deduction concern."
},
"q_3_10": {
"evaluator_selection_option": "N/A",
"mark_distribution": "-",
"detail": "Agent asked permission before hold: 'অপেক্ষা রাখতে পারি?' and thanked the customer after return."
},
"q_4_1": {
"evaluator_selection_option": "No",
"mark_distribution": 0,
"detail": "Agent used 'বুঝাতে পেরেছি?' earlier, but not near the end of the call; the call ended without a final comprehension/further-help check."
},
"q_4_2": {
"evaluator_selection_option": "No",
"mark_distribution": 0,
"detail": "No professional closing with thanks/good wishes/salam at the end; call ended while moving to another issue."
},
"q_5_1": {
"evaluator_selection_option": "N/A",
"mark_distribution": "-",
"detail": "No CRM/CQC/wrap-up metadata provided, so CRM capture cannot be verified."
}
}