When AI Struggles, Human Judgment Provides the Solution

Last Updated：

2026-03-27

Prompt changes stop AI misrouting vague emails, routing them to staff review and improving response speed.

1. Issue: Errors in AI-Driven Respondent Selection

A trading company handles a wide array of office essentials, including copy machines and printers. To streamline their office phone reception, the company adopted a call answering service. Calls to their main number are handled by an operator, and the outcomes are shared with the administration department via email. An AI-powered system has been integrated to further enhance efficiency by prioritizing calls and selecting appropriate respondents.

However, this system presents a persistent challenge. When operator-generated emails are addressed to a vague recipient address such as “To Support Staff” or “To HR Department,” the AI attempts to force-select a respondent from its list of respondents.

This often results in response requests being misdirected to entirely unrelated individuals. Crucially, when the AI incorrectly selects a respondent for a critical inquiry, it can significantly delay the request from reaching the appropriate person, leading to slow response times.

While a certain degree of AI error is unavoidable, there is a clear demand for improvements to prevent the occurrence of unnecessary operational burdens.

2. Solution: Modifying the AI Prompt to Output “No Respondent”

To mitigate erroneous AI-driven respondent selection, the process owner modified the AI’s prompt. Specifically, when an email from an operator contains a vague recipient designation like “To [Department] Staff”, the AI is now configured to output a hyphen (“-“), signifying that there is no respondent, rather than attempting to select a candidate from the list of respondents.

Furthermore, a mechanism has been implemented to automatically branch to the “1. Respondent Determination” step whenever the AI outputs a hyphen. This configuration ensures that emails with ambiguous recipient designations, particularly those containing critical inquiries, are automatically routed to a member of the administration department to manually select the respondent.

Preventing incorrect AI judgments in ambiguous cases shortens staff identification time and improves customer response speed.

Before

View details of the workflow diagram

x1. Acquire Email Content

Upon receipt of an email with the “phone” label in Gmail, its content is read.

x2. Importance Assessment by AI

The AI analyzes the email content to determine if it is a sales inquiry. The data item “Judgment Result” is set to “not important” for sales inquiries and “important” otherwise.

x3. Acquire Respondent List

User information registered in the workflow platform (Respondent List) is retrieved.

x4. Respondent Determination by AI

The AI analyzes the email content and selects a respondent (email address) from the Respondent List obtained in “x3. Acquire Respondent List.”

x5. Set Respondent

Based on the respondent (email address) selected in the “x4. Respondent Determination by AI” automated process, the handler for the “2. Record Response Outcome” process is set.

g1. AI Judgment Result

The path is selected based on the value of the “Judgment Result” data item:
– If “not important”, proceed to “1. Respondent Determination”.
– If “important”, proceed to the email transmission event (“m1. Urgent Response Request”).

m1. Urgent Response Request

An email is sent to administration department members and the respondent determined in “x3. Respondent Determination by AI”, prompting them to handle the “2. Record Response Outcome” process.

1. Respondent Determination

An administration department member determines the “Respondent” based on the email content. The respondent determined in “x4. Respondent Determination by AI” is pre-filled.

2. Record Response Outcome

The respondent reviews the email content and records how the inquiry was handled.

After