The Risks of Relying on AI: Lessons from Air Canada’s Chatbot Debacle

Originally published by Truyo.

In the era of artificial intelligence (AI), companies are increasingly relying on automated systems to streamline operations and enhance customer service. However, a recent incident involving Air Canada’s AI-powered chatbot serves as a stark reminder of the risks associated with relying solely on AI technology, particularly when it comes to customer interactions and policy enforcement.

The Incident

A Canadian customer recently found themself in a frustrating predicament when they sought clarification on Air Canada’s bereavement rates following the death of a family member. The customer consulted the company’s AI-powered chatbot for guidance and received a response advising them to submit a ticket for a reduced bereavement rate within 90 days of issuance.

Relying on the chatbot’s advice, the customer proceeded to book a ticket and later requested a refund, only to discover that Air Canada’s actual policy did not align with the chatbot’s guidance. Despite initially refusing to honor the chatbot’s promise, Air Canada offered the customer a $200 credit for future use but declined to issue a refund.

Unsatisfied with the outcome, the customer took the matter to the Civil Resolution Tribunal (CRT), arguing that Air Canada should be held accountable for the chatbot’s misleading advice. In an unprecedented move, Air Canada contended that the chatbot was a separate legal entity responsible for its own actions, marking a notable attempt to evade liability for AI-generated interactions.

Outcome and Implications

Following a fierce dispute, the CRT ruled in favor of the customer, compelling Air Canada to issue a partial refund of $650.88 and cover the customer’s CRT fees. This decision underscores the accountability of companies for the actions of their AI systems, forecasting the result of future cases involving AI-driven customer interactions.

Mitigating AI Risks

The Air Canada chatbot debacle highlights several key lessons for companies leveraging AI technology:

Transparency and Accuracy: Companies must ensure that AI-powered systems provide accurate and transparent information to customers. Misleading or erroneous guidance can lead to legal disputes and reputational damage.

Policy Alignment: AI systems should align with company policies and procedures to avoid discrepancies between automated responses and official guidelines. Regular audits and updates are essential to maintain consistency and compliance.

Legal Liability: Companies cannot absolve themselves of responsibility for AI-generated interactions. Legal frameworks must evolve to address the accountability of companies for the actions of their AI systems, clarifying liability and mitigating legal risks.

Continuous Improvement: The Air Canada case underscores the importance of ongoing monitoring and improvement of AI systems. Companies should invest in training data, algorithmic refinement, and quality assurance measures to enhance the accuracy and reliability of AI-driven interactions.

The Air Canada chatbot incident serves as a cautionary tale for companies navigating the complexities of AI integration in public-facing systems. While a tool like an AI-powered chatbot offers tremendous potential for efficiency and innovation, companies must approach AI implementation with caution and accountability. By prioritizing transparency, policy alignment, and legal compliance, companies can mitigate the risks associated with AI-driven interactions and uphold their commitment to customer satisfaction and integrity.

---

About the Author

Dan Clarke is a former Intel® executive with numerous leadership roles who was pulled into the privacy space after a call from Intel® anticipating GDPR’s implementation. Dan’s privacy expertise comes from developing the Truyo platform, which automates compliance with current and emerging privacy laws for enterprise-level companies. Clarke is a privacy thought leader involved in Arizona, Texas, and federal privacy legislation. Dan helped Truyo step into the AI Governance realm by developing the first comprehensive AI Governance Platform and creating a 5 Steps to Defensible AI Governance workshop that's been conducted with enterprise companies across the United States.