By D. Maan, Jadetimes News
Early Adoption of Chatbots in Government
Long before the advent of ChatGPT, governments were eager to use chatbots to automate their services and provide advice. According to Colin van Noordt, a researcher on AI in government based in the Netherlands, those early chatbots were simpler and had limited conversational abilities. However, the emergence of generative AI in the past two years has revived the vision of more efficient public services, where human like advisors are available around the clock to answer questions about benefits, taxes, and other areas of government interaction with the public.
Potential and Pitfalls of Generative AI
Generative AI is sophisticated enough to provide human like responses and, if trained on sufficient quality data, could theoretically handle a wide range of questions about government services. However, generative AI is also known for producing mistakes or nonsensical answers, often referred to as hallucinations. In the UK, the Government Digital Service (GDS) tested a ChatGPT based chatbot called GOV.UK chat, designed to answer citizens' questions on various government services. While nearly 70% of trial participants found the responses useful, there were instances where the system generated incorrect information and presented it as fact, raising concerns about misplaced confidence in its accuracy.
International Experiments with AI Chatbots
Other countries are also exploring generative AI based systems. In 2023, Portugal launched the Justice Practical Guide, a chatbot created to answer basic questions on topics like marriage and divorce. Funded by the European Union’s Recovery and Resilience Facility (RRF), the €1.3m ($1.4m; £1.1m) project is based on OpenAI’s GPT4.0 language model. In its first 14 months, the guide received 28,608 questions. While it handled straightforward queries well, it struggled with more complex ones, such as whether a person under 18 but married can set up a company.
Addressing Trust and Accuracy Issues
Despite its potential, the Justice Practical Guide still faces challenges in terms of trustworthiness, with rare but significant incorrect responses. A source from the Portuguese Ministry of Justice acknowledges these limitations but expresses hope that they will be overcome with improvements in the confidence level of the answers. As governments continue to iterate and refine these systems, the focus remains on enhancing accuracy and reliability to ensure they meet the high standards required for public service platforms.
Caution Advised with AI Chatbots
Due to inherent flaws, many experts, including Colin van Noordt, urge caution with AI chatbots. Van Noordt warns that problems arise when chatbots are used to replace human workers solely to cut costs. Instead, he suggests they should be viewed as an additional service to quickly access information. Sven Nyholm, an ethics professor at Munich’s Ludwig Maximilians University, emphasizes that chatbots cannot replace the accountability and moral responsibility of civil servants. He also points out that while new chatbots may appear intelligent, their occasional mistakes can be humorous or even dangerous if relied upon for critical advice.
Estonia's Approach to Digital Services
Estonia offers an alternative approach to digitizing public services, being a leader in this field since the early 1990s. The country introduced a digital ID card in 2002, allowing citizens to access state services. Estonia is now developing a suite of chatbots for state services under the name Bürokratt. Unlike ChatGPT or Google's Gemini, these chatbots use Natural Language Processing (NLP) rather than Large Language Models (LLMs). NLP breaks down requests into small segments and identifies key words to infer user intent. If Bürokratt's chatbot cannot answer a query, it is handed over to a customer support agent.
Comparing NLP and LLM based Chatbots
While NLP models used by Estonia offer greater control and transparency, they are limited in mimicking human speech and detecting nuances in language. Colin van Noordt explains that early chatbots, which forced citizens to choose options for questions, allowed for better control and transparency. In contrast, LLM based chatbots, like those using ChatGPT, offer more conversational quality and nuanced answers but at the cost of less control and consistency. These chatbots can provide different answers to the same question, highlighting the trade off between conversational ability and system control.
