Latest News

ChatGPT and Google Translate still have flaws – Find out what happened when it rewrote key parts of constitutions

AI could water down the human rights of citizens, if governments use it to write and translate their constitutions, according to an analysis by The Spanish Group, a leading certified translation company.

This follows a series of recent developments in Brazil when its National Justice Council translated its 1988 constitution into an indigenous language for the first time, and Councilman Ramiro Rosário received criticism for using AI to write a water meter bill.

Language and translation expert Salvador Ordorica warns against using free AI-powered translation tools for major legal and constitutional documents because their inaccuracy could potentially leave citizens’ human rights open to interpretation.

This is based on new research which reveals the extent to which tools like ChatGPT and Google Translate struggle to accurately translate relatively simple text.

Researchers compared the AI translations of constitutions including Germany, Italy and Spain to the official English translations held on record.

AI made a series of errors, in some cases rewording the founding documents in a way that creates uncertainty about the fundamental human rights of citizens worldwide.

Across Europe, Italy has hit the headlines for amending its constitution, with Prime Minister Giorgia Meloni enshrining a requirement to promote a sports culture nationwide. When asked to translate articles 1-3 of the Italian Constitution, ChatGPT changed the original word ‘duty’ to ‘task’, relaxing the Italian government’s obligations to its people.

The official translation by the Senate of the Republic of Italy:

“It is the duty of the Republic to remove those obstacles of an economic or social nature which constrain the freedom and equality of citizens”.

ChatGPT version:

“It is the Republic’s task to remove the economic and social obstacles that, in fact, limit the freedom and equality of citizens.”

And when translating the Spanish Constitution, ratified in 1978 shortly after the end of Franco’s rule, the word ‘well-being’ became ‘good’.

Translation by the Official Gazette of Spain:

“The Spanish Nation, desiring to establish justice, liberty and security, and to promote the well-being of all its members.”

Google Translate version:

“The Spanish Nation, desiring to establish justice, liberty and security and promote the good of all its members.”

As Germany prepares to mark its Basic Law constitution’s 75th anniversary next month, one of the most concerning blunders occurred when AI translated Article 6.4 of the Basic Law for the Federal Republic of Germany, which deals with the rights of children.

Official translation by the German Bundestag:

“Children born outside of marriage shall be provided by legislation with the same opportunities for physical and mental development and for their position in society as are enjoyed by those born within marriage.”

ChatGPT version:

“Illegitimate children are to be provided by legislation with the same conditions for their physical and mental development and their status in society as legitimate children.”

Google Translate version:

“Illegitimate children are to be provided by legislation with the same conditions for their physical and mental development and their status in society as legitimate children.”

According to the UK Government, referring to children as illegitimate is a view that changed in 1926[1], 23 years before Germany’s Basic Law document was agreed.

In another example of poetic licence, Chat GPT changed the word ‘undermine’ to ‘infringe’ when translating a section on human rights in the French Constitution.

Similar errors were detected in the German translation with ‘interfered’ becoming ‘encroached’ when using Chat GPT.

Founder of professional translation services provider The Spanish Group, Salvador Ordorica a fluent Spanish speaker, said of the Spanish mistranslation:

“While the change might appear subtle, it is a substantial change to what is the foundational document that affects the lives of almost 48 million people. In a legal context, it could be argued that ‘well-being’ and ‘good’ do not mean the same thing at all.”

In the Polish translation of the Constitution, both Chat GPT and Google Translate used ‘belong’ instead of ‘vested’ whilst with Chat GPT “freedoms and rights of persons and citizens” became “freedoms and rights of man and citizen” with the latter creating potential for women’s rights to be ignored.

Whilst only a subtle difference, the benefit of human translators is emphasised with the small but significant difference between “Romanians living abroad” and “Romanians abroad” as per the Chat GPT translation of the Romanian Constitution.

On average, AI translation altered just over 14% of all the text it was asked to translate. Most errors were found in the Spanish, German, French and Romanian translations, with Dutch, Turkish, Polish and Russian being the translations with the fewest errors.

Mr. Ordorica, who speaks several languages and oversees the translation of thousands of documents every month, said:

“Both of these tools were developed in the USA, and interestingly, there is an inverse relationship between the language’s usage in the United States and the standard of the translations. Spanish, German, French and Italian are among the most spoken European languages but were also the translations most likely to contain errors. Where the languages are less widely spoken in the USA, error frequency was lower, suggesting that AI tools may be suffering from some degree of Americanisation-bias.”

He believes the research shows the pitfalls of relying on free tools.

“These might seem like trivial semantic nuances, but when it comes to important documents, accuracy and precision matters. We translate hundreds/thousands of official documents every month and we are keenly aware of how small differences can have a huge impact on whether a translation serves its purpose.

“Even slight mistranslations can be the difference between someone having their citizenship application accepted or rejected, so details matter”.

“My advice is simple; AI is fine for low-stakes tasks like translating a menu or learning a few phrases to use on holiday. But it should never be relied on for high stakes tasks that can have an impact on your life.”

Research Methodology

For the study, a direct comparison was made between: the native language Constitution as provided by Governmental websites or similar, plus the same sources’ English translation. The native language version was then inputted to ChatGPT and Google Translate respectively with the key differences being highlighted. The Constitutions selected for translation were based on the largest country by population as per 2024 data.

[1] https://www.parliament.uk/business/publications/research/olympic-britain/housing-and-home-life/kid-and-kin/