It’s actually an interesting question. For English speakers anyway, results from the US Army School of Languages in Monterey, California, showed that Chinese, Japanese and Korean are the three hardest.
I believe that Chinese was the worst of all. The main problem with Chinese is that it has an ungodly and unwieldy writing system that is very difficult to learn.
Literate Chinese are supposed to know 3,000 characters. The government has set a goal of 80% of Chinese learning 3,000 characters or more. That’s never going to happen. At the bottom end, you are supposed to know 1,000 characters, but a lot of Chinese don’t even know that many. A very literate Chinese should know 10,000 characters, but I doubt if 1% know that many. There are about 30,000 Chinese characters, but I doubt if anyone knows all of them.
Many rural Chinese learned to read and write but then forgot it once they got to be about 40 or so. They are in the fields all day, and they don’t read and write much, so they just forget.
There is a good argument out there that the Chinese writing system is so unwieldy that it actually harms Chinese competitiveness and the economy. Economic calculations have actually been made of how much damage this orthographic system costs.
When the Communists took over, they introduced a pinyin system in addition to simplifying the Chinese characters. The simplification was a great idea, but the language was still very hard to read and write. The pinyin Romanization is only one of many that have been introduced over time. I don’t think that any of them have worked out well. It is said that the Chinese language being written enabled ~3,500 Chinese dialects to all speak to each other, but now that everyone is learning Putonghua anyway, why not just write Putonghua in pinyin?
There is a problem with writing the other major Chinese languages using the system designed for Mandarin. It is not so easy to write Cantonese, Min, Hakka,Wu, Xiang and Hui using the Mandarin character set. One problem is that Cantonese for instance has quite a few words that lack Chinese characters. To some extent this is true with the other languages also. Min has a Romanization scheme, but most Min speakers don’t know how to use it. The whole idea of writing something other than Putonghua using the traditional Chinese character set introduces all sorts of minefields.
I believe that alternate character sets or additions to the character sets have been introduced for some of the other languages. Due to tones, it gets hard to write Chinese using a Romanization scheme easily, since you have to put all sorts of diacritics on the letters in the isolating language of Chinese.
Something similar is happening with Japanese, which after all uses a character set borrowed from the Chinese.
After the war there were attempts to introduce a Romanization system to Japanese. However, by 1960 or so, Japanese nationalism had returned to Japan, at least to the extent that this Romanization system was seen as a Western imperial affront to the mystical Japanese super-culture. Since then, things have only been on the downswing. Japan actually uses three different kinds of symbol sets, Katakana, Kanji and Haragana. There are Chinese characters mixed into all of this stuff.
Even beyond that, the Japanese language, though not tonal, is mind-bafflingly complex. There are rules, but then there are tons of exceptions to those rules, but the exceptions are not really taught. A native speaker just more or less unconsciously figures out the rules and the exceptions in the course of growing up Japanese.
Anyway, this rule-exception mix is so chaotic and senseless that it’s almost impossible to codify it somehow and then teach it to non-native speakers. The Japanese also have strange concepts like using different counting systems when counting different types of things.
As if speaking it alone were not complicated enough, there is that writing system. Once again, reasonable people have figured out that the convoluted logographics costs the Japanese economy quite a bit per year. But the logographics is seen by the hyperethnocentric Japanese now as a mystical part of their Super-race and Super-culture, and they will not allow anyone to lay hands on it, especially not pesky Western imperialists who occupied their land tried to shove the West down their throats.
The Korean logographic system, Hangul, is actually excellent, and is one of the most logical alphabets ever devised, or so say scholars. It uses a limited character set like English. However, in some way that I am not familiar with, the Korean language is not easy at all for English speakers to learn. Whatever it is, it is not the writing system.
In terms of European languages, Finnish and Hungarian are said to be Godawful languages to learn. British diplomats who were placed all over Europe were notorious for refusing posts in Hungary due to the difficulty in learning the language. Finnish of course is one of the most case-marked major languages on Earth, with 14-15 different cases. Coming from a language like English that does not mark case very much, that must be awfully hard.
In a on the Internet, the following languages were thought to be the hardest to learn: