Chinese is really hard to learn if you are referring with their own manuscript. Even those some Chinese people as well they have difficulty n learning lots of things.
I had a friend of mine in Malaysia actually a Malaysian-Chinese guy who had not learned the Chinese manuscript but he can speak it. He sent his daughter to school to learn Chinese and it was funny when some of the Chinese letters he sees he ask his 5 year-old daughter on what does it says.
About in how to speak Chinese, I believe it's not that difficult although Mandarin has no past, present, past participle in their grammar that's why I thought most of them they have some difficulties to adapt English language.