Chinese | General tool for Chinese parsing

I must say the module pyttsx3 looks like it's not responding well to language changes. The synthesizer is aweful and something was missing.

Until I encountered gtts lib.

In order to get all supported languages use the following: print(gtts.lang.tts_langs())

Which will output:

In this case, I'll use geom_text instead of annotate, since it allows you to have subset of your data.

To extract the chinese words only, We could use str_extract: extracting all non latin characters with "[:alpha:]+":

• Don't use ToCharArray() when working with non-"ASCII" (or at least, non-Latin-style) text.
• Instead, use TextElementEnumerator.
  • Here's why.

In Unicode, the 𧜏 character has a code-point of U+2770F which is outside the range supported by a single 16-bit UTF-16 value (i.e. 2 bytes, a single .NET Char value), so UTF-16 uses a pair of separate 16-bit values known as a surrogate pair to represent it:

I found an extremely hacky solution that seems to work. However, it seems absurd that there's no way to simply set the locale of a label, so if anyone finds something I missed, please post an answer.

The trick relies on the fact that the Hiragino font displays kanji using Japanese encoding rather than Chinese encoding by default. However, the font looks like shit for English text, so I have to search every string in every label for Japanese substrings and manually change the font using NSMutableAttributedString. The font is also completely broken so I had to find another workaround to fix that.

wchar_t is not the same as char16_t. wchar_t are 2 byte characters on windows, but (usually) 4 byte characters on linux. This is like the int vs. int16_t problem. The standard does not define wchar_t.

So the question is not what format specifier to use with wprintf. It's rather how to convert a char16_t string to a wchar_t string.

Under Windows you might get away with simply casting a char16_t to a wchar_t, which is what happens implicitly with wprintf, since it does not actually validate it's parameters. The warning C4477 is just a little help by the (Visual Studio?) compiler hinting at your problem.

But on other platforms you have to actually convert the string.

So the best solution would be something like this: wprintf("%ls", boost::utf16_to_wchar_t(chinese)); (I am just throwing in boost here, since they have conversion functions. I don't know the exact function to use).

Or alternatively use wchar_t escape sequences and define your chinese as a wchar_t* string.

Puctuation marks are scattered about in different Unicode code blocks.

The Unicode® Standard
Version 14.0 – Core Specification

Chapter 6
Writing Systems and Punctuation
https://www.unicode.org/versions/latest/ch06.pdf

Punctuation. The rest of this chapter deals with a special case: punctuation marks, which tend to be scattered about in different blocks and which may be used in common by many scripts. Punctuation characters occur in several widely separated places in the blocks, including Basic Latin, Latin-1 Supplement, General Punctuation, Supplemental Punctuation, and CJK Symbols and Punctuation. There are also occasional punctuation characters in blocks for specific scripts.

Here are two of your examples,

〜 Wave Dash U+301C

。Ideographic Full Stop U+3002

If you are using Mui v5, the syntax is slightly different:

pandas string methods allow accessing values from lists/tuples/dictionaries: