utf8.js | Convert JavaScript strings to a UTF8 byte array

We all know UTF-8 is hard. I exported my messages from Facebook and the resulting JSON file escaped all non-ascii characters to unicode code points.

I am looking for an easy way to unescape these unicode code points to regular old UTF-8. I also would love to use PowerShell.

I tried

I have been trying to understanding why the need for encoding/decoding to UTF-8 happens all over the place in JavaScript land, and learned that JavaScript uses UTF-16 encoding.

Let’s talk about Javascript string encoding

So I'm assuming that's why a library such as utf8.js exists, to convert between UTF-16 and UTF-8.

But then at the end he provides some insights:

Encoding in Node is extremely confusing, and difficult to get right. It helps, though, when you realize that Javascript string types will always be encoded as UTF-16, and most of the other places strings in RAM interact with sockets, files, or byte arrays, the string gets re-encoded as UTF-8.

This is all massively inefficient, of course. Most strings are representable as UTF-8, and using two bytes to represent their characters means you are using more memory than you need to, as well as paying an O(n) tax to re-encode the string any time you encounter a HTTP or filesystem boundary.

That reminded me of the in the HTML , which I never really thought too much about, other than "you need this to get text working properly".

Now I'm wondering, which this question is about, if that tag tells JavaScript to do UTF-8 encoding. That would then mean that when you create strings in JavaScript, they would be UTF-8 encoded rather than UTF-16. Or if I'm wrong there, what exactly it is doing. If it is telling JavaScript to use UTF-8 encoding instead of UTF-16 (which I guess would be considered the "default"), then that would mean you don't need to be paying that O(n) tax over doing conversions between UTF-8 and UTF-16, which would mean a performance improvement. Wondering if I am understanding correctly, or if not, what I am missing.

I faced a weird problem today after trying to decode a utf8 formatted string. It's being fetched through stream as an array of strings but formatted in utf8 somehow (I'm using fast-csv). However as you can see in the console if I log it directly it shows the correct version but when it's inside an object literal it's back to utf8 encoded version.