output-formatters | Apply transformations to structured data | Reflection library

Ok, apparently I hadn't read deep enough in to the docs, here's where the answer can be found:

From https://www.crummy.com/software/BeautifulSoup/bs4/doc/#encodings:

The problem is that the snippet of code provided to BS is so short, that BeautifulSoup's sub-library Unicode, Dammit, doesn't have enough info to properly guess the encoding.

Unicode, Dammit guesses correctly most of the time, but sometimes it makes mistakes. ...you can avoid mistakes and delays by passing it to the BeautifulSoup constructor as from_encoding.

So the key is to add from_encoding="UTF-8" to each time the BS is constructed:

soup = BeautifulSoup(soup.renderContents(), "html.parser", from_encoding="UTF-8")

BeautifulSoup tried to preserve the newlines and multiple spaces you had in the attribute values in the input HTML.

One workaround here would be to iterate over the element attributes and clean them up prior to prettifying - removing the newlines and replacing multiple consecutive spaces with a single space: