Using chardet to convert arbitrary byte strings to Unicode

chardet is a fantastic module for finding the encoding of arbitrary byte strings. You can combine this with a check for a BOM to pretty reliably turn them into Unicode. Edit: Thanks to Kirit's comment below, I added code to check for UTF-32. import chardet def bytes2unicode(bytes, errors='replace'):     """Convert a byte string into […]