"syntaxerror: Non-ASCII character ..." or "SyntaxError: Non-UTF-8 code starting with ..." trying to use non-ASCII text in a Python script

Handling non-ASCII characters with UTF-8

When you're dancing with non-ASCII characters, like the pound sign (£), your Python script must be prepared to tango. Python 3 comes to the dance floor with UTF-8 boots, while Python 2 requires some style advice.

Here's the deal:

1. Choose UTF-8 encoding as your first dance partner, unless you have specific reasons to pick another partner.
2. Call out your dance partner with a magic comment if you're not dancing with UTF-8.
3. Ensure the save format of your file is cute with your dance partner. Just match the declared encoding.

Alternate dance partners and outlying moves

Sometimes the salsa of UTF-8 in Python 2 doesn't quite match your rhythm. Try a tango with Latin-1 (latin-1). Announce your partner like this:

# coding: latin-1
# Dancing the Latin-1 tango, cha-cha-chow!


Performing a flamenco with mixed encodings or rarely used characters? Unicode escape sequences are your rose-in-mouth move.

Steps to stay in rhythm:

1. Validation check—make sure you're not dancing the Jitterbug by accident. Use a hex editor.
2. Declare your dance partner—don't leave your partner guessing, it may lead to tripping over feet (errors).
3. Dance consistently—don't change partners mid-dance. Stick to one encoding for the file.

Diving deeper: The encoding quandary

Maintaining consistency for non-ASCII character handling

Who wants a jumbled mess of characters when you can have perfect harmony? To achieve that:

• Make sure that the source file's encoding agrees with its encoding declaration.
• When in doubt, opt for Unicode literals, e.g., u"£".
• When ASCII is your only option, escape sequences like \xa3 for the pound sign can save the day.

Safeguarding against common pitfalls

The encoding declaration is powerful but not almighty.

Keep the following cautions in mind:

• The encoding comment should be in the first or second line of the file.
• Saving in a different encoding can still cause a SyntaxError, even with the correct declaration.
• Ensure your text editor is in sync with your encoding preferences, particularly with UTF-8.

Debugging made easy

Still facing encoding errors? Here's how to troubleshoot:

• Re-check the script's save format—has your file really been saved in UTF-8 or the encoding declared?
• Check third-party libraries or modules—they may not follow the same encoding rules.

Understanding encodings Becoming familiar with character encodings can save you a lot of time. The PEP references and online encoding guides mentioned in the references below are great places to start.