Python’s interpreter is far more lenient with variable handling than most other programming languages. However, you’ll still run into situations where variable types cause conflicts with each other. This is especially true of bytes-like objects and stings. The cannot use a string pattern on a bytes-like object error is a perfect example of this phenomenon. But we can also fix it fairly easily.
What Is the Error Actually Stating?
The error message can be somewhat confusing at first. It states that we “cannot use a string pattern on a bytes-like object”“. But the actual code producing that error typically uses what appears to be strings of text.
The typical explanation is that the Python code is actually working with two variable types rather than one. These are strings and bytes-like objects. This is an easy mistake to make since both strings and bytes-like objects can behave and look quite similar to each other. It’s quite easy to see output that looks like a string, assume that’s what it is, and use common string pattern manipulation on it. But if it’s actually a bytes-like object you’ll receive the “cannot use a string pattern on a bytes-like object” error message.
A Deeper Dive Into Strings, Byte-Like Objects, and Patterns
One of the more confusing elements of a byte-like object is its distinction from Python’s other data types. Everything on a computer can be essentially reduced to bytes. So how do strings which consist of bytes differ from a byte-like object containing text?
The answer ultimately comes down to a definition based on whether or not Python’s system understands the data it’s working with. Strings in the 3.x branch of Python are encoded with Unicode. And bytes-like objects might also contain Unicode encoded text. The difference between these two examples stems from the fact that Python’s interpreter understands that the string is using the Unicode standard. Strings essentially come with a notification to translate that data using Unicode. But a bytes-like object is essentially treated as an unknown entity. And this is why string manipulation methods don’t work on bytes-like objects. Python’s interpreter doesn’t know if the object contains standard Unicode data. You can define the bytes-like object as such, but doing so essentially redefines it into a string.
Of course bytes-like objects can consist of far more than just text. But error messages referring to the interaction of strings and bytes-like objects usually stem from bytes-like objects which contain text. For example, the popular urllib library can take a string value for URLs and returns what appears to be plain text. But that text is actually a bytes-like object. Trying to work with the returned data will result in the “cannot use a string pattern on a bytes-like object” error. Consider the following example.
import urllib.request
import re
with urllib.request.urlopen(“https://google.com”) as source:
point = source.read()
match = “title(.*?)/title”
neededData = re.search(match, point)
print(“Title = {}.”.format(neededData))
Note that you may want to put opening and closing tags around the two title keywords in the match variable assignment. They’re left without HTML tags here for ease in formatting since the data we’re grabbing is secondary to actually demonstrating and fixing the underlying datatype error.
We begin by importing the urllib and regular expression libraries. We’ll use these to scrape some basic information from Google’s default search page. Next, we try to run a simple regular expression on the returned data to find the page’s title. It should be fairly simple as it’s set between the two title tags. But the script fails with our Python error. The fix is fairly easy since we just need to clear up Python’s confusion about the data types.
How To Fix the Error
We essentially have two solid options to help Python’s interpreter understand the data we’re working with. We can either change the string to a bytes-like object or change the bytes-like object into a string. Lets begin by converting the string into a bytes-like object. You can use the previous example code, but change the match definition to the following line.
match = bytes(“title(.*?)/title”, “utf-8”)
This is fairly similar to the original assignment. But the text used for the regular expression is first fed through the bytes function. Bytes takes two parameters. The first is the actual text to encode into a bytes-like object. And the second explains the nature of the data being sent. Python’s strings use UTF-8 encoding, so we pass that along with the data. The regular expression is now working with two compatible byte-like objects, so we’ll see the rest of the script run cleanly and return the title information from Google’s search page.
We can also go the opposite route and convert the bytes-like object into a string. Load up the original example code again. But you’ll want to replace this line.
point = source.read()
Swap that line out with the following code.
print(source.headers[‘content-type’])
point = source.read().decode( source.headers[‘content-type’].split(“=”, 1)[1] )
The first of these new lines isn’t really necessary to fix the error. Instead, it’s there to explain the following line’s formatting. We make a call to urllib’s headers function and pass the results to print. We now see some information about what we’re dealing with. The fact that the data is HTML isn’t surprising. But we also receive the text encoding used within the content. This is the information that we’ll be working with on the next line.
We make an assignment to the point variable again. But this time we use the decode function on the source variable. Decode, as the name suggests, decodes a bytes-like object into a string. But we need to know how the text is encoded before we can do so. This is where the headers call comes into play. Encode will default to UTF-8 decoding if no variable is passed. But we can’t be sure that we’re dealing with UTF-8. So we instead need to grab the descriptive text from headers.
We then narrow the header data string down to a substring containing the information following the equal sign. This is the encoding type. We can pass that information as a string to the encode function. And this will create a properly formatted encode with the correct data. The rest of the script will now run error-free because the regular expression using a string is being used on a string.