Files over the Internet

The file containing the data you want could be located on a computer half a world away or even in the cloud. Provided the file is accessible over the Internet, though, you can read it just as you do a local file. For example, the Hopedale data not only exists on your computers, but it’s also, at the time of writing, available online.

The location of an object, such as a data file, on the Internet is specified by a Uniform Resource Locator (URL). A URL is a string that consists of the access protocol name (usually https:// or http://), the remote computer name (for example, robjhyndman.com), and the file path at the remote computer (for example, /tsdldata/ecology1/hopedale.dat). It informs your computer on how to request a remote resource, where to request it, and what exactly to request.

The urllib.request module contains a function called urlopen that opens a URL for reading. urlopen returns a file-like object that you can use as if you were reading a local file.

There’s a hitch: because there are many kinds of files (images, music, videos, text, and more), the file-like object’s read and readline methods both return a type you haven’t yet encountered: bytes.

What’s a Byte?

Computer hardware stores information as bits, binary digits representing ones and zeros. Bits are grouped into units of eight, referred to as bytes. All data, like text, sounds, and pixels, are ultimately encoded as sequences of bytes. Programming languages provide tools and abstractions that allow you to work with these bytes as higher-level data types, such as integers, characters, strings, images, and files.

Working With Bytes

When dealing with type bytes, such as a piece of information returned by a call to the urllib.urlrequest.read function, you need to decode it. To decode it, you need to know how it was encoded.

Common encoding schemes are described in the online Python documentation. One of the most common encodings is UTF-8, a character encoding designed to represent Unicode.

The Hopedale data on the web is encoded using UTF-8. This program reads the URL and uses the string method decode to decode the bytes object into a string:

​	​from​ ​urllib.request​ ​import​ urlopen
​	url = ​'https://robjhyndman.com/tsdldata/ecology1/hopedale.dat'​
​	​with​ urlopen(url) ​as​ doc:
​	​for​ line ​in​ doc:
​	line = line.strip().decode(​'utf-8'​)
​	​print​(line)

Отправить