Appendix I — The os Module

I.1 Filesystem Operations

The os module helps us access and manipulate the file system.

I.1.1 Current Working Directory

Detecting the name of the current working directory, using the getcwd function:

import os

os.getcwd()

'/content'

We see in Google Colab the default working directory is called “content”.

I.1.2 Listing Files

Listing all files and folders that exist in a given directory (for example in the “content” directory where we are right now), using the listdir function:

os.listdir("/content")

['.config', 'sample_data']

We see there is a “sample_data” directory.

After further inspection, we see it contains some example text and data files:

os.listdir("/content/sample_data")

['anscombe.json',
 'README.md',
 'california_housing_train.csv',
 'mnist_train_small.csv',
 'mnist_test.csv',
 'california_housing_test.csv']

Note

So far we have used an absolute file reference, but since we are already in the “content” directory, it is possible to use a relative file references instead. These references are relative to the “content” directory, where we are right now.

os.listdir()

['.config', 'sample_data']

os.listdir("sample_data")

['anscombe.json',
 'README.md',
 'california_housing_train.csv',
 'mnist_train_small.csv',
 'mnist_test.csv',
 'california_housing_test.csv']

For the remainder of this chapter we will continue using relative references, for simplicity.

I.1.3 Detecting Directories and Files

Checking to see whether a given directory or file exists, using the isdir and isfile functions from the os.path sub-module:

os.path.isdir("sample_data")

True

os.path.isfile("sample_data/README.md")

True

I.1.4 Reading and Writing Files

See the Text File Operations chapter for examples of how to read and write text files using the open function.

See Getting Started with Pandas for examples of how to read and write tabular data files using the pandas package.

I.1.5 Deleting Files

filepath = "sample_data/anscombe.json"

# verifying the file exists:
os.path.isfile(filepath)

True

Deleting a file, using the remove function:

os.remove(filepath)

# verifying the file was deleted:
os.path.isfile(filepath)

False

I.1.6 Creating Directories

Creating a new directory using the makedirs function:

os.makedirs("my_data", exist_ok=True)

# verifying the "my_data" directory got created:
os.listdir()

['.config', 'my_data', 'sample_data']

I.1.7 Deleting Directories

To delete an empty directory, we can use the rmdir function from the os module, however it only works for empty directories and throws an error if the directory does not exist. So for a more robust solution, we can use the rmtree function from the shutil module:

#os.rmdir("my_data")

from shutil import rmtree

rmtree("my_data", ignore_errors=True)

# verifying the "my_data" directory got deleted:
os.listdir()

['.config', 'sample_data']