Vinay Garg Assistant Professor

Lesson 29: File processing and the os module

File processing means reading a file line by line and doing something with each line. The os module adds the filesystem around it: check whether a file exists, list a folder, rename, and delete.

Python · Lesson 29 of 29

A dark code window listing files with os.listdir and renaming one

In short

  • Processing a file usually means one loop over its lines, with `strip()` to remove the newline.
  • `readline()` reads one line at a time; `tell()` reports the current position in the file.
  • `os.path.exists`, `os.path.getsize` and `os.path.join` work with paths without opening anything.
  • `os.listdir`, `os.rename`, `os.remove` and `os.mkdir` change the filesystem, so test on copies first.

1

pass over a file needed for most processing

this lesson

4

filesystem calls in this lesson: listdir, rename, remove, mkdir

Python docs

Processing a file

File processing is the loop you write after opening: read one line, clean it, do something with it. Counting words, skipping comments, parsing a CSV by hand — all the same shape.

The program

# process.py — lines, positions, and the os module

import os

with open("data.txt", "w") as f:
    f.write("alpha\nbeta\ngamma\n")

with open("data.txt", "r") as f:
    lines = f.readlines()
print(len(lines), lines[0].strip())      # 3 alpha

with open("data.txt", "r") as f:
    print(f.readline().strip())          # alpha
    print(f.tell())                      # 6: position after the first line

with open("data.txt", "r") as f:
    for number, line in enumerate(f, start=1):
        print(number, line.strip())

print(os.path.exists("data.txt"))        # True
print(os.path.getsize("data.txt"))       # 17 bytes
print(os.path.splitext("data.txt"))      # ('data', '.txt')
print("data.txt" in os.listdir("."))     # True

os.rename("data.txt", "final.txt")
print(os.path.exists("final.txt"))       # True
os.remove("final.txt")
print(os.path.exists("final.txt"))       # False

How it works, step by step

  1. readlines() returns the file as a list of lines, each keeping its newline.
  2. strip() removes that newline so the printed line looks normal.
  3. readline() reads one line and moves the position; tell() then reports the byte offset — 6 after "alpha\n".
  4. Looping over the file with enumerate(f, start=1) gives a line number without a separate counter.
  5. os.path.exists() and getsize() only look; they do not open the file.
  6. os.rename() and os.remove() change the filesystem, which is why the checks after each one matter.

What it prints

3 alpha
alpha
6
1 alpha
2 beta
3 gamma
True
17
('data', '.txt')
True
True
False

The os calls worth knowing

CallDoes
os.getcwd()Current working directory
os.listdir(path)Names of the items in a folder
os.path.exists(p)True when the path exists
os.path.getsize(p)Size in bytes
os.path.join(a, b)Builds a path with the right separator
os.mkdir(p), os.makedirs(p)Create one folder, or a chain
os.rename(old, new)Rename or move
os.remove(p), os.rmdir(p)Delete a file, delete an empty folder

Always join paths with os.path.join

import os

folder = "records"
name = "marks.txt"
print(os.path.join(folder, name))   # records\marks.txt on Windows, records/marks.txt on Linux

Hard-coding a slash works on one system and breaks on the other. join() uses the separator of the machine the program is running on.

Common mistakes

  • Assuming the current folder. A script run from another place looks in that place; use an absolute path or os.path.dirname(__file__).
  • Calling os.remove() on a missing file: FileNotFoundError. Check with exists() first.
  • Deleting files before testing on copies. There is no undo and no recycle bin.

Practice

Try these before moving on. Each one uses only what this lesson covered.

  1. Write three lines to `data.txt`, then print each line with its line number using `enumerate`.
  2. Count the words and the lines in a file you create yourself.
  3. List every file in the current folder whose name ends with `.py` using `os.listdir`.
  4. Rename `data.txt` to `final.txt`, check that it exists, then delete it.

Sources

Frequently asked questions

What is file processing in Python?

Reading a file and working through its contents, usually one line at a time: cleaning each line, counting, splitting, or building records from it.

What is the os module used for?

Interacting with the operating system: listing folders, checking whether a path exists, getting file sizes, creating and removing directories, renaming files, and reading environment variables.

How do I check whether a file exists?

Use os.path.exists("file.txt"). It returns True or False without opening the file, which lets you avoid FileNotFoundError.

Why use os.path.join instead of writing a slash?

Because Windows and Linux use different separators. os.path.join builds the correct path for the machine the code runs on.

Vinay Garg

Vinay Garg

Assistant Professor of Computer Science at Multani Mal Modi College, Patiala. Twenty-five years of teaching web programming, and a habit of building the things I teach about.