FH

File Handling in Python

Understanding text files, random files, and file handling operations in Python

Learning Objectives

By the end of this lesson, you will be able to:

  • Understand why text files are used in programming
  • Open, read, write, and close text files in Python using appropriate modes
  • Use the with statement for automatic file handling
  • Combine arrays (lists) with text files to process data efficiently
  • Understand the concept of random files and their advantages
  • Perform file processing operations on serial, sequential, and random files
  • Use pseudocode for random file operations (OPENFILE, SEEK, GETRECORD, PUTRECORD, CLOSEFILE)
  • Use Python's pickle module for serializing and deserializing objects
  • Work with binary file modes for random file handling
  • Implement complete file handling solutions for real-world scenarios

Key Terms

Text File

A file containing plain text data that can be read by humans and programs

File Mode

Specifies how a file is opened: read ('r'), write ('w'), append ('a'), or binary ('b')

Serial File

Records are stored one after another; must be read sequentially from start

Sequential File

Records stored in order (e.g., alphabetical); can be read from any point but sequentially

Random File

Allows direct access to any record using a pointer/position

Record Structure

A composite data type defining fields for storing related data

Pickle

Python module for serializing (pickling) and deserializing (unpickling) objects

Serialization

Converting a Python object into a binary format for storage (pickling)

Deserialization

Converting binary data back into a Python object (unpickling)

SEEK

Moves the file pointer to a specific position in a random file

Array (List)

Python data structure used to store and process collections of data

Binary Mode

File mode ('rb', 'wb', 'r+b') for handling non-text files like pickled objects

Text File Handling

Text files are used in high-level programming languages like Python to store data that needs to persist between program runs. They allow programs to save output, store structured or unstructured data, and facilitate data exchange between different programs or systems.

Real-Life Example: Student Records System

Think of a school management system that needs to:

  • Save student names in a file for future reference
  • Store exam results that can be analyzed later
  • Log system activities for debugging purposes
  • Export data to share with other systems

All these tasks use text file handling operations!

Why Use Text Files?

Persistence

Data remains after program ends

Data Exchange

Share data between different programs

Processing

Read, write, and update content easily

File Opening Modes

Mode Description If File Exists If File Doesn't Exist
'r' (read) Open file for reading only Opens successfully Returns an error
'w' (write) Open file for writing only Overwrites the file Creates a new file
'a' (append) Open file for appending only Adds to end of file Creates a new file
'r+' (read+) Open file for reading and writing Opens successfully Returns an error

File Operations Simulation

How it works: This simulation shows how Python interacts with files. The file pointer moves as you read or write data. Notice how different modes affect the file content.

1. Opening and Closing Files

To work with a file, you need to open it first using the open() function and close it using close() method.

# Basic file opening and closing
file = open("example.txt", "w")  # Open file in write mode
# Perform operations...
file.close()                   # Always close the file

2. Writing to a File

To write data into a file, open it in 'w' (write) mode. If the file doesn't exist, Python will create it.

1
Open file in write mode
2
Write data using write()
3
Close the file
# Task: Create a file called data.txt and write "Python is awesome!" into it

file = open("data.txt", "w")
file.write("Python is awesome!")
file.close()

3. Reading from a File

You can read content from a file using the read() or readline() method.

# Read the whole file
file = open("example.txt", "r")
content = file.read()
print(content)
file.close()
# Read line by line
file = open("example.txt", "r")
for line in file:
    print(line)
file.close()
# Task: Read the content of data.txt and print it

file = open("data.txt", "r")
content = file.read()
print(content)
file.close()

4. Appending to a File

To add new content to the end of an existing file, open it in 'a' (append) mode.

# Task: Append "Python makes coding fun!" to data.txt

file = open("data.txt", "a")  # Open in append mode
file.write("\nPython makes coding fun!")  # Add new content
file.close()

5. Using the with Statement

The with statement is the best way to handle files because it automatically closes the file when you're done.

# Task: Use the with statement to read the content of data.txt

with open("data.txt", "r") as file:
    content = file.read()
    print(content)
# File automatically closes here

Combining Text Files with Arrays in Python

In Python, you can use arrays (lists) to manage and process data from text files efficiently. You can read data from text files into arrays, manipulate the data, and write it back to a file.

Reading Data into Arrays

# Read numbers from a file into an array
numbers = []
with open("numbers.txt", "r") as file:
    for line in file:
        numbers.append(int(line.strip()))
print(numbers)  # Output: [10, 20, 30, 40, 50]

Writing Arrays to Files

# Write numbers from an array into a file
numbers = [10, 20, 30, 40, 50]
with open("output.txt", "w") as file:
    for number in numbers:
        file.write(f"{number}\n")

Activity 1: Student Records System

Problem: Write a program that:

  1. Creates a file called students.txt and writes three student names into it
  2. Reads and prints all the names
  3. Appends a new student name to the file

Use appropriate file handling techniques including the with statement.

Solution:
# Step 1: Write three names to the file
with open("students.txt", "w") as file:
    file.write("Alice\n")
    file.write("Bob\n")
    file.write("Charlie\n")

# Step 2: Read and print all names
with open("students.txt", "r") as file:
    for line in file:
        print(line.strip())

# Step 3: Append a new name
with open("students.txt", "a") as file:
    file.write("Diana\n")

Explanation: This program demonstrates all three basic file operations: writing, reading, and appending. The with statement ensures files are properly closed.

Activity 2: Processing Data with Arrays

Problem: Write a Python program that reads numbers from numbers.txt and:

  1. Creates two arrays: one for even numbers and one for odd numbers
  2. Saves the even numbers to even_numbers.txt
  3. Saves the odd numbers to odd_numbers.txt

Sample numbers.txt content: 10, 3, 8, 5, 12 (each on new line)

Solution:
# Open the file and read numbers
file = open("numbers.txt", "r")
numbers = []
for line in file:
    number = int(line.strip())
    numbers.append(number)
file.close()

# Separate even and odd numbers
even_numbers = []
odd_numbers = []

for num in numbers:
    if num % 2 == 0:
        even_numbers.append(num)
    else:
        odd_numbers.append(num)

# Write even numbers to file
file = open("even_numbers.txt", "w")
for number in even_numbers:
    file.write(str(number) + "\n")
file.close()

# Write odd numbers to file
file = open("odd_numbers.txt", "w")
for number in odd_numbers:
    file.write(str(number) + "\n")
file.close()

Explanation: This shows how to read data from a file into an array, process it (separate even/odd), and write the results to different files.

Check Your Understanding: Text File Handling

Answer
  • [1 mark] To store data that needs to persist between runs of a program
  • [1 mark] To save output or logs for future reference
  • [Additional] To store structured/unstructured data and enable data exchange between programs
Answer
  • [1 mark] The existing file is overwritten (all previous content is lost)
  • [Additional] If the file doesn't exist, Python creates a new file
Answer
  • [1 mark] It automatically closes the file when you're done
  • [1 mark] It ensures proper resource management even if an error occurs
  • [Additional] It makes code cleaner and reduces the risk of forgetting to close files
Answer
  • [1 mark] Open the file in read mode using open("filename.txt", "r")
  • [1 mark] Use a for loop to iterate through the file object: for line in file:
  • [Additional] Alternatively, use readline() or readlines() methods
Answer
  • [1 mark] 'w' (write) mode overwrites the entire file if it exists
  • [1 mark] 'a' (append) mode adds new content to the end of the existing file
  • [Additional] Both create a new file if it doesn't exist
Answer
  • [1 mark] Read data from text files into arrays using loops or list comprehensions
  • [1 mark] Process the data in the array (filter, sort, calculate)
  • [1 mark] Write the processed data back to text files
  • [Additional] Examples: Separate even/odd numbers, filter words by length, combine data from multiple files

Random Files

Random files allow direct access to specific records by using a pointer to locate them. Each record is stored in a structured format and can be retrieved or updated without sequentially reading the entire file.

Direct Access

Access any record directly using its position without reading previous records

Binary Format

Data stored in binary format using Python's pickle module

Types of File Organisation

Serial Files

Records stored one after another; must be read sequentially from start

Example:

Log files, transaction records

Sequential Files

Records stored in order (e.g., alphabetical); can be read from any point but sequentially

Example:

Phone directories, dictionaries

Random Files

Allows direct access to any record using a pointer/position

Example:

Databases, indexed files

Real-Life Example: Library Management System

Problem
  • Library needs to store thousands of book records
  • Need to quickly find, update, or delete any book
  • Don't want to read all records to find one book
  • Need to maintain book availability counts
Random File Solution
  • Each book stored at a specific position
  • Use ISBN or book ID to calculate position
  • Directly access any book in constant time
  • Update availability without rewriting entire file

Random Files Using Pseudocode

Random File Record Visualization

How it works: Each record has a fixed position in the file. The SEEK command moves the pointer to a specific position, and GETRECORD/PUTRECORD read/write records at that position.

Pseudocode File Commands

OPENFILE

Opens the file in RANDOM mode

SEEK

Moves the pointer to a specific record position

GETRECORD

Reads a record from the file at current pointer

PUTRECORD

Writes a record to the file at current pointer

CLOSEFILE

Closes the file after use

Record Structure Example

// Student record structure
TYPE Student
    DECLARE LastName : STRING
    DECLARE FirstName : STRING
    DECLARE DateOfBirth : DATE
    DECLARE YearGroup : INTEGER
    DECLARE FormGroup : CHAR
ENDTYPE

Adding a New Record

// Pseudocode for adding a new student record
DECLARE NewStudent : Student

// Assign values to the new student's fields
NewStudent.LastName ← "Doe"
NewStudent.FirstName ← "John"
NewStudent.DateOfBirth ← 15/03/2005
NewStudent.YearGroup ← 11
NewStudent.FormGroup ← 'B'

// Open the file in RANDOM mode
OPENFILE "StudentFile.dat" FOR RANDOM

// Move the pointer to position 5 and write the record
SEEK "StudentFile.dat", 5
PUTRECORD "StudentFile.dat", NewStudent

// Close the file
CLOSEFILE "StudentFile.dat"

Updating an Existing Record

// Pseudocode for updating a student record
DECLARE ExistingStudent : Student

// Open the file in RANDOM mode
OPENFILE "StudentFile.dat" FOR RANDOM

// Move to position 3 and read the record
SEEK "StudentFile.dat", 3
GETRECORD "StudentFile.dat", ExistingStudent

// Update the YearGroup of the student
ExistingStudent.YearGroup ← ExistingStudent.YearGroup + 1

// Write the updated record back to the same position
SEEK "StudentFile.dat", 3
PUTRECORD "StudentFile.dat", ExistingStudent

// Close the file
CLOSEFILE "StudentFile.dat"

Random Files Using Python (Pickle Module)

The pickle module in Python is used for serializing and deserializing Python objects. Serialization (or "pickling") converts a Python object into a binary format for storage. Deserialization (or "unpickling") converts binary data back into a Python object.

Pickle Methods

pickle.dump()

Saves (serializes) an object to a binary file

import pickle
data = {"name": "Alice", "age": 30}
with open("data.pkl", "wb") as file:
    pickle.dump(data, file)
pickle.load()

Loads (deserializes) an object from a binary file

import pickle
with open("data.pkl", "rb") as file:
    loaded_data = pickle.load(file)
print(loaded_data)

Binary File Modes

wb
Write Binary

Open for writing in binary mode (used for pickling)

rb
Read Binary

Open for reading in binary mode (used for unpickling)

r+b
Read/Write Binary

Open for reading and writing in binary mode (random access)

Complete Python Example

# Python example: Random file handling with pickle
import pickle

# Define a Book class
class Book:
    def __init__(self, title, author, year_published, isbn, copies_available):
        self.title = title
        self.author = author
        self.year_published = year_published
        self.isbn = isbn
        self.copies_available = copies_available
    
    def __str__(self):
        return f"Title: {self.title}, Author: {self.author}, Year: {self.year_published}, ISBN: {self.isbn}, Copies: {self.copies_available}"

# Create a book object
book = Book("To Kill a Mockingbird", "Harper Lee", 1960, "9780060935467", 5)

# Write the book to position 3 in the file
with open("LibraryDatabase.dat", "r+b") as file:
    file.seek(3 * 128)  # Move pointer to position 3 (assuming 128 bytes per record)
    pickle.dump(book, file)

# Read the book back from position 3
with open("LibraryDatabase.dat", "rb") as file:
    file.seek(3 * 128)  # Move pointer to position 3
    retrieved_book = pickle.load(file)
    print(retrieved_book)

Important Notes About Pickle

  • Security Warning: Never unpickle data from untrusted sources - it may execute malicious code
  • Compatibility: Pickled files are Python-specific and might not work across different Python versions
  • Fixed Record Size: In random files, ensure consistent record sizes for accurate positioning with seek()
  • Use Cases: Save program settings, store intermediate data, save/load game progress

Activity 3: Library Record Structure

Task: A library uses a random file to store its book collection. Each book record includes the following fields:

  • CopiesAvailable: INTEGER
  • Title: STRING
  • Author: STRING
  • YearPublished: INTEGER
  • ISBN: STRING

1. Create the record structure in pseudocode
2. Write pseudocode to add the book "Pride and Prejudice" by Jane Austen at position 7 with 12 copies available

Solution:
// 1. Record structure
TYPE Book
    DECLARE CopiesAvailable : INTEGER
    DECLARE Title : STRING
    DECLARE Author : STRING
    DECLARE YearPublished : INTEGER
    DECLARE ISBN : STRING
ENDTYPE
// 2. Adding a new book
DECLARE NewBook : Book

// Assign values
NewBook.CopiesAvailable ← 12
NewBook.Title ← "Pride and Prejudice"
NewBook.Author ← "Jane Austen"
NewBook.YearPublished ← 1813
NewBook.ISBN ← "9780141199078"

// Open file and add record
OPENFILE "Library.dat" FOR RANDOM
SEEK "Library.dat", 7
PUTRECORD "Library.dat", NewBook
CLOSEFILE "Library.dat"

Activity 4: Pickle Module Practice

Task:

  1. Define a custom class Product with fields: name (string), price (float), stock (integer)
  2. Create an instance with: name="Laptop", price=1500.00, stock=10
  3. Save this object to a file named product.pkl
  4. Load the object from product.pkl and print its details
Solution:
import pickle

# 1. Define Product class
class Product:
    def __init__(self, name, price, stock):
        self.name = name
        self.price = price
        self.stock = stock
    
    def __str__(self):
        return f"Product: {self.name}, Price: ${self.price}, Stock: {self.stock}"

# 2. Create instance
product = Product("Laptop", 1500.00, 10)

# 3. Save to file
with open("product.pkl", "wb") as file:
    pickle.dump(product, file)

# 4. Load from file and print
with open("product.pkl", "rb") as file:
    loaded_product = pickle.load(file)
    print(loaded_product)

Explanation: This demonstrates how to use pickle to save and load custom class objects. Note the use of binary modes ('wb' for writing, 'rb' for reading).

Check Your Understanding: Random Files

Answer
  • [1 mark] Direct access to any record without reading previous records
  • [1 mark] Faster retrieval and updating of specific records
  • [Additional] More efficient for databases and systems requiring frequent random access
Answer
  • [1 mark] Moves the file pointer to a specific record position
  • [1 mark] Allows direct access to that record for reading or writing
  • [Additional] Example: SEEK "StudentFile.dat", 5 moves to record position 5
Answer
  • [1 mark] The process of converting a Python object into a binary format
  • [1 mark] Also called "pickling" - allows objects to be stored in files or transmitted
  • [Additional] The opposite process (converting binary back to object) is deserialization or unpickling
Answer
  • [1 mark] pickle stores data in binary format, not plain text
  • [1 mark] Binary mode ensures no character encoding issues occur during reading/writing
  • [Additional] Modes: 'wb' for writing, 'rb' for reading, 'r+b' for reading and writing
Answer
  • [1 mark] A composite data type defining fields for storing related data
  • [1 mark] Provides a blueprint for how data is organized in the file
  • [1 mark] Ensures consistent record size for accurate positioning with SEEK
  • [Additional] Example: Student record with LastName, FirstName, YearGroup fields
Answer
  • [1 mark] Never unpickle data from untrusted sources
  • [1 mark] Pickled data can execute arbitrary code during unpickling
  • [Additional] Only unpickle data from trusted sources you created or verified

Key Takeaways

  • Text files persist data between program runs and enable data exchange between systems
  • Always close files after use, or use the with statement for automatic closure
  • File modes matter: 'r' for reading, 'w' for writing (overwrites), 'a' for appending
  • Arrays (lists) combined with files allow efficient data processing: read → process → write
  • Random files provide direct access to records using pointers, unlike sequential access
  • Pseudocode for random files uses: OPENFILE, SEEK, GETRECORD, PUTRECORD, CLOSEFILE
  • Record structures define data organization and ensure consistent record sizes
  • Python's pickle module serializes objects to binary format for storage
  • Binary file modes ('rb', 'wb', 'r+b') are required when using pickle
  • Security warning: Never unpickle data from untrusted sources
  • Real-world applications: Student records, library systems, configuration files, data logging
  • Cambridge 9618 syllabus requires understanding of serial, sequential, and random file organization methods

Question Bank

Marking Scheme & Answer
  • [1 mark] To store data that needs to persist between runs of a program
  • [1 mark] To save output or logs for future reference
  • [1 mark] To enable data exchange between different programs or systems
  • [Additional] To store structured or unstructured data (configurations, user input, records)
Marking Scheme & Answer
# Writing scores to file
with open("scores.txt", "w") as file:
    file.write("85\n")
    file.write("92\n")
    file.write("78\n")

# Reading and printing scores
with open("scores.txt", "r") as file:
    for line in file:
        print(line.strip())
  • [1 mark] Correct file opening in write mode with with statement
  • [1 mark] Writing three scores with newline characters
  • [1 mark] Correct file opening in read mode
  • [1 mark] Reading and printing each line with strip()
Marking Scheme & Answer
Sequential Files:
  • Records stored one after another
  • Must be read from start to find a record
  • Efficient for processing all records
  • Example: Log files, transaction records
Random Files:
  • Direct access to any record
  • Use pointers/position to locate records
  • Efficient for accessing specific records
  • Example: Databases, indexed files
Key Difference: Sequential requires reading from start, random allows direct access to any position.
Marking Scheme & Answer
DECLARE ExistingBook : Book

// Open the file in RANDOM mode
OPENFILE "Library.dat" FOR RANDOM

// Move to position 5 and read the record
SEEK "Library.dat", 5
GETRECORD "Library.dat", ExistingBook

// Update the number of copies
ExistingBook.CopiesAvailable ← 20

// Write the updated record back to the same position
SEEK "Library.dat", 5
PUTRECORD "Library.dat", ExistingBook

// Close the file
CLOSEFILE "Library.dat"
  • [1 mark] Declaring variable of correct record type
  • [1 mark] Opening file in RANDOM mode
  • [1 mark] Using SEEK to move to position 5
  • [1 mark] GETRECORD to read and PUTRECORD to write
  • [1 mark] Updating field and closing file
Marking Scheme & Answer
  • [1 mark] pickle serializes Python objects into binary format for storage
  • [1 mark] Used with seek() to position the file pointer at specific locations
  • [1 mark] pickle.dump() writes objects to binary files at current pointer position
  • [1 mark] pickle.load() reads objects from binary files at current pointer position
  • [Additional] Files must be opened in binary mode ('rb', 'wb', 'r+b')
Marking Scheme & Answer
with open("data.txt", "r") as file:
    numbers = [int(line.strip()) for line in file]

max_value = max(numbers)
min_value = min(numbers)

with open("max.txt", "w") as file:
    file.write(f"Maximum: {max_value}\n")

with open("min.txt", "w") as file:
    file.write(f"Minimum: {min_value}\n")
  • [1 mark] Reading numbers from file into a list using list comprehension
  • [1 mark] Using max() and min() functions correctly
  • [1 mark] Writing maximum value to "max.txt" with descriptive text
  • [1 mark] Writing minimum value to "min.txt" with descriptive text
  • [1 mark] Using with statements for proper file handling
Marking Scheme & Answer
Serial Files:
  • Records stored consecutively
  • Must read from start
  • Example: System log files, transaction records
Sequential Files:
  • Records in order (e.g., alphabetical)
  • Can start reading from any point but sequentially
  • Example: Phone directory, dictionary
Random Files:
  • Direct access to any record
  • Use pointer/position to locate
  • Example: Database records, indexed files
Cambridge Syllabus: These methods are defined under File Processing in the 9618 syllabus.
Marking Scheme & Answer
DECLARE Pupil : Student
DECLARE Position : INTEGER

// Open the file in RANDOM mode
OPENFILE "StudentFile.dat" FOR RANDOM

// Shift records from position 20 to 10 (backward)
FOR Position ← 20 TO 10 STEP -1
    // Read the record at the current position
    SEEK "StudentFile.dat", Position
    GETRECORD "StudentFile.dat", Pupil
    
    // Write the record to the next position
    SEEK "StudentFile.dat", Position + 1
    PUTRECORD "StudentFile.dat", Pupil
NEXT Position

// Close the file
CLOSEFILE "StudentFile.dat"
  • [1 mark] Open file in RANDOM mode
  • [1 mark] Loop through positions in reverse (20 to 10)
  • [1 mark] Read record at current position and write to next position
  • [1 mark] Close file after operation
Marking Scheme & Answer
  • [1 mark] Online compilers often don't allow local file creation or access
  • [1 mark] IDEs let you create and manage files easily on your computer
  • [Additional] IDEs provide better debugging tools and file management features for file handling tasks
Marking Scheme & Answer
import pickle

class Book:
    def __init__(self, title, author, year_published, isbn, copies_available):
        self.title = title
        self.author = author
        self.year_published = year_published
        self.isbn = isbn
        self.copies_available = copies_available

def update_book(filename, position, new_copies):
    with open(filename, 'r+b') as file:
        # Read the record
        file.seek(position * 128)  # Assuming 128 bytes per record
        book = pickle.load(file)
        
        # Update the number of copies
        book.copies_available = new_copies
        
        # Write the updated record
        file.seek(position * 128)
        pickle.dump(book, file)

# Example usage
update_book("LibraryDatabase.dat", 5, 20)
  • [1 mark] Importing pickle module
  • [1 mark] Defining Book class with appropriate attributes
  • [1 mark] Opening file in 'r+b' mode (read/write binary)
  • [1 mark] Using seek() to position file pointer
  • [1 mark] Using pickle.load() to read and pickle.dump() to write
  • [1 mark] Correctly updating the copies_available attribute