Learning Objectives
By the end of this lesson, you will be able to:
- Understand why text files are used in programming
- Open, read, write, and close text files in Python using appropriate modes
- Use the
withstatement for automatic file handling - Combine arrays (lists) with text files to process data efficiently
- Understand the concept of random files and their advantages
- Perform file processing operations on serial, sequential, and random files
- Use pseudocode for random file operations (OPENFILE, SEEK, GETRECORD, PUTRECORD, CLOSEFILE)
- Use Python's
picklemodule for serializing and deserializing objects - Work with binary file modes for random file handling
- Implement complete file handling solutions for real-world scenarios
Key Terms
Text File
A file containing plain text data that can be read by humans and programs
File Mode
Specifies how a file is opened: read ('r'), write ('w'), append ('a'), or binary ('b')
Serial File
Records are stored one after another; must be read sequentially from start
Sequential File
Records stored in order (e.g., alphabetical); can be read from any point but sequentially
Random File
Allows direct access to any record using a pointer/position
Record Structure
A composite data type defining fields for storing related data
Pickle
Python module for serializing (pickling) and deserializing (unpickling) objects
Serialization
Converting a Python object into a binary format for storage (pickling)
Deserialization
Converting binary data back into a Python object (unpickling)
SEEK
Moves the file pointer to a specific position in a random file
Array (List)
Python data structure used to store and process collections of data
Binary Mode
File mode ('rb', 'wb', 'r+b') for handling non-text files like pickled objects
Text File Handling
Text files are used in high-level programming languages like Python to store data that needs to persist between program runs. They allow programs to save output, store structured or unstructured data, and facilitate data exchange between different programs or systems.
Real-Life Example: Student Records System
Think of a school management system that needs to:
- Save student names in a file for future reference
- Store exam results that can be analyzed later
- Log system activities for debugging purposes
- Export data to share with other systems
All these tasks use text file handling operations!
Why Use Text Files?
Persistence
Data remains after program ends
Data Exchange
Share data between different programs
Processing
Read, write, and update content easily
File Opening Modes
| Mode | Description | If File Exists | If File Doesn't Exist |
|---|---|---|---|
'r' (read) |
Open file for reading only | Opens successfully | Returns an error |
'w' (write) |
Open file for writing only | Overwrites the file | Creates a new file |
'a' (append) |
Open file for appending only | Adds to end of file | Creates a new file |
'r+' (read+) |
Open file for reading and writing | Opens successfully | Returns an error |
File Operations Simulation
How it works: This simulation shows how Python interacts with files. The file pointer moves as you read or write data. Notice how different modes affect the file content.
1. Opening and Closing Files
To work with a file, you need to open it first using the open() function and close it using close() method.
# Basic file opening and closing
file = open("example.txt", "w") # Open file in write mode
# Perform operations...
file.close() # Always close the file
2. Writing to a File
To write data into a file, open it in 'w' (write) mode. If the file doesn't exist, Python will create it.
# Task: Create a file called data.txt and write "Python is awesome!" into it
file = open("data.txt", "w")
file.write("Python is awesome!")
file.close()
3. Reading from a File
You can read content from a file using the read() or readline() method.
# Read the whole file
file = open("example.txt", "r")
content = file.read()
print(content)
file.close()
# Read line by line
file = open("example.txt", "r")
for line in file:
print(line)
file.close()
# Task: Read the content of data.txt and print it
file = open("data.txt", "r")
content = file.read()
print(content)
file.close()
4. Appending to a File
To add new content to the end of an existing file, open it in 'a' (append) mode.
# Task: Append "Python makes coding fun!" to data.txt
file = open("data.txt", "a") # Open in append mode
file.write("\nPython makes coding fun!") # Add new content
file.close()
5. Using the with Statement
The with statement is the best way to handle files because it automatically closes the file when you're done.
# Task: Use the with statement to read the content of data.txt
with open("data.txt", "r") as file:
content = file.read()
print(content)
# File automatically closes here
Combining Text Files with Arrays in Python
In Python, you can use arrays (lists) to manage and process data from text files efficiently. You can read data from text files into arrays, manipulate the data, and write it back to a file.
Reading Data into Arrays
# Read numbers from a file into an array
numbers = []
with open("numbers.txt", "r") as file:
for line in file:
numbers.append(int(line.strip()))
print(numbers) # Output: [10, 20, 30, 40, 50]
Writing Arrays to Files
# Write numbers from an array into a file
numbers = [10, 20, 30, 40, 50]
with open("output.txt", "w") as file:
for number in numbers:
file.write(f"{number}\n")
Activity 1: Student Records System
Problem: Write a program that:
- Creates a file called
students.txtand writes three student names into it - Reads and prints all the names
- Appends a new student name to the file
Use appropriate file handling techniques including the with statement.
Solution:
# Step 1: Write three names to the file
with open("students.txt", "w") as file:
file.write("Alice\n")
file.write("Bob\n")
file.write("Charlie\n")
# Step 2: Read and print all names
with open("students.txt", "r") as file:
for line in file:
print(line.strip())
# Step 3: Append a new name
with open("students.txt", "a") as file:
file.write("Diana\n")
Explanation: This program demonstrates all three basic file operations: writing, reading, and appending. The with statement ensures files are properly closed.
Activity 2: Processing Data with Arrays
Problem: Write a Python program that reads numbers from numbers.txt and:
- Creates two arrays: one for even numbers and one for odd numbers
- Saves the even numbers to
even_numbers.txt - Saves the odd numbers to
odd_numbers.txt
Sample numbers.txt content: 10, 3, 8, 5, 12 (each on new line)
Solution:
# Open the file and read numbers
file = open("numbers.txt", "r")
numbers = []
for line in file:
number = int(line.strip())
numbers.append(number)
file.close()
# Separate even and odd numbers
even_numbers = []
odd_numbers = []
for num in numbers:
if num % 2 == 0:
even_numbers.append(num)
else:
odd_numbers.append(num)
# Write even numbers to file
file = open("even_numbers.txt", "w")
for number in even_numbers:
file.write(str(number) + "\n")
file.close()
# Write odd numbers to file
file = open("odd_numbers.txt", "w")
for number in odd_numbers:
file.write(str(number) + "\n")
file.close()
Explanation: This shows how to read data from a file into an array, process it (separate even/odd), and write the results to different files.
Check Your Understanding: Text File Handling
1. Why are text files used in programming? [2 marks]
Answer
- [1 mark] To store data that needs to persist between runs of a program
- [1 mark] To save output or logs for future reference
- [Additional] To store structured/unstructured data and enable data exchange between programs
2. What happens when you open a file in 'w' mode if it already exists? [1 mark]
Answer
- [1 mark] The existing file is overwritten (all previous content is lost)
- [Additional] If the file doesn't exist, Python creates a new file
3. What is the advantage of using the with statement when working with files? [2 marks]
Answer
- [1 mark] It automatically closes the file when you're done
- [1 mark] It ensures proper resource management even if an error occurs
- [Additional] It makes code cleaner and reduces the risk of forgetting to close files
4. How do you read a file line by line in Python? [2 marks]
Answer
- [1 mark] Open the file in read mode using
open("filename.txt", "r") - [1 mark] Use a for loop to iterate through the file object:
for line in file: - [Additional] Alternatively, use
readline()orreadlines()methods
5. What is the difference between 'w' and 'a' file modes? [2 marks]
Answer
- [1 mark] 'w' (write) mode overwrites the entire file if it exists
- [1 mark] 'a' (append) mode adds new content to the end of the existing file
- [Additional] Both create a new file if it doesn't exist
6. How can arrays (lists) be combined with text files? [3 marks]
Answer
- [1 mark] Read data from text files into arrays using loops or list comprehensions
- [1 mark] Process the data in the array (filter, sort, calculate)
- [1 mark] Write the processed data back to text files
- [Additional] Examples: Separate even/odd numbers, filter words by length, combine data from multiple files
Random Files
Random files allow direct access to specific records by using a pointer to locate them. Each record is stored in a structured format and can be retrieved or updated without sequentially reading the entire file.
Direct Access
Access any record directly using its position without reading previous records
Binary Format
Data stored in binary format using Python's pickle module
Types of File Organisation
Serial Files
Records stored one after another; must be read sequentially from start
Log files, transaction records
Sequential Files
Records stored in order (e.g., alphabetical); can be read from any point but sequentially
Phone directories, dictionaries
Random Files
Allows direct access to any record using a pointer/position
Databases, indexed files
Real-Life Example: Library Management System
Problem
- Library needs to store thousands of book records
- Need to quickly find, update, or delete any book
- Don't want to read all records to find one book
- Need to maintain book availability counts
Random File Solution
- Each book stored at a specific position
- Use ISBN or book ID to calculate position
- Directly access any book in constant time
- Update availability without rewriting entire file
Random Files Using Pseudocode
Random File Record Visualization
How it works: Each record has a fixed position in the file. The SEEK command moves the pointer to a specific position, and GETRECORD/PUTRECORD read/write records at that position.
Pseudocode File Commands
OPENFILE
Opens the file in RANDOM mode
SEEK
Moves the pointer to a specific record position
GETRECORD
Reads a record from the file at current pointer
PUTRECORD
Writes a record to the file at current pointer
CLOSEFILE
Closes the file after use
Record Structure Example
// Student record structure
TYPE Student
DECLARE LastName : STRING
DECLARE FirstName : STRING
DECLARE DateOfBirth : DATE
DECLARE YearGroup : INTEGER
DECLARE FormGroup : CHAR
ENDTYPE
Adding a New Record
// Pseudocode for adding a new student record
DECLARE NewStudent : Student
// Assign values to the new student's fields
NewStudent.LastName ← "Doe"
NewStudent.FirstName ← "John"
NewStudent.DateOfBirth ← 15/03/2005
NewStudent.YearGroup ← 11
NewStudent.FormGroup ← 'B'
// Open the file in RANDOM mode
OPENFILE "StudentFile.dat" FOR RANDOM
// Move the pointer to position 5 and write the record
SEEK "StudentFile.dat", 5
PUTRECORD "StudentFile.dat", NewStudent
// Close the file
CLOSEFILE "StudentFile.dat"
Updating an Existing Record
// Pseudocode for updating a student record
DECLARE ExistingStudent : Student
// Open the file in RANDOM mode
OPENFILE "StudentFile.dat" FOR RANDOM
// Move to position 3 and read the record
SEEK "StudentFile.dat", 3
GETRECORD "StudentFile.dat", ExistingStudent
// Update the YearGroup of the student
ExistingStudent.YearGroup ← ExistingStudent.YearGroup + 1
// Write the updated record back to the same position
SEEK "StudentFile.dat", 3
PUTRECORD "StudentFile.dat", ExistingStudent
// Close the file
CLOSEFILE "StudentFile.dat"
Random Files Using Python (Pickle Module)
The pickle module in Python is used for serializing and deserializing Python objects. Serialization (or "pickling") converts a Python object into a binary format for storage. Deserialization (or "unpickling") converts binary data back into a Python object.
Pickle Methods
pickle.dump()
Saves (serializes) an object to a binary file
import pickle
data = {"name": "Alice", "age": 30}
with open("data.pkl", "wb") as file:
pickle.dump(data, file)
pickle.load()
Loads (deserializes) an object from a binary file
import pickle
with open("data.pkl", "rb") as file:
loaded_data = pickle.load(file)
print(loaded_data)
Binary File Modes
Write Binary
Open for writing in binary mode (used for pickling)
Read Binary
Open for reading in binary mode (used for unpickling)
Read/Write Binary
Open for reading and writing in binary mode (random access)
Complete Python Example
# Python example: Random file handling with pickle
import pickle
# Define a Book class
class Book:
def __init__(self, title, author, year_published, isbn, copies_available):
self.title = title
self.author = author
self.year_published = year_published
self.isbn = isbn
self.copies_available = copies_available
def __str__(self):
return f"Title: {self.title}, Author: {self.author}, Year: {self.year_published}, ISBN: {self.isbn}, Copies: {self.copies_available}"
# Create a book object
book = Book("To Kill a Mockingbird", "Harper Lee", 1960, "9780060935467", 5)
# Write the book to position 3 in the file
with open("LibraryDatabase.dat", "r+b") as file:
file.seek(3 * 128) # Move pointer to position 3 (assuming 128 bytes per record)
pickle.dump(book, file)
# Read the book back from position 3
with open("LibraryDatabase.dat", "rb") as file:
file.seek(3 * 128) # Move pointer to position 3
retrieved_book = pickle.load(file)
print(retrieved_book)
Important Notes About Pickle
- Security Warning: Never unpickle data from untrusted sources - it may execute malicious code
- Compatibility: Pickled files are Python-specific and might not work across different Python versions
- Fixed Record Size: In random files, ensure consistent record sizes for accurate positioning with
seek() - Use Cases: Save program settings, store intermediate data, save/load game progress
Activity 3: Library Record Structure
Task: A library uses a random file to store its book collection. Each book record includes the following fields:
- CopiesAvailable: INTEGER
- Title: STRING
- Author: STRING
- YearPublished: INTEGER
- ISBN: STRING
1. Create the record structure in pseudocode
2. Write pseudocode to add the book "Pride and Prejudice" by Jane Austen at position 7 with 12 copies available
Solution:
// 1. Record structure
TYPE Book
DECLARE CopiesAvailable : INTEGER
DECLARE Title : STRING
DECLARE Author : STRING
DECLARE YearPublished : INTEGER
DECLARE ISBN : STRING
ENDTYPE
// 2. Adding a new book
DECLARE NewBook : Book
// Assign values
NewBook.CopiesAvailable ← 12
NewBook.Title ← "Pride and Prejudice"
NewBook.Author ← "Jane Austen"
NewBook.YearPublished ← 1813
NewBook.ISBN ← "9780141199078"
// Open file and add record
OPENFILE "Library.dat" FOR RANDOM
SEEK "Library.dat", 7
PUTRECORD "Library.dat", NewBook
CLOSEFILE "Library.dat"
Activity 4: Pickle Module Practice
Task:
- Define a custom class
Productwith fields: name (string), price (float), stock (integer) - Create an instance with: name="Laptop", price=1500.00, stock=10
- Save this object to a file named
product.pkl - Load the object from
product.pkland print its details
Solution:
import pickle
# 1. Define Product class
class Product:
def __init__(self, name, price, stock):
self.name = name
self.price = price
self.stock = stock
def __str__(self):
return f"Product: {self.name}, Price: ${self.price}, Stock: {self.stock}"
# 2. Create instance
product = Product("Laptop", 1500.00, 10)
# 3. Save to file
with open("product.pkl", "wb") as file:
pickle.dump(product, file)
# 4. Load from file and print
with open("product.pkl", "rb") as file:
loaded_product = pickle.load(file)
print(loaded_product)
Explanation: This demonstrates how to use pickle to save and load custom class objects. Note the use of binary modes ('wb' for writing, 'rb' for reading).
Check Your Understanding: Random Files
1. What is the main advantage of random files over sequential files? [2 marks]
Answer
- [1 mark] Direct access to any record without reading previous records
- [1 mark] Faster retrieval and updating of specific records
- [Additional] More efficient for databases and systems requiring frequent random access
2. What does the SEEK command do in pseudocode for random files? [2 marks]
Answer
- [1 mark] Moves the file pointer to a specific record position
- [1 mark] Allows direct access to that record for reading or writing
- [Additional] Example:
SEEK "StudentFile.dat", 5moves to record position 5
3. What is serialization in the context of Python's pickle module? [2 marks]
Answer
- [1 mark] The process of converting a Python object into a binary format
- [1 mark] Also called "pickling" - allows objects to be stored in files or transmitted
- [Additional] The opposite process (converting binary back to object) is deserialization or unpickling
4. Why must files be opened in binary mode when using pickle? [2 marks]
Answer
- [1 mark]
picklestores data in binary format, not plain text - [1 mark] Binary mode ensures no character encoding issues occur during reading/writing
- [Additional] Modes: 'wb' for writing, 'rb' for reading, 'r+b' for reading and writing
5. What is a record structure and why is it important for random files? [3 marks]
Answer
- [1 mark] A composite data type defining fields for storing related data
- [1 mark] Provides a blueprint for how data is organized in the file
- [1 mark] Ensures consistent record size for accurate positioning with
SEEK - [Additional] Example: Student record with LastName, FirstName, YearGroup fields
6. What security concern exists when using the pickle module? [2 marks]
Answer
- [1 mark] Never unpickle data from untrusted sources
- [1 mark] Pickled data can execute arbitrary code during unpickling
- [Additional] Only unpickle data from trusted sources you created or verified
Key Takeaways
- Text files persist data between program runs and enable data exchange between systems
- Always close files after use, or use the
withstatement for automatic closure - File modes matter: 'r' for reading, 'w' for writing (overwrites), 'a' for appending
- Arrays (lists) combined with files allow efficient data processing: read → process → write
- Random files provide direct access to records using pointers, unlike sequential access
- Pseudocode for random files uses:
OPENFILE,SEEK,GETRECORD,PUTRECORD,CLOSEFILE - Record structures define data organization and ensure consistent record sizes
- Python's
picklemodule serializes objects to binary format for storage - Binary file modes ('rb', 'wb', 'r+b') are required when using
pickle - Security warning: Never unpickle data from untrusted sources
- Real-world applications: Student records, library systems, configuration files, data logging
- Cambridge 9618 syllabus requires understanding of serial, sequential, and random file organization methods
Question Bank
1. Explain three reasons why text files are used in programming. [3 marks]
Marking Scheme & Answer
- [1 mark] To store data that needs to persist between runs of a program
- [1 mark] To save output or logs for future reference
- [1 mark] To enable data exchange between different programs or systems
- [Additional] To store structured or unstructured data (configurations, user input, records)
2. Write Python code to create a file called "scores.txt", write three scores (85, 92, 78) each on a new line, then read and print them. [4 marks]
Marking Scheme & Answer
# Writing scores to file
with open("scores.txt", "w") as file:
file.write("85\n")
file.write("92\n")
file.write("78\n")
# Reading and printing scores
with open("scores.txt", "r") as file:
for line in file:
print(line.strip())
- [1 mark] Correct file opening in write mode with
withstatement - [1 mark] Writing three scores with newline characters
- [1 mark] Correct file opening in read mode
- [1 mark] Reading and printing each line with
strip()
3. Compare sequential and random file organization methods. [4 marks]
Marking Scheme & Answer
Sequential Files:
- Records stored one after another
- Must be read from start to find a record
- Efficient for processing all records
- Example: Log files, transaction records
Random Files:
- Direct access to any record
- Use pointers/position to locate records
- Efficient for accessing specific records
- Example: Databases, indexed files
4. Write pseudocode to update a book record at position 5 in a library file, changing the CopiesAvailable to 20. [5 marks]
Marking Scheme & Answer
DECLARE ExistingBook : Book
// Open the file in RANDOM mode
OPENFILE "Library.dat" FOR RANDOM
// Move to position 5 and read the record
SEEK "Library.dat", 5
GETRECORD "Library.dat", ExistingBook
// Update the number of copies
ExistingBook.CopiesAvailable ← 20
// Write the updated record back to the same position
SEEK "Library.dat", 5
PUTRECORD "Library.dat", ExistingBook
// Close the file
CLOSEFILE "Library.dat"
- [1 mark] Declaring variable of correct record type
- [1 mark] Opening file in RANDOM mode
- [1 mark] Using SEEK to move to position 5
- [1 mark] GETRECORD to read and PUTRECORD to write
- [1 mark] Updating field and closing file
5. Explain how Python's pickle module is used in random file handling. [4 marks]
Marking Scheme & Answer
- [1 mark]
pickleserializes Python objects into binary format for storage - [1 mark] Used with
seek()to position the file pointer at specific locations - [1 mark]
pickle.dump()writes objects to binary files at current pointer position - [1 mark]
pickle.load()reads objects from binary files at current pointer position - [Additional] Files must be opened in binary mode ('rb', 'wb', 'r+b')
6. Write a Python program that reads numbers from "data.txt", finds the maximum and minimum values, and writes them to "max.txt" and "min.txt". [5 marks]
Marking Scheme & Answer
with open("data.txt", "r") as file:
numbers = [int(line.strip()) for line in file]
max_value = max(numbers)
min_value = min(numbers)
with open("max.txt", "w") as file:
file.write(f"Maximum: {max_value}\n")
with open("min.txt", "w") as file:
file.write(f"Minimum: {min_value}\n")
- [1 mark] Reading numbers from file into a list using list comprehension
- [1 mark] Using
max()andmin()functions correctly - [1 mark] Writing maximum value to "max.txt" with descriptive text
- [1 mark] Writing minimum value to "min.txt" with descriptive text
- [1 mark] Using
withstatements for proper file handling
7. What are the three file organization methods mentioned in the Cambridge 9618 syllabus? Give an example use case for each. [6 marks]
Marking Scheme & Answer
Serial Files:
- Records stored consecutively
- Must read from start
- Example: System log files, transaction records
Sequential Files:
- Records in order (e.g., alphabetical)
- Can start reading from any point but sequentially
- Example: Phone directory, dictionary
Random Files:
- Direct access to any record
- Use pointer/position to locate
- Example: Database records, indexed files
8. Describe the process of moving records in a random file (e.g., from position 20 to 10). [4 marks]
Marking Scheme & Answer
DECLARE Pupil : Student
DECLARE Position : INTEGER
// Open the file in RANDOM mode
OPENFILE "StudentFile.dat" FOR RANDOM
// Shift records from position 20 to 10 (backward)
FOR Position ← 20 TO 10 STEP -1
// Read the record at the current position
SEEK "StudentFile.dat", Position
GETRECORD "StudentFile.dat", Pupil
// Write the record to the next position
SEEK "StudentFile.dat", Position + 1
PUTRECORD "StudentFile.dat", Pupil
NEXT Position
// Close the file
CLOSEFILE "StudentFile.dat"
- [1 mark] Open file in RANDOM mode
- [1 mark] Loop through positions in reverse (20 to 10)
- [1 mark] Read record at current position and write to next position
- [1 mark] Close file after operation
9. Why is it recommended to use an IDE like PyCharm or VS Code instead of online compilers for file handling? [2 marks]
Marking Scheme & Answer
- [1 mark] Online compilers often don't allow local file creation or access
- [1 mark] IDEs let you create and manage files easily on your computer
- [Additional] IDEs provide better debugging tools and file management features for file handling tasks
10. Write a Python function that uses pickle to update the number of copies available for a book at a specific position in a library file. [6 marks]
Marking Scheme & Answer
import pickle
class Book:
def __init__(self, title, author, year_published, isbn, copies_available):
self.title = title
self.author = author
self.year_published = year_published
self.isbn = isbn
self.copies_available = copies_available
def update_book(filename, position, new_copies):
with open(filename, 'r+b') as file:
# Read the record
file.seek(position * 128) # Assuming 128 bytes per record
book = pickle.load(file)
# Update the number of copies
book.copies_available = new_copies
# Write the updated record
file.seek(position * 128)
pickle.dump(book, file)
# Example usage
update_book("LibraryDatabase.dat", 5, 20)
- [1 mark] Importing pickle module
- [1 mark] Defining Book class with appropriate attributes
- [1 mark] Opening file in 'r+b' mode (read/write binary)
- [1 mark] Using
seek()to position file pointer - [1 mark] Using
pickle.load()to read andpickle.dump()to write - [1 mark] Correctly updating the copies_available attribute