# Python shutil Module: Complete Guide to File Operations

**URL:** https://www.compsmag.com/blogs/python-shutil/
**Author:** Sumit Chauhan
**Published:** 2026-07-20
**Updated:** 2026-07-20
**Categories:** Blogs
**Tags:** Guides, Python
**Reading Time:** 45 min

---

Python 3.6+

Intermediate

This guide teaches you everything about Python's `shutil` module from basic file copying to advanced directory management. Whether you are writing your first backup script or building a production deployment pipeline, you will learn how to handle files and directories safely, efficiently, and cross-platform. We will cover every function, compare shutil with similar modules, and build real-world tools you can use immediately.

Table of Contents

Step 1: What is shutil?

Step 2: Why Do You Need It?

Step 3: Should You Install It?

Step 4: System Requirements

Step 5: Installation & Setup

Step 6: Verification

Step 7: First Test

Step 8: Real-World Example

Step 9: Troubleshooting

Step 10: Common Mistakes

Step 11: Best Practices

Step 12: Security Considerations

Step 13: Performance Tips

Step 14: Advanced Usage

Step 15: Alternatives & Comparisons

FAQ

## Step 1: What is shutil?

`shutil` is a Python standard library module for high-level file and directory operations. The name stands for "shell utilities." It is built on top of lower-level modules such as `os` and `pathlib` and provides convenient, cross-platform functions for copying, moving, archiving, and deleting files and directories.

Think of `shutil` as the Python equivalent of common command-line tools like `cp`, `mv`, `rm -r`, and `tar` but wrapped in a way that works identically on Windows, macOS, and Linux. You do not need to write different code for different operating systems. One function call handles the complexity for you.

When you call `shutil.copy("file.txt", "backup/file.txt")`, Python handles the underlying system calls regardless of whether you are on Windows, macOS, or Linux. On Windows, it might use the Windows API copy function. On Linux, it might use the `sendfile` system call for zero-copy transfers. You do not need to know these details. That is the power of an abstraction layer.

Figure 1: shutil sits above os and pathlib, providing a higher-level interface to the operating system kernel.

### Core Functions at a Glance

Before diving deep, here is a quick reference of the most commonly used shutil functions. Bookmark this table. You will come back to it often.

| Function | Purpose | Preserves Metadata | Cross-Platform |
| --- | --- | --- | --- |
| `shutil.copy()` | Copy a file | Permissions only | Yes |
| `shutil.copy2()` | Copy a file with all metadata | Timestamps, permissions, attributes | Yes |
| `shutil.copyfile()` | Copy file contents only | None | Yes |
| `shutil.copytree()` | Copy an entire directory recursively | Depends on copy_function | Yes |
| `shutil.move()` | Move or rename files and directories | Preserved on same filesystem | Yes |
| `shutil.rmtree()` | Delete an entire directory recursively | N/A (deletion) | Yes |
| `shutil.disk_usage()` | Get disk usage statistics | N/A | Yes |
| `shutil.make_archive()` | Create ZIP, TAR, and other archives | N/A | Yes |
| `shutil.unpack_archive()` | Extract archives | N/A | Yes |
| `shutil.which()` | Find an executable in PATH | N/A | Yes |

Each of these functions is designed to handle edge cases that you might not think about at first. For example, `shutil.copy()` automatically creates the destination directory if it does not exist. `shutil.move()` detects whether the source and destination are on the same filesystem and uses the fastest method available a simple rename if they are on the same drive, or a copy-and-delete if they are on different drives.

## Step 2: Why Do You Need It?

Every developer eventually needs to manipulate files and directories programmatically. It is not a matter of if - it is a matter of when. Here are the most common scenarios where shutil becomes indispensable.

### 2.1 Back Up Data Before Deployment

Before deploying a new version of your application, you should back up the current deployment. If the new version breaks something, you can restore from the backup in minutes instead of hours. shutil makes this a single function call: `shutil.copytree("production_app", "production_app_backup_20260720")`.

### 2.2 Clean Up Temporary Files

Build systems, test suites, and web applications generate temporary files. Left unchecked, these files consume disk space and slow down your system. A scheduled script using `shutil.rmtree()` can clean up old cache directories, log files, and build artifacts automatically.

### 2.3 Move Applications Between Environments

DevOps pipelines often need to move application bundles from a staging server to a production server. shutil handles this cross-platform, so the same deployment script works on your Windows laptop, your Linux CI runner, and your macOS production machine.

### 2.4 Create Archives for Distribution

When you need to distribute your application or send a dataset to a colleague, `shutil.make_archive()` creates a ZIP or TAR file in one line. No need to learn the intricacies of the `zipfile` or `tarfile` modules unless you need advanced features.

### 2.5 Monitor and Manage Disk Space

`shutil.disk_usage("/")` returns the total, used, and free space on a drive. You can use this to trigger cleanup scripts when disk space drops below a threshold, preventing system crashes due to full disks.

Tip

shutil is not just for scripts. It is also commonly used inside web applications, data pipelines, DevOps automation, and desktop applications. Any time you need to manipulate the filesystem beyond simple reading and writing, shutil is the right tool. Even machine learning pipelines use shutil to organize training datasets and model checkpoints.

## Step 3: Should You Install It?

**No installation is required.** `shutil` is part of the Python standard library. It ships with every Python installation starting from Python 2.3 and is available immediately after you install Python on any operating system.

This is one of the biggest advantages of `shutil`: you do not need to worry about dependency management, virtual environments, or package conflicts. It is always there, ready to use. You do not need a `requirements.txt` entry. You do not need to pin versions. You do not need to worry about compatibility with other packages.

This makes shutil particularly valuable in constrained environments like Docker containers, embedded systems, CI/CD pipelines, and systems where you want to minimize external dependencies. If Python runs, shutil runs.

Important

Because shutil is part of the standard library, you should never try to install it with pip. Running `pip install shutil` will either fail or install an unrelated third-party package that happens to have the same name. Always use `import shutil` directly in your code.

## Step 4: System Requirements

shutil works on any system that runs Python. Because it is a pure Python module built on top of the standard library, it has minimal system requirements. Here is what you need:

| Requirement | Minimum | Recommended | Notes |
| --- | --- | --- | --- |
| Python Version | 3.6+ | 3.11+ | 3.8+ required for dirs_exist_ok |
| Operating System | Windows 7+, macOS 10.12+, Linux 3.x+ | Windows 10/11, macOS 13+, Ubuntu 22.04+ | All major platforms supported |
| Disk Space | None | None | Part of Python installation |
| Memory | Minimal | Depends on file sizes | Large copies may need RAM for buffering |
| Permissions | Read/Write to target directories | Same | May need admin for system directories |
| Network | None | None | Local filesystem operations only |

Some advanced features require newer Python versions. The `dirs_exist_ok` parameter for `copytree()`, which allows overwriting existing directories, was added in Python 3.8. If you are on an older version, you will need to manually remove the destination directory before copying. The `shutil.copy()` function's `follow_symlinks` parameter works on all supported versions.

For most use cases, any Python 3.6+ installation will handle everything you need. If you are starting a new project, use Python 3.11 or newer to take advantage of the latest optimizations and features.

## Step 5: Installation & Setup

Since shutil is built into Python, the only thing you need is Python itself. If you do not have Python installed, follow the steps for your operating system below. We will cover Windows, macOS, Linux, and Docker. Choose the method that matches your system.

### 5.1 Installing Python on Windows

**Step A:** Open your web browser and navigate to the official Python website at python.org/downloads. You will see a large yellow button that says "Download Python 3.x.x" where x.x is the latest stable release version number.

**Step B:** Click the download button. The installer file (typically named something like `python-3.11.4-amd64.exe`) will download to your Downloads folder.

**Step C:** Run the downloaded installer. On the first screen, at the bottom, you will see two important checkboxes. The second one says **"Add python.exe to PATH"**. You must check this box. It is not checked by default, and if you skip this step, you will not be able to run Python from the command line later. This is the most common mistake Windows users make when installing Python.

**Step D:** Click the "Install Now" button. The installer will copy Python files to your system, register Python file associations, and add Python to your system PATH. This takes about two to five minutes depending on your computer speed.

**Step E:** When the installation completes, you will see a screen that says "Setup was successful." Click "Close."

Warning

If you forget to check "Add python.exe to PATH" during installation, you will need to either reinstall Python (recommended) or manually add it to your system PATH environment variable. To verify PATH was set correctly, open Command Prompt after installation and type `python --version`. If you see a version number, PATH is correct.

### 5.2 Installing Python on macOS

macOS ships with an older version of Python (Python 2.7 on older systems), but you should install the latest Python 3 for modern development. There are two common methods.

**Method 1: Official Installer (Best for Beginners)**

Visit python.org/downloads/macos and download the macOS 64-bit universal2 installer. This installer works on both Intel and Apple Silicon Macs. Open the downloaded .pkg file and follow the installation wizard. The installer will automatically add Python to your PATH. After installation, open Terminal and verify by running `python3 --version`.

**Method 2: Homebrew (Best for Developers)**

Homebrew is a package manager for macOS. If you do not have it installed, open Terminal and run this command:

```
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```

This downloads and installs Homebrew. It will ask for your password because it needs to create directories in /usr/local. After Homebrew installs, install Python with:

```
brew install python
```

This installs the latest Python 3 and makes it available as both `python3` and `python` in your Terminal.

### 5.3 Installing Python on Linux (Ubuntu/Debian)

Most Linux distributions ship with Python pre-installed. To check if Python is installed and what version you have, open a terminal and run:

```
python3 --version
```

If you see output like `Python 3.10.12`, Python is installed. If the command is not found, or if you need a newer version, install it using your distribution's package manager.

On Ubuntu and Debian-based systems:

```
sudo apt update
sudo apt install python3 python3-pip python3-venv
```

On Fedora, RHEL, and CentOS:

```
sudo dnf install python3 python3-pip
```

On Arch Linux:

```
sudo pacman -S python python-pip
```

### 5.4 Using Python with Docker

If you prefer containerized environments or want to test shutil without installing Python on your host system, you can use Docker. First, install Docker from docker.com. Then run Python in a container:

```
docker run -it --rm python:3.11 python3 -c "import shutil; print(shutil.__file__)"
```

This command does several things. `docker run` starts a new container. `-it` makes it interactive. `--rm` removes the container when it exits. `python:3.11` is the official Python Docker image. The rest is the command to run inside the container: import shutil and print its file path. If you see a path like `/usr/local/lib/python3.11/shutil.py`, everything is working.

You can also start an interactive Python shell inside a container:

```
docker run -it --rm python:3.11 python3
```

This drops you into a Python REPL where you can experiment with shutil interactively.

## Step 6: Verification

After installing Python, you need to verify two things: that Python is installed correctly, and that shutil is available. This section walks you through both checks on every platform.

### 6.1 Open a Terminal

A terminal is a text-based interface where you type commands and the computer responds. Every operating system has one.

| Operating System | How to Open Terminal | Alternative Method |
| --- | --- | --- |
| Windows | Press Win + R, type cmd, press Enter | Right-click Start button, select "Terminal" or "Windows PowerShell" |
| macOS | Press Cmd + Space, type Terminal, press Enter | Open Applications > Utilities > Terminal |
| Linux | Press Ctrl + Alt + T | Search for "Terminal" in applications menu |

### 6.2 Verify Python Installation

Type the following command and press Enter:

```
python --version
```

On Linux and macOS, the command might be `python3` instead of `python`. This is because some systems reserve the `python` command for Python 2 (which is obsolete). If the first command fails with "command not found," try:

```
python3 --version
```

You should see output similar to this:

```
Python 3.11.4
```

The exact version number will vary depending on when you installed Python. Any version 3.6 or higher will work for everything in this guide.

### 6.3 Verify shutil is Available

Start the Python interactive shell by typing:

```
python
```

Or on Linux and macOS:

```
python3
```

You will see the Python version and a prompt that looks like `>>>`. This is the Python REPL (Read-Eval-Print Loop), where you can type Python code and see results immediately.

At the prompt, type these two lines:

```
>>> import shutil
>>> print(shutil.__file__)
```

Press Enter after each line. You should see output like:

```
/usr/lib/python3.11/shutil.py
```

The exact path will vary depending on your operating system and Python installation location. On Windows, it might look like `C:\Users\YourName\AppData\Local\Programs\Python\Python311\lib\shutil.py`. On macOS with Homebrew, it might be `/opt/homebrew/Cellar/python@3.11/3.11.4/Frameworks/Python.framework/Versions/3.11/lib/python3.11/shutil.py`. The important thing is that you see a path. That means shutil is installed and ready.

To see all available functions in shutil, type:

```
>>> dir(shutil)
```

This prints a list of all names defined in the shutil module. To read the documentation for a specific function, use the built-in help system:

```
>>> help(shutil.copy)
```

Press `q` to exit the help viewer. To exit the Python shell, type `exit()` or press Ctrl+D (Linux/macOS) or Ctrl+Z then Enter (Windows).

Tip

The Python interactive shell is your best friend when learning. You can experiment with shutil functions, see what they return, and understand how they work before writing full scripts. Make it a habit to test new functions in the REPL first.

## Step 7: First Test

Now it is time to write your first shutil script. This exercise will create a test file, copy it, move it, create a directory, check disk usage, and clean up. Follow along on your own computer. Do not just read instead type the commands and run the code. That is how you learn.

### 7.1 Create a Working Directory

Open your terminal and create a folder for this tutorial. Keeping everything in one folder makes cleanup easy.

On Linux and macOS:

```
mkdir ~/shutil-tutorial
cd ~/shutil-tutorial
```

On Windows Command Prompt:

```
mkdir %USERPROFILE%\shutil-tutorial
cd %USERPROFILE%\shutil-tutorial
```

On Windows PowerShell:

```
New-Item -ItemType Directory -Path "$env:USERPROFILE\shutil-tutorial"
Set-Location "$env:USERPROFILE\shutil-tutorial"
```

### 7.2 Create a Test File

Create a simple text file named hello.txt. This is the file we will copy and move.

On Linux and macOS:

```
echo "Hello, shutil!" > hello.txt
```

On Windows Command Prompt:

```
echo Hello, shutil! > hello.txt
```

On Windows PowerShell:

```
"Hello, shutil!" | Out-File hello.txt
```

### 7.3 Write Your First shutil Script

Create a file named first_test.py using any text editor. Notepad works on Windows. TextEdit works on macOS (set it to plain text mode). Nano or Vim work on Linux. VS Code is excellent on all platforms. Paste the following code and save the file in your shutil-tutorial directory.

```
# first_test.py - Your first shutil script
# This script demonstrates basic shutil operations

import shutil
import os

# Step 1: Copy the file
# shutil.copy(src, dst) copies the file at 'src' to 'dst'
# If dst is a directory, the file is copied into it with the same name
shutil.copy("hello.txt", "hello_copy.txt")
print("Step 1: Copied hello.txt to hello_copy.txt")

# Step 2: Move (rename) the copied file
# shutil.move(src, dst) moves 'src' to 'dst'
# This is like renaming a file or moving it to a different folder
shutil.move("hello_copy.txt", "hello_moved.txt")
print("Step 2: Moved hello_copy.txt to hello_moved.txt")

# Step 3: Create a backup directory
# shutil does NOT have a function to create a single directory
# We use os.makedirs() which creates parent directories too
# exist_ok=True prevents an error if the directory already exists
os.makedirs("backup", exist_ok=True)
print("Step 3: Created backup/ directory")

# Step 4: Copy the file into the backup directory
# When dst is a directory, shutil.copy puts the file inside it
shutil.copy("hello.txt", "backup/hello.txt")
print("Step 4: Copied hello.txt into backup/")

# Step 5: Check disk usage of the current directory
# shutil.disk_usage(path) returns a named tuple: (total, used, free)
# All values are in bytes, so we divide by 1024**3 to get gigabytes
usage = shutil.disk_usage(".")
print(f"Step 5: Disk usage: {usage.used / (1024**3):.2f} GB used, {usage.free / (1024**3):.2f} GB free")

# Step 6: Clean up - remove the moved file
# os.remove() deletes a single file
# We use os.remove here because shutil does not have a single-file delete
os.remove("hello_moved.txt")
print("Step 6: Removed hello_moved.txt")

# Step 7: List what remains in our directory
print("\nFinal directory contents:")
for item in os.listdir("."):
    print(f"  {item}")
```

### 7.4 Run the Script

Save the file and run it from your terminal. Make sure you are still in the shutil-tutorial directory.

```
python first_test.py
```

On Linux and macOS, if the above fails:

```
python3 first_test.py
```

Expected output:

```
Step 1: Copied hello.txt to hello_copy.txt
Step 2: Moved hello_copy.txt to hello_moved.txt
Step 3: Created backup/ directory
Step 4: Copied hello.txt into backup/
Step 5: Disk usage: 245.67 GB used, 512.34 GB free
Step 6: Removed hello_moved.txt

Final directory contents:
  hello.txt
  backup
  first_test.py
```

Congratulations! You have successfully used shutil to copy, move, and manage files. The hello_copy.txt file was created, then renamed to hello_moved.txt, then deleted. The original hello.txt remains, and a copy exists inside the backup directory. You also checked how much disk space is available on your drive.

Tip

Notice that shutil.copy() and shutil.move() do not print anything when they succeed. They follow the Unix philosophy: silence means success. That is why we added print() statements after each operation, so we can see what happened. In production code, you would use the logging module instead of print().

## Step 8: Real-World Example

Now that you understand the basics, let us build something you can actually use: a production-ready project backup script. This script will back up a directory, compress it into a ZIP archive, rotate old backups to save disk space, and handle errors gracefully. This is the kind of tool you might run every night via a cron job or scheduled task.

Figure 2: A complete backup workflow using shutil and related standard library modules.

### 8.1 The Complete Backup Script

Create a file named backup_project.py in your shutil-tutorial directory. This script is production-ready: it validates inputs, checks disk space, ignores unnecessary files, creates a timestamped ZIP archive, cleans up temporary files, and rotates old backups.

```
#!/usr/bin/env python3
# backup_project.py - A production-ready backup script using shutil
# Usage: python backup_project.py

import shutil
import os
import sys
import logging
from datetime import datetime
from pathlib import Path

# Configure logging so we can track what happens
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('backup.log'),
        logging.StreamHandler()
    ]
)
logger = logging.getLogger(__name__)

def backup_project(source_dir, backup_dir, max_backups=5):
    """
    Backs up a project directory by copying it and creating a ZIP archive.

    This function performs the following steps:
    1. Validates that the source directory exists
    2. Calculates the size of the source directory
    3. Checks that enough disk space is available
    4. Creates the backup directory if it does not exist
    5. Generates a timestamped backup name
    6. Copies the entire source directory (skipping junk files)
    7. Creates a ZIP archive from the copied directory
    8. Removes the temporary copied directory
    9. Rotates old backups, keeping only the most recent N

    Args:
        source_dir: Path to the project directory to back up.
        backup_dir: Path where backups will be stored.
        max_backups: Maximum number of backups to keep (oldest are deleted).

    Returns:
        Path to the created ZIP archive as a string.
    """

    # --- Step 1: Validate inputs ---
    source_path = Path(source_dir).resolve()
    backup_path = Path(backup_dir).resolve()

    logger.info(f"Starting backup of {source_path}")

    if not source_path.exists():
        raise FileNotFoundError(f"Source directory not found: {source_path}")

    if not source_path.is_dir():
        raise NotADirectoryError(f"Source path is not a directory: {source_path}")

    # --- Step 2: Check available disk space ---
    # Calculate total size of all files in source directory
    source_size = sum(
        f.stat().st_size for f in source_path.rglob('*') if f.is_file()
    )

    # Get free disk space on the backup drive
    disk_usage = shutil.disk_usage(backup_path)

    # Require at least 2x the source size for safety margin
    # We need space for the copy AND the ZIP archive
    required_space = source_size * 2

    if disk_usage.free < required_space:
        raise OSError(
            f"Insufficient disk space. Need {required_space / (1024**2):.1f} MB, "
            f"have {disk_usage.free / (1024**2):.1f} MB free."
        )

    logger.info(f"Source size: {source_size / (1024**2):.1f} MB")
    logger.info(f"Free space: {disk_usage.free / (1024**2):.1f} MB")

    # --- Step 3: Create backup directory ---
    # parents=True creates parent directories if they don't exist
    # exist_ok=True prevents error if directory already exists
    backup_path.mkdir(parents=True, exist_ok=True)

    # --- Step 4: Generate timestamped backup name ---
    # datetime.now() gets current date and time
    # strftime formats it as YYYYMMDD_HHMMSS
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
    project_name = source_path.name
    backup_name = f"{project_name}_backup_{timestamp}"
    temp_backup_dir = backup_path / backup_name

    logger.info(f"Creating backup: {backup_name}")

    # --- Step 5: Copy the entire directory ---
    # shutil.copytree() copies a directory and all its contents recursively
    # ignore_patterns skips files matching the given patterns
    # This keeps backups small by excluding cache and build files
    shutil.copytree(
        source_path,
        temp_backup_dir,
        ignore=shutil.ignore_patterns(
            '__pycache__', '*.pyc', '*.pyo',
            '.git', '.gitignore', '.svn',
            'node_modules', 'vendor',
            '.env', '.venv', 'venv',
            '*.log', '*.tmp', '*.swp',
            '.DS_Store', 'Thumbs.db',
            'build', 'dist', '*.egg-info'
        ),
        dirs_exist_ok=True
    )

    logger.info(f"Copied {source_path} to {temp_backup_dir}")

    # --- Step 6: Create ZIP archive ---
    # shutil.make_archive(base_name, format, root_dir)
    # Creates a ZIP file from the copied directory
    # The archive will be named: backup_name.zip
    archive_path = backup_path / backup_name
    shutil.make_archive(
        base_name=str(archive_path),
        format='zip',
        root_dir=temp_backup_dir
    )

    logger.info(f"Created archive: {archive_path}.zip")

    # --- Step 7: Remove the temporary copied directory ---
    # shutil.rmtree() deletes a directory and all contents recursively
    # Be careful with this function - it permanently deletes files
    shutil.rmtree(temp_backup_dir)
    logger.info(f"Cleaned up temporary directory")

    # --- Step 8: Rotate old backups ---
    # List all ZIP files in backup directory
    # Sort by modification time (newest first)
    zip_files = sorted(
        backup_path.glob("*.zip"),
        key=lambda f: f.stat().st_mtime,
        reverse=True
    )

    # Delete backups beyond max_backups limit
    if len(zip_files) > max_backups:
        old_backups = zip_files[max_backups:]
        for old in old_backups:
            old.unlink()  # Path.unlink() deletes a file
            logger.info(f"Deleted old backup: {old.name}")

    logger.info(f"Keeping {min(len(zip_files), max_backups)} most recent backups")

    return str(archive_path) + ".zip"


if __name__ == "__main__":
    # Example usage - modify these paths for your project
    try:
        result = backup_project(
            source_dir="./my_project",
            backup_dir="./backups",
            max_backups=5
        )
        logger.info(f"Backup complete: {result}")
        print(f"\nSuccess! Backup created at: {result}")
    except Exception as e:
        logger.error(f"Backup failed: {e}", exc_info=True)
        print(f"\nError: {e}", file=sys.stderr)
        sys.exit(1)
```

Best Practice

Always validate inputs, check disk space, and use `ignore_patterns` to skip unnecessary files (like `__pycache__`, `.git`, and `node_modules`). This keeps your backups small and fast. The `dirs_exist_ok=True` parameter (Python 3.8+) allows `copytree()` to overwrite existing directories, which is useful for incremental backups. Using the logging module instead of print() gives you timestamped, level-filtered output that can be saved to files for audit trails.

### 8.2 Running the Script

First, create a sample project to back up. This simulates a real project with source files, documentation, and cache directories.

```
mkdir my_project
cd my_project
echo "print('Hello World')" > main.py
echo "# My Project" > README.md
mkdir src
echo "# Source code" > src/__init__.py
echo "def helper(): pass" > src/utils.py
mkdir __pycache__
echo "cached" > __pycache__/cache.pyc
echo "cached" > __pycache__/module.pyc
mkdir .git
echo "ref: refs/heads/main" > .git/HEAD
cd ..
```

Now run the backup script:

```
python backup_project.py
```

Expected output:

```
2026-07-20 14:30:52,341 - INFO - Starting backup of /home/user/shutil-tutorial/my_project
2026-07-20 14:30:52,342 - INFO - Source size: 0.0 MB
2026-07-20 14:30:52,343 - INFO - Free space: 512345.6 MB
2026-07-20 14:30:52,344 - INFO - Creating backup: my_project_backup_20260720_143052
2026-07-20 14:30:52,350 - INFO - Copied /home/user/shutil-tutorial/my_project to /home/user/shutil-tutorial/backups/my_project_backup_20260720_143052
2026-07-20 14:30:52,380 - INFO - Created archive: /home/user/shutil-tutorial/backups/my_project_backup_20260720_143052.zip
2026-07-20 14:30:52,381 - INFO - Cleaned up temporary directory
2026-07-20 14:30:52,382 - INFO - Keeping 1 most recent backups
2026-07-20 14:30:52,383 - INFO - Backup complete: /home/user/shutil-tutorial/backups/my_project_backup_20260720_143052.zip

Success! Backup created at: /home/user/shutil-tutorial/backups/my_project_backup_20260720_143052.zip
```

Notice that the __pycache__ directory and .git directory were not included in the backup. The ignore_patterns function filtered them out. This keeps the backup small and avoids including files that can be regenerated (cache) or are managed elsewhere (version control).

## Step 9: Troubleshooting

Even with a well-designed module like shutil, things can go wrong. Files disappear. Permissions change. Disks fill up. Here is how to diagnose and fix the most common issues you will encounter.

| Error / Symptom | Root Cause | Solution |
| --- | --- | --- |
| `FileNotFoundError` | The source file or directory does not exist, or the path is incorrect. | Verify the path with `os.path.exists()` or `Path.exists()`. Use absolute paths to avoid working-directory confusion. |
| `PermissionError` | Insufficient permissions to read the source or write to the destination. | Run with elevated permissions (admin/sudo) if necessary. Check file ownership with `ls -la` (Linux/macOS) or file properties (Windows). |
| `shutil.SameFileError` | Source and destination are the same file. | Use `os.path.abspath()` to resolve paths before comparing. Add a guard: `if os.path.samefile(src, dst): return`. |
| `shutil.Error` with copytree | One or more files failed to copy inside a directory tree. | Wrap copytree in try/except and log the specific file that failed. Check for permission issues on individual files. |
| `FileExistsError` with copytree | The destination directory already exists (Python < 3.8). | Upgrade to Python 3.8+ and use `dirs_exist_ok=True`. On older versions, delete the destination first or use a different path. |
| `OSError: No space left on device` | Not enough free disk space for the operation. | Use `shutil.disk_usage()` before large operations. Always leave a safety margin (2x the source size). |
| Symlinks not copied correctly | shutil follows symlinks by default, copying the target file instead of the link. | Use `follow_symlinks=False` in copy/copy2/copytree to preserve symlinks as links. |
| Special files skipped | shutil ignores special files (devices, sockets, FIFOs) by default for safety. | Use a custom copy function with `copy_function` parameter, or handle special files separately with os-level operations. |
| Windows path issues | Backslashes in paths cause escape sequence problems in strings. | Use raw strings (`r"C:\path"`), forward slashes (`"C:/path"`), or pathlib which handles this automatically. |
| Long path errors on Windows | Windows has a 260-character path limit by default. | Enable long path support in Windows 10/11 Group Policy, or use the `\\\\?\\` prefix for absolute paths. |

### 9.1 Debugging shutil Operations

When a shutil operation fails, wrap it in a try/except block to capture detailed error information. The shutil.Error exception is special and it contains a list of (src, dst, error) tuples, one for each file that failed:

```
# debugging_shutil.py - How to debug shutil errors properly
import shutil
import traceback

try:
    shutil.copytree("source_dir", "dest_dir")
except shutil.Error as e:
    # shutil.Error.args[0] is a list of (src, dst, error) tuples
    print(f"Copytree failed with {len(e.args[0])} errors:")
    for src, dst, error in e.args[0]:
        print(f"  Failed to copy {src} to {dst}: {error}")
except PermissionError as e:
    print(f"Permission denied: {e}")
    print("Try running with administrator/sudo privileges.")
except Exception as e:
    print(f"Unexpected error: {e}")
    traceback.print_exc()
```

Developer Note

The `shutil.Error` exception is unique because its `args[0]` contains a list of tuples, each representing one failed file operation. This design allows copytree() to continue copying other files even when some fail, and then report all failures at the end. This is much better than failing on the first error and leaving you guessing about how many files were affected.

## Step 10: Common Mistakes

Here are the most frequent mistakes developers make when using shutil, and how to avoid them. Learning from these mistakes now will save you hours of debugging later.

### Mistake 1: Confusing copy() with copy2()

**The Problem:** Using copy() when you need to preserve timestamps and other metadata.

```
# WRONG: This does NOT preserve creation/modification timestamps
# The copied file will have a new timestamp (now)
shutil.copy("important.doc", "backup/important.doc")
```

**The Fix:** Use copy2() when metadata matters. This preserves timestamps, permissions, and extended attributes.

```
# RIGHT: This preserves ALL metadata including timestamps
# The copied file will have the same timestamp as the original
shutil.copy2("important.doc", "backup/important.doc")
```

### Mistake 2: Forgetting That rmtree() Is Permanent

**The Problem:** Calling rmtree() on a path without validation. This permanently deletes files. There is no Recycle Bin. There is no undo.

```
# DANGEROUS: This permanently deletes everything in the path
# If path is "/", you could destroy your system
shutil.rmtree("/")  # NEVER DO THIS
```

**The Fix:** Always validate the path and add safety checks against system directories.

```
import os

def safe_rmtree(path):
    """Safely remove a directory tree with validation."""
    abs_path = os.path.abspath(path)

    # Prevent deleting critical system directories
    forbidden = [
        "/", "/usr", "/bin", "/sbin",
        "/lib", "/lib64", "/etc", "/var", "/opt", "/boot",
        "C:\\", "C:\\Windows", "C:\\Program Files",
        "C:\\ProgramData", "C:\\Users", "C:\\System32",
        os.path.expanduser("~")
    ]

    if abs_path in forbidden:
        raise ValueError(f"Refusing to delete protected path: {abs_path}")

    if not os.path.exists(abs_path):
        print(f"Path does not exist: {abs_path}")
        return

    shutil.rmtree(abs_path)
    print(f"Deleted: {abs_path}")
```

### Mistake 3: Not Handling Existing Directories with copytree()

**The Problem:** Calling copytree() without handling the case where the destination already exists.

```
# WRONG: This fails if 'backup' already exists
# FileExistsError: [Errno 17] File exists: 'backup'
shutil.copytree("project", "backup")
```

**The Fix:** Use dirs_exist_ok=True (Python 3.8+) or check and remove first.

```
# RIGHT for Python 3.8+: Overwrites existing directory
shutil.copytree("project", "backup", dirs_exist_ok=True)

# RIGHT for older Python: Remove destination first
if os.path.exists("backup"):
    shutil.rmtree("backup")
shutil.copytree("project", "backup")
```

### Mistake 4: Ignoring Return Values from disk_usage()

**The Problem:** Not checking disk space before large copy operations. Your script might crash halfway through, leaving partial files and a corrupted destination.

```
# WRONG: Risky - could run out of disk space mid-operation
# Leaving partial/corrupted files in the destination
shutil.copytree("huge_project", "backup/huge_project")
```

**The Fix:** Always check available space first with a safety margin.

```
# RIGHT: Check space before copying
usage = shutil.disk_usage(".")
source_size = sum(
    os.path.getsize(f) for f in Path("huge_project").rglob("*")
    if f.is_file()
)

# Require 2x space: one for the copy, one for safety margin
if usage.free < source_size * 2:
    raise OSError("Not enough disk space for backup")

shutil.copytree("huge_project", "backup/huge_project")
```

### Mistake 5: Assuming move() Is Always Fast

**The Problem:** Assuming shutil.move() is always an instant rename operation. When source and destination are on different filesystems (different drives or partitions), move() performs a full copy followed by a delete. This can take minutes for large files and temporarily uses double the disk space.

```
# WRONG ASSUMPTION: This might copy then delete if on different drives
# If /home is on a different partition than /mnt, this is SLOW
shutil.move("/home/user/file.txt", "/mnt/external/file.txt")
```

**The Fix:** Check if source and destination are on the same filesystem before moving.

```
import os

src = "/home/user/file.txt"
dst = "/mnt/external/file.txt"

# Check if on same filesystem by comparing device IDs
src_dev = os.stat(src).st_dev
dst_dev = os.stat(os.path.dirname(dst)).st_dev

if src_dev != dst_dev:
    print("Warning: Cross-filesystem move will copy then delete")
    file_size = os.path.getsize(src)
    dst_usage = shutil.disk_usage(os.path.dirname(dst))
    if dst_usage.free < file_size:
        raise OSError("Not enough space on destination filesystem")

shutil.move(src, dst)
```

## Step 11: Best Practices

Follow these guidelines to write robust, maintainable, and safe shutil code. These practices come from years of production experience and will save you from common pitfalls.

### 11.1 Always Use Absolute Paths

Relative paths depend on the current working directory, which can change unexpectedly when your script is run from different locations or via cron jobs. Always resolve paths to absolute form before performing operations.

```
from pathlib import Path

# Good: Resolve to absolute path immediately
src = Path("data/file.txt").resolve()
dst = Path("backup/file.txt").resolve()

# Now src and dst are absolute, regardless of working directory
print(f"Source: {src}")
print(f"Destination: {dst}")

shutil.copy(src, dst)
```

### 11.2 Combine shutil with pathlib

pathlib provides object-oriented path manipulation that is cleaner than string concatenation. Use it for path construction and validation, then pass the resulting Path objects to shutil functions. This is the modern Python way.

```
from pathlib import Path
import shutil
from datetime import datetime

base_dir = Path.home() / "projects"
source = base_dir / "app"
backup = base_dir / "backups" / datetime.now().strftime("%Y%m%d")

# pathlib handles directory creation
backup.mkdir(parents=True, exist_ok=True)

# shutil handles the copy
shutil.copytree(source, backup / source.name, dirs_exist_ok=True)
```

### 11.3 Use ignore_patterns to Skip Junk Files

When copying projects, always skip cache files, build artifacts, and version control directories. This keeps your backups small, fast, and free of files that can be regenerated.

```
shutil.copytree(
    "my_project",
    "my_project_backup",
    ignore=shutil.ignore_patterns(
        '__pycache__', '*.pyc', '*.pyo',  # Python cache
        '.git', '.gitignore', '.svn',    # Version control
        'node_modules', 'vendor',        # Dependencies
        '.env', '.venv', 'venv',      # Virtual environments
        '*.log', '*.tmp', '*.swp',   # Temporary files
        '.DS_Store', 'Thumbs.db',       # OS metadata
        'build', 'dist', '*.egg-info' # Build artifacts
    )
)
```

### 11.4 Log All File Operations

Use Python's built-in logging module instead of print() for production code. Logging gives you timestamps, severity levels, and the ability to write to files simultaneously with console output.

```
import logging
import shutil

# Configure logging with both file and console output
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('backup.log'),    # Write to file
        logging.StreamHandler()                  # Also print to console
    ]
)
logger = logging.getLogger(__name__)

# Use logger instead of print throughout your code
logger.info("Starting backup of %s", source_dir)
try:
    shutil.copytree(source_dir, dest_dir)
    logger.info("Backup completed successfully")
except Exception as e:
    logger.error("Backup failed: %s", e, exc_info=True)
```

### 11.5 Validate Before You Act

Always verify that source paths exist and destination paths are safe before performing destructive operations. This prevents half-completed operations and data loss.

```
def validate_backup_operation(source, destination):
    """Validate paths before performing backup operations."""
    source_path = Path(source).resolve()
    dest_path = Path(destination).resolve()

    if not source_path.exists():
        raise FileNotFoundError(f"Source does not exist: {source_path}")

    if not source_path.is_dir():
        raise NotADirectoryError(f"Source is not a directory: {source_path}")

    # Prevent overwriting the source with itself
    if dest_path.exists() and dest_path.samefile(source_path):
        raise ValueError("Source and destination cannot be the same path")

    return source_path, dest_path
```

Best Practice Summary

- Always use absolute paths with `Path.resolve()`
- Combine pathlib with shutil for modern, readable code
- Skip unnecessary files with `ignore_patterns()` to keep operations fast
- Use the logging module, not print(), in production code
- Validate all inputs before performing operations
- Check disk space before large copy operations
- Handle exceptions gracefully with try/except blocks
- Never use user input directly as file paths without sanitization

## Step 12: Security Considerations

File operations are inherently risky. A single bug can delete critical data, expose sensitive information, or allow an attacker to read files they should not access. Here is how to keep your shutil code secure.

### 12.1 Path Traversal Protection

Never use user-provided paths directly. An attacker could provide a path like `../../../etc/passwd` to access files outside the intended directory. Always sanitize and validate paths.

```
from pathlib import Path

def safe_path(base_dir, user_input):
    """
    Ensure user_input stays within base_dir.
    Prevents path traversal attacks like ../../../etc/passwd
    """
    base = Path(base_dir).resolve()
    # resolve() follows symlinks and makes the path absolute
    target = (base / user_input).resolve()

    # target must be inside base or equal to base
    # str.startswith() checks this relationship
    if not str(target).startswith(str(base)):
        raise ValueError(f"Path traversal detected: {user_input}")

    return target

# Safe usage in a web application
user_file = safe_path("/var/app/uploads", request.form["filename"])
shutil.copy(user_file, "/var/app/processed")
```

### 12.2 Protect Against Accidental System Deletion

When writing scripts that delete directories, add hard guards against system paths. A typo in a path could destroy your operating system.

```
import os
import shutil

PROTECTED_PATHS = {
    "/", "/bin", "/sbin", "/usr", "/lib", "/lib64",
    "/etc", "/var", "/opt", "/boot", "/dev",
    "C:\\", "C:\\Windows", "C:\\Program Files",
    "C:\\ProgramData", "C:\\Users", "C:\\System32",
    os.path.expanduser("~")
}

def safe_rmtree(path):
    """Delete a directory tree with system path protection."""
    abs_path = os.path.abspath(path)

    if abs_path in PROTECTED_PATHS:
        raise PermissionError(
            f"Refusing to delete protected path: {abs_path}"
        )

    for protected in PROTECTED_PATHS:
        if abs_path.startswith(protected + os.sep):
            raise PermissionError(
                f"Refusing to delete path inside protected directory: {abs_path}"
            )

    shutil.rmtree(abs_path)
    print(f"Deleted: {abs_path}")
```

### 12.3 Handle Symlinks Carefully

Symlinks can be exploited to read or write files outside the intended directory. By default, shutil follows symlinks, which means it copies the target file, not the link itself. To preserve symlinks as links and prevent following malicious links:

```
# Preserve symlinks as links, do not follow them
# This prevents symlink-based path traversal attacks
shutil.copytree(
    "source_dir",
    "dest_dir",
    symlinks=True,           # Copy symlink as symlink, not target
    ignore_dangling_symlinks=True  # Skip broken symlinks safely
)
```

Security Note

When processing files from untrusted sources (user uploads, downloaded archives), always validate paths, check file sizes, and scan for malicious content. shutil does not validate file contents - it only manipulates files. Never run `shutil.rmtree()` on paths derived directly from user input without thorough sanitization. A path like `../../../` could delete your entire system if passed to rmtree().

## Step 13: Performance Tips

shutil is already optimized, but you can make your file operations faster and more efficient by understanding how it works under the hood and applying these techniques.

| Technique | When to Use | Performance Impact |
| --- | --- | --- |
| `copyfile()` instead of `copy()` | When you only need contents, not permissions | Slightly faster, fewer system calls |
| `copytree()` with `ignore_patterns` | Copying projects with many cache/build files | Dramatically faster, smaller output |
| Same-filesystem `move()` | Moving files within the same drive | Instant (rename only), no data copy |
| Batch operations | Processing many small files | Reduce overhead by grouping operations |
| Memory-mapped files | Copying very large files | Better I/O performance for multi-GB files |
| Multiprocessing | Copying many large directories | Parallel I/O on multi-core systems |

### 13.1 Fast Same-Filesystem Moves

When source and destination are on the same filesystem, shutil.move() performs a simple rename operation at the filesystem level. No data is copied. This is instant even for multi-terabyte directories because only the directory entry is updated.

```
import os

src = "/home/user/projects/old_name"
dst = "/home/user/projects/new_name"

# Check if on same filesystem by comparing device IDs
# st_dev is the device ID of the filesystem containing the file
if os.stat(src).st_dev == os.stat(os.path.dirname(dst)).st_dev:
    print("Same filesystem - move will be instant (rename only)")
else:
    print("Different filesystems - move will copy then delete")

shutil.move(src, dst)
```

### 13.2 Parallel Copying with Multiprocessing

For copying many large files, use Python's multiprocessing module to parallelize I/O operations. This is especially effective when copying between different physical drives.

```
from multiprocessing import Pool
import shutil
from pathlib import Path

def copy_file(args):
    """Helper function for parallel copying."""
    src, dst = args
    shutil.copy2(src, dst)
    return dst

def parallel_copy(source_dir, dest_dir, workers=4):
    """Copy files in parallel using multiple processes."""
    source = Path(source_dir)
    dest = Path(dest_dir)
    dest.mkdir(parents=True, exist_ok=True)

    # Build list of (src, dst) tuples for all files
    tasks = []
    for src_file in source.rglob("*"):
        if src_file.is_file():
            rel_path = src_file.relative_to(source)
            dst_file = dest / rel_path
            dst_file.parent.mkdir(parents=True, exist_ok=True)
            tasks.append((str(src_file), str(dst_file)))

    # Copy in parallel using a process pool
    with Pool(processes=workers) as pool:
        results = pool.map(copy_file, tasks)

    return results

# Usage: copy with 8 parallel workers
parallel_copy("./large_project", "./backup", workers=8)
```

Tip

Parallel copying helps most when copying many medium-to-large files across different physical drives. For SSD-to-SSD copies on the same drive, parallel I/O may not help because the drive controller already optimizes sequential access. In fact, too many parallel operations can cause contention and slow things down. Start with 4 workers and benchmark before increasing.

## Step 14: Advanced Usage

Once you have mastered the basics, these advanced techniques will help you handle complex scenarios that come up in production environments.

### 14.1 Custom Copy Functions

You can pass a custom copy function to copytree() to control exactly how each file is copied. This is useful for progress tracking, filtering, encryption, or special handling of certain file types.

```
import shutil
import os

def copy_with_progress(src, dst, *, follow_symlinks=True):
    """Copy a file and print progress information."""
    size = os.path.getsize(src)
    size_mb = size / 1024 / 1024
    print(f"Copying {os.path.basename(src)} ({size_mb:.1f} MB)...")
    shutil.copy2(src, dst, follow_symlinks=follow_symlinks)
    print(f"Done: {os.path.basename(src)}")

# Use custom copy function in copytree
# Every file will be copied using our progress-tracking function
shutil.copytree(
    "source_project",
    "dest_project",
    copy_function=copy_with_progress,
    ignore=shutil.ignore_patterns('*.tmp')
)
```

### 14.2 Incremental Backups with copytree()

Use dirs_exist_ok=True (Python 3.8+) to update an existing backup directory, copying only files that have changed since the last backup. This is much faster than full backups for large projects.

```
import shutil
import os
from pathlib import Path

def incremental_backup(source, backup):
    """Update backup directory with changed files only."""
    source_path = Path(source)
    backup_path = Path(backup)

    files_copied = 0
    files_skipped = 0

    for src_file in source_path.rglob("*"):
        rel_path = src_file.relative_to(source_path)
        dst_file = backup_path / rel_path

        if src_file.is_dir():
            dst_file.mkdir(parents=True, exist_ok=True)
            continue

        # Copy only if file is new or modified
        # Compare modification timestamps
        if not dst_file.exists() or \
           src_file.stat().st_mtime > dst_file.stat().st_mtime:
            dst_file.parent.mkdir(parents=True, exist_ok=True)
            shutil.copy2(src_file, dst_file)
            files_copied += 1
        else:
            files_skipped += 1

    print(f"Incremental backup complete: {files_copied} copied, {files_skipped} unchanged")
    return files_copied, files_skipped

incremental_backup("./project", "./project_backup")
```

### 14.3 Archive Format Comparison

shutil.make_archive() supports multiple formats. Choose the right one for your use case based on compression needs, platform compatibility, and speed requirements.

| Format | Extension | Compression | Preserves Permissions | Best For |
| --- | --- | --- | --- | --- |
| zip | .zip | Deflate | Limited | Cross-platform sharing, Windows compatibility |
| tar | .tar | None | Yes | Quick bundling without compression |
| gztar | .tar.gz | gzip | Yes | Linux/macOS, good balance of speed and size |
| bztar | .tar.bz2 | bzip2 | Yes | Maximum compression, slower speed |
| xztar | .tar.xz | xz | Yes | Best compression ratio, slowest speed |

```
# Create different archive formats for comparison
shutil.make_archive("backup", 'zip', "project")      # backup.zip
shutil.make_archive("backup", 'gztar', "project")   # backup.tar.gz
shutil.make_archive("backup", 'bztar', "project")   # backup.tar.bz2
shutil.make_archive("backup", 'xztar', "project")   # backup.tar.xz
```

### 14.4 Using which() for Cross-Platform Tool Discovery

shutil.which() finds executables in the system PATH, similar to the Unix which command. This is invaluable for writing cross-platform scripts that need to call external tools.

```
import shutil
import subprocess

# Find python executable path
python_path = shutil.which("python3")
print(f"Python found at: {python_path}")

# Check if git is installed before using it
if shutil.which("git"):
    result = subprocess.run(
        ["git", "--version"],
        capture_output=True,
        text=True,
        check=True
    )
    print(result.stdout.strip())
else:
    print("Git is not installed or not in PATH")
    print("Please install Git from https://git-scm.com/downloads")
```

## Step 15: Alternatives & Comparisons

shutil is powerful, but it is not the only tool for file operations in Python. Understanding when to use each module will make you a more effective developer and help you choose the right tool for each job.

Figure 3: Module selection guide for Python file operations.

### 15.1 shutil vs os

| Feature | shutil | os |
| --- | --- | --- |
| Copy files | Yes (copy, copy2, copyfile) | No |
| Move files | Yes (move) | Rename only (rename) |
| Delete folders recursively | Yes (rmtree) | No (rmdir only empty dirs) |
| Create directories | No | Yes (mkdir, makedirs) |
| Environment variables | No | Yes (environ, getenv) |
| Process management | No | Yes (system, spawn) |
| Low-level filesystem | Limited | Extensive |

**When to use os:** Use os when working with the operating system itself environment variables, process IDs, permissions, and low-level path operations. Use shutil when you need to actually manipulate files and directories.

```
import os

# Create nested directories
os.makedirs("path/to/new/dir", exist_ok=True)

# Rename a file (same filesystem only)
os.rename("old.txt", "new.txt")

# Check environment variable
api_key = os.getenv("API_KEY")
```

### 15.2 shutil vs pathlib

| Feature | pathlib | shutil |
| --- | --- | --- |
| Read/write files | Yes (read_text, write_text) | No |
| Path manipulation | Excellent (OOP approach) | None |
| Copy files | Limited | Excellent |
| Move files | Limited | Excellent |
| Delete folders recursively | No | Yes (rmtree) |

**When to use pathlib:** Use pathlib for path construction, existence checks, and basic file I/O. They are commonly used together in modern Python code.

```
from pathlib import Path
import shutil

src = Path("photo.jpg")
dst = Path("backup")

# pathlib for path operations
if src.exists() and src.stat().st_size > 0:
    dst.mkdir(parents=True, exist_ok=True)
    # shutil for the actual copy
    shutil.copy2(src, dst / src.name)
```

### 15.3 shutil vs zipfile/tarfile

Use shutil.make_archive() for simple archive creation. Use zipfile or tarfile when you need fine-grained control over compression levels, passwords, selective file inclusion, or metadata preservation.

```
import zipfile
from pathlib import Path

# shutil: Simple one-line archive creation
shutil.make_archive("backup", 'zip', "project")

# zipfile: Fine-grained control
# Choose compression level, add files selectively
with zipfile.ZipFile(
    "secure_backup.zip",
    'w',
    zipfile.ZIP_DEFLATED,
    compresslevel=9
) as zf:
    for file in Path("project").rglob("*.py"):
        zf.write(file, arcname=file.name)
```

### 15.4 shutil vs send2trash

shutil.rmtree() permanently deletes files. If users might need to recover deleted files, use the third-party send2trash package instead. This sends files to the Recycle Bin on Windows or the Trash on macOS and Linux.

```
# Install first: pip install send2trash
from send2trash import send2trash

# Sends to Recycle Bin / Trash instead of permanent deletion
send2trash("notes.txt")
send2trash("old_project_folder")
```

| Feature | shutil.rmtree() | send2trash |
| --- | --- | --- |
| Permanently deletes | Yes | No |
| Sends to Recycle Bin/Trash | No | Yes |
| Recoverable | No | Yes (until bin is emptied) |
| Installation required | No (built-in) | Yes (pip install) |

### 15.5 shutil vs subprocess

Some developers use subprocess to call system commands like cp or rsync. This is usually unnecessary and less safe than using shutil. subprocess spawns a new process, which is slower and introduces shell injection risks if not used carefully.

| Aspect | shutil | subprocess |
| --- | --- | --- |
| Cross-platform | Yes (native Python) | No (commands vary by OS) |
| Calls external tools | No | Yes |
| Safer (no shell injection) | Yes | Risk if not used carefully |
| Faster startup | Yes | No (spawns new process) |
| Platform-specific features | Limited | Full access to OS tools |

**Recommendation:** Prefer shutil for file operations. Only use subprocess when you specifically need platform-specific utilities like rsync, robocopy, or custom shell scripts that have no Python equivalent.

### 15.6 Which Module Should You Use? Quick Reference

| Task | Recommended Module |
| --- | --- |
| Copy files | shutil |
| Move files | shutil |
| Delete directory tree | shutil |
| Create/delete directories | os or pathlib |
| Manipulate file paths | pathlib |
| Search files by pattern | glob or pathlib.Path.glob() |
| Read/write text files | pathlib |
| ZIP archive (advanced control) | zipfile |
| TAR archive (advanced control) | tarfile |
| Send files to Recycle Bin | send2trash |
| Execute shell commands | subprocess |

## Conclusion

Python's shutil module is an essential tool in every developer's toolkit. It provides a clean, cross-platform API for the most common file and directory operations like copying, moving, archiving, and deleting without requiring any external dependencies. Because it is part of the standard library, you can rely on it being available on any system that runs Python.

The key takeaways from this guide:

- **shutil is built-in** - No installation required. Import it and use it immediately on any platform.
- **Know your functions** - Use copy() for permissions, copy2() for metadata, copytree() for directories, and rmtree() for deletion. Each function is optimized for a specific use case.
- **Combine with pathlib** - Use pathlib for path construction and shutil for operations. They complement each other perfectly and represent modern Python best practices.
- **Safety first** - Always validate paths, check disk space, and protect against accidental deletion of system directories. Never trust user input as a file path.
- **Log everything** - Use Python's logging module in production code for traceability, debugging, and audit trails.
- **Understand the alternatives** - Know when to use os, pathlib, zipfile, tarfile, send2trash, or subprocess instead of or alongside shutil.

With the knowledge from this guide, you can confidently build backup scripts, deployment pipelines, file management tools, and any other application that manipulates the filesystem. Start with the simple examples, then progressively incorporate the advanced techniques as your needs grow. The skills you have learned here transfer directly to real-world DevOps, data engineering, and system administration tasks.

Next Steps

Practice by building your own backup script, then extend it with incremental backups, email notifications, and cloud upload integration. The skills you have learned here transfer directly to real-world DevOps, data engineering, and system administration tasks. Consider also learning about [pathlib](/python-pathlib-guide), [os](/python-os-module-guide), and [logging](/python-logging-guide) to round out your file manipulation toolkit.

## Frequently Asked Questions

### What is shutil in Python?

shutil is a Python standard library module for high-level file and directory operations. It provides convenient functions for copying, moving, archiving, and deleting files and directories, built on top of lower-level modules like os and pathlib. It is cross-platform and requires no installation.

### Is shutil included with Python?

Yes, shutil is part of the Python standard library and requires no installation. It is available immediately after installing Python on any platform including Windows, macOS, and Linux. You should never run pip install shutil as this may install an unrelated third-party package.

### What is the difference between shutil.copy() and shutil.copy2()?

shutil.copy() copies a file and preserves permissions only. shutil.copy2() copies a file and preserves all metadata including timestamps, permissions, and other file attributes. Use copy2() when you need to maintain the original file's metadata such as creation and modification dates.

### Should I use shutil or os for file operations?

Use shutil for high-level file operations like copying, moving, and deleting directory trees. Use os for low-level operating system interactions like creating directories, managing environment variables, and process management. They complement each other and are commonly used together in production code.

### How do I delete a directory recursively with shutil?

Use shutil.rmtree(path) to delete an entire directory and all its contents recursively. Be careful as this permanently deletes files and cannot be undone. For safer deletion that sends files to the Recycle Bin or Trash, consider the third-party send2trash package instead.

### How do I preserve symlinks when copying with shutil?

Use the symlinks=True parameter in shutil.copytree() to preserve symlinks as links rather than following them. Also use ignore_dangling_symlinks=True to safely skip broken symlinks. For individual files, use follow_symlinks=False in shutil.copy() or shutil.copy2().

### Can shutil copy files between different drives?

Yes, shutil can copy files between different drives and filesystems. However, shutil.move() will perform a copy-and-delete operation when moving between different filesystems, which is slower than a same-filesystem rename. Always check available disk space before cross-filesystem operations since they temporarily require space on both drives.

### What archive formats does shutil.make_archive() support?

shutil.make_archive() supports zip, tar, gztar (tar.gz), bztar (tar.bz2), and xztar (tar.xz) formats. ZIP is best for cross-platform sharing. gztar offers a good balance of compression and speed for Linux/macOS. xztar provides the best compression ratio but is the slowest.

## Related Articles

- [Python pathlib: Object-Oriented Path Manipulation](/python-pathlib-guide)
- [Python os Module: Operating System Interface](/python-os-module-guide)
- [Python File Handling: Read, Write, and Manage Files](/python-file-handling-guide)
- [Python Logging: Production-Ready Logging Strategies](/python-logging-guide)
- [Python subprocess: Running External Commands Safely](/python-subprocess-guide)

## External References

- [Python Official Documentation: shutil](https://docs.python.org/3/library/shutil.html)
- [Python Official Documentation: pathlib](https://docs.python.org/3/library/pathlib.html)
- [Python Official Documentation: os](https://docs.python.org/3/library/os.html)
- [Real Python: High-Level File Operations in Python](https://realpython.com/python-shutil/)

---

*End of Article*