Bruce Ho

What is a Lockfile?

4 min read

TopicsPythonTooling

A lockfile is a text file that records the exact versions of every single dependency and sub-dependency your project needs to run.

The purpose of a lockfile is to provide reproducibility. If you give your lockfile to a coworker, or send it to a production server, they can download the exact same bytes you did and avoid the “it works on my machine” problem.

What is the difference between requirements.txt vs lockfiles?

Historically, Python lacked a dedicated lockfile which forced requirements.txt to perform two conflicting roles:

  1. Declaring Abstract Dependencies: Manually listing top-level package needs (e.g., requests>=2.0).
  2. Locking Concrete Dependencies: Developers would run pip freeze > requirements.txt to capture the exact state of their environment (e.g., requests==2.31.0, urllib3==1.26.16).

Making one file (i.e., requirements.txt) perform these two separate jobs causes a bunch of problems. Running pip freeze would overwrite your hand-written top-level requirements with a large list of hardcoded versions. Once overwritten, you can no longer distinguish between a package you explicitly chose and a sub-dependency that package brought along. This turns future security patches and routine package updates into a really frustrating and error-prone process.

To solve the overwrite problem before lockfiles existed, teams typically adopted a multi-file convention:

  • requirements.in (Human-edited) for manually listing top-level packages.
  • requirements.txt for capturing dependencies and was generated from the .in file.
  • requirements-dev.txt for listing dependencies for local development, testing, and formatting (e.g., pytest, ruff) that is not needed in the production server.

pip-tools provided a set of command-line utilities to specifically manage the requirements.in to requirements.txt workflow.

Example Lockfile

# uv.lock (Machine-generated, do not edit manually)
version = 1
 
[[package]]
name = "requests"
version = "2.31.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
    { name = "certifi" },
    { name = "charset-normalizer" },
    { name = "idna" },
    { name = "urllib3" },
]
sdist = { url = "https://files.pythonhosted.org/.../requests-2.31.0.tar.gz", hash = "sha256:5615...", size = 110283 }
 
[[package]]
name = "urllib3"
version = "2.2.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/.../urllib3-2.2.1.tar.gz", hash = "sha256:450b...", size = 262843 }
 
# ... (other sub-dependencies omitted for brevity)

Enter pyproject.toml

While the pip-tools multi-file convention successfully solved the overwrite problem, it was a community workaround rather than an official standard. pyproject.toml was introduced as the official replacement for the hand-written requirements.in file by providing a single place to declare your project’s metadata, build system, and dependencies.

Example TOML file

[project]
name = "my-awesome-app"
version = "0.1.0"
description = "A modern Python application"
authors = [{ name = "Your Name", email = "you@example.com" }]
requires-python = ">=3.10"
 
# 1. THE TOP-LEVEL PACKAGES (Replaces requirements.in)
# These are your abstract, top-level dependencies.
dependencies = [
    "requests>=2.0",
    "flask>=3.0",
]
 
# 2. THE DEV TOOLS (Replaces requirements-dev.txt)
# These are only installed when you explicitly ask for them (e.g., during local development).
[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "ruff>=0.3.0",
]
 
# 3. THE BUILD SYSTEM (Optional but common)
# Tells tools exactly how to package your code if you are building a library.
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Modern Python Packaging: What Should You Use Today?

If you are starting a new project, uv is the current gold standard. It is desgined as a drop-in replacement for pip, pip-tools, virtual environment managers (venv, virtualenv) and python version managers (pyenv).

The uv Quick Start Guide

1. Create a New Project

uv init my-project-name
cd my-project-name

This instantly generates a standard pyproject.toml, a README, and a .python-version file to lock down your environment from day one.

2. Add a Dependency (replaces pip install & pip-compile)

uv add requests

In a single step, uv downloads the package, registers the abstract dependency in pyproject.toml, initializes your virtual environment (.venv), and generates a deterministic uv.lock file.

3. Add a Development Tool (replaces requirements-dev.txt)

uv add --dev pytest

This partitions local tools (like testing frameworks or linters) under a dedicated development group in your pyproject.toml, keeping production slim.

4. Run Your Code

uv run python main.py

uv run guarantees your script executes against your exact locked dependencies without requiring you to manually source or activate a .venv. Another benefit is that if your pyproject.toml changes, uv will automatically sync the environment before execution to prevent out-of-date crashes. (Note: If you still prefer the old workflow, manual activation scripts are still available inside .venv/bin/).

5. Sync an Environment (replaces pip install -r requirements.txt)

uv sync

This reads uv.lock and mirrors the exact state into your local virtual environment, adding missing packages and deleting any stray ones. This ensures any teammate who clones your repository can replicate your environment.