How Do You Set Environment Variables in Python?
Setting environment variables is a fundamental skill for any Python developer aiming to create flexible, secure, and easily configurable applications. Whether you’re managing sensitive credentials, configuring different settings for development and production, or simply streamlining your workflow, understanding how to set environment variables in Python can greatly enhance your coding practice. This article will guide you through the essential concepts and practical approaches to effectively manage environment variables within your Python projects.
Environment variables act as a bridge between your operating system and your Python code, allowing you to store and access configuration data outside of your source files. This separation not only improves security by keeping sensitive information out of your codebase but also promotes portability and scalability. By mastering environment variables, you can ensure that your applications adapt seamlessly to different environments without the need for hard-coded changes.
In the following sections, we will explore various methods to set and retrieve environment variables in Python, discuss best practices, and highlight common pitfalls to avoid. Whether you’re a beginner or looking to refine your skills, this guide will equip you with the knowledge to confidently manage environment variables and build more robust Python applications.
Setting Environment Variables Using the `os` Module
In Python, the `os` module provides built-in functions to interact with the operating system, including managing environment variables. You can set environment variables programmatically within a Python script using `os.environ`. This is particularly useful when you want to configure your application’s behavior without relying on external shell commands.
To set an environment variable, assign a value to the desired key in `os.environ`, which behaves like a dictionary:
“`python
import os
os.environ[‘MY_VARIABLE’] = ‘some_value’
“`
This change affects the current process and any child processes spawned from it. However, it does not modify the environment variables of the parent shell or system-wide settings. Therefore, the variable will only persist during the runtime of the script or its child processes.
You can also check if an environment variable exists and retrieve its value safely:
“`python
my_var = os.environ.get(‘MY_VARIABLE’, ‘default_value’)
“`
This returns `’default_value’` if `’MY_VARIABLE’` is not set.
Key points when using `os.environ`:
- Changes apply only to the current Python process and its subprocesses.
- Environment variables set this way are not persistent; they disappear after the script ends.
- Use this method when you want to temporarily set variables for subprocesses or within the scope of a running program.
Using `.env` Files with `python-dotenv` for Environment Variable Management
Managing environment variables directly in code can become cumbersome and insecure, especially when dealing with sensitive data such as API keys or database credentials. Using a `.env` file to store environment variables is a common practice. The `python-dotenv` package facilitates loading these variables into your Python environment seamlessly.
To use `python-dotenv`:
- Install the package via pip:
“`
pip install python-dotenv
“`
- Create a `.env` file in your project directory with key-value pairs:
“`
SECRET_KEY=abc123
DATABASE_URL=postgres://user:password@localhost/dbname
“`
- Load the `.env` file in your Python script:
“`python
from dotenv import load_dotenv
import os
load_dotenv() By default, looks for a .env file in the current directory
secret_key = os.getenv(‘SECRET_KEY’)
database_url = os.getenv(‘DATABASE_URL’)
“`
`load_dotenv()` reads the `.env` file and sets the environment variables for the running process. This approach keeps sensitive information out of your source code, improving security and configurability.
Advantages of using `.env` files with `python-dotenv`:
- Centralized management of environment variables.
- Easy to switch configurations between development, testing, and production.
- Keeps secrets out of version control if `.env` is added to `.gitignore`.
Setting Environment Variables in Different Operating Systems
Environment variable configuration varies depending on the operating system. While Python allows you to set variables at runtime, you might want to set them at the system or user level for persistence.
Below is a summary of how environment variables are set in common OS environments:
| Operating System | Shell/Method | Command or File | Scope |
|---|---|---|---|
| Linux / macOS | Bash / Zsh |
export VAR_NAME="value"Add to ~/.bashrc, ~/.zshrc, or ~/.profile
|
User-level persistent |
| Windows (Command Prompt) | cmd.exe |
setx VAR_NAME "value"Or via System Properties > Environment Variables GUI |
User or system-level persistent |
| Windows (PowerShell) | PowerShell | $Env:VAR_NAME = "value" (session only) |
Current session only |
When environment variables are set outside Python, they become accessible within Python scripts executed in the same environment, allowing you to retrieve them using `os.getenv()` or `os.environ`.
Best Practices for Managing Environment Variables in Python Projects
Effective environment variable management enhances security, portability, and maintainability of Python applications. Consider the following best practices:
- Avoid hardcoding secrets in source code: Use environment variables or secrets management tools.
- Use `.env` files during development: This simplifies switching environments without modifying code.
- Add `.env` files to `.gitignore`: Prevent accidental commits of sensitive data.
- Use `os.getenv()` to read variables: It allows setting default values and handles missing variables gracefully.
- Validate environment variables on startup: Ensure all required variables are set to avoid runtime errors.
- Document environment variables: Maintain a sample `.env.example` file listing required variables for new developers.
- Use virtual environments: Keeps dependencies isolated and environment variables scoped.
By following these guidelines, your Python applications will be more secure and easier to configure across different environments.
Setting Environment Variables in Python
Environment variables provide a way to configure your Python applications externally from the source code, enabling better security and flexibility. Python offers multiple approaches to set and access these variables, both temporarily within the runtime and persistently across sessions.
Using the `os` Module to Set Environment Variables Temporarily
The `os` module in Python allows you to manipulate environment variables for the duration of the program’s execution. This method does not affect the system environment variables permanently.
“`python
import os
Setting an environment variable
os.environ[‘MY_VARIABLE’] = ‘my_value’
Accessing the environment variable
print(os.environ[‘MY_VARIABLE’]) Output: my_value
“`
- Changes made using `os.environ` only persist while the Python process is running.
- Modifications do not propagate to subprocesses unless explicitly passed.
- Accessing a non-existent variable via `os.environ[‘VAR’]` raises a `KeyError`; use `os.environ.get(‘VAR’)` to avoid this.
Setting Environment Variables Persistently on Different Operating Systems
To make environment variables available beyond the Python runtime, set them at the operating system level.
| Operating System | Method | Example |
|---|---|---|
| Linux/macOS (bash) | Export in shell configuration |
export MY_VARIABLE="my_value" |
| Windows (Command Prompt) | Set environment variable temporarily |
set MY_VARIABLE=my_value |
| Windows (PowerShell) | Set environment variable temporarily |
$env:MY_VARIABLE = "my_value" |
| Windows (Persistent) | System Environment Variables GUI or `setx` command |
setx MY_VARIABLE "my_value" |
- After setting variables persistently, restart your terminal or IDE to apply the changes.
- Persistent environment variables are accessible by any process launched after the change.
Loading Environment Variables from a `.env` File Using `python-dotenv`
For projects requiring multiple environment variables, using a `.env` file is a common practice. This file contains key-value pairs that can be loaded into the environment at runtime.
- Create a `.env` file in your project root:
“`
MY_VARIABLE=my_value
DATABASE_URL=postgres://user:password@localhost/dbname
“`
- Install the `python-dotenv` package:
“`bash
pip install python-dotenv
“`
- Load the variables in your Python script:
“`python
from dotenv import load_dotenv
import os
load_dotenv() By default, loads from .env in current directory
my_var = os.getenv(‘MY_VARIABLE’)
print(my_var) Output: my_value
“`
- Using `python-dotenv` keeps sensitive information out of your codebase.
- The `.env` file should be added to `.gitignore` to prevent committing secrets.
- `load_dotenv()` can accept a path argument to specify a custom location for the `.env` file.
Best Practices for Managing Environment Variables in Python
- Security: Never hardcode sensitive information such as API keys or passwords directly in the source code.
- Portability: Use `.env` files or OS-level environment variables to maintain consistency across development, testing, and production environments.
- Error Handling: Always check for the presence of expected environment variables and provide meaningful error messages or defaults.
“`python
import os
api_key = os.getenv(‘API_KEY’)
if not api_key:
raise EnvironmentError(‘API_KEY environment variable is missing.’)
“`
- Documentation: Clearly document required environment variables for your project in README files or configuration guides.
- Use Virtual Environments: When working on multiple projects, isolate environment variables and dependencies using virtual environments or containers.
Accessing Environment Variables in Subprocesses
When spawning subprocesses, environment variables can be explicitly passed to ensure they are available.
“`python
import subprocess
import os
env = os.environ.copy()
env[‘MY_VARIABLE’] = ‘my_value’
result = subprocess.run([‘python’, ‘some_script.py’], env=env)
“`
- Passing the modified environment dictionary ensures child processes inherit the variables.
- Omitting the `env` argument causes the subprocess to inherit the parent’s environment unchanged.
Using Environment Variables with Configuration Libraries
Many configuration libraries integrate environment variable support to simplify application setup.
| Library | Feature Description | Example Usage |
|---|---|---|
| `pydantic` | Supports environment variable-based settings | `Settings = BaseSettings` |
| `dynaconf` | Manages layered configuration including env vars | `settings.setenv(‘MY_VAR’)` |
| `configparser` | Primarily file-based, can be combined with env vars | Manual merging with `os.environ` |
Example with `pydantic`:
“`python
from pydantic import BaseSettings
class Settings(BaseSettings):
api_key: str
class Config:
env_file = “.env”
settings = Settings()
print(settings.api_key)
“`
- These libraries provide validation, type enforcement, and default values.
- Using them enhances maintainability and reduces boilerplate code related to environment management.