- Category
- Developer Tools
- Rank
- No. 1206Tools index
Previous survey · No. 1201 ·
- Pricing
- Open Source
- Type
- TOOL
- Use case
- Data Processing · Security & Identity
- Interfaces
- CLI
- GitHub
- 149 stars
- Latest release
- v0.1.2
- Date
About
A Python CLI and library that scans SQL databases to detect sensitive columns, masks them with deterministic realistic fakes, and validates that masking succeeded row-by-row before data is copied to lower-security environments. It supports pattern-based detection, optional local or remote LLM-assisted detection, a seed map for consistent masking across snapshots, and works across PostgreSQL, MySQL, SQL Server, Oracle, and SQLite via SQLAlchemy.
What it does
This is a command-line tool and Python library for scrubbing a database before it leaves production. It inspects each column for personal information such as names, emails, phone or card numbers, replaces matches with realistic fake values, then re-checks the table afterward to prove nothing slipped through. Fakes come from a private seed, so a given input always produces the same output everywhere, which lets tables that reference each other still join correctly once masked. Deciding what counts as sensitive combines a human override list, a memory of prior decisions, statistical checks on sampled values, and optionally an AI model for the rest. Unclassifiable columns are reported, not skipped.
Why it's ranked here
Masking only writes to disk behind an explicit apply flag, an incomplete scan refuses to proceed, and completeness gets checked row by row against an untouched copy afterward. It is MIT licensed, published to the Python package index, with a continuous-integration badge in its documentation. The honest caveat: its automated test suite currently exercises only one database engine, across many Python versions, on two operating systems. Postgres and MySQL run through the same code path but are not yet covered by that suite, only by early adopters. That gap is disclosed in the documentation rather than left for a user to discover.
What's good
The fail-closed defaults are the strongest part: masking never writes without an explicit flag, a column that cannot be classified is reported as unknown rather than assumed safe, and a sensitive primary key gets a loud warning instead of a silent skip. Generated values are typed correctly, not just string-shaped: dates stay real calendar dates, credit card numbers pass a Luhn check, an integer column never receives a string. Validation distinguishes a genuine pass from an unprovable one; a database without a usable key gets a warning, not a green checkmark, and strict mode turns that warning into a failure for a build pipeline.
Tradeoffs
The project is still labeled beta, and the test suite matches that: continuous integration currently runs against one database engine only. The other four share the same connection code, but that support rests on early adopters, not an automated suite yet. Determinism, the feature that keeps joins intact, is also the main weakness: anyone who knows the seed and can guess a value can recompute its mask, so the seed and the consistency store need the same protection as the data itself. Free text is flagged at the column level at best; a phone number buried inside a paragraph will not be caught.
How to use it well
This fits a pipeline that turns production data into a dev, staging, demo or analytics copy, wired in right before that copy is handed out, with verification run as a gate that fails the build rather than a step someone can skip. Set a private seed and pull the consistency-store salt from an environment variable instead of accepting the public defaults. Treat any column reported as unknown as a manual decision, not noise, and pick a blunt strategy such as blanking or redacting for free-text fields detection cannot see inside. Anyone targeting an engine other than SQLite should test thoroughly first rather than assume the same guarantees.
Technical notes+
src/dbmask/cli.py's mask command overrides config.masking.dry_run to the inverse of the apply flag before invoking Runner, closing a documented bug where dry_run: false in YAML could write while the CLI printed a dry-run banner. src/dbmask/connectors/__init__.py exposes one Connector/SQLConnector pair reused across every SQLAlchemy dialect; docs/architecture.md describes the apply path as keyset-paginated (iter_pages) to avoid a SQLite SHARED-lock deadlock from interleaved reads and writes. src/dbmask/masking/__init__.py exports MaskingEngine, MaskContext and SeedStore, with strategy randomness from seeded_rng(value, ctx.seed). docs/detection.md documents PatternMatcher(min_ratio, min_samples, date_order) computing its ratio over SQL DISTINCT-sampled values. src/dbmask/validation/__init__.py's Validator runs row_count, schema_elements and masking_completeness checks; docs/validation.md documents pk_row_limit (default 5000 rows per column) bounding the primary-key-aligned check.
Observed
- License
- MIT (LICENSE file referenced from pyproject.toml)
- Language / version support
- Python, requires-python >=3.9, classifiers list support through 3.14
- Packaging
- Installable from PyPI; core runtime dependencies are limited to SQLAlchemy, PyYAML and click, with database drivers and LLM SDKs shipped as separate optional extras
- Interfaces
- CLI (console-script entry point) and a Python library API
- Platform / database support
- OS Independent per package classifiers; connects to PostgreSQL, MySQL/MariaDB, SQL Server, Oracle and SQLite through a single SQLAlchemy-based connector
- Test coverage scope
- A test directory is present and referenced from the architecture docs, but the docs state the automated suite exercises SQLite only; PostgreSQL and MySQL integration tests are documented as not yet part of that suite
- Security process
- docs/security-model.md states what dbmask defends against (casual exposure, breach amplification, fail-closed failures) versus what it does not (deterministic-seed guessing, linkage re-identification, free-text leakage, primary-key sensitivity)
Read from README.md, pyproject.toml, requirements.txt, docs/index.md, src/dbmask/__init__.py, src/dbmask/cli.py, docs/architecture.md, docs/detection.md, docs/security-model.md, docs/strategies.md, docs/validation.md, src/dbmask/connectors/__init__.py, src/dbmask/detection/__init__.py, src/dbmask/masking/__init__.py, src/dbmask/validation/__init__.py.
What it can do
Scan SQL databases to detect sensitive columns
SQL database → List of detected sensitive columns
Mask sensitive columns with deterministic realistic fake data
Sensitive columns → Masked data
Validate that masking succeeded row-by-row
Masked database → Validation report
Detect sensitive data using local or remote LLM assistance
Database schema/data → Sensitive column detection results
Maintain a seed map for consistent masking across snapshots
Seed map → Consistent masked values across runs
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.