Advanded Software Tools for the Reliability of Industrial Datasets
ASTRID
Advanced Software Tools for Reliable Industrial Datasets
Open-source tooling to assess and improve the reliability of datasets used in industrial AI systems.
Table of Contents
- What is ASTRID?
- Key Features
- Architecture
- The Five Reliability Dimensions
- Installation
- Quick Start
- Customising Metric Weights
- Headless APIs and CLI
- Analyzers Reference
- Health Score & Grading
- Policy Gate Presets
- Configuration Reference
- Exporting Results
- Standards Alignment
- Contributing
- License
- Citation
- Acknowledgements
What is ASTRID?
Industrial AI systems are only as reliable as the data they are trained and validated on. Yet in practice, datasets used in manufacturing, energy, healthcare, and other safety-critical sectors routinely suffer from hidden problems: missing values that accumulate silently across sensor channels, label leakage between training and test splits, personally-identifiable information embedded in column names, distribution drift across production batches, and demographic disparities that surface only after deployment. Detecting and quantifying these issues before a model reaches production is both technically difficult and frequently neglected.
ASTRID provides a unified, browser-based quality-assessment platform for three of the most common data modalities in industrial AI: tabular datasets (CSV, Parquet, Excel), time-series recordings, and image archives (ZIP files). Each analyzer runs a comprehensive battery of checks spanning data quality, security, reliability, robustness, and fairness—and synthesises the results into a single, interpretable health score together with a letter grade and a ranked list of remediation recommendations.
The tool is aimed at ML engineers who want a fast first-pass audit before committing a dataset to a training run, data scientists who need evidence of dataset fitness for regulated applications, and AI safety teams responsible for documenting compliance with emerging standards such as the EU AI Act and ISO/IEC 25012. ASTRID is intentionally dependency-light and runs entirely locally: no data is sent to external services, making it suitable for confidential or commercially-sensitive datasets.
Within a broader MLOps workflow, ASTRID fits naturally at the data-validation gate that precedes model training. Its JSON export can be stored as a dataset artefact alongside model cards and experiment metadata, enabling reproducible audits and drift monitoring across dataset versions.
Key Features
- Multi-modal support — dedicated analyzers for tabular data (CSV/Parquet/Excel), time-series (CSV/Parquet), and image datasets (ZIP archives with optional metadata CSV).
- Five weighted reliability dimensions — every dataset is evaluated across Quality, Security, Reliability, Robustness, and Fairness, each with multiple concrete checks, plus a sixth Transparency data-card dimension (zero default weight on tabular/time-series; a small default weight on images).
- Configurable metric weights — users can adjust the contribution of each dimension to the composite health score directly from the sidebar, with automatic normalisation so values do not need to sum to exactly 100.
- Automated recommendations — after each analysis run ASTRID generates a prioritised list of plain-language remediation actions.
- HTML report export — a self-contained HTML report can be downloaded after every analysis and shared with colleagues or archived as compliance evidence.
- EU AI Act evidence mapping — tabular, time-series, and image reports map observed metrics to selected EU AI Act evidence areas such as data governance, record-keeping, transparency, and robustness.
- ISO/IEC 25012 evidence mapping — tabular, time-series, and image reports also map observed metrics to data-quality characteristics such as completeness, consistency, traceability, confidentiality, and availability.
- PII detection — heuristic regex-based scanning flags columns or file paths that may contain personally-identifiable information such as email addresses, phone numbers, and national identifiers.
- Drift detection — Kolmogorov–Smirnov statistics measure distribution shift between data slices (e.g., first vs. last temporal segment), flagging datasets where the underlying distribution has changed.
- Fairness analysis — group-level disparity checks measure positive-rate differences across protected or demographic subgroups, surfacing potential bias before training.
- Split-leakage detection — row-hash and perceptual-hash cross-split checks identify samples that appear in both training and test partitions.
- Transparency scoring — dataset documentation completeness and traceability coverage are measured and included in the overall score (image analyzer).
Architecture
astrid/
├── app.py # Streamlit home page
├── astrid_cli.py # Headless command-line interface
├── astrid_core.py # Reusable tabular analysis API
├── astrid_timeseries.py # Reusable time-series analysis API
├── astrid_images.py # Reusable image analysis API
├── astridimageio.py # Resource-safe image archive ingestion
├── audit_history.py # Audit persistence and policy gates
├── utils.py # Shared scoring, reporting, and UI helpers
├── pages/
│ ├── 01_Tabular.py # Tabular dataset analyzer
│ ├── 02TimeSeries.py # Time-series dataset analyzer
│ ├── 03_Images.py # Image dataset analyzer
│ ├── 04Driftexperimental.py # Experimental drift tracker
│ ├── 05AuditHistory.py # Saved-run review and comparison
│ └── 06CrossDataset_Drift.py # Dataset drift comparison
├── tests/ # Automated tests
├── experiments/ # Reproducible research experiments
├── docs/ # Jekyll documentation site
└── pyproject.toml # Package metadata and dependenciesapp.py is the Streamlit entry point and renders the landing page with navigation cards for each analyzer.
utils.py is the shared library imported by every page. It contains the CSS design system, the computehealthscore scoring engine (with DEFAULT_WEIGHTS), helper functions for HTML report generation, PII pattern matching, statistical utilities, and common widget renderers.
Each analyzer has a reusable module shared by its Streamlit page, Python callers, and the CLI: astridcore.py for tabular data, astridtimeseries.py for time series, and astrid_images.py for image archives.
The Five Reliability Dimensions
| Dimension | Default Weight | What it measures | Key checks performed | |-----------|---------------|------------------|----------------------| | Quality | 35 % | Structural correctness and completeness of the data | Missingness rate, exact duplicate rows, split leakage (row-hash), annotation schema consistency, class balance entropy, metadata completeness | | Security | 25 % | Confidentiality and privacy risk | PII heuristic scanning (email, phone, ID patterns), EXIF GPS metadata in images, suspicious sample detection, source-concentration HHI | | Reliability | 20 % | Temporal and distributional stability | Kolmogorov–Smirnov drift between first/last slices, cadence irregularity, duplicate timestamp rate, inter-annotator agreement (Cohen's κ) | | Robustness | 10 % | Resilience to noise and edge cases | MAD-based row anomaly scoring (p99), image-feature outlier rate, condition-coverage gaps across operational subsets | | Fairness | 10 % | Equitable representation across groups | Positive-rate disparity across protected attributes, representation Jensen–Shannon divergence, label-parity gap, missingness disparity by group |
Note: The image analyzer additionally tracks a Transparency dimension (documentation completeness and traceability). On the image page it is exposed as a configurable weight in the sidebar (default 8 / 100); on the tabular and time-series analyzers it is not measured.
Installation
Prerequisites
- Python 3.9 or later
- Git
Step-by-step
# 1. Clone the repository
git clone https://github.com/jorge-martinez-gil/astrid.git
cd astrid
2. Create and activate a virtual environment
python -m venv .venv
On Linux / macOS:
source .venv/bin/activate
On Windows:
.venv\Scripts\activate
3. Install ASTRID and its dependencies
python -m pip install -e .
4. Launch the application
streamlit run app.pyThe application opens automatically in your default browser at http://localhost:8501.
Quick Start
Tabular and time-series datasets
- Navigate to Tabular Analyzer or Time Series Analyzer from the home page.
- Upload your dataset file (CSV, Parquet, or Excel).
- In the sidebar, select a threshold preset (Balanced / Strict / Lenient) and configure column roles (label, split, time, group).
- Optionally expand ⚖️ Score Weights to adjust dimension contributions (see Customising Metric Weights).
- Click 🔬 Run analysis.
- Read the Verdict card at the top—it summarises the overall finding, lists the key issues, and shows recommended actions.
- Explore the dimension tabs (Quality, Security, Reliability, Robustness, Fairness, Transparency, Security) for detailed metric values and evidence.
- Switch to the Export tab and click ⬇ Download HTML to save a self-contained report.
- Open Audit History to review saved runs against policy gate presets or custom thresholds.
Image datasets
- Navigate to Image Analyzer from the home page.
- Upload a ZIP file containing your images. Optionally also upload a metadata CSV with label, split, group, and source columns.
- Configure column roles in the sidebar.
- Optionally expand ⚖️ Score Weights to adjust dimension contributions.
- Click Run analysis and wait while ASTRID scans each image.
- Review the property score bars and detailed tabs. Download the HTML report from the Export tab.
Customising Metric Weights
Default weights
| Dimension | Default weight | |-------------|---------------| | Quality | 35 | | Security | 25 | | Reliability | 20 | | Robustness | 10 | | Fairness | 10 | | Total | 100 |
Using the ⚖️ Score Weights sidebar expander
In every analyzer, open the ⚖️ Score Weights expander in the left sidebar (located just above the Run button). You will see five sliders—one per dimension—each initialised to the default value above.
- Drag any slider to increase or decrease that dimension's contribution.
- A live sum indicator shows the current total. If it is not 100, ASTRID displays an amber notice and automatically normalises the weights before computing the score, so you never need to make them add up to exactly 100.
- Click Reset to defaults to restore all sliders to the values in the table above.
How normalisation works
Internally, before computing the health score, ASTRID divides each supplied weight by the sum of all weights and multiplies by 100:
norm_weight[dim] = weight[dim] / sum(all weights) × 100This means that only the relative proportions matter. Setting Quality = 70, Security = 50, Reliability = 40, Robustness = 20, Fairness = 20 gives exactly the same result as setting them to 35, 25, 20, 10, 10.
Example use cases
- Security-critical domain (e.g., medical records, financial data): raise Security to 50 and lower Robustness and Fairness to 5 each. The health score will penalise PII findings more heavily.
- Academic benchmark where fairness is paramount: increase Fairness to 40 and lower Security to 10. The score will reflect demographic disparity more strongly.
- Manufacturing sensor data without labelled groups: set Fairness to 0 (all weight is redistributed to the other four dimensions automatically).
Headless APIs and CLI
Command-line audits and history
After installing the package, every analyzer can run headlessly:
astrid audit data.csv --label target --split split --policy "Balanced (default)" --save-history
astrid audit-timeseries sensors.parquet --time timestamp --entity device_id --json time-series.json
astrid audit-images images.zip --metadata labels.csv --label class --split split --json images.jsonUse --json, --markdown, or --html to export reports, --history-dir to save audit records outside the default local history folder, and --exit-on-fail to turn policy failures into a non-zero CI exit code:
astrid audit data.csv --json report.json --history-dir auditrunsci --quietSaved CLI audits appear in the same Audit History workflow as Streamlit-created runs.
Python API
The high-level file APIs return the report, score, grade, findings, recommendations, audit record, and optional policy result in one dictionary:
from astridcore import analyzetabular_file
from astridimages import analyzeimage_file
from astridtimeseries import analyzetimeseries_file
tabular = analyzetabularfile("data.csv") timeseries = analyzetimeseries_file("sensors.parquet") images = analyzeimagefile( "images.zip", metadata_path="labels.csv", )
print(tabular["score"], time_series["score"], images["score"])
For in-memory workflows, use analyzetimeseriesdataframe with a pandas DataFrame or analyzeimagearchive with ZIP bytes. Lower-level scoring remains available through computehealthscore in utils.py. To override inferred column roles and thresholds, build and pass a configuration with maketabularconfig, maketimeseriesconfig, or makeimageconfig.
Analyzers Reference
01 — Tabular Analyzer
Accepted file formats: CSV, Parquet, Excel (.xls, .xlsx)
Checks performed:
- Overall and per-column missingness rate
- Exact duplicate row rate
- Row-hash cross-split leakage (train/test contamination)
- PII heuristic scan across text columns
- Numeric distribution drift (KS statistic, first vs. last temporal slice)
- MAD-based row anomaly scoring
- Suspected label-noise rate with ranked row-level review candidates
- Group fairness: positive-rate disparity across user-selected group columns
- Inter-annotator agreement (Cohen's κ) when annotator columns are supplied
- Rare-category detection in categorical columns
- Schema consistency across splits
- Verdict card (PASS / WARNING / FAIL) with reasons and recommendations
- Health score ring chart (0–100) with dimension breakdown bars
- Per-dimension detail tabs with metric tables, histograms, and evidence lists
- Downloadable JSON report and self-contained HTML report
02 — Time Series Analyzer
Accepted file formats: CSV, Parquet
Checks performed:
- All tabular checks above, with temporal awareness
- Timestamp parsability and duplicate timestamp rate
- Cadence irregularity (coefficient of variation of inter-sample intervals)
- Entity-level statistics when entity columns are selected
- Temporal label-noise assessment using sensor, context, and timestamp-derived features
- Time-slice mode (day / week / month / quarter) for drift computation
- Same as Tabular Analyzer, plus time-axis health metrics (cadence, gaps)
03 — Image Analyzer (Experimental)
Accepted file formats: ZIP archive containing images (JPEG, PNG, BMP, TIFF, WEBP, GIF); optional metadata CSV or Parquet
Checks performed:
- Image readability (corrupt / unreadable file detection)
- Resolution audit (low-resolution image rate)
- Exact duplicate detection (SHA-256 hash) and perceptual near-duplicate detection (pHash)
- Cross-split hash leakage
- Conflicting-label detection on duplicate images
- Suspected label-noise rate with ranked image review candidates
- Class balance (normalised entropy)
- Annotator agreement (Cohen's κ) on annotation files
- KS-based feature drift between temporal slices
- Provenance coverage (source column completeness)
- Image-feature outlier detection (MAD on pixel statistics; Isolation Forest when
scikit-learnis available) - Condition-coverage gaps across operational subsets
- Group fairness: missingness disparity, representation JSD, label-parity gap
- Datasheet completeness (documentation fields)
- Traceability coverage (ID column completeness)
- EXIF GPS privacy check
- PII-like pattern detection in file paths and names
- Suspicious sample rate (statistical outlier images)
- Source-concentration HHI (dataset diversity)
Output:
- Property score bars for six dimensions (Quality, Reliability, Robustness, Fairness, Transparency, Security)
- Detailed tabs per dimension
- Downloadable HTML report
Health Score & Grading
Every analysis produces an integer health score between 0 and 100. The score is the weighted sum of per-dimension raw fractions, where each fraction measures how far that dimension is from ideal (1.0) versus worst-case (0.0).
| Grade | Score range | Meaning | |-------|-------------|---------| | A | 90 – 100 | Excellent — dataset is production-ready with minor or no issues | | B | 80 – 89 | Good — a few low-severity issues; address before long training runs | | C | 70 – 79 | Acceptable — noticeable issues that should be resolved; proceed with caution | | D | 60 – 69 | Poor — significant problems likely to degrade model performance or safety | | F | 0 – 59 | Failing — critical issues detected; do not use for production training without remediation |
The score breakdown bars on the Overview tab show each dimension's contribution as a percentage of its maximum possible weight.
Policy Gate Presets
The policy gate controls the final PASS/FAIL decision. It is separate from the health score and can be relaxed or tightened for different review contexts.
| Preset | Intended use | |--------|--------------| | Strict production | Release gate for high-risk or regulated datasets | | Balanced (default) | Default review gate for routine audits; tolerant of noisy development datasets while keeping PII strict | | Lenient development | Development gate that relaxes noisy quality, leakage, fairness, and drift checks while keeping PII strict | | Exploratory research | Research-only gate for stress tests and demos; not recommended for deployment decisions |
Analyzers save audit records with their metric snapshots. The Audit History page owns policy gate review and can re-review saved runs under any preset or custom thresholds.
Configuration Reference
| Parameter | Description | Default | |-----------|-------------|---------| | Threshold preset | Selects Balanced, Strict, or Lenient threshold profiles for all checks simultaneously | Balanced | | Policy gate preset | Audit History setting for Strict production, Balanced, Lenient development, or Exploratory research pass/fail gates | Balanced | | Drift KS threshold | KS statistic above which numeric distribution shift is flagged | 0.30 (Balanced) | | PII hit-rate threshold | Regex hit-rate above which a text column is flagged as potentially containing PII | 0.01 (Balanced) | | Label column | Column containing the prediction target; used for leakage and fairness checks | Auto-detected | | Split column | Column indicating train/val/test membership; used for leakage detection | Auto-detected | | Time column | Column containing timestamps; used for temporal drift and cadence checks | Auto-detected | | Group columns | Columns defining demographic or operational subgroups for fairness analysis | Auto-detected | | Random state | Random seed for any stochastic sub-steps (e.g., sub-sampling for PII scan) | 7 |
Threshold values for Strict and Lenient presets:
| Parameter | Strict | Balanced | Lenient | |-----------|--------|----------|---------| | Drift KS threshold | 0.20 | 0.30 | 0.40 | | PII hit-rate threshold | 0.005 | 0.01 | 0.02 |
Exporting Results
JSON report
Every analyzer makes the full analysis report available as a structured JSON object. Use the Export tab → ⬇ Download JSON to save it. The JSON captures all metric values, threshold settings, file fingerprints, and per-dimension evidence and is suitable for archiving alongside model cards or experiment tracking systems.
HTML report
The ⬇ Download HTML button on the Export tab generates a self-contained HTML file that can be opened in any browser without an internet connection. It includes the verdict, health score, dimension breakdown, and a summary of findings and recommendations. Share this file with stakeholders or attach it to compliance documentation.
EU AI Act evidence report
Each analyzer includes an EU AI Act Evidence section and downloadable Markdown/JSON evidence reports. These reports link ASTRID results to selected areas of Regulation (EU) 2024/1689, including Article 9 risk management, Article 10 data governance, Articles 11-12 documentation and record-keeping, Article 13 transparency, and Article 15 accuracy, robustness, and cybersecurity. The mapping is technical evidence only and is not a legal opinion or compliance certification.
ISO/IEC 25012 evidence report
Each analyzer also includes an ISO/IEC 25012 section and downloadable Markdown/JSON evidence reports. These reports link ASTRID results to the ISO/IEC 25012 data-quality model characteristics, including completeness, consistency, credibility, currentness, confidentiality, traceability, understandability, availability, portability, and recoverability. The mapping is technical evidence only and is not a formal conformity assessment or certification.
Standards Alignment
ASTRID's metrics and checks are designed with the following standards and regulatory frameworks in mind:
| Standard / Framework | Relevance | |----------------------|-----------| | EU AI Act (Annex IV, Art. 10) | Data governance requirements for high-risk AI: relevance, representativeness, freedom from errors, completeness | | ISO/IEC 25012 | Data quality model: completeness, consistency, accuracy, currentness, accessibility, compliance | | NIST AI RMF (GOVERN, MAP, MEASURE, MANAGE) | Dataset risk identification, measurement, and management practices |
ASTRID does not provide legal compliance certification. The outputs are technical evidence to support human-led compliance assessments.
Contributing
Contributions are welcome. See CONTRIBUTING.md for the development setup and required checks.
To contribute:
- Fork the repository on GitHub.
- Create a feature branch:
git checkout -b feature/my-improvement. - Make your changes and add tests where applicable.
- Open a pull request describing your changes.
Please report security vulnerabilities privately as described in SECURITY.md.
License
ASTRID is released under the MIT License.
Acknowledgements
The Advanced Software Tools for Reliable Industrial Datasets (ASTRID) action has received funding from the EU, via oc1-2025-TIS-01 issued and implemented by the ENFIELD project, under the grant agreement No 101120657.
