Skip to content
SolvesEverythingNotebook
SOLVESEVERYTHING

Validation

How we check that the numbers are right. Statistical results are compared with established software — R, limma, SciPy, statsmodels, pingouin, NumPy and the original method packages — at stated tolerances, and the checks run automatically before every update goes live.

65 numerical checks — most against an independent implementation in reference software — across 5 workspaces are listed below, each with the date it was last run. Every record also lists what has not been verified yet, because a validation page that only lists successes is not useful. Synthetic test data are used so results can be checked exactly; biological validation on your own data remains your judgement.

Jump to: Metabolic Integrator · FACS–scRNA Integration · Paper Explorer · Postdoc Toolkit · RNA Atlas

Metabolic Integrator

Metabolic Integrator (Tool 17): scRNA-seq + metabolomics integration through Human-GEM / Mouse-GEM

2026-10-25 · All tests in tests/toolkit-metabo.test.mjs, tests/toolkit-metabo-v2.test.mjs pass

Numerical checks

  • Benjamini–Hochberg adjustment matches R p.adjust(method = 'BH')
  • Welch t-test, one-sided one-sample t-test and hypergeometric tail match SciPy (ttest_ind equal_var=False, ttest_1samp, hypergeom.sf)
  • OLS differential abundance with covariates and rank-deficiency detection match statsmodels
  • Condition-adjusted (partial) Spearman correlation matches pingouin
  • Cell-level counts → pseudobulk log2 CPM reproduces the stated formula
Biology and behaviour checks (4)
  • Curated reference links behave as expected (ACOD1 → itaconate; IDO1 upstream of kynurenine; cofactors flagged)
  • Plasma-membrane transporters match known physiology
  • Synthetic data with planted sources/uptake are recovered; noise does not reorder candidates
  • Bulk protein-degradation reactions are not counted as producers of single amino acids; no cell type is proposed without gene evidence (both found with public data and fixed)
Not yet verified (3)
  • Biological validation on users' own datasets
  • Completeness of the reference metabolic networks
  • Rendered appearance in every browser

FACS–scRNA Integration

FACS–scRNA Integration (Tool 18): statistics for FlowJo exports linked to single-cell RNA

2026-10-25 · All tests in tests/toolkit-flow.test.mjs pass

Numerical checks

  • Quasi-binomial GLM for frequencies with event counts matches statsmodels (Binomial, var_weights = trials, scale = Pearson X²)
  • PCA variances and scores match NumPy SVD
  • Paired designs reproduce SciPy ttest_rel
  • Tie-aware Spearman permutation p values match SciPy permutation_test (exact enumeration for small n)
Biology and behaviour checks (2)
  • Marker → gene mapping is species-aware and never confuses CD44hi with CD4 or naive cells
  • Synthetic demo recovers planted effects with the intended models; confounded covariates are dropped; negative MFIs use asinh
Not yet verified (2)
  • Users' own FlowJo gating and compensation (performed in FlowJo)
  • Rendered appearance in every browser

Paper Explorer

Paper Explorer (Tool 19): rule-based, extractive analysis and comparison of papers

2026-10-25 · All tests in tests/toolkit-paper.test.mjs, tests/toolkit-ui.test.mjs, tests/toolkit-links.test.mjs pass
Extraction and logic checks (4)
  • Synthetic JATS and PDF fixtures: sections, citations, statistics, figure references, abbreviations, reagents, accessions and RRIDs are extracted as written
  • Results at a glance: knockout, treatment and no-change rows; recipient strains, marker genes and speculative clauses are excluded
  • Comparison logic: a loss-of-function result with a readout increased agrees with a treatment that lowers it; regulators are bridged only by a stated relation or a curated enzyme → product link
  • Manual checks on open-access eLife, Nature Communications and PNAS articles, including a knockout-versus-treatment pair that agrees once designs are put on one footing
Not yet verified (2)
  • Extraction accuracy on every journal format (rules can miss or misread sentences; every output links to its sentence)
  • Scanned PDFs without a text layer (not supported)

Postdoc Toolkit

Browser-only knockout consequence viewer and Tool 12 handoff

2026-10-20
Live annotation checks (5)
  • Mouse Acod1 ENSMUST00000022722.7 / ENSMUSP00000022722; WT 488 aa; GRCm39
  • Human ACOD1 ENST00000377462; WT 481 aa; GRCh38
  • Human reverse-strand HBB ENST00000335295; WT 147 aa; GRCh38
  • Human reverse-strand alternative HBB ENST00000647020.1; WT 147 aa; GRCh38
  • Expanded lookup + cDNA/CDS/protein/protein-feature endpoints returned successfully; response CORS allowed origin *
Not yet verified (4)
  • Rendered target browser and actual browser CORS flow
  • Actual PNG rasterization pixels/download in a browser
  • Experimental knockout/Cre/NMD outcomes
  • Exhaustive annotation variants and large-gene performance

lab dilution

2026-10-20
Behaviour checks (7)
  • all eight live calculators
  • unit and molecular-weight conversion
  • serial dilution retained-volume conservation
  • antibody master mix and viability-adjusted cell seeding
  • local favorites JSON import/export and storage denial
  • invalid results clear and block export
  • Tool 5 link and catalog registration

RNA Atlas

Native single-cell cleanup/integration/Leiden/t-SNE/DPT, specimen composition and Plot Studio

2026-10-20 · 141 of 146 automated tests passed

Numerical checks

  • Every per-capture Scrublet observed score and threshold flag matches independent direct package fits, tolerance 1e-12
  • Every Harmony corrected coordinate matches independent direct package fit, tolerance 1e-10
  • Known separated Leiden graph, independent exact weighted graph partition quality, tolerance 1e-8, and seeded repeatability
  • Every TSNE coordinate matches direct sklearn, tolerance 1e-10
  • Every diffusion coordinate and DPT pseudotime matches independent Scanpy Gaussian-neighbor/diffmap/DPT fit, tolerance 1e-7
  • Specimen counts/fractions preserve technical-capture grouping and zero populations; all tested limma outputs match independent real-R reference fit
  • All implemented Plot Studio statistical families match independent actual R stats calls and exported reproduction expressions
  • Nested-unit aggregation, matched/repeated completeness, confounding, missingness, exact risk sets/tied event/censoring and descriptive Kaplan-Meier hand cases
  • SVG finite geometry, physical width, panel labels and active-content rejection; actual PNG pHYs resolution and CRC replacement
Workflow checks (9)
  • Real loopback HTTP token/origin/job security and actual Python job execution
  • Explicit browser pairing/review/transfer consent using actual worker and real method packages through test UI transport
  • All-cell results, owned metadata, hash/threshold checks, dependent-result invalidation and raw-count preservation
  • CSV/JSON/R-code/figure exports and reproducible private local job archives
  • Worker/checkpoint/JSON.gz roundtrip, corruption rejection and failed-operation rollback
  • RNA gene handoff creates specimen observations; DE handoff preserves model provenance and blocks treating genes as biological replicates
  • Plot Studio editable grid, real browser-R model, figure export, saved/reordered figure boards and no raw-table duplication in job ledger
  • Companion ZIP integrity and exact source-byte matching, with Python source/requirements included
  • Existing bulk/pathway/native-format/persistence/14-tool regressions
Not yet verified (6)
  • Updated Docker image build
  • Downloaded OS launchers
  • Actual desktop-browser loopback/local-network permission
  • Rendered browser layout and rasterization/file dialogs
  • Large-data and 25k-cell capacity/performance
  • Native R integrations skipped in this environment

Local SoupX ambient correction and own-reference CellTypist annotation, alongside upload repairs

2026-10-20 · 159 automated tests

Numerical checks

  • Every tested CellTypist query score matches independent package calls at atol 1e-12
  • Specimen-disjoint holdout, explicit Unassigned thresholds and zero-library rejection
  • No model-download network access during reference training/prediction
  • Every SoupX auto/manual corrected CSR pointer/index/value and rho matches independent calls
  • All supplied called droplets excluded from ambient pool; mismatched counts rejected
  • Real authenticated loopback HTTP count uploads execute CellTypist and SoupX
  • Worker attachment, compact provenance, JSON.GZ/binary restore, tamper/stale guards and parent/copy independence
  • DOM upload/mapping, consent invalidation and pairing gates
  • Existing gzip, browser-runtime, large-CSV and RNA/toolkit regressions

Reference package versions: CellTypist 1.7.1, SoupX 1.6.2, scikit-learn 1.8.0, SciPy 1.17.0, NumPy 2.3.5

Not yet verified (6)
  • Rendered target browser
  • Actual browser-to-loopback permission
  • Docker image build and OS launchers
  • User expression/Seurat files
  • Large-data capacities
  • Biological benchmark reproduction

Not implemented: DecontX, Pretrained public-model discovery, RNA velocity, CellChat/NicheNet, Mixed-effects condition models, Unlimited native Seurat jobs

Bundled human and mouse Reactome pathway annotations

2026-10-19 · 108 of 113 automated tests passed
Behaviour checks (12)
  • four asset checksums and counts
  • species-specific gene identifiers
  • download integrity guards
  • identifier coverage
  • species mismatch
  • exact-match version suffix handling
  • actual worker ORA and GSEA
  • source provenance
  • failed-load rollback
  • project restore
  • unmatched background retention
  • UI-to-worker loading/zero-match/ORA/export
Not yet verified (3)
  • rendered target-browser appearance
  • actual target-browser download dialogs
  • new native fgsea pairing execution

Bulk differential-results CSV identifier mapping and validation

2026-10-19 · 105 of 110 automated tests passed
Behaviour checks (15)
  • stable-ID priority
  • optional repeated/missing gene symbols
  • exact duplicate and missing row diagnostics
  • multiple comparisons
  • collision-free pair keys
  • independent per-field length limits
  • live mapping UI
  • stale response protection
  • safe text rendering
  • actual worker file upload
  • DE statistics and NA preservation
  • CSV export
  • snapshot restore
  • failed-import rollback
  • UI-to-worker volcano workflow
Not yet verified (3)
  • user's actual CSV
  • rendered target browser
  • native browser download dialogs

Population subset workspaces and positive gene-program scoring

2026-10-20 · 126 of 131 automated tests passed

Numerical checks

  • Independent actual-R rank(-values,ties.method=average) plus stated formula, 4 cells x 3 signatures x 4 rank cutoffs; absolute tolerance 1e-12
  • Implicit zero ties, tied positive values, rank capping and missing-gene skip policy
  • Per-cell score invariance after cell subsetting with unchanged feature universe
  • Every-gene sparse subset values and original observation IDs match parent
  • Specimen means and cell counts match source values
Workflow checks (11)
  • Reviewed text and uploaded GMT; exact identifiers, duplicate-symbol ambiguity and positive-only guards
  • Unknown logged input, invalid ranks, memory gates and missing genes
  • Coverage confirmation before scoring; scores for all retained cells
  • Graph coexistence and fresh subset refitting; parent label renaming without collisions
  • Input/output hash, finite score and mapping validation on restore
  • JSON.GZ and binary project restore; failed-operation rollback
  • Actual UI handlers and actual worker for coverage, scoring, plots and CSV export
  • Subset receives a separate IndexedDB identity and original project remains intact
  • Current specimen export requires refreshed plot choices
  • Cell-level distributions and descriptive specimen summaries have no condition P values
  • Existing bulk, pathway, imports, single-cell core and 14-tool regression suite
Not yet verified (6)
  • Rendered browser appearance
  • Target-browser file dialogs
  • 25000-cell performance
  • Actual peak memory and planning-estimate accuracy
  • Full R UCell package equivalence beyond the positive-signature rank formula
  • Optional native-R integrations without configured Rscript

Sample-level GSVA/ssGSEA and continuous pathway-score comparisons

2026-10-20

Numerical checks

  • All scores in 21 independent raw/linear/logged GSVA/ssGSEA cases match direct calls at atol 1e-12
  • ssGSEA alpha 0/.25/1 and normalized/unnormalized options
  • All score-model effects/SE/t/P/BH match independent simple, batch and fixed-subject fits at atol 1e-12
  • Reverse contrasts invert effects and retain P
  • Actual authorized loopback expression upload, R calculation, archive and deletion
  • Negative transformed values, exact sample/binary order, identifier/design guards
  • Worker attachment preserves original expression and gene DE results
  • JSON.GZ/binary persistence, output/input hash tamper and stale-library rejection
  • Actual-R DOM workflow for consent, four plot types, display-only regrouping and exports
  • 240 single cells become eight biological pseudobulk columns
  • Bundled mouse annotation metadata survives native JSON roundtrip using only synthetic expression
Not yet verified (6)
  • Rendered target browser
  • Actual browser local-network permission
  • Docker build and OS launchers
  • User files
  • Biological benchmark reproduction
  • Large-data capacity

Bounded browser-local single-cell core preview; partial roadmap phase

2026-10-20 · 119 of 124 automated tests passed

Numerical checks

  • Four tied/untied unequal-size rank-sum cases versus actual R wilcox.test(exact=FALSE,correct=TRUE); absolute P tolerance 1e-7
  • MAD scaling versus actual R mad
  • Sparse Poisson residual variances versus independent dense formula, including zeros
  • All-cell PCA projection versus dense centered/scaled X times fitted loadings
  • Separated clique Louvain labels and known modularity 0.5
  • Independent Euclidean silhouette example
  • BH across all marker comparisons and normalized-linear mean fold changes
  • Every-gene pseudobulk raw sums by biological specimen
Behaviour checks (14)
  • Deterministic seeded graph across resolutions and every retained cell
  • QC fractions/complexity, small/zero-MAD strata and counts-only checks
  • Marker confirmation, all-tied/missing-statistic behavior and unknown logged scale
  • Manual mapping preserves original labels and requires explicit replacement
  • Pseudobulk raw-count/replication/covariate guards
  • Graph/marker/annotation checkpoint, JSON.GZ and binary project restore
  • Corrupted hashes, labels and graph diagnostics reject transactionally
  • Failed operation preserves previous analysis
  • QC/features invalidate dependent owned labels and analyses
  • Actual UI handlers with actual analysis worker
  • Separate derived bulk dataset retains original scRNA IndexedDB record
  • Parent save guard waits for saves and blocks quota failures/missing originals
  • Existing bulk/pathway/import/native-format/toolkit regression suite
  • All bundled graph dependency license notices collected
Not yet verified (5)
  • Rendered browser appearance and target-device interactions
  • 25000-cell speed/memory
  • Browser-specific OPFS/WORKERFS behavior
  • Optional native R integrations without RNA_ATLAS_NATIVE_RSCRIPT
  • Independent experimental scientific benchmark equivalence

Not implemented: doublet detection, ambient correction, Harmony, Leiden, automatic reference annotation, differential abundance, advanced trajectory/communication, full Plot Studio

Browser TF activity from user-uploaded signed networks

2026-10-20

Numerical checks

  • Independent R stats::lm matches all tested scores for counts, linear and signed logged input at absolute tolerance 1e-8
  • Exported expression + record JSON + R script reproduce scores, including CSV-protected identifiers
  • Network sign reversal negates scores; positive weight rescaling preserves scores
  • Constant expression and exact linear fits yield null with explicit reasons
  • All retained single cells scored; zero-only target genes excluded and reported
  • Actual worker preserves raw expression and gene DE results
  • JSON snapshot/binary portable roundtrips, forged coverage/hash rejection and atomic recovery rollback
  • DOM workflow uses pasted and file networks, coverage review and explicit confirmation, four plot families, display-only regrouping and all exports
  • HTML parsing confirms TF page is separate, with four cards; complete suite checks unique IDs and references
Not yet verified (5)
  • Rendered target browser
  • User datasets
  • Biological benchmarking
  • Large-data/device performance
  • Unchanged companion Docker and OS launchers

Biological-sample TF score comparisons

2026-10-20

Numerical checks

  • Every effect, moderated SE, t, P and BH value matches independently constructed R model.matrix / limma fits at absolute tolerance 1e-9
  • Simple, batch plus continuous-covariate and fixed paired-subject designs
  • Reverse contrasts invert effects and retain FDR
  • Exported R contrast code executed with the exported score CSV and reproduces effects
  • No residual variation, cells, confounding, duplicate subject/condition, missing/constant score and saturation guards
Workflow checks (5)
  • Actual browser R runtime in Node, not a model mock
  • DOM TF scoring → design review → sampling confirmation → real fit → plots → CSV/JSON/R exports
  • Worker preserves raw expression and imported gene DE values
  • Model hashes, methods and packages in provenance; JSON and portable binary roundtrip
  • Invalid model recovery rolls back; new score hash invalidates the model
Not yet verified (5)
  • Rendered target browser
  • User datasets
  • Native-R version parity
  • Biological benchmarks
  • Large-data/device performance

Found a discrepancy?

If a result differs from your reference software, write to contact@solveseverything.com with a small synthetic example, the settings you used and the expected value.