SOLVESEVERYTHING
Validation
How we check that the numbers are right. Statistical results are compared with established software — R, limma, SciPy, statsmodels, pingouin, NumPy and the original method packages — at stated tolerances, and the checks run automatically before every update goes live.
65 numerical checks — most against an independent implementation in reference software — across 5 workspaces are listed below, each with the date it was last run. Every record also lists what has not been verified yet, because a validation page that only lists successes is not useful. Synthetic test data are used so results can be checked exactly; biological validation on your own data remains your judgement.
Jump to: Metabolic Integrator · FACS–scRNA Integration · Paper Explorer · Postdoc Toolkit · RNA Atlas
FACS–scRNA Integration (Tool 18): statistics for FlowJo exports linked to single-cell RNA
2026-10-25 · All tests in tests/toolkit-flow.test.mjs pass
Numerical checks
- Quasi-binomial GLM for frequencies with event counts matches statsmodels (Binomial, var_weights = trials, scale = Pearson X²)
- PCA variances and scores match NumPy SVD
- Paired designs reproduce SciPy ttest_rel
- Tie-aware Spearman permutation p values match SciPy permutation_test (exact enumeration for small n)
Biology and behaviour checks (2)
- Marker → gene mapping is species-aware and never confuses CD44hi with CD4 or naive cells
- Synthetic demo recovers planted effects with the intended models; confounded covariates are dropped; negative MFIs use asinh
Not yet verified (2)
- Users' own FlowJo gating and compensation (performed in FlowJo)
- Rendered appearance in every browser
Paper Explorer (Tool 19): rule-based, extractive analysis and comparison of papers
2026-10-25 · All tests in tests/toolkit-paper.test.mjs, tests/toolkit-ui.test.mjs, tests/toolkit-links.test.mjs pass
Extraction and logic checks (4)
- Synthetic JATS and PDF fixtures: sections, citations, statistics, figure references, abbreviations, reagents, accessions and RRIDs are extracted as written
- Results at a glance: knockout, treatment and no-change rows; recipient strains, marker genes and speculative clauses are excluded
- Comparison logic: a loss-of-function result with a readout increased agrees with a treatment that lowers it; regulators are bridged only by a stated relation or a curated enzyme → product link
- Manual checks on open-access eLife, Nature Communications and PNAS articles, including a knockout-versus-treatment pair that agrees once designs are put on one footing
Not yet verified (2)
- Extraction accuracy on every journal format (rules can miss or misread sentences; every output links to its sentence)
- Scanned PDFs without a text layer (not supported)
Browser-only knockout consequence viewer and Tool 12 handoff
2026-10-20
Live annotation checks (5)
- Mouse Acod1 ENSMUST00000022722.7 / ENSMUSP00000022722; WT 488 aa; GRCm39
- Human ACOD1 ENST00000377462; WT 481 aa; GRCh38
- Human reverse-strand HBB ENST00000335295; WT 147 aa; GRCh38
- Human reverse-strand alternative HBB ENST00000647020.1; WT 147 aa; GRCh38
- Expanded lookup + cDNA/CDS/protein/protein-feature endpoints returned successfully; response CORS allowed origin *
Not yet verified (4)
- Rendered target browser and actual browser CORS flow
- Actual PNG rasterization pixels/download in a browser
- Experimental knockout/Cre/NMD outcomes
- Exhaustive annotation variants and large-gene performance
lab dilution
2026-10-20
Behaviour checks (7)
- all eight live calculators
- unit and molecular-weight conversion
- serial dilution retained-volume conservation
- antibody master mix and viability-adjusted cell seeding
- local favorites JSON import/export and storage denial
- invalid results clear and block export
- Tool 5 link and catalog registration
Native single-cell cleanup/integration/Leiden/t-SNE/DPT, specimen composition and Plot Studio
2026-10-20 · 141 of 146 automated tests passed
Numerical checks
- Every per-capture Scrublet observed score and threshold flag matches independent direct package fits, tolerance 1e-12
- Every Harmony corrected coordinate matches independent direct package fit, tolerance 1e-10
- Known separated Leiden graph, independent exact weighted graph partition quality, tolerance 1e-8, and seeded repeatability
- Every TSNE coordinate matches direct sklearn, tolerance 1e-10
- Every diffusion coordinate and DPT pseudotime matches independent Scanpy Gaussian-neighbor/diffmap/DPT fit, tolerance 1e-7
- Specimen counts/fractions preserve technical-capture grouping and zero populations; all tested limma outputs match independent real-R reference fit
- All implemented Plot Studio statistical families match independent actual R stats calls and exported reproduction expressions
- Nested-unit aggregation, matched/repeated completeness, confounding, missingness, exact risk sets/tied event/censoring and descriptive Kaplan-Meier hand cases
- SVG finite geometry, physical width, panel labels and active-content rejection; actual PNG pHYs resolution and CRC replacement
Workflow checks (9)
- Real loopback HTTP token/origin/job security and actual Python job execution
- Explicit browser pairing/review/transfer consent using actual worker and real method packages through test UI transport
- All-cell results, owned metadata, hash/threshold checks, dependent-result invalidation and raw-count preservation
- CSV/JSON/R-code/figure exports and reproducible private local job archives
- Worker/checkpoint/JSON.gz roundtrip, corruption rejection and failed-operation rollback
- RNA gene handoff creates specimen observations; DE handoff preserves model provenance and blocks treating genes as biological replicates
- Plot Studio editable grid, real browser-R model, figure export, saved/reordered figure boards and no raw-table duplication in job ledger
- Companion ZIP integrity and exact source-byte matching, with Python source/requirements included
- Existing bulk/pathway/native-format/persistence/14-tool regressions
Not yet verified (6)
- Updated Docker image build
- Downloaded OS launchers
- Actual desktop-browser loopback/local-network permission
- Rendered browser layout and rasterization/file dialogs
- Large-data and 25k-cell capacity/performance
- Native R integrations skipped in this environment
Local SoupX ambient correction and own-reference CellTypist annotation, alongside upload repairs
2026-10-20 · 159 automated tests
Numerical checks
- Every tested CellTypist query score matches independent package calls at atol 1e-12
- Specimen-disjoint holdout, explicit Unassigned thresholds and zero-library rejection
- No model-download network access during reference training/prediction
- Every SoupX auto/manual corrected CSR pointer/index/value and rho matches independent calls
- All supplied called droplets excluded from ambient pool; mismatched counts rejected
- Real authenticated loopback HTTP count uploads execute CellTypist and SoupX
- Worker attachment, compact provenance, JSON.GZ/binary restore, tamper/stale guards and parent/copy independence
- DOM upload/mapping, consent invalidation and pairing gates
- Existing gzip, browser-runtime, large-CSV and RNA/toolkit regressions
Reference package versions: CellTypist 1.7.1, SoupX 1.6.2, scikit-learn 1.8.0, SciPy 1.17.0, NumPy 2.3.5
Not yet verified (6)
- Rendered target browser
- Actual browser-to-loopback permission
- Docker image build and OS launchers
- User expression/Seurat files
- Large-data capacities
- Biological benchmark reproduction
Not implemented: DecontX, Pretrained public-model discovery, RNA velocity, CellChat/NicheNet, Mixed-effects condition models, Unlimited native Seurat jobs
Bundled human and mouse Reactome pathway annotations
2026-10-19 · 108 of 113 automated tests passed
Behaviour checks (12)
- four asset checksums and counts
- species-specific gene identifiers
- download integrity guards
- identifier coverage
- species mismatch
- exact-match version suffix handling
- actual worker ORA and GSEA
- source provenance
- failed-load rollback
- project restore
- unmatched background retention
- UI-to-worker loading/zero-match/ORA/export
Not yet verified (3)
- rendered target-browser appearance
- actual target-browser download dialogs
- new native fgsea pairing execution
Bulk differential-results CSV identifier mapping and validation
2026-10-19 · 105 of 110 automated tests passed
Behaviour checks (15)
- stable-ID priority
- optional repeated/missing gene symbols
- exact duplicate and missing row diagnostics
- multiple comparisons
- collision-free pair keys
- independent per-field length limits
- live mapping UI
- stale response protection
- safe text rendering
- actual worker file upload
- DE statistics and NA preservation
- CSV export
- snapshot restore
- failed-import rollback
- UI-to-worker volcano workflow
Not yet verified (3)
- user's actual CSV
- rendered target browser
- native browser download dialogs
Population subset workspaces and positive gene-program scoring
2026-10-20 · 126 of 131 automated tests passed
Numerical checks
- Independent actual-R rank(-values,ties.method=average) plus stated formula, 4 cells x 3 signatures x 4 rank cutoffs; absolute tolerance 1e-12
- Implicit zero ties, tied positive values, rank capping and missing-gene skip policy
- Per-cell score invariance after cell subsetting with unchanged feature universe
- Every-gene sparse subset values and original observation IDs match parent
- Specimen means and cell counts match source values
Workflow checks (11)
- Reviewed text and uploaded GMT; exact identifiers, duplicate-symbol ambiguity and positive-only guards
- Unknown logged input, invalid ranks, memory gates and missing genes
- Coverage confirmation before scoring; scores for all retained cells
- Graph coexistence and fresh subset refitting; parent label renaming without collisions
- Input/output hash, finite score and mapping validation on restore
- JSON.GZ and binary project restore; failed-operation rollback
- Actual UI handlers and actual worker for coverage, scoring, plots and CSV export
- Subset receives a separate IndexedDB identity and original project remains intact
- Current specimen export requires refreshed plot choices
- Cell-level distributions and descriptive specimen summaries have no condition P values
- Existing bulk, pathway, imports, single-cell core and 14-tool regression suite
Not yet verified (6)
- Rendered browser appearance
- Target-browser file dialogs
- 25000-cell performance
- Actual peak memory and planning-estimate accuracy
- Full R UCell package equivalence beyond the positive-signature rank formula
- Optional native-R integrations without configured Rscript
Sample-level GSVA/ssGSEA and continuous pathway-score comparisons
2026-10-20
Numerical checks
- All scores in 21 independent raw/linear/logged GSVA/ssGSEA cases match direct calls at atol 1e-12
- ssGSEA alpha 0/.25/1 and normalized/unnormalized options
- All score-model effects/SE/t/P/BH match independent simple, batch and fixed-subject fits at atol 1e-12
- Reverse contrasts invert effects and retain P
- Actual authorized loopback expression upload, R calculation, archive and deletion
- Negative transformed values, exact sample/binary order, identifier/design guards
- Worker attachment preserves original expression and gene DE results
- JSON.GZ/binary persistence, output/input hash tamper and stale-library rejection
- Actual-R DOM workflow for consent, four plot types, display-only regrouping and exports
- 240 single cells become eight biological pseudobulk columns
- Bundled mouse annotation metadata survives native JSON roundtrip using only synthetic expression
Not yet verified (6)
- Rendered target browser
- Actual browser local-network permission
- Docker build and OS launchers
- User files
- Biological benchmark reproduction
- Large-data capacity
Bounded browser-local single-cell core preview; partial roadmap phase
2026-10-20 · 119 of 124 automated tests passed
Numerical checks
- Four tied/untied unequal-size rank-sum cases versus actual R wilcox.test(exact=FALSE,correct=TRUE); absolute P tolerance 1e-7
- MAD scaling versus actual R mad
- Sparse Poisson residual variances versus independent dense formula, including zeros
- All-cell PCA projection versus dense centered/scaled X times fitted loadings
- Separated clique Louvain labels and known modularity 0.5
- Independent Euclidean silhouette example
- BH across all marker comparisons and normalized-linear mean fold changes
- Every-gene pseudobulk raw sums by biological specimen
Behaviour checks (14)
- Deterministic seeded graph across resolutions and every retained cell
- QC fractions/complexity, small/zero-MAD strata and counts-only checks
- Marker confirmation, all-tied/missing-statistic behavior and unknown logged scale
- Manual mapping preserves original labels and requires explicit replacement
- Pseudobulk raw-count/replication/covariate guards
- Graph/marker/annotation checkpoint, JSON.GZ and binary project restore
- Corrupted hashes, labels and graph diagnostics reject transactionally
- Failed operation preserves previous analysis
- QC/features invalidate dependent owned labels and analyses
- Actual UI handlers with actual analysis worker
- Separate derived bulk dataset retains original scRNA IndexedDB record
- Parent save guard waits for saves and blocks quota failures/missing originals
- Existing bulk/pathway/import/native-format/toolkit regression suite
- All bundled graph dependency license notices collected
Not yet verified (5)
- Rendered browser appearance and target-device interactions
- 25000-cell speed/memory
- Browser-specific OPFS/WORKERFS behavior
- Optional native R integrations without RNA_ATLAS_NATIVE_RSCRIPT
- Independent experimental scientific benchmark equivalence
Not implemented: doublet detection, ambient correction, Harmony, Leiden, automatic reference annotation, differential abundance, advanced trajectory/communication, full Plot Studio
Browser TF activity from user-uploaded signed networks
2026-10-20
Numerical checks
- Independent R stats::lm matches all tested scores for counts, linear and signed logged input at absolute tolerance 1e-8
- Exported expression + record JSON + R script reproduce scores, including CSV-protected identifiers
- Network sign reversal negates scores; positive weight rescaling preserves scores
- Constant expression and exact linear fits yield null with explicit reasons
- All retained single cells scored; zero-only target genes excluded and reported
- Actual worker preserves raw expression and gene DE results
- JSON snapshot/binary portable roundtrips, forged coverage/hash rejection and atomic recovery rollback
- DOM workflow uses pasted and file networks, coverage review and explicit confirmation, four plot families, display-only regrouping and all exports
- HTML parsing confirms TF page is separate, with four cards; complete suite checks unique IDs and references
Not yet verified (5)
- Rendered target browser
- User datasets
- Biological benchmarking
- Large-data/device performance
- Unchanged companion Docker and OS launchers
Biological-sample TF score comparisons
2026-10-20
Numerical checks
- Every effect, moderated SE, t, P and BH value matches independently constructed R model.matrix / limma fits at absolute tolerance 1e-9
- Simple, batch plus continuous-covariate and fixed paired-subject designs
- Reverse contrasts invert effects and retain FDR
- Exported R contrast code executed with the exported score CSV and reproduces effects
- No residual variation, cells, confounding, duplicate subject/condition, missing/constant score and saturation guards
Workflow checks (5)
- Actual browser R runtime in Node, not a model mock
- DOM TF scoring → design review → sampling confirmation → real fit → plots → CSV/JSON/R exports
- Worker preserves raw expression and imported gene DE values
- Model hashes, methods and packages in provenance; JSON and portable binary roundtrip
- Invalid model recovery rolls back; new score hash invalidates the model
Not yet verified (5)
- Rendered target browser
- User datasets
- Native-R version parity
- Biological benchmarks
- Large-data/device performance
Found a discrepancy?
If a result differs from your reference software, write to contact@solveseverything.com with a small synthetic example, the settings you used and the expected value.