# -*- coding: utf-8 -*-
"""
TPC-PY --- TFK zone schemas: canonical serialization, per-zone digests,
successor classification, schema validators (TPC step 2, increment 4)
=====================================================================
STATUS: SELECTED D92.1, D98.1, D99.1, D100.1; units D103.1, D104.1, D105.1
(2026-09-22); values VERIFIED in both residences. Standard library only;
Python 3.8+; Windows/Linux.
SCHEMA 0.5.0 (tpc_zone 0.5.0, proposed; D176.1 and D177.1, taken 2026-09-26): one
digest move carries four fields. A LAW carries ROLE, required: EOM(<NAME>),
CONSTRAINT, CONSTITUTIVE, INVARIANT or DEFINITION, spelled exactly so, with no
space; in EOM(<NAME>) the NAME is a DOF of the TPK (T1 refuses otherwise). A DOF may
carry ROLE, whose only value is INPUT (a DOF the kernel takes as given: forcing,
noise). The TPK may carry a PARAMETERS part ahead of the six kinds, an object whose
only field is D_VARIABLES, ONE or SEVERAL; an absent PARAMETERS or D_VARIABLES means
ONE and serializes exactly as ONE. The TMS PARAMETERS may carry STRUCTURE_CHECK, ON or
OFF; absent means ON and serializes exactly as ON (the IC2_POLICY precedent). Every
other consistency of a ROLE is the structure check's finding, not a refusal (D176.3,
S31). T1 also runs tpc_form's D-variable rules (D177.1, tpc_form.d_variable_check):
the second argument s of every D in the TPK's forms is a DOF that no CONSTRAINT
targets, and under ONE every D uses one s. Positions in the preimage, by the existing
conventions (required fields in their listed order, then the optional ones): LAW
NAME, STATEMENT, REFS, ROLE, EQUATION; DOF NAME, DIMENSION, ROLE; the TPK part
PARAMETERS after the schema tag and before the kinds, as ["PARAMETERS",
[D_VARIABLES, <value>]], the TMS PARAMETERS' shape; STRUCTURE_CHECK after
PROMOTION_FLOOR. The schema version in every preimage moved from 0.4.0 to 0.5.0, so
every zone digest moved (fixtures only; no UWC exists).
S15 (tpc_zone 0.4.1, proposed; schema 0.4.0 unchanged): a DIMENSION's
repeated bases add (TPS 2.5.12); a sum outside -9999..9999 would give a
canonical token the grammar refuses ("L9999 L1" -> "L10000"), so such a
DIMENSION is refused - "DIMENSION <D>: an exponent is out of range", as
tpc_units refuses a UNIT - never altered (D99, D103). The canonical token
test is bounded to four digits alike. Every zone S15 admits serializes as
before, so no admitted zone's digest moves and the schema tag stays.
SCHEMA 0.4.0 (D108.1, increment 8, tpm_increment_8 0.1.0): the OPTIONAL
declared form. A LAW may carry EQUATION and a BOUNDARY may carry
CONDITION beside its prose; a SIGNATURE may carry FORM and IMAGE_FORM
(both or neither) beside IMAGE. Each parses under the closed grammar of
tpc_form and enters the preimage in its canonical spelling, so two
spellings of one form digest alike; only spelling is canonical ("x*2"
and "2*x" are different statements). A form that does not parse, or
that names what its REFS (OVER) does not list, or that leaves a REFS
(OVER) member unnamed, is refused at zone entry, never altered. An
entry without a form serializes as it did under 0.3.0; the schema
version in every preimage moved from 0.3.0 to 0.4.0, so every zone
digest moved once more (fixtures only; no UWC exists).
SCHEMA 0.3.0 (D104.1, tpm_zone_schemas 0.3.0): UNIT is a parsed field.
A TPK CONSTANT's UNIT, and the UNIT a CONSTRAINT with ACTION FIX now
requires, parse under the unit grammar of tpc_units (D103.1) and are
commensurable with the DIMENSION of the CONSTANT or of the TARGET DOF;
REMOVE forbids UNIT. The preimage carries the unit's canonical spelling,
so two spellings of one unit digest alike; only spelling is canonical
("5 cm" and "0.05 m" are different statements). The schema version in
every preimage moved from 0.2.0 to 0.3.0, so every zone digest moved.
WHAT IT FIXES (D98.1, D99.1, D100.1, D92.1):
Zone schemas for the three TFK zones (TPS 2.1-2.4; guide 3.1, 4.2, 4.3).
TPK kinds DOF, CONSTANT, LAW, ASSUMPTION, CONSTRAINT, BOUNDARY, each
with its required fields. LAW, ASSUMPTION and BOUNDARY carry
REFS; CONSTRAINT carries TARGET. Every citation is explicit, so
T1's resolution clause and T5 (a), (c) are mechanical (D98).
Schema 0.5.0: a PARAMETERS part (D_VARIABLES) ahead of the kinds;
ROLE on LAW (required) and on DOF (INPUT, optional).
TrM HEADER (ENVIRONMENT_INVENTORY, SIGNATURE, TRANSFORMATIONS), then
ENTRIES with the identification fields, DIMENSION included
(TPS 2.3.11; guide 4.3; PMP-TPM VOCABULARY).
TMS PARAMETERS (STEP_CONVENTION, K, N_X, S_MAX; IC2_POLICY optional,
GAP when absent, per TPS 2.3.7; PROMOTION_FLOOR optional, F2 when
absent, per D92.1; STRUCTURE_CHECK optional, ON when absent, per
D176.1), named CONSTANTS (SR9 C5), YARDSTICKS in the closed
ordering form (D100).
Canonical serialization (D99), on the ratified D97.1 primitive:
- a type-tagged recursive netstring: string -> "s"+bytes,
integer -> "i"+base-10, list -> "l"+enc(children). The tags make the
encoding injective over every tree, validated or not;
- each zone opens with its zone tag and the schema version;
- kinds in fixed order; entries sorted by NAME (ENVIRONMENT_QUANTITY for
the environment inventory, DOMAIN for limiting cases); set-valued fields
(REFS, OVER, BEARS) sorted; a yardstick KEY keeps its order;
- NORMALIZED, NOT REFUSED: line endings to LF and text to Unicode
NFC, as MediaWiki stores them; DIMENSION to its
canonical base-exponent form; decimals (VALUE, UNCERTAINTY, K) to
their canonical mantissa-exponent form; yardstick terms to their
canonical spelling; an absent IC2_POLICY is serialized as GAP, an
absent PROMOTION_FLOOR as F2, an absent STRUCTURE_CHECK as ON and an
absent TPK D_VARIABLES as ONE. Two spellings of one meaning digest
alike. The validators refuse what cannot be parsed and what cannot
be seen.
TEXT SAFETY (tpm_zone_schemas 0.2.0 section 6): every string is refused
if, after line-ending normalization, it holds a code point of Unicode
general category C (control, format, surrogate, private use,
unassigned) other than TAB and LF, or U+2028 / U+2029. What a human
inspects is then what is stored and digested: no bidi override, no
zero-width or tag character, no control. Problem messages escape such
code points as <U+XXXX>. Validators never raise on malformed input.
Digests are SHA-256, lowercase hex. D(TFK) is the ratified
combine_digests([D(TPK), D(TrM), D(TMS)]) (D97.1, unchanged).
DIGESTS ARE DEFINED OVER WELL-FORMED ZONES ONLY. TPX runs the validators
first; a zone with a problem is returned to its author (TPS 5.5). An unknown
or mutable field is a PROBLEM, never silently dropped: content outside the
digest would be content the chain does not cover.
"""
import hashlib
import re
import unicodedata
import tpc_digest as G
def _units():
"""tpc_units, imported on first use: it imports this module's helpers."""
import tpc_units
return tpc_units
def _form():
"""tpc_form, imported on first use: it imports this module's helpers."""
import tpc_form
return tpc_form
TPC_ZONE_VERSION = "0.5.1" # proposed PATCH with TPC-PY 0.2.0 (C-MS-5: TPC-PHP reads the JSON as written, a {} apart from a []); schema 0.5.0
SCHEMA_VERSION = "0.5.0"
TPK_KINDS = ["DOF", "CONSTANT", "LAW", "ASSUMPTION", "CONSTRAINT", "BOUNDARY"]
TPK_FIELDS = {
"DOF": (["NAME", "DIMENSION"], ["ROLE"]),
"CONSTANT": (["NAME", "DIMENSION"], ["VALUE", "UNIT", "UNCERTAINTY"]),
"LAW": (["NAME", "STATEMENT", "REFS", "ROLE"], ["EQUATION"]),
"ASSUMPTION": (["NAME", "PROPOSITION", "REFS"], []),
"CONSTRAINT": (["NAME", "TARGET", "ACTION", "STATEMENT"], ["VALUE", "UNIT"]),
"BOUNDARY": (["NAME", "PREDICATE", "REFS"], ["CONDITION"]),
}
TPK_PARTS = ["PARAMETERS"] # schema 0.5.0: ahead of the six kinds
TPK_PARAM_FIELDS = ([], ["D_VARIABLES"])
D_VARIABLES_VALUES = ["ONE", "SEVERAL"] # D177.1
D_VARIABLES_DEFAULT = "ONE" # absent serializes as ONE
DOF_ROLES = ["INPUT"] # D176.1: a DOF the kernel takes as given
LAW_ROLE_WORDS = ["CONSTRAINT", "CONSTITUTIVE", "INVARIANT", "DEFINITION"] # and EOM(<NAME>)
ROLE_GRAMMAR = "is not EOM(<DOF>), CONSTRAINT, CONSTITUTIVE, INVARIANT or DEFINITION"
TRM_PARTS = ["HEADER", "ENTRIES"]
TRM_HEADER = ["ENVIRONMENT_INVENTORY", "SIGNATURE", "TRANSFORMATIONS"]
ENV_FIELDS = (["ENVIRONMENT_QUANTITY", "DIMENSION"], [])
SIG_FIELDS = (["NAME", "OVER", "IMAGE"], ["FORM", "IMAGE_FORM"])
TRF_FIELDS = (["NAME", "STATEMENT"], [])
ENTRY_FIELDS = (["NAME", "DOF", "DOMAIN", "ENVIRONMENT_QUANTITY", "IDENTIFICATION",
"FALSIFIER", "FALSIFIER_CLASS", "BEARS", "DIMENSION"], [])
TMS_PARTS = ["PARAMETERS", "CONSTANTS", "YARDSTICKS"]
PARAM_FIELDS = (["STEP_CONVENTION", "K", "N_X", "S_MAX"], ["IC2_POLICY", "PROMOTION_FLOOR", "STRUCTURE_CHECK"])
IC2_DEFAULT = "GAP" # TPS 2.3.7: a TMS MAY tighten to REJECT
PROMOTION_DEFAULT = "F2" # D92.1: F2 where the UWC does not say otherwise
STRUCTURE_DEFAULT = "ON" # D176.1: ON when absent, serialized ON
STRUCTURE_VALUES = ["ON", "OFF"]
INT_MAX = 1000000 # N_X, S_MAX: both residences hold it natively
TMSC_FIELDS = (["NAME", "VALUE"], [])
YARD_FIELDS = (["NAME", "KEY", "LIMITING"], [])
LIM_FIELDS = (["DOMAIN", "KEY"], [])
SET_FIELDS = {"REFS", "OVER", "BEARS"} # sorted in serialization
SEQ_FIELDS = {"KEY"} # order kept; terms canonicalized
OBJ_LIST_FIELDS = {"LIMITING"}
INT_FIELDS = {"N_X", "S_MAX"}
DIM_FIELDS = {"DIMENSION"} # normalized
DEC_FIELDS = {"VALUE", "UNCERTAINTY", "K"} # normalized
UNIT_FIELDS = {"UNIT"} # parsed; canonical spelling (D104.1)
FORM_FIELDS = {"EQUATION", "CONDITION", "FORM", "IMAGE_FORM"} # parsed; canonical spelling (D108.1)
MUTABLE = ["STATE", "SCRUTINY", "RIDER", "NOTE", "CHECKS_OWED", "RECORDS"] # TPS 2.1.3
FALSIFIER_CLASSES = ["F1", "F2", "F3"] # F0 inadmissible (guide C4)
DIM_BASES = ["L", "M", "T", "I", "TH", "N", "J"]
# \Z, not $: "$" also matches before a trailing newline. PHP mirrors with \z.
# Exponents carry at most four digits, so both residences parse them as
# native integers without overflow.
_DIM_TOKEN = re.compile(r"^(TH|L|M|T|I|N|J)(-?[0-9]{1,4})\Z")
_DIM_CANON_TOKEN = re.compile(r"^(TH|L|M|T|I|N|J)(-?[1-9][0-9]{0,3})\Z") # S15: four digits, as _DIM_TOKEN
DIM_EXP_MAX = 9999 # S15: a canonical exponent stays inside the grammar (D99, D103)
_DECIMAL = re.compile(r"^(-?)([0-9]+)(?:\.([0-9]+))?(?:[eE]([-+]?[0-9]{1,4}))?\Z")
_IDENT = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*\Z")
_EOM_ROLE = re.compile(r"^EOM\(([A-Za-z_][A-Za-z0-9_]*)\)\Z") # D176.1: EOM(<NAME>), no space
# Unicode White_Space (PropList.txt), listed so both residences agree exactly:
# str.strip() and PHP trim() use different sets and would diverge on NBSP.
_WS = set(chr(c) for c in list(range(0x09, 0x0E)) + [0x20, 0x85, 0xA0, 0x1680]
+ list(range(0x2000, 0x200B)) + [0x2028, 0x2029, 0x202F, 0x205F, 0x3000])
# ---- canonical forms -------------------------------------------------------
def _eol(s):
"""Line endings as MediaWiki stores them: CRLF and lone CR become LF."""
return s.replace("\r\n", "\n").replace("\r", "\n")
def _text(s):
"""The canonical text of a string: LF line endings, then NFC."""
return unicodedata.normalize("NFC", _eol(s))
def _disallowed(ch):
"""General category C except TAB and LF, plus U+2028 and U+2029."""
if ch in ("\t", "\n"):
return False
if ch in ("
", "
"):
return True
return unicodedata.category(ch)[0] == "C"
def _first_bad(s):
"""The first disallowed code point of s after line-ending
normalization, as U+XXXX, or None."""
for ch in _eol(s):
if _disallowed(ch):
return "U+%04X" % ord(ch)
return None
def _show(msg):
"""A problem message with every disallowed code point escaped."""
return "".join("<U+%04X>" % ord(c) if _disallowed(c) else c for c in msg)
def _blank(s):
"""True iff s is empty or every code point is Unicode White_Space."""
return all(c in _WS for c in s)
def _is_ident(s):
return isinstance(s, str) and bool(_IDENT.match(s))
def canon_dimension(exps):
"""Canonical DIMENSION string from {base: exponent}; zero exponents drop."""
toks = [b + str(int(exps[b])) for b in DIM_BASES if b in exps and int(exps[b]) != 0]
return " ".join(toks) if toks else "1"
def _dim_sums(s):
"""{base: summed exponent} of a DIMENSION's tokens, {} for "1", or None
when a token does not parse."""
if not isinstance(s, str):
return None
if s == "1":
return {}
exps = {}
for tok in s.split(" "):
m = _DIM_TOKEN.match(tok)
if not m:
return None
exps[m.group(1)] = exps.get(m.group(1), 0) + int(m.group(2))
return exps
def dimension_out_of_range(s):
"""S15: True iff every token of s parses but a base's summed exponent lies
outside -DIM_EXP_MAX..DIM_EXP_MAX, so its canonical form would leave the
grammar."""
e = _dim_sums(s)
return e is not None and any(abs(v) > DIM_EXP_MAX for v in e.values())
def normalize_dimension(s):
"""Canonical form of a DIMENSION, or None if it does not parse.
Accepted: "1", or tokens <BASE><exponent> over L M T I TH N J separated
by single spaces, in any order; a base may repeat (exponents add); an
exponent may be zero. Canonical: bases in that order, each once, zero
exponents dropped, "1" when none remain. S15: a sum outside
-9999..9999 would canonicalize outside the grammar, so the DIMENSION is
refused (None), never altered (D99)."""
e = _dim_sums(s)
if e is None or any(abs(v) > DIM_EXP_MAX for v in e.values()):
return None
return canon_dimension(e)
def is_canonical_dimension(s):
"""True iff s is already in canonical form."""
if not isinstance(s, str):
return False
if s == "1":
return True
last = -1
for tok in s.split(" "):
m = _DIM_CANON_TOKEN.match(tok)
if not m:
return False
idx = DIM_BASES.index(m.group(1))
if idx <= last:
return False
last = idx
return True
def canon_decimal(s):
"""Canonical form of a decimal string, or None if it does not parse.
Accepted: -?digits(.digits)?([eE][-+]?d{1,4})?. Canonical: "0" for zero;
otherwise [-]M with "e"E appended when E is not 0, M an integer without
leading or trailing zeros: "0.050" -> "5e-2", "1500" -> "15e2",
"-0" -> "0". Exact: string arithmetic only, no floating point. Trailing
zeros carry no precision here; UNCERTAINTY carries it."""
if not isinstance(s, str):
return None
m = _DECIMAL.match(s)
if not m:
return None
sign, ip, fp, ex = m.group(1), m.group(2), m.group(3) or "", m.group(4)
digits = (ip + fp).lstrip("0")
if digits == "":
return "0"
e = (int(ex) if ex else 0) - len(fp)
t = len(digits) - len(digits.rstrip("0"))
digits = digits[:len(digits) - t]
e += t
return sign + digits + ("e" + str(e) if e != 0 else "")
def _dec_sign(s):
"""-1, 0 or 1 for a decimal string; 0 if it does not parse."""
c = canon_decimal(s)
if c is None or c == "0":
return 0
return -1 if c.startswith("-") else 1
def _is_pos_finite(s):
"""A positive decimal that is a positive finite double: SR3 uses k as a
double, so 1e-400 (-> 0.0) and 1e400 (-> inf) are refused rather than
silently changing what SR3 computes."""
if _dec_sign(s) <= 0:
return False
f = float(s)
return f > 0 and f != float("inf")
# ---- the closed yardstick term grammar (D100) --------------------------------
# term := SR1 | A | S | FIELD:<ident> | NEG:<term> | WSUM:<wt>(;<wt>)*
# wt := <constant>*<atom> atom := A | S | FIELD:<ident>
# A KEY is a list of terms compared lexicographically, lower is closer, and
# MUST begin with SR1. A WSUM names each atom at most once. Weights are TMS
# CONSTANTS by name; there are no literal numbers (SR9 C5). SR1 is 0 for a
# record passing SR1 and 1 for one failing it; A is a(r); S is s(r).
def _parse_atom(a):
if a in ("A", "S"):
return True
return a.startswith("FIELD:") and _is_ident(a[6:])
def _parse_term(t):
"""(well_formed, uses_sr1, constants, canonical, a_contributions).
a_contributions lists (sign, weight constant or None) for each place A
enters the term; _a_sign turns it into the sign of A's coefficient.
NEG prefixes are counted, not recursed into, so no input depth can
exhaust the stack; an even count cancels (NEG:NEG:x is x)."""
n, base = 0, t
while base.startswith("NEG:"):
base = base[4:]
n += 1
ok, sr1, cs, c, contrib = _parse_base(base)
if not ok:
return False, False, [], t, []
if n % 2:
return True, sr1, cs, "NEG:" + c, [(-s, k) for s, k in contrib]
return True, sr1, cs, c, contrib
def _parse_base(t):
"""A term that does not begin with NEG:."""
if t in ("SR1", "A", "S"):
return True, t == "SR1", [], t, ([(1, None)] if t == "A" else [])
if t.startswith("FIELD:"):
return _is_ident(t[6:]), False, [], t, []
if t.startswith("WSUM:"):
wts, cs, contrib, atoms = [], [], [], set()
for wt in t[5:].split(";"):
i = wt.find("*")
if i <= 0:
return False, False, [], t, []
c, a = wt[:i], wt[i + 1:]
if not _is_ident(c) or not _parse_atom(a) or a in atoms:
return False, False, [], t, []
atoms.add(a)
wts.append((a, c))
cs.append(c)
if a == "A":
contrib.append((1, c))
canon = "WSUM:" + ";".join(c + "*" + a for a, c in sorted(wts))
return True, False, cs, canon, contrib
return False, False, [], t, []
def canonical_term(t):
"""Canonical spelling of a well-formed term (WSUM weights sorted by atom,
double negation removed); a malformed term is returned unchanged."""
return _parse_term(t)[3]
def _a_sign(contrib, consts):
"""Sign of the coefficient of A in a term. A WSUM names A at most once
and NEG only flips, so each term has at most one contribution and its
sign is exact without arithmetic."""
total = 0
for s, k in contrib:
total += s * (1 if k is None else _dec_sign(consts.get(k, "0")))
return (total > 0) - (total < 0)
# ---- canonical serialization (D99) ------------------------------------------
def _node(x):
if isinstance(x, bool):
raise TypeError("zone serialization: bool is not a permitted value")
if isinstance(x, str):
return b"s" + _text(x).encode("utf-8")
if isinstance(x, int):
return b"i" + str(x).encode("ascii")
if isinstance(x, list):
return b"l" + G._enc([_node(c) for c in x])
raise TypeError("zone serialization: unsupported value %r" % type(x))
def _entry_list(e, fields):
out = []
for f in fields:
if f in e:
v = e[f]
if f in SET_FIELDS:
v = sorted(_text(x) for x in v)
elif f in SEQ_FIELDS:
v = [canonical_term(_text(x)) for x in v]
elif f in OBJ_LIST_FIELDS:
v = [_entry_list(o, LIM_FIELDS[0]) for o in
sorted(v, key=lambda o: _text(o["DOMAIN"]))]
elif f in DIM_FIELDS:
v = normalize_dimension(v)
elif f in DEC_FIELDS:
v = canon_decimal(v)
elif f in UNIT_FIELDS:
r = _units().unit_parse(v)
v = r[1] if r[0] == "OK" else None
elif f in FORM_FIELDS:
r = _form().parse_form(v, _form().MODE_OF_FIELD[f])
v = r[1] if r[0] == "OK" else None
out += [f, v]
return out
def _sorted_entries(lst, fields, keyfield="NAME"):
return [_entry_list(e, fields) for e in sorted(lst, key=lambda e: _text(e[keyfield]))]
def _tree_tpk(tpk):
pm = dict(tpk.get("PARAMETERS", {}))
pm.setdefault("D_VARIABLES", D_VARIABLES_DEFAULT) # absent serializes as ONE (D177.1)
t = ["TPK", SCHEMA_VERSION, ["PARAMETERS", _entry_list(pm, TPK_PARAM_FIELDS[1])]]
for kind in TPK_KINDS:
req, opt = TPK_FIELDS[kind]
t.append([kind, _sorted_entries(tpk.get(kind, []), req + opt)])
return t
def _tree_trm(trm):
h = trm["HEADER"]
return ["TrM", SCHEMA_VERSION,
["HEADER",
["ENVIRONMENT_INVENTORY", _sorted_entries(h["ENVIRONMENT_INVENTORY"], ENV_FIELDS[0],
"ENVIRONMENT_QUANTITY")],
["SIGNATURE", _sorted_entries(h["SIGNATURE"], SIG_FIELDS[0] + SIG_FIELDS[1])],
["TRANSFORMATIONS", _sorted_entries(h["TRANSFORMATIONS"], TRF_FIELDS[0])]],
["ENTRIES", _sorted_entries(trm["ENTRIES"], ENTRY_FIELDS[0])]]
def _tree_tms(tms):
pm = dict(tms["PARAMETERS"])
pm.setdefault("IC2_POLICY", IC2_DEFAULT)
pm.setdefault("PROMOTION_FLOOR", PROMOTION_DEFAULT)
pm.setdefault("STRUCTURE_CHECK", STRUCTURE_DEFAULT) # absent serializes as ON (D176.1)
return ["TMS", SCHEMA_VERSION,
["PARAMETERS", _entry_list(pm, PARAM_FIELDS[0] + PARAM_FIELDS[1])],
["CONSTANTS", _sorted_entries(tms.get("CONSTANTS", []), TMSC_FIELDS[0])],
["YARDSTICKS", _sorted_entries(tms.get("YARDSTICKS", []), YARD_FIELDS[0])]]
def preimage_tpk(tpk):
return _node(_tree_tpk(tpk))
def preimage_trm(trm):
return _node(_tree_trm(trm))
def preimage_tms(tms):
return _node(_tree_tms(tms))
def zone_preimage(zone, value):
"""The exact preimage of one zone, as UTF-8 text (for audit and vectors)."""
f = {"TPK": preimage_tpk, "TrM": preimage_trm, "TMS": preimage_tms}[zone]
return f(value).decode("utf-8")
def d_tpk(tpk):
return hashlib.sha256(preimage_tpk(tpk)).hexdigest()
def d_trm(trm):
return hashlib.sha256(preimage_trm(trm)).hexdigest()
def d_tms(tms):
return hashlib.sha256(preimage_tms(tms)).hexdigest()
def d_tfk_of(tfk):
"""D(TFK) of a whole TFK {"TPK":..., "TrM":..., "TMS":...} (D97.1 rule)."""
return G.combine_digests([d_tpk(tfk["TPK"]), d_trm(tfk["TrM"]), d_tms(tfk["TMS"])])
def classify_successor(a, b):
"""TPS 2.1.5: IDENTICAL; YARDSTICK_ONLY (D(TPK), D(TrM) equal, D(TMS)
moved); CORRESPONDENCE_CHANGED (D(TPK) equal, D(TrM) moved);
KERNEL_CHANGED (D(TPK) moved)."""
if d_tpk(a["TPK"]) != d_tpk(b["TPK"]):
return "KERNEL_CHANGED"
if d_trm(a["TrM"]) != d_trm(b["TrM"]):
return "CORRESPONDENCE_CHANGED"
if d_tms(a["TMS"]) != d_tms(b["TMS"]):
return "YARDSTICK_ONLY"
return "IDENTICAL"
# ---- validators ---------------------------------------------------------------
# Every validator returns [pass, sorted de-duplicated problems] and never
# raises on malformed input: a problem is returned to the author (TPS 5.5).
def _obj(x):
return isinstance(x, dict)
def _list(x):
return isinstance(x, list)
def _entries(label, x, probs):
"""The object entries of a list-valued part; problems for the rest."""
if not _list(x):
probs.append(label + " is not a list")
return []
out = []
for e in x:
if _obj(e):
out.append(e)
else:
probs.append(label + ": entry is not an object")
return out
def _type_ok(f, v):
if f in SET_FIELDS or f in SEQ_FIELDS:
return _list(v) and all(isinstance(x, str) for x in v)
if f in OBJ_LIST_FIELDS:
return _list(v) and all(_obj(x) for x in v)
if f in INT_FIELDS:
return isinstance(v, int) and not isinstance(v, bool)
return isinstance(v, str)
def _check_fields(tag, e, req, opt, probs):
allowed = set(req) | set(opt)
for f in sorted(e):
if f in allowed:
continue
if f in MUTABLE:
probs.append(tag + ": mutable field " + f + " is not permitted in the TFK")
else:
probs.append(tag + ": unknown field " + f)
for f in req:
if f not in e:
probs.append(tag + ": missing field " + f)
for f in req + opt:
if f not in e:
continue
v = e[f]
if not _type_ok(f, v):
probs.append(tag + ": field " + f + " has the wrong type")
continue
strs = [v] if isinstance(v, str) else (v if f in SET_FIELDS or f in SEQ_FIELDS else [])
for x in strs: # first disallowed code point, in order
bad = _first_bad(x)
if bad is not None:
probs.append(tag + ": field " + f + " contains disallowed character " + bad)
break
if f in SET_FIELDS and len(set(v)) != len(v):
probs.append(tag + ": field " + f + " has duplicate entries")
elif f in DIM_FIELDS and normalize_dimension(v) is None:
if dimension_out_of_range(v): # S15
probs.append(tag + ": DIMENSION " + v + ": an exponent is out of range")
else:
probs.append(tag + ": DIMENSION " + v + " is not a dimension")
elif f in DEC_FIELDS and canon_decimal(v) is None:
probs.append(tag + ": " + f + " " + v + " is not a decimal")
elif f in UNIT_FIELDS:
r = _units().unit_parse(v)
if r[0] != "OK":
probs.append(tag + ": " + r[1])
elif f in FORM_FIELDS:
r = _form().parse_form(v, _form().MODE_OF_FIELD[f])
if r[0] != "OK":
probs.append(tag + ": " + f + " does not parse: " + r[1])
if "NAME" in req and isinstance(e.get("NAME"), str) and not _is_ident(e["NAME"]):
probs.append(tag + ": NAME " + e["NAME"] + " is not an identifier")
def _name(e, field="NAME"):
v = e.get(field)
return v if isinstance(v, str) else "?"
def _dupes(names):
seen, d = set(), set()
for n in names:
if n is None:
continue
if n in seen:
d.add(n)
seen.add(n)
return sorted(d)
def _names(entries, field="NAME", nfc=False):
out = []
for e in entries:
v = e.get(field)
out.append((_text(v) if nfc else v) if isinstance(v, str) else None)
return out
def _done(probs):
probs = sorted(set(_show(p) for p in probs))
return [not probs, probs]
def t1_well_formed(tpk):
"""Kernel test T1 WELL-FORMEDNESS (guide 3.6) under the D98 TPK schema:
every entry carries its kind's fields and no others; every NAME is an
identifier and unique across the kernel; every citation resolves to the
kind it must; DOF and CONSTANT carry a DIMENSION that parses; decimals
parse; no entry cites itself; a declared form (EQUATION, CONDITION)
parses and names exactly the REFS members (D108.1, S5: both
directions). Schema 0.5.0: the PARAMETERS part carries D_VARIABLES ONE or
SEVERAL only; a DOF's ROLE is INPUT; a LAW's ROLE, required, is in its closed
grammar and EOM(<NAME>) names a DOF; the D-variable rules of D177.1
(tpc_form.d_variable_check). [pass, sorted problems]."""
if not _obj(tpk):
return _done(["TPK: zone is not an object"])
probs = []
for k in sorted(tpk):
if k not in TPK_KINDS and k not in TPK_PARTS:
probs.append("TPK: unknown kind " + k)
if "PARAMETERS" in tpk:
pm = tpk["PARAMETERS"]
if not _obj(pm):
probs.append("TPK: PARAMETERS is not an object")
else:
_check_fields("TPK PARAMETERS", pm, TPK_PARAM_FIELDS[0], TPK_PARAM_FIELDS[1], probs)
dv = pm.get("D_VARIABLES")
if isinstance(dv, str) and dv not in D_VARIABLES_VALUES:
probs.append("TPK PARAMETERS: D_VARIABLES " + dv + " is not ONE or SEVERAL")
entries = []
for kind in TPK_KINDS:
req, opt = TPK_FIELDS[kind]
for e in _entries("TPK kind " + kind, tpk.get(kind, []), probs):
tag = kind + ":" + _name(e)
_check_fields(tag, e, req, opt, probs)
entries.append((kind, _name(e), tag, e))
for n in _dupes(_names([e for _, _, _, e in entries])):
probs.append("TPK: duplicate name " + n)
kind_of, dim_of = {}, {}
for kind, n, _, e in entries:
if isinstance(e.get("NAME"), str) and n not in kind_of:
kind_of[n] = kind
if isinstance(e.get("DIMENSION"), str):
dim_of[n] = normalize_dimension(e["DIMENSION"])
for kind, n, tag, e in entries:
role = e.get("ROLE")
if kind == "DOF" and isinstance(role, str) and role not in DOF_ROLES:
probs.append(tag + ": ROLE " + role + " is not INPUT")
if kind == "LAW" and isinstance(role, str) and role not in LAW_ROLE_WORDS:
m = _EOM_ROLE.match(role)
if not m:
probs.append(tag + ": ROLE " + role + " " + ROLE_GRAMMAR)
elif kind_of.get(m.group(1)) != "DOF":
probs.append(tag + ": ROLE " + role + " names " + m.group(1) + ", which is not a DOF")
refs = e.get("REFS")
if kind in ("LAW", "ASSUMPTION", "BOUNDARY") and refs is not None and _type_ok("REFS", refs):
for r in sorted(set(refs)):
if r == n:
probs.append(tag + ": cites itself")
elif r not in kind_of:
probs.append(tag + ": REFS " + r + " does not resolve")
elif kind_of[r] not in ("DOF", "CONSTANT"):
probs.append(tag + ": REFS " + r + " is a " + kind_of[r] + ", not a DOF or CONSTANT")
field = "EQUATION" if kind == "LAW" else ("CONDITION" if kind == "BOUNDARY" else None)
f = e.get(field) if field else None
if isinstance(f, str) and _first_bad(f) is None:
r = _form().form_names(f, _form().MODE_OF_FIELD[field])
if r[0] == "OK":
named, rs = set(r[1]), set(refs)
for x in sorted(named - rs):
probs.append(tag + ": " + field + " names " + x + ", which is not in REFS")
for x in sorted(rs - named):
probs.append(tag + ": REFS " + x + " is not named in " + field)
if kind == "CONSTRAINT":
t = e.get("TARGET")
if isinstance(t, str):
if t == n:
probs.append(tag + ": cites itself")
elif kind_of.get(t) != "DOF":
probs.append(tag + ": TARGET " + t + " is not a DOF")
a = e.get("ACTION")
if isinstance(a, str):
if a not in ("REMOVE", "FIX"):
probs.append(tag + ": ACTION " + a + " is not REMOVE or FIX")
else:
if a == "FIX" and "VALUE" not in e:
probs.append(tag + ": FIX requires VALUE")
if a == "FIX" and "UNIT" not in e:
probs.append(tag + ": FIX requires UNIT")
if a == "REMOVE" and "VALUE" in e:
probs.append(tag + ": REMOVE forbids VALUE")
if a == "REMOVE" and "UNIT" in e:
probs.append(tag + ": REMOVE forbids UNIT")
if a == "FIX" and isinstance(t, str) and kind_of.get(t) == "DOF" and t != n:
_unit_against(tag, e.get("UNIT"), dim_of.get(t), " of TARGET " + t, probs)
if kind == "CONSTANT":
if "VALUE" in e and "UNIT" not in e:
probs.append(tag + ": VALUE requires UNIT")
if "UNIT" in e and "VALUE" not in e:
probs.append(tag + ": UNIT requires VALUE")
if "UNCERTAINTY" in e and "VALUE" not in e:
probs.append(tag + ": UNCERTAINTY requires VALUE")
u = e.get("UNCERTAINTY")
if isinstance(u, str) and _dec_sign(u) < 0:
probs.append(tag + ": UNCERTAINTY " + u + " is negative")
d = e.get("DIMENSION")
_unit_against(tag, e.get("UNIT"), normalize_dimension(d) if isinstance(d, str) else None,
"", probs)
probs += _form().d_variable_check(tpk)[1] # D177.1: the D variables
return _done(probs)
def _unit_against(tag, unit, dimension, whose, probs):
"""C-32 at zone entry (D104.1): a UNIT that parses must be commensurable
with the DIMENSION it states a value of. Silent when either does not
parse: that is reported once, where it is found."""
if not isinstance(unit, str) or dimension is None:
return
r = _units().unit_parse(unit)
if r[0] == "OK" and r[2] != dimension:
probs.append(tag + ": UNIT " + unit + " has dimension " + r[2] + ", not " + dimension + whose)
def _tpk_names(tpk, kinds):
out = set()
if not _obj(tpk):
return out
for kind in kinds:
lst = tpk.get(kind, [])
if _list(lst):
for e in lst:
if _obj(e) and isinstance(e.get("NAME"), str):
out.add(e["NAME"])
return out
def trm_well_formed(trm, tpk):
"""TrM schema check against its TPK (structural only; the isomorphism
checks IC1-IC5 are separate). [pass, sorted problems]."""
if not _obj(trm):
return _done(["TrM: zone is not an object"])
probs = []
for p in sorted(trm):
if p not in TRM_PARTS:
probs.append("TrM: unknown part " + p)
for p in TRM_PARTS:
if p not in trm:
probs.append("TrM: missing part " + p)
dofs = _tpk_names(tpk, ["DOF"])
bearable = _tpk_names(tpk, ["LAW", "CONSTRAINT"])
quantities = set()
h = trm.get("HEADER", {})
if not _obj(h):
probs.append("TrM: HEADER is not an object")
else:
for p in sorted(h):
if p not in TRM_HEADER:
probs.append("TrM HEADER: unknown part " + p)
for p in TRM_HEADER:
if p not in h:
probs.append("TrM HEADER: missing part " + p)
env = _entries("TrM ENVIRONMENT_INVENTORY", h.get("ENVIRONMENT_INVENTORY", []), probs)
for e in env:
tag = "ENV:" + _name(e, "ENVIRONMENT_QUANTITY")
_check_fields(tag, e, ENV_FIELDS[0], ENV_FIELDS[1], probs)
q = e.get("ENVIRONMENT_QUANTITY")
if isinstance(q, str) and not _is_ident(q):
probs.append(tag + ": ENVIRONMENT_QUANTITY " + q + " is not an identifier")
for n in _dupes(_names(env, "ENVIRONMENT_QUANTITY")):
probs.append("TrM: duplicate environment quantity " + n)
quantities = set(n for n in _names(env, "ENVIRONMENT_QUANTITY") if n is not None)
sig = _entries("TrM SIGNATURE", h.get("SIGNATURE", []), probs)
for e in sig:
tag = "SIGNATURE:" + _name(e)
_check_fields(tag, e, SIG_FIELDS[0], SIG_FIELDS[1], probs)
if "OVER" in e and _type_ok("OVER", e["OVER"]):
for d in sorted(set(e["OVER"])):
if d not in dofs:
probs.append(tag + ": OVER " + d + " is not a TPK DOF")
if "FORM" in e and "IMAGE_FORM" not in e:
probs.append(tag + ": FORM requires IMAGE_FORM")
if "IMAGE_FORM" in e and "FORM" not in e:
probs.append(tag + ": IMAGE_FORM requires FORM")
f = e.get("FORM")
if isinstance(f, str) and _first_bad(f) is None and "OVER" in e and _type_ok("OVER", e["OVER"]):
r = _form().form_names(f, "form")
if r[0] == "OK":
named, ov = set(r[1]), set(e["OVER"])
for x in sorted(named - ov):
probs.append(tag + ": FORM names " + x + ", which is not in OVER")
for x in sorted(ov - named):
probs.append(tag + ": OVER " + x + " is not named in FORM")
f = e.get("IMAGE_FORM")
if isinstance(f, str) and _first_bad(f) is None:
r = _form().form_names(f, "form")
if r[0] == "OK":
for x in sorted(set(r[1]) - quantities):
probs.append(tag + ": IMAGE_FORM names " + x + ", which is not in the ENVIRONMENT INVENTORY")
for n in _dupes(_names(sig)):
probs.append("TrM: duplicate signature name " + n)
trf = _entries("TrM TRANSFORMATIONS", h.get("TRANSFORMATIONS", []), probs)
for e in trf:
_check_fields("TRANSFORMATION:" + _name(e), e, TRF_FIELDS[0], TRF_FIELDS[1], probs)
for n in _dupes(_names(trf)):
probs.append("TrM: duplicate transformation name " + n)
ents = _entries("TrM ENTRIES", trm.get("ENTRIES", []), probs)
for e in ents:
tag = "ENTRY:" + _name(e)
_check_fields(tag, e, ENTRY_FIELDS[0], ENTRY_FIELDS[1], probs)
d = e.get("DOF")
if isinstance(d, str) and d not in dofs:
probs.append(tag + ": DOF " + d + " is not a TPK DOF")
q = e.get("ENVIRONMENT_QUANTITY")
if isinstance(q, str) and q not in quantities:
probs.append(tag + ": ENVIRONMENT_QUANTITY " + q + " is not in the ENVIRONMENT INVENTORY")
dom = e.get("DOMAIN")
if isinstance(dom, str) and _blank(dom):
probs.append(tag + ": DOMAIN is empty")
fal = e.get("FALSIFIER")
if isinstance(fal, str) and _blank(fal):
probs.append(tag + ": FALSIFIER is empty")
c = e.get("FALSIFIER_CLASS")
if isinstance(c, str):
if c == "F0":
probs.append(tag + ": FALSIFIER_CLASS F0 is inadmissible")
elif c not in FALSIFIER_CLASSES:
probs.append(tag + ": FALSIFIER_CLASS " + c + " is not one of F1 F2 F3")
if "BEARS" in e and _type_ok("BEARS", e["BEARS"]):
for b in sorted(set(e["BEARS"])):
if b not in bearable:
probs.append(tag + ": BEARS " + b + " is not a TPK LAW or CONSTRAINT")
for n in _dupes(_names(ents)):
probs.append("TrM: duplicate entry name " + n)
return _done(probs)
def _check_key(tag, key, consts, probs, c2):
"""KEY rules (D100): non-empty; begins with SR1 (SR9 C1 by construction);
SR1 nowhere else; every term well-formed; every weight a TMS constant
(SR9 C5). With c2 set (a main KEY, not a limiting case): the first term
whose value moves with A must move up with it, so adding an assumption
never lowers the key (SR9 C2, static)."""
if not key:
probs.append(tag + ": KEY is empty")
return
if key[0] != "SR1":
probs.append(tag + ": KEY must begin with SR1")
decided = False
for i, t in enumerate(key):
ok, sr1, cs, _, contrib = _parse_term(t)
if not ok:
probs.append(tag + ": term " + t + " is not well-formed")
continue
if sr1 and i > 0:
probs.append(tag + ": SR1 may appear only as the first term")
missing = [c for c in cs if c not in consts]
for c in missing:
probs.append(tag + ": constant " + c + " is not a TMS constant")
if c2 and not decided and not missing:
s = _a_sign(contrib, consts)
if s != 0:
decided = True
if s < 0:
probs.append(tag + ": KEY ranks a record higher when an assumption is added (SR9 C2)")
def tms_well_formed(tms):
"""TMS schema check (D98, D100). [pass, sorted problems]."""
if not _obj(tms):
return _done(["TMS: zone is not an object"])
probs = []
for p in sorted(tms):
if p not in TMS_PARTS:
probs.append("TMS: unknown part " + p)
if "PARAMETERS" not in tms: # CONSTANTS, YARDSTICKS default to empty
probs.append("TMS: missing part PARAMETERS")
pm = tms.get("PARAMETERS", {})
if not _obj(pm):
probs.append("TMS: PARAMETERS is not an object")
elif "PARAMETERS" in tms:
_check_fields("PARAMETERS", pm, PARAM_FIELDS[0], PARAM_FIELDS[1], probs)
sc = pm.get("STEP_CONVENTION")
if isinstance(sc, str) and _blank(sc):
probs.append("PARAMETERS: STEP_CONVENTION is empty")
k = pm.get("K")
if isinstance(k, str) and canon_decimal(k) is not None and not _is_pos_finite(k):
probs.append("PARAMETERS: K " + k + " is not a positive decimal in double range")
nx = pm.get("N_X")
if "N_X" in pm and _type_ok("N_X", nx) and nx < 2:
probs.append("PARAMETERS: N_X must be at least 2")
if "N_X" in pm and _type_ok("N_X", nx) and nx > INT_MAX:
probs.append("PARAMETERS: N_X must be at most %d" % INT_MAX)
sm = pm.get("S_MAX")
if "S_MAX" in pm and _type_ok("S_MAX", sm) and sm < 1:
probs.append("PARAMETERS: S_MAX must be at least 1")
if "S_MAX" in pm and _type_ok("S_MAX", sm) and sm > INT_MAX:
probs.append("PARAMETERS: S_MAX must be at most %d" % INT_MAX)
fl = pm.get("PROMOTION_FLOOR")
if isinstance(fl, str) and fl not in FALSIFIER_CLASSES:
probs.append("PARAMETERS: PROMOTION_FLOOR " + fl + " is not F1, F2 or F3")
pol = pm.get("IC2_POLICY")
if isinstance(pol, str) and pol not in ("GAP", "REJECT"):
probs.append("PARAMETERS: IC2_POLICY " + pol + " is not GAP or REJECT")
sc = pm.get("STRUCTURE_CHECK")
if isinstance(sc, str) and sc not in STRUCTURE_VALUES:
probs.append("PARAMETERS: STRUCTURE_CHECK " + sc + " is not ON or OFF")
cons = _entries("TMS CONSTANTS", tms.get("CONSTANTS", []), probs)
for e in cons:
_check_fields("CONSTANT:" + _name(e), e, TMSC_FIELDS[0], TMSC_FIELDS[1], probs)
for n in _dupes(_names(cons)):
probs.append("TMS: duplicate constant name " + n)
consts = {}
for e in cons:
if isinstance(e.get("NAME"), str) and isinstance(e.get("VALUE"), str):
consts.setdefault(e["NAME"], e["VALUE"])
yards = _entries("TMS YARDSTICKS", tms.get("YARDSTICKS", []), probs)
for e in yards:
tag = "YARDSTICK:" + _name(e)
_check_fields(tag, e, YARD_FIELDS[0], YARD_FIELDS[1], probs)
if "KEY" in e and _type_ok("KEY", e["KEY"]):
_check_key(tag, e["KEY"], consts, probs, True)
lim = e.get("LIMITING")
if "LIMITING" in e and _type_ok("LIMITING", lim):
for le in lim:
ltag = tag + " LIMITING " + _name(le, "DOMAIN")
_check_fields(ltag, le, LIM_FIELDS[0], LIM_FIELDS[1], probs)
dom = le.get("DOMAIN")
if isinstance(dom, str) and _blank(dom):
probs.append(ltag + ": DOMAIN is empty")
if "KEY" in le and _type_ok("KEY", le["KEY"]):
_check_key(ltag, le["KEY"], consts, probs, False) # C2-exempt in its DOMAIN
for n in _dupes(_names(lim, "DOMAIN", nfc=True)):
probs.append(tag + ": duplicate limiting DOMAIN " + n)
for n in _dupes(_names(yards)):
probs.append("TMS: duplicate yardstick name " + n)
return _done(probs)
