Summary
Inside make(), provide self.upstream — a pre-constructed Diagram.trace(self & key) — as a property on auto-populated tables. This gives make() convenient, provenance-safe access to pre-restricted ancestor table expressions.
Context
Discussion: #1232
Depends on: #1423 (Diagram.trace())
Related: #242 (farfetch)
Design
The make(self, key) signature stays unchanged. self.upstream is simply a new property available during make() execution:
def make(self, key):
# Read from upstream — pre-restricted by key
session_date = self.upstream[Session].fetch1("session_date")
traces = self.upstream[ExtractTraces].to_arrays("trace")
# All query operators work — upstream[Table] returns a QueryExpression
self.upstream[Session].aggr(self.upstream[Scan], n='count(scan_id)')
# Write to self
self.insert1({**key, "result": compute(traces)})
Read from self.upstream, write to self. The read/write boundary is the provenance boundary.
self.upstream only exposes declared ancestors and their part tables — requesting anything else raises an error
key is passed as before — no signature changes, no redundancy
make_kwargs work unchanged
- Existing code that doesn't use
self.upstream is completely unaffected
Implementation
- The framework sets
self.upstream = Diagram.trace(self & key) before calling make()
self.upstream[TableClass] returns pre-restricted QueryExpression objects with full fetch capabilities
- Requesting a table outside the declared ancestor graph raises
DataJointError
Summary
Inside
make(), provideself.upstream— a pre-constructedDiagram.trace(self & key)— as a property on auto-populated tables. This givesmake()convenient, provenance-safe access to pre-restricted ancestor table expressions.Context
Discussion: #1232
Depends on: #1423 (
Diagram.trace())Related: #242 (farfetch)
Design
The
make(self, key)signature stays unchanged.self.upstreamis simply a new property available duringmake()execution:Read from
self.upstream, write toself. The read/write boundary is the provenance boundary.self.upstreamonly exposes declared ancestors and their part tables — requesting anything else raises an errorkeyis passed as before — no signature changes, no redundancymake_kwargswork unchangedself.upstreamis completely unaffectedImplementation
self.upstream = Diagram.trace(self & key)before callingmake()self.upstream[TableClass]returns pre-restrictedQueryExpressionobjects with full fetch capabilitiesDataJointError