Validate a semantic model¶
A successful refresh says the model ran, not that its measure is right. Add an Assumption to the working Parcel model, then reconcile the additional source-backed model with independent Warehouse SQL.
In Power BI-only mode, without a Catalogue, DAX validations run directly against the selected model using source/file mode. Results return to the caller; Weaver records no installed Test history.
With a Catalogue, Build can install authored validations. Installed Test runs record results in the Catalogue, and Health assesses them alongside other managed work. Source/file mode remains a deliberate option in Catalogue-backed projects too; the commands below distinguish the two.
Choose the model you are checking¶
- ParcelWarehouse already reads the managed Warehouse: go straight to Compare DAX with source SQL. Keep the source and model you just loaded; no inline-data setup or new provisioning is needed.
- You are checking the native inline Parcel model: add the DAX Assumption to that existing model and retain your overlay.
Start from the model you already deployed¶
For the Assumption, continue the native model: Parcel has inline Import data, Delivery Count evaluates to 2, and its same-name overlay hides the relationship foreign key. Keep that overlay and your existing workspace configuration. You add one DAX file, not another project or deployment setup.
If you are starting here, generate and deploy the native Parcel example, then add the Assumption below. Keep any existing overlay.
DAX queries need model Read and Build permissions and an identity permitted by the tenant’s Execute Queries setting. Reconciliation also needs SELECT access to the Warehouse source. Use existing permissions; do not change tenant settings for this exercise.
1. Add the DAX Assumption¶
Create PowerBI/Parcel/assumptions/Parcel/Parcel.PositiveDeliveryCount.dax:
/*
Assumption ID: Parcel.PositiveDeliveryCount
Description: The delivery count is positive.
*/
EVALUATE FILTER(ROW("DeliveryCount", [Delivery Count]), [DeliveryCount] <= 0)
The filename, metadata ID and assumptions directory agree. Parcel in the directory is the logical model name, whatever the physical model is called in a given workspace. An Assumption returns violating rows; zero rows passes. DAX validation reference owns the complete metadata rules.
2. Run the authored query¶
weaver check
weaver test SemanticModel/Parcel --source . --file 'PowerBI/Parcel/assumptions/Parcel/Parcel.PositiveDeliveryCount.dax'
The command returned:
Test
Read validations 0.1s
Execute 1.1s
✓ Test 1.2s
test passed
Test summary
1 passed
0 failed
The Assumption passed with zero violations and exit 0. You do not need Build merely to edit a source query: this run evaluates the new file against the already deployed model.
The file is evaluated against the model you already deployed. This source run does not install a validation or write catalogue history. Test modes explains installing a validation when you want recorded estate results.
Optional: make the measure fail, then restore your model
3. Produce a finding without losing your overlay¶
Use this declaration for the optional failure exercise:
Save a copy of the overlay so you can put it back:
Then add the failure declaration to the end of PowerBI/Parcel/Parcel.tmdl, below the existing model Model section. If the overlay already has a table Delivery section, for example from shared policy, change the measure there instead; a second declaration of the same table is refused. If you have no Parcel.tmdl yet, create it with just the failure declaration and delete it afterwards instead of restoring.
Deploy that changed measure and run the same source Assumption:
weaver check
weaver build --item SemanticModel/Parcel
weaver load SemanticModel/Parcel
weaver test SemanticModel/Parcel --source . --file 'PowerBI/Parcel/assumptions/Parcel/Parcel.PositiveDeliveryCount.dax'
The command returned:
Test
Read validations 0.1s
Execute 1.0s
✗ Test 1.1s
test failed
failed Assumption SemanticModel/Parcel/Parcel.PositiveDeliveryCount (1 violation)
Test summary
0 passed
1 failed
SemanticModel/Parcel/Parcel.PositiveDeliveryCount:
{'DeliveryCount': -1}
Test returned one violating row, DeliveryCount = -1, and exit 1. This is a finding, not a query failure. A permission or execution failure produces an invalid node and also exits 1.
4. Restore the prior model and rerun¶
Put the saved overlay back, then deploy and rerun:
mv Parcel.tmdl.saved PowerBI/Parcel/Parcel.tmdl
weaver check
weaver build --item SemanticModel/Parcel
weaver load SemanticModel/Parcel
weaver test SemanticModel/Parcel --source . --file 'PowerBI/Parcel/assumptions/Parcel/Parcel.PositiveDeliveryCount.dax'
The command returned:
Test
Read validations 0.1s
Execute 1.0s
✓ Test 1.1s
test passed
Test summary
1 passed
0 failed
The Assumption passed again, and the overlay is back as it was, including its foreign-key rule.
--file explicitly selects source/file mode even when the configuration contains a catalogue. --source . identifies the project; it does not force file mode by itself. Without a configured catalogue, --source . alone uses source mode. Source/file mode records no installed _.TestStatus or Test history. Repeat --file for paths/directories/globs and --name for case-insensitive whole-validation regex seeds; each supplied name pattern must match.
To install this Assumption in a catalogue-backed project, Build the model, then run Test without --file. Installed mode reads the validation definition Build recorded in the catalogue and records its result for Health; an edited local DAX file changes nothing until the next Build.
The command returned:
Test
Read catalogue 2.6s
Workspace Parcel Development
Catalogue Warehouse/Catalogue
Targets
SemanticModel/Parcel
Build run graph 0.0s
→ Test SemanticModel/Parcel/Parcel.PositiveDeliveryCount
✓ Test SemanticModel/Parcel/Parcel.PositiveDeliveryCount 0.5s
Execute 0.7s
Record what the run did 0.7s
✓ Test 4.0s
test passed
Test summary
1 passed
0 failed
Workflow: 49d590fc6481430fa5e513ee217a93c3
The command returned:
{'status': 'passed',
'nodes': [{'logical_id': 'SemanticModel/Parcel/Parcel.PositiveDeliveryCount',
'kind': 'assumption',
'physical_target': 'SemanticModel/Parcel',
'primitive_kind': 'semantic_validation',
'dispatch_location': '',
'status': 'passed',
'executed': True,
'messages': [],
'started_at': '2026-10-17T08:02:41.781454+00:00',
'finished_at': '2026-10-17T08:02:41.993875+00:00',
'violation_count': 0,
'error_message': None}],
'totals': {'planned': 1,
'executed': 1,
'passed': 1,
'failed': 0,
'invalid': 0,
'missing_count': 0,
'unexpected_count': 0,
'violation_count': 0},
'workflow_id': '[run-id]',
'started_at': '2026-10-17T08:02:41.633362+00:00',
'finished_at': '2026-10-17T08:02:42.486617+00:00'}
Compare DAX with source SQL¶
Use the already loaded managed-source model in this same project. ParcelWarehouse reads Operations’ CurrentStatus View; the native Parcel model remains inline. Do not compare the inline Delivery data with an unrelated Warehouse result.
Add a measure to the existing CurrentStatus declaration in PowerBI/Parcel/ParcelWarehouse.tmdl. Its complete contents become:
model Model
culture: en-US
defaultMode: import
annotation 'Weaver.BaseSemanticModels' = Parcel
table CurrentStatus
annotation 'Weaver.Source' = Warehouse/Operations/Parcel.CurrentStatus
measure 'Managed Parcel Count' = COUNTROWS(CurrentStatus)
Create PowerBI/Parcel/tests/ParcelWarehouse/Parcel.CurrentStatusReconciles.dax:
/*
Test ID: Parcel.CurrentStatusReconciles
Description: Model parcel count agrees with the managed Warehouse View.
Expected source: Warehouse/Operations
Expected SQL: |
SELECT COUNT_BIG(*) AS ParcelCount FROM [Parcel].[CurrentStatus]
*/
EVALUATE ROW("ParcelCount", [Managed Parcel Count])
Build installs this model’s measure and Test; refresh it, then compare the model count with independent SQL. Both should return 2:
weaver check
weaver build --item SemanticModel/ParcelWarehouse --bind-data-sources
weaver load SemanticModel/ParcelWarehouse
weaver test SemanticModel/ParcelWarehouse --name 'Parcel[.]CurrentStatusReconciles'
The command returned:
Test
Read catalogue 2.4s
Workspace Parcel Development
Catalogue Warehouse/Catalogue
Targets
SemanticModel/ParcelWarehouse
Build run graph 0.0s
→ Test SemanticModel/ParcelWarehouse/Parcel.CurrentStatusReconciles
✓ Test SemanticModel/ParcelWarehouse/Parcel.CurrentStatusReconciles 1.2s
Execute 1.4s
Record what the run did 1.8s
✓ Test 5.6s
test passed
Test summary
1 passed
0 failed
Workflow: 85af66f2a2424f7cbf428e309465ff1f
The Test ran the DAX query against the refreshed model and the Expected SQL against Operations, and both returned 2. Health now records the result:
Inspect the recorded model and validation evidence
This is installed mode. Inspect CurrentStatus under logical SemanticModel/ParcelWarehouse in _.SemanticModelTable, and the _.Dependency row whose authored Dependency reference is Warehouse/Operations/Parcel.CurrentStatus. Source mode/access belongs in the table metadata; source identity belongs in Dependency. This generated Import partition uses SQL access.
Inspect the installed validation in _.SemanticModelTest and its result in _.TestStatus. Model refresh and Test results sit beside Warehouse state in the same catalogue. With no item selector, Health covers the managed estate, not arbitrary workspace items.
Optional: inspect a DAX-to-SQL discrepancy and restore it
Produce and correct a discrepancy¶
Save a copy, then change COUNTROWS(CurrentStatus) to COUNTROWS(CurrentStatus) + 1 in the measure. Keep the source annotation and the rest of the file:
weaver build --item SemanticModel/ParcelWarehouse --bind-data-sources
weaver load SemanticModel/ParcelWarehouse
weaver test SemanticModel/ParcelWarehouse --name 'Parcel[.]CurrentStatusReconciles'
The command returned:
Test
Read catalogue 2.8s
Workspace Parcel Development
Catalogue Warehouse/Catalogue
Targets
SemanticModel/ParcelWarehouse
Build run graph 0.0s
→ Test SemanticModel/ParcelWarehouse/Parcel.CurrentStatusReconciles
✗ Test SemanticModel/ParcelWarehouse/Parcel.CurrentStatusReconciles (1 missing, 1 unexpected) 1.1s
Execute 1.3s
Record what the run did 0.7s
✗ Test 4.8s
test failed
failed Test SemanticModel/ParcelWarehouse/Parcel.CurrentStatusReconciles (1 missing, 1 unexpected)
Test summary
0 passed
1 failed
Workflow: c6cca9bede6b49db8cbb20e63f27d9a4
SemanticModel/ParcelWarehouse/Parcel.CurrentStatusReconciles:
{'_weaver_side': 'expected', '_weaver_sk': 1, 'ParcelCount': 2}
{'_weaver_side': 'actual', '_weaver_sk': 2, 'ParcelCount': 3}
The command returned:
{'status': 'failed',
'nodes': [{'logical_id': 'SemanticModel/ParcelWarehouse/Parcel.CurrentStatusReconciles',
'kind': 'test',
'physical_target': 'SemanticModel/ParcelWarehouse',
'primitive_kind': 'semantic_validation',
'dispatch_location': '',
'status': 'failed',
'executed': True,
'messages': [],
'started_at': '2026-10-17T08:05:26.467209+00:00',
'finished_at': '2026-10-17T08:05:27.010639+00:00',
'missing_count': 1,
'unexpected_count': 1,
'failure_count': 2,
'error_message': None}],
'totals': {'planned': 1,
'executed': 1,
'passed': 0,
'failed': 1,
'invalid': 0,
'missing_count': 1,
'unexpected_count': 1,
'violation_count': 0},
'workflow_id': '[run-id]',
'started_at': '2026-10-17T08:05:26.293818+00:00',
'finished_at': '2026-10-17T08:05:27.447042+00:00'}
The expected row (ParcelCount 2, from SQL) is missing and the actual row (3, from DAX) is unexpected. Restore the file and repeat the three commands; the Test passes again:
This edits the additional model’s measure, not Warehouse data or the native Parcel model. Access or query errors make the validation invalid, not a discrepancy; installed _.TestStatus records them as Error.
Read the estate result¶
In a catalogue-backed project, weaver test without item selectors runs installed Lakehouse, Warehouse and model validations. Run and inspect the estate combines those outcomes with source loads and model refreshes. Exact diagnostics and recording boundaries belong in Test behaviour.