Fix: allow --select-model to plan a model deletion#5759
Fix: allow --select-model to plan a model deletion#5759viniciusnunest wants to merge 3 commits intoSQLMesh:mainfrom
Conversation
When a model file is deleted locally but still exists in the deployed environment, --select-model now correctly plans the removal instead of raising PlanError. Fixes SQLMesh#5741 Signed-off-by: vinicius <viniciusnunes.tavares@gmail.com>
93e5928 to
da9a69c
Compare
|
Thanks for your submission @viniciusnunest. Does this solution assume that the selector only contains deleted models or not deleted models? What if there was a mix? |
|
@eakmanrq Good question. It handles both cases. The new code path ( When there is a mix (some active + some deleted), |
| all_selected_models = self.expand_model_selections( | ||
| model_selections, models={**env_models, **self._models} | ||
| ) |
There was a problem hiding this comment.
Is this what you need since it uses the env models? I'm wondering if you could return this alongside the current UniqueKeyDict and then use that and remove the need for expand_model_selections_with_env? Ex:
models_override, selected_fqns = model_selector.select_models(...)
if not backfill_models:
selected_deletion_fqns = selected_fqns - set(self._models)
Thanks I understand now. Can you make sure tests cover this case? |
Refactor select_models to return matched FQNs alongside the model dict so context._plan can detect deletions without a separate env lookup. Remove expand_model_selections_with_env, add mixed selection test coverage. Fixes SQLMesh#5741 Signed-off-by: vinicius <viniciusnunes.tavares@gmail.com>
|
Good call, refactored. Also added test coverage for the mixed case (active + deleted in the same selection). The mixed scenario was already working before but now it is explicitly tested, |
Description
Fixes #5741
Right now if you delete a model file locally and try to plan its removal with
--select-model=<deleted-model>, you get:The problem is that
expand_model_selectionsonly looks at local models. Sincethe file is gone, it finds nothing,
backfill_modelsends up empty, and_plan()raises. The workaround was adding some unrelated active model to theselection, which works but it is not great.
The thing is, the underlying machinery already handles deletions fine. When
backfill_modelshas at least one entry, deleted models show up correctly incontext_diff.removed_snapshots. So the fix is about making the selector andplan validation aware that "no local matches" can still be valid when matching
models pending deletion in the deployed environment.
Changes:
_load_env_models()fromselect_models()to avoid duplicatingthe environment loading logic
expand_model_selections_with_env()that expands selections againstboth local and environment models
_plan(), whenbackfill_modelsis empty, it now checks whether theselection matched models in the environment that were deleted locally. If so,
PlanErroris suppressed and the deletions surface viaremoved_snapshotsraises
PlanErroras beforeTest Plan
test_expand_model_selections_with_env: deleted models found in env, localmodels still work, wildcards match, non-existent returns empty
test_expand_model_selections_with_env_fallback: fallback environment is usedwhen target env is missing
test_expand_model_selections_with_env_expired: expired environments ignoredtest_plan_select_model_deleted_model: end-to-end with the sushi example,deploys to prod, deletes model file, reloads, runs plan and checks that the
model appears in
removed_snapshotstest_plan_select_model_deleted_model_still_rejects_nonexistent: negativecase, makes sure garbage input still fails
Checklist
make styleand fixed any issuesmake fast-test)git commit -s) per the DCO