Asset selection examples
This page contains common example asset selection queries and their implementation in Python, CLI, and the Dagster UI. For a full explanation of the filters, layers, operands, and functions that you can use to construct your own queries, see "Asset selection syntax reference".
The examples in this section use the following asset graph to demonstrate how to use the selection syntax:
Select all assets on the path between two assets
key:"raw_data_b"+ and +key:"summary_stats_2"
Selects all assets on the path from the raw_data_b
asset to the summary_stats_2
asset.
- Dagster UI
- Python
- CLI
raw_data_b_summary_stats_2_job = define_asset_job(
name="raw_data_b_summary_stats_2_job", selection='key:"raw_data_b"+ and +key:"summary_stats_2"'
)
dagster asset list --select 'key:"raw_data_b"+ and +key:"summary_stats_2"'
dagster asset materialize --select 'key:"raw_data_b"+ and +key:"summary_stats_2"'
Select all assets on the path between two sets of assets
(key:"raw_data_a" or key:"raw_data_b")+ and +(key:"a_b_c_for_sales" or key:"b_c_for_sales")
Selects all assets on the paths between the raw_data_a
or raw_data_b
assets and the a_b_c_for_sales
and b_c_for_sales
assets.
- Dagster UI
- Python
- CLI
assets_on_path_job = define_asset_job(
name="assets_on_path_job", selection='(key:"raw_data_a" or key:"raw_data_b")+ and +(key:"a_b_c_for_sales" or key:"b_c_for_sales")'
)
dagster asset list --select '(key:"raw_data_a" or key:"raw_data_b")+ and +(key:"a_b_c_for_sales" or key:"b_c_for_sales")'
dagster asset materialize --select '(key:"raw_data_a" or key:"raw_data_b")+ and +(key:"a_b_c_for_sales" or key:"b_c_for_sales")'
Select all assets on the path between two sets of assets by tag
tag:"private"+ and +tag:"public"
Selects all assets on the paths between assets tagged with private
and assets tagged with public
.
- Dagster UI
- Python
- CLI
private_public_assets_job = define_asset_job(
name="private_public_assets_job", selection='tag:"private"+ and +tag:"public"'
)
dagster asset list --select 'tag:"private"+ and +tag:"public"'
dagster asset materialize --select 'tag:"private"+ and +tag:"public"'
Select all assets on the path between two groups of assets
group:"sensitive_data"+ and +group:"public_data"
Selects all assets on the paths from the sensitive_data
group to the public_data
group.
- Dagster UI
- Python
- CLI
sensitive_to_public_asset_job = define_asset_job(
name="sensitive_to_public_asset_job", selection='group:"sensitive_data"+ and +group:"public_data"'
)
dagster asset list --select 'group:"sensitive_data"+ and +group:"public_data"'
dagster asset materialize --select 'group:"sensitive_data"+ and +group:"public_data"'
Select assets between two assets that go through the "middle" asset
key:"raw_data_c"+ and +key:"combo_a_b_c_data"+ and +key:"summary_stats_1"
Selects all assets on the path between the raw_data_c
asset and summary_stats_1
asset that go through the combo_a_b_c_data
asset.
- Dagster UI
- Python
- CLI
middle_asset_job = define_asset_job(
name="middle_asset_job", selection='key:"raw_data_c"+ and +key:"combo_a_b_c_data"+ and +key:"summary_stats_1"'
)
dagster asset list --select 'key:"raw_data_c"+ and +key:"combo_a_b_c_data"+ and +key:"summary_stats_1"'
dagster asset materialize --select 'key:"raw_data_c"+ and +key:"combo_a_b_c_data"+ and +key:"summary_stats_1"'
Select assets with multiple key components
To select an asset with a key containing multiple components, such as a prefix, insert slashes (/
) between the components:
key:"my_prefix/raw_data_a"
This example selects the my_prefix/raw_data_a
asset, which is defined below:
@dg.asset(
key_prefix="my_prefix",
...
)
def raw_data_a() -> None:
...
- Dagster UI
- Python
- CLI
my_prefix_job = define_asset_job(name="my_prefix_job", selection='key:"my_prefix/raw_data_a"')
dagster asset list --select 'key:"my_prefix/raw_data_a"'
dagster asset materialize --select 'key:"my_prefix/raw_data_a"'
Select multiple assets with or
To select multiple assets, use the or
operand. The assets don't have to be dependent on each other. This example selects the summary_stats_1
and summary_stats_2
assets:
key:"summary_stats_1" or key:"summary_stats_2"
- Dagster UI
- Python
- CLI
summary_stats_assets_job = define_asset_job(
name="summary_stats_assets_job", selection='key:"summary_stats_1" or key:"summary_stats_2"'
)
dagster asset list --select 'key:"summary_stats_1" or key:"summary_stats_2"'
dagster asset materialize --select 'key:"summary_stats_1" or key:"summary_stats_2"'
Select downstream assets with a filter
key:"combo_a_b_c_data"+ and kind:"csv"
Selects one layer downstream of combo_a_b_c_data
, limited to assets of kind csv
.
- Dagster UI
- Python
- CLI
downstream_csv_job = define_asset_job(
name="downstream_csv_job", selection='key:"combo_a_b_c_data"+ and kind:"csv"'
)
dagster asset list --select 'key:"combo_a_b_c_data"+ and kind:"csv"'
dagster asset materialize --select 'key:"combo_a_b_c_data"+ and kind:"csv"'
Select assets without a specific tag
owner:"nora.dagster@example.com" and not tag:"customer_data"
Selects all assets owned by nora.dagster@example.com
excluding any assets tagged with customer_data
.
- Dagster UI
- Python
- CLI
data_eng_not_private_job = define_asset_job(
name="data_eng_not_private_job", selection='owner:"nora.dagster@example.com" and not tag:"customer_data"'
)
dagster asset list --select 'owner:"nora.dagster@example.com" and not tag:"customer_data"'
dagster asset materialize --select 'owner:"nora.dagster@example.com" and not tag:"customer_data"'
Select assets with parentheses grouping and filters
Select a parentheses group or
assets that belong in a named group
(owner:"team:sales" and kind:"csv") or group:"public_data"
Selects assets that are either owned by the sales team and of kind csv
, or that belong to the public_data
group.
- Dagster UI
- Python
- CLI
sales_csv_public_job = define_asset_job(
name="sales_csv_public_job", selection='(owner:"team:sales" and kind:"csv") or group:"public_data"'
)
dagster asset list --select '(owner:"team:sales" and kind:"csv") or group:"public_data"'
dagster asset materialize --select '(owner:"team:sales" and kind:"csv") or group:"public_data"'
Exclude a parentheses group with not
and select assets that match a key
wildcard
not (kind:"s3" or kind:"csv") and key:"raw*"
Selects assets whose keys contain raw
and are not of the kind s3
or csv
.
- Dagster UI
- Python
- CLI
not_s3_csv_raw_job = define_asset_job(
name="not_s3_csv_raw_job", selection='not (kind:"s3" or kind:"csv") and key:"raw*"'
)
dagster asset list --select 'not (kind:"s3" or kind:"csv") and key:"raw*"'
dagster asset materialize --select 'not (kind:"s3" or kind:"csv") and key:"raw*"'
Select roots or sinks
The examples in this section operate on a selection of assets in the public_data
group:
Select all root assets within an asset selection
roots(group:"public_data")
Selects root assets within the public_data
group.
- Dagster UI
- Python
- CLI
roots_within_public_data_job = define_asset_job(
name="roots_within_public_data_job", selection='roots(group:"public_data")'
)
dagster asset list --select 'roots(group:"public_data")'
dagster asset materialize --select 'roots(group:"public_data")'
Select all sink assets within an asset selection
sinks(group:"public_data")
Selects sink assets within the public_data
group.
- Dagster UI
- Python
- CLI
sinks_within_public_data_job = define_asset_job(
name="sinks_within_public_data_job", selection='sinks(group:"public_data")'
)
dagster asset list --select 'sinks(group:"public_data")'
dagster asset materialize --select 'sinks(group:"public_data")'
Select all root assets that feed into an asset selection
roots(+group:"public_data")
Selects root assets that feed into the public_data
group, but do not belong to that group.
- Dagster UI
- Python
- CLI
roots_feed_to_public_data_job = define_asset_job(
name="roots_feed_to_public_data_job", selection='roots(+group:"public_data")'
)
dagster asset list --select 'roots(+group:"public_data")'
dagster asset materialize --select 'roots(+group:"public_data")'
Select all sink assets that depend on assets in a selection
sinks(group:"public_data"+)
Selects sink assets that depend on assets in the public_data
group, but do not belong to that group.
- Dagster UI
- Python
- CLI
sinks_feed_to_public_data_job = define_asset_job(
name="sinks_feed_to_public_data_job", selection='sinks(group:"public_data"+)'
)
dagster asset list --select 'sinks(group:"public_data"+)'
dagster asset materialize --select 'sinks(group:"public_data"+)'