Sign in Dashboard Get started
  1. Packages
  2. Packages
  3. Databricks Provider
  4. API Docs
  5. getFeatureEngineeringKafkaConfig
Viewing docs for Databricks v1.96.0
published on Thursday, Jun 18, 2026 by Pulumi
Schema (JSON)
pulumi/pulumi-databricks

databricks.getFeatureEngineeringKafkaConfig

databricks logo
Viewing docs for Databricks v1.96.0
published on Thursday, Jun 18, 2026 by Pulumi
Schema (JSON)
pulumi/pulumi-databricks

    Private Preview

    Using getFeatureEngineeringKafkaConfig

    Two invocation forms are available. The direct form accepts plain arguments and either blocks until the result value is available, or returns a Promise-wrapped result. The output form accepts Input-wrapped arguments and returns an Output-wrapped result.

    function getFeatureEngineeringKafkaConfig(args: GetFeatureEngineeringKafkaConfigArgs, opts?: InvokeOptions): Promise<GetFeatureEngineeringKafkaConfigResult>
function getFeatureEngineeringKafkaConfigOutput(args: GetFeatureEngineeringKafkaConfigOutputArgs, opts?: InvokeOptions): Output<GetFeatureEngineeringKafkaConfigResult>
    def get_feature_engineering_kafka_config(name: Optional[str] = None,
                                         provider_config: Optional[GetFeatureEngineeringKafkaConfigProviderConfig] = None,
                                         opts: Optional[InvokeOptions] = None) -> GetFeatureEngineeringKafkaConfigResult
def get_feature_engineering_kafka_config_output(name: pulumi.Input[Optional[str]] = None,
                                         provider_config: pulumi.Input[Optional[GetFeatureEngineeringKafkaConfigProviderConfigArgs]] = None,
                                         opts: Optional[InvokeOptions] = None) -> Output[GetFeatureEngineeringKafkaConfigResult]
    func LookupFeatureEngineeringKafkaConfig(ctx *Context, args *LookupFeatureEngineeringKafkaConfigArgs, opts ...InvokeOption) (*LookupFeatureEngineeringKafkaConfigResult, error)
func LookupFeatureEngineeringKafkaConfigOutput(ctx *Context, args *LookupFeatureEngineeringKafkaConfigOutputArgs, opts ...InvokeOption) LookupFeatureEngineeringKafkaConfigResultOutput

    > Note: This function is named LookupFeatureEngineeringKafkaConfig in the Go SDK.

    public static class GetFeatureEngineeringKafkaConfig 
{
    public static Task<GetFeatureEngineeringKafkaConfigResult> InvokeAsync(GetFeatureEngineeringKafkaConfigArgs args, InvokeOptions? opts = null)
    public static Output<GetFeatureEngineeringKafkaConfigResult> Invoke(GetFeatureEngineeringKafkaConfigInvokeArgs args, InvokeOptions? opts = null)
}
    public static CompletableFuture<GetFeatureEngineeringKafkaConfigResult> getFeatureEngineeringKafkaConfig(GetFeatureEngineeringKafkaConfigArgs args, InvokeOptions options)
public static Output<GetFeatureEngineeringKafkaConfigResult> getFeatureEngineeringKafkaConfig(GetFeatureEngineeringKafkaConfigArgs args, InvokeOptions options)

    fn::invoke:
  function: databricks:index/getFeatureEngineeringKafkaConfig:getFeatureEngineeringKafkaConfig
  arguments:
    # arguments dictionary
    data "databricks_getfeatureengineeringkafkaconfig" "name" {
    # arguments
}

    The following arguments are supported:

    Name string
    Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    ProviderConfig GetFeatureEngineeringKafkaConfigProviderConfig
    Configure the provider for management through account provider.
    Name string
    Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    ProviderConfig GetFeatureEngineeringKafkaConfigProviderConfig
    Configure the provider for management through account provider.
    name string
    Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    provider_config object
    Configure the provider for management through account provider.
    name String
    Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    providerConfig GetFeatureEngineeringKafkaConfigProviderConfig
    Configure the provider for management through account provider.
    name string
    Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    providerConfig GetFeatureEngineeringKafkaConfigProviderConfig
    Configure the provider for management through account provider.
    name str
    Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    provider_config GetFeatureEngineeringKafkaConfigProviderConfig
    Configure the provider for management through account provider.
    name String
    Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    providerConfig Property Map
    Configure the provider for management through account provider.

    getFeatureEngineeringKafkaConfig Result

    The following output properties are available:

    AuthConfig GetFeatureEngineeringKafkaConfigAuthConfig
    (AuthConfig) - Authentication configuration for connection to topics
    BackfillSource GetFeatureEngineeringKafkaConfigBackfillSource
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    BootstrapServers string
    (string) - A comma-separated list of host/port pairs pointing to Kafka cluster
    ExtraOptions Dictionary<string, string>
    (object) - Catch-all for miscellaneous options. Keys should be source options or Kafka consumer options (kafka.*)
    Id string
    The provider-assigned unique ID for this managed resource.
    IngestionConfig GetFeatureEngineeringKafkaConfigIngestionConfig
    (IngestionConfig) - Configuration for ingesting Kafka data into a Databricks-managed Delta table
    KeySchema GetFeatureEngineeringKafkaConfigKeySchema
    (SchemaConfig) - Schema configuration for extracting message keys from topics. At least one of keySchema and valueSchema must be provided
    Name string
    (string) - Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    SubscriptionMode GetFeatureEngineeringKafkaConfigSubscriptionMode
    (SubscriptionMode) - Options to configure which Kafka topics to pull data from
    ValueSchema GetFeatureEngineeringKafkaConfigValueSchema
    (SchemaConfig) - Schema configuration for extracting message values from topics. At least one of keySchema and valueSchema must be provided
    ProviderConfig GetFeatureEngineeringKafkaConfigProviderConfig
    AuthConfig GetFeatureEngineeringKafkaConfigAuthConfig
    (AuthConfig) - Authentication configuration for connection to topics
    BackfillSource GetFeatureEngineeringKafkaConfigBackfillSource
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    BootstrapServers string
    (string) - A comma-separated list of host/port pairs pointing to Kafka cluster
    ExtraOptions map[string]string
    (object) - Catch-all for miscellaneous options. Keys should be source options or Kafka consumer options (kafka.*)
    Id string
    The provider-assigned unique ID for this managed resource.
    IngestionConfig GetFeatureEngineeringKafkaConfigIngestionConfig
    (IngestionConfig) - Configuration for ingesting Kafka data into a Databricks-managed Delta table
    KeySchema GetFeatureEngineeringKafkaConfigKeySchema
    (SchemaConfig) - Schema configuration for extracting message keys from topics. At least one of keySchema and valueSchema must be provided
    Name string
    (string) - Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    SubscriptionMode GetFeatureEngineeringKafkaConfigSubscriptionMode
    (SubscriptionMode) - Options to configure which Kafka topics to pull data from
    ValueSchema GetFeatureEngineeringKafkaConfigValueSchema
    (SchemaConfig) - Schema configuration for extracting message values from topics. At least one of keySchema and valueSchema must be provided
    ProviderConfig GetFeatureEngineeringKafkaConfigProviderConfig
    auth_config object
    (AuthConfig) - Authentication configuration for connection to topics
    backfill_source object
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    bootstrap_servers string
    (string) - A comma-separated list of host/port pairs pointing to Kafka cluster
    extra_options map(string)
    (object) - Catch-all for miscellaneous options. Keys should be source options or Kafka consumer options (kafka.*)
    id string
    The provider-assigned unique ID for this managed resource.
    ingestion_config object
    (IngestionConfig) - Configuration for ingesting Kafka data into a Databricks-managed Delta table
    key_schema object
    (SchemaConfig) - Schema configuration for extracting message keys from topics. At least one of keySchema and valueSchema must be provided
    name string
    (string) - Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    subscription_mode object
    (SubscriptionMode) - Options to configure which Kafka topics to pull data from
    value_schema object
    (SchemaConfig) - Schema configuration for extracting message values from topics. At least one of keySchema and valueSchema must be provided
    provider_config object
    authConfig GetFeatureEngineeringKafkaConfigAuthConfig
    (AuthConfig) - Authentication configuration for connection to topics
    backfillSource GetFeatureEngineeringKafkaConfigBackfillSource
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    bootstrapServers String
    (string) - A comma-separated list of host/port pairs pointing to Kafka cluster
    extraOptions Map<String,String>
    (object) - Catch-all for miscellaneous options. Keys should be source options or Kafka consumer options (kafka.*)
    id String
    The provider-assigned unique ID for this managed resource.
    ingestionConfig GetFeatureEngineeringKafkaConfigIngestionConfig
    (IngestionConfig) - Configuration for ingesting Kafka data into a Databricks-managed Delta table
    keySchema GetFeatureEngineeringKafkaConfigKeySchema
    (SchemaConfig) - Schema configuration for extracting message keys from topics. At least one of keySchema and valueSchema must be provided
    name String
    (string) - Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    subscriptionMode GetFeatureEngineeringKafkaConfigSubscriptionMode
    (SubscriptionMode) - Options to configure which Kafka topics to pull data from
    valueSchema GetFeatureEngineeringKafkaConfigValueSchema
    (SchemaConfig) - Schema configuration for extracting message values from topics. At least one of keySchema and valueSchema must be provided
    providerConfig GetFeatureEngineeringKafkaConfigProviderConfig
    authConfig GetFeatureEngineeringKafkaConfigAuthConfig
    (AuthConfig) - Authentication configuration for connection to topics
    backfillSource GetFeatureEngineeringKafkaConfigBackfillSource
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    bootstrapServers string
    (string) - A comma-separated list of host/port pairs pointing to Kafka cluster
    extraOptions {[key: string]: string}
    (object) - Catch-all for miscellaneous options. Keys should be source options or Kafka consumer options (kafka.*)
    id string
    The provider-assigned unique ID for this managed resource.
    ingestionConfig GetFeatureEngineeringKafkaConfigIngestionConfig
    (IngestionConfig) - Configuration for ingesting Kafka data into a Databricks-managed Delta table
    keySchema GetFeatureEngineeringKafkaConfigKeySchema
    (SchemaConfig) - Schema configuration for extracting message keys from topics. At least one of keySchema and valueSchema must be provided
    name string
    (string) - Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    subscriptionMode GetFeatureEngineeringKafkaConfigSubscriptionMode
    (SubscriptionMode) - Options to configure which Kafka topics to pull data from
    valueSchema GetFeatureEngineeringKafkaConfigValueSchema
    (SchemaConfig) - Schema configuration for extracting message values from topics. At least one of keySchema and valueSchema must be provided
    providerConfig GetFeatureEngineeringKafkaConfigProviderConfig
    auth_config GetFeatureEngineeringKafkaConfigAuthConfig
    (AuthConfig) - Authentication configuration for connection to topics
    backfill_source GetFeatureEngineeringKafkaConfigBackfillSource
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    bootstrap_servers str
    (string) - A comma-separated list of host/port pairs pointing to Kafka cluster
    extra_options Mapping[str, str]
    (object) - Catch-all for miscellaneous options. Keys should be source options or Kafka consumer options (kafka.*)
    id str
    The provider-assigned unique ID for this managed resource.
    ingestion_config GetFeatureEngineeringKafkaConfigIngestionConfig
    (IngestionConfig) - Configuration for ingesting Kafka data into a Databricks-managed Delta table
    key_schema GetFeatureEngineeringKafkaConfigKeySchema
    (SchemaConfig) - Schema configuration for extracting message keys from topics. At least one of keySchema and valueSchema must be provided
    name str
    (string) - Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    subscription_mode GetFeatureEngineeringKafkaConfigSubscriptionMode
    (SubscriptionMode) - Options to configure which Kafka topics to pull data from
    value_schema GetFeatureEngineeringKafkaConfigValueSchema
    (SchemaConfig) - Schema configuration for extracting message values from topics. At least one of keySchema and valueSchema must be provided
    provider_config GetFeatureEngineeringKafkaConfigProviderConfig
    authConfig Property Map
    (AuthConfig) - Authentication configuration for connection to topics
    backfillSource Property Map
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    bootstrapServers String
    (string) - A comma-separated list of host/port pairs pointing to Kafka cluster
    extraOptions Map<String>
    (object) - Catch-all for miscellaneous options. Keys should be source options or Kafka consumer options (kafka.*)
    id String
    The provider-assigned unique ID for this managed resource.
    ingestionConfig Property Map
    (IngestionConfig) - Configuration for ingesting Kafka data into a Databricks-managed Delta table
    keySchema Property Map
    (SchemaConfig) - Schema configuration for extracting message keys from topics. At least one of keySchema and valueSchema must be provided
    name String
    (string) - Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name
    subscriptionMode Property Map
    (SubscriptionMode) - Options to configure which Kafka topics to pull data from
    valueSchema Property Map
    (SchemaConfig) - Schema configuration for extracting message values from topics. At least one of keySchema and valueSchema must be provided
    providerConfig Property Map

    Supporting Types

    GetFeatureEngineeringKafkaConfigAuthConfig

    MtlsConfig GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfig
    (MtlsConfig) - Mutual-TLS authentication. See MtlsConfig
    UcServiceCredentialName string
    (string) - Name of the Unity Catalog service credential. This value will be set under the option databricks.serviceCredential
    MtlsConfig GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfig
    (MtlsConfig) - Mutual-TLS authentication. See MtlsConfig
    UcServiceCredentialName string
    (string) - Name of the Unity Catalog service credential. This value will be set under the option databricks.serviceCredential
    mtls_config object
    (MtlsConfig) - Mutual-TLS authentication. See MtlsConfig
    uc_service_credential_name string
    (string) - Name of the Unity Catalog service credential. This value will be set under the option databricks.serviceCredential
    mtlsConfig GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfig
    (MtlsConfig) - Mutual-TLS authentication. See MtlsConfig
    ucServiceCredentialName String
    (string) - Name of the Unity Catalog service credential. This value will be set under the option databricks.serviceCredential
    mtlsConfig GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfig
    (MtlsConfig) - Mutual-TLS authentication. See MtlsConfig
    ucServiceCredentialName string
    (string) - Name of the Unity Catalog service credential. This value will be set under the option databricks.serviceCredential
    mtls_config GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfig
    (MtlsConfig) - Mutual-TLS authentication. See MtlsConfig
    uc_service_credential_name str
    (string) - Name of the Unity Catalog service credential. This value will be set under the option databricks.serviceCredential
    mtlsConfig Property Map
    (MtlsConfig) - Mutual-TLS authentication. See MtlsConfig
    ucServiceCredentialName String
    (string) - Name of the Unity Catalog service credential. This value will be set under the option databricks.serviceCredential

    GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfig

    KeyPasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeyPasswordRef
    (SecretScopeReference) - Secret-scope reference for the private key password. Often the same value as the keystore password (keytool's default), but provided as a separate field because Apache Kafka requires it as a distinct option (kafka.ssl.key.password)
    KeystoreLocation string
    (string) - Unity Catalog volume path to the JKS keystore file containing the client certificate and private key. e.g. "/Volumes////client.jks". The materialization compute must have read permission on this volume
    KeystorePasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeystorePasswordRef
    (SecretScopeReference) - Secret-scope reference for the JKS keystore password
    TruststoreLocation string
    (string) - Unity Catalog volume path to the JKS truststore file containing the CA certificate(s) trusted to verify the Kafka broker's server certificate. e.g. "/Volumes////truststore.jks"
    TruststorePasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigTruststorePasswordRef
    (SecretScopeReference) - Secret-scope reference for the JKS truststore password
    DisableHostnameVerification bool
    (boolean) - Set to true only when the broker certificate's SAN intentionally does not match the connection endpoint — for example when reaching the cluster through a PrivateLink endpoint whose DNS name is not in the broker certificate. Skipping the hostname check removes a defense against man-in-the-middle attacks; do not enable casually. mTLS client authentication is unaffected by this option.
    KeyPasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeyPasswordRef
    (SecretScopeReference) - Secret-scope reference for the private key password. Often the same value as the keystore password (keytool's default), but provided as a separate field because Apache Kafka requires it as a distinct option (kafka.ssl.key.password)
    KeystoreLocation string
    (string) - Unity Catalog volume path to the JKS keystore file containing the client certificate and private key. e.g. "/Volumes////client.jks". The materialization compute must have read permission on this volume
    KeystorePasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeystorePasswordRef
    (SecretScopeReference) - Secret-scope reference for the JKS keystore password
    TruststoreLocation string
    (string) - Unity Catalog volume path to the JKS truststore file containing the CA certificate(s) trusted to verify the Kafka broker's server certificate. e.g. "/Volumes////truststore.jks"
    TruststorePasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigTruststorePasswordRef
    (SecretScopeReference) - Secret-scope reference for the JKS truststore password
    DisableHostnameVerification bool
    (boolean) - Set to true only when the broker certificate's SAN intentionally does not match the connection endpoint — for example when reaching the cluster through a PrivateLink endpoint whose DNS name is not in the broker certificate. Skipping the hostname check removes a defense against man-in-the-middle attacks; do not enable casually. mTLS client authentication is unaffected by this option.
    key_password_ref object
    (SecretScopeReference) - Secret-scope reference for the private key password. Often the same value as the keystore password (keytool's default), but provided as a separate field because Apache Kafka requires it as a distinct option (kafka.ssl.key.password)
    keystore_location string
    (string) - Unity Catalog volume path to the JKS keystore file containing the client certificate and private key. e.g. "/Volumes////client.jks". The materialization compute must have read permission on this volume
    keystore_password_ref object
    (SecretScopeReference) - Secret-scope reference for the JKS keystore password
    truststore_location string
    (string) - Unity Catalog volume path to the JKS truststore file containing the CA certificate(s) trusted to verify the Kafka broker's server certificate. e.g. "/Volumes////truststore.jks"
    truststore_password_ref object
    (SecretScopeReference) - Secret-scope reference for the JKS truststore password
    disable_hostname_verification bool
    (boolean) - Set to true only when the broker certificate's SAN intentionally does not match the connection endpoint — for example when reaching the cluster through a PrivateLink endpoint whose DNS name is not in the broker certificate. Skipping the hostname check removes a defense against man-in-the-middle attacks; do not enable casually. mTLS client authentication is unaffected by this option.
    keyPasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeyPasswordRef
    (SecretScopeReference) - Secret-scope reference for the private key password. Often the same value as the keystore password (keytool's default), but provided as a separate field because Apache Kafka requires it as a distinct option (kafka.ssl.key.password)
    keystoreLocation String
    (string) - Unity Catalog volume path to the JKS keystore file containing the client certificate and private key. e.g. "/Volumes////client.jks". The materialization compute must have read permission on this volume
    keystorePasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeystorePasswordRef
    (SecretScopeReference) - Secret-scope reference for the JKS keystore password
    truststoreLocation String
    (string) - Unity Catalog volume path to the JKS truststore file containing the CA certificate(s) trusted to verify the Kafka broker's server certificate. e.g. "/Volumes////truststore.jks"
    truststorePasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigTruststorePasswordRef
    (SecretScopeReference) - Secret-scope reference for the JKS truststore password
    disableHostnameVerification Boolean
    (boolean) - Set to true only when the broker certificate's SAN intentionally does not match the connection endpoint — for example when reaching the cluster through a PrivateLink endpoint whose DNS name is not in the broker certificate. Skipping the hostname check removes a defense against man-in-the-middle attacks; do not enable casually. mTLS client authentication is unaffected by this option.
    keyPasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeyPasswordRef
    (SecretScopeReference) - Secret-scope reference for the private key password. Often the same value as the keystore password (keytool's default), but provided as a separate field because Apache Kafka requires it as a distinct option (kafka.ssl.key.password)
    keystoreLocation string
    (string) - Unity Catalog volume path to the JKS keystore file containing the client certificate and private key. e.g. "/Volumes////client.jks". The materialization compute must have read permission on this volume
    keystorePasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeystorePasswordRef
    (SecretScopeReference) - Secret-scope reference for the JKS keystore password
    truststoreLocation string
    (string) - Unity Catalog volume path to the JKS truststore file containing the CA certificate(s) trusted to verify the Kafka broker's server certificate. e.g. "/Volumes////truststore.jks"
    truststorePasswordRef GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigTruststorePasswordRef
    (SecretScopeReference) - Secret-scope reference for the JKS truststore password
    disableHostnameVerification boolean
    (boolean) - Set to true only when the broker certificate's SAN intentionally does not match the connection endpoint — for example when reaching the cluster through a PrivateLink endpoint whose DNS name is not in the broker certificate. Skipping the hostname check removes a defense against man-in-the-middle attacks; do not enable casually. mTLS client authentication is unaffected by this option.
    key_password_ref GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeyPasswordRef
    (SecretScopeReference) - Secret-scope reference for the private key password. Often the same value as the keystore password (keytool's default), but provided as a separate field because Apache Kafka requires it as a distinct option (kafka.ssl.key.password)
    keystore_location str
    (string) - Unity Catalog volume path to the JKS keystore file containing the client certificate and private key. e.g. "/Volumes////client.jks". The materialization compute must have read permission on this volume
    keystore_password_ref GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeystorePasswordRef
    (SecretScopeReference) - Secret-scope reference for the JKS keystore password
    truststore_location str
    (string) - Unity Catalog volume path to the JKS truststore file containing the CA certificate(s) trusted to verify the Kafka broker's server certificate. e.g. "/Volumes////truststore.jks"
    truststore_password_ref GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigTruststorePasswordRef
    (SecretScopeReference) - Secret-scope reference for the JKS truststore password
    disable_hostname_verification bool
    (boolean) - Set to true only when the broker certificate's SAN intentionally does not match the connection endpoint — for example when reaching the cluster through a PrivateLink endpoint whose DNS name is not in the broker certificate. Skipping the hostname check removes a defense against man-in-the-middle attacks; do not enable casually. mTLS client authentication is unaffected by this option.
    keyPasswordRef Property Map
    (SecretScopeReference) - Secret-scope reference for the private key password. Often the same value as the keystore password (keytool's default), but provided as a separate field because Apache Kafka requires it as a distinct option (kafka.ssl.key.password)
    keystoreLocation String
    (string) - Unity Catalog volume path to the JKS keystore file containing the client certificate and private key. e.g. "/Volumes////client.jks". The materialization compute must have read permission on this volume
    keystorePasswordRef Property Map
    (SecretScopeReference) - Secret-scope reference for the JKS keystore password
    truststoreLocation String
    (string) - Unity Catalog volume path to the JKS truststore file containing the CA certificate(s) trusted to verify the Kafka broker's server certificate. e.g. "/Volumes////truststore.jks"
    truststorePasswordRef Property Map
    (SecretScopeReference) - Secret-scope reference for the JKS truststore password
    disableHostnameVerification Boolean
    (boolean) - Set to true only when the broker certificate's SAN intentionally does not match the connection endpoint — for example when reaching the cluster through a PrivateLink endpoint whose DNS name is not in the broker certificate. Skipping the hostname check removes a defense against man-in-the-middle attacks; do not enable casually. mTLS client authentication is unaffected by this option.

    GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeyPasswordRef

    Key string
    (string) - The key within the scope
    Scope string
    (string) - The Databricks secret scope name
    Key string
    (string) - The key within the scope
    Scope string
    (string) - The Databricks secret scope name
    key string
    (string) - The key within the scope
    scope string
    (string) - The Databricks secret scope name
    key String
    (string) - The key within the scope
    scope String
    (string) - The Databricks secret scope name
    key string
    (string) - The key within the scope
    scope string
    (string) - The Databricks secret scope name
    key str
    (string) - The key within the scope
    scope str
    (string) - The Databricks secret scope name
    key String
    (string) - The key within the scope
    scope String
    (string) - The Databricks secret scope name

    GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigKeystorePasswordRef

    Key string
    (string) - The key within the scope
    Scope string
    (string) - The Databricks secret scope name
    Key string
    (string) - The key within the scope
    Scope string
    (string) - The Databricks secret scope name
    key string
    (string) - The key within the scope
    scope string
    (string) - The Databricks secret scope name
    key String
    (string) - The key within the scope
    scope String
    (string) - The Databricks secret scope name
    key string
    (string) - The key within the scope
    scope string
    (string) - The Databricks secret scope name
    key str
    (string) - The key within the scope
    scope str
    (string) - The Databricks secret scope name
    key String
    (string) - The key within the scope
    scope String
    (string) - The Databricks secret scope name

    GetFeatureEngineeringKafkaConfigAuthConfigMtlsConfigTruststorePasswordRef

    Key string
    (string) - The key within the scope
    Scope string
    (string) - The Databricks secret scope name
    Key string
    (string) - The key within the scope
    Scope string
    (string) - The Databricks secret scope name
    key string
    (string) - The key within the scope
    scope string
    (string) - The Databricks secret scope name
    key String
    (string) - The key within the scope
    scope String
    (string) - The Databricks secret scope name
    key string
    (string) - The key within the scope
    scope string
    (string) - The Databricks secret scope name
    key str
    (string) - The key within the scope
    scope str
    (string) - The Databricks secret scope name
    key String
    (string) - The key within the scope
    scope String
    (string) - The Databricks secret scope name

    GetFeatureEngineeringKafkaConfigBackfillSource

    DeltaTableName string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    DeltaTableSource GetFeatureEngineeringKafkaConfigBackfillSourceDeltaTableSource
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    DeltaTableName string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    DeltaTableSource GetFeatureEngineeringKafkaConfigBackfillSourceDeltaTableSource
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    delta_table_name string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    delta_table_source object
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    deltaTableName String
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    deltaTableSource GetFeatureEngineeringKafkaConfigBackfillSourceDeltaTableSource
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    deltaTableName string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    deltaTableSource GetFeatureEngineeringKafkaConfigBackfillSourceDeltaTableSource
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    delta_table_name str
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    delta_table_source GetFeatureEngineeringKafkaConfigBackfillSourceDeltaTableSource
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    deltaTableName String
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    deltaTableSource Property Map
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored

    GetFeatureEngineeringKafkaConfigBackfillSourceDeltaTableSource

    FullName string
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    DataframeSchema string
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    EntityColumns List<string>
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    FilterCondition string
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    TimeseriesColumn string
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    TransformationSql string
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    FullName string
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    DataframeSchema string
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    EntityColumns []string
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    FilterCondition string
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    TimeseriesColumn string
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    TransformationSql string
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    full_name string
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    dataframe_schema string
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    entity_columns list(string)
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    filter_condition string
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    timeseries_column string
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    transformation_sql string
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    fullName String
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    dataframeSchema String
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    entityColumns List<String>
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    filterCondition String
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    timeseriesColumn String
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    transformationSql String
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    fullName string
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    dataframeSchema string
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    entityColumns string[]
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    filterCondition string
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    timeseriesColumn string
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    transformationSql string
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    full_name str
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    dataframe_schema str
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    entity_columns Sequence[str]
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    filter_condition str
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    timeseries_column str
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    transformation_sql str
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    fullName String
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    dataframeSchema String
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    entityColumns List<String>
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    filterCondition String
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    timeseriesColumn String
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    transformationSql String
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe

    GetFeatureEngineeringKafkaConfigIngestionConfig

    BackfillJobId int
    (integer) - The ID of the Databricks Job that performs the historical backfill of the ingestion Delta table
    IngestionDestination GetFeatureEngineeringKafkaConfigIngestionConfigIngestionDestination
    (IngestionDestination) - Destination for the Databricks-managed Delta table that holds an offline copy of the streaming data for querying and training. This table contains both 1) forward-filled data from the Stream and 2) backfilled data from the BackfillSource (if provided). This table is created and managed by Databricks and is deleted when the Stream is deleted
    IngestionJobId int
    (integer) - The ID of the Databricks Job that performs the forward-fill ingestion
    IngestionPipelineId string
    (string) - The ID of the SDP pipeline that continuously copies new events from the streaming source into the ingestion Delta table
    BackfillSource GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSource
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    DeduplicationColumns List<string>
    (list of string) - Column paths used to identify duplicate rows during ingestion; only one row per distinct combination of these values is kept. Use dot notation for nested fields (e.g. value.user_id). Empty list means every column is compared
    BackfillJobId int
    (integer) - The ID of the Databricks Job that performs the historical backfill of the ingestion Delta table
    IngestionDestination GetFeatureEngineeringKafkaConfigIngestionConfigIngestionDestination
    (IngestionDestination) - Destination for the Databricks-managed Delta table that holds an offline copy of the streaming data for querying and training. This table contains both 1) forward-filled data from the Stream and 2) backfilled data from the BackfillSource (if provided). This table is created and managed by Databricks and is deleted when the Stream is deleted
    IngestionJobId int
    (integer) - The ID of the Databricks Job that performs the forward-fill ingestion
    IngestionPipelineId string
    (string) - The ID of the SDP pipeline that continuously copies new events from the streaming source into the ingestion Delta table
    BackfillSource GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSource
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    DeduplicationColumns []string
    (list of string) - Column paths used to identify duplicate rows during ingestion; only one row per distinct combination of these values is kept. Use dot notation for nested fields (e.g. value.user_id). Empty list means every column is compared
    backfill_job_id number
    (integer) - The ID of the Databricks Job that performs the historical backfill of the ingestion Delta table
    ingestion_destination object
    (IngestionDestination) - Destination for the Databricks-managed Delta table that holds an offline copy of the streaming data for querying and training. This table contains both 1) forward-filled data from the Stream and 2) backfilled data from the BackfillSource (if provided). This table is created and managed by Databricks and is deleted when the Stream is deleted
    ingestion_job_id number
    (integer) - The ID of the Databricks Job that performs the forward-fill ingestion
    ingestion_pipeline_id string
    (string) - The ID of the SDP pipeline that continuously copies new events from the streaming source into the ingestion Delta table
    backfill_source object
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    deduplication_columns list(string)
    (list of string) - Column paths used to identify duplicate rows during ingestion; only one row per distinct combination of these values is kept. Use dot notation for nested fields (e.g. value.user_id). Empty list means every column is compared
    backfillJobId Integer
    (integer) - The ID of the Databricks Job that performs the historical backfill of the ingestion Delta table
    ingestionDestination GetFeatureEngineeringKafkaConfigIngestionConfigIngestionDestination
    (IngestionDestination) - Destination for the Databricks-managed Delta table that holds an offline copy of the streaming data for querying and training. This table contains both 1) forward-filled data from the Stream and 2) backfilled data from the BackfillSource (if provided). This table is created and managed by Databricks and is deleted when the Stream is deleted
    ingestionJobId Integer
    (integer) - The ID of the Databricks Job that performs the forward-fill ingestion
    ingestionPipelineId String
    (string) - The ID of the SDP pipeline that continuously copies new events from the streaming source into the ingestion Delta table
    backfillSource GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSource
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    deduplicationColumns List<String>
    (list of string) - Column paths used to identify duplicate rows during ingestion; only one row per distinct combination of these values is kept. Use dot notation for nested fields (e.g. value.user_id). Empty list means every column is compared
    backfillJobId number
    (integer) - The ID of the Databricks Job that performs the historical backfill of the ingestion Delta table
    ingestionDestination GetFeatureEngineeringKafkaConfigIngestionConfigIngestionDestination
    (IngestionDestination) - Destination for the Databricks-managed Delta table that holds an offline copy of the streaming data for querying and training. This table contains both 1) forward-filled data from the Stream and 2) backfilled data from the BackfillSource (if provided). This table is created and managed by Databricks and is deleted when the Stream is deleted
    ingestionJobId number
    (integer) - The ID of the Databricks Job that performs the forward-fill ingestion
    ingestionPipelineId string
    (string) - The ID of the SDP pipeline that continuously copies new events from the streaming source into the ingestion Delta table
    backfillSource GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSource
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    deduplicationColumns string[]
    (list of string) - Column paths used to identify duplicate rows during ingestion; only one row per distinct combination of these values is kept. Use dot notation for nested fields (e.g. value.user_id). Empty list means every column is compared
    backfill_job_id int
    (integer) - The ID of the Databricks Job that performs the historical backfill of the ingestion Delta table
    ingestion_destination GetFeatureEngineeringKafkaConfigIngestionConfigIngestionDestination
    (IngestionDestination) - Destination for the Databricks-managed Delta table that holds an offline copy of the streaming data for querying and training. This table contains both 1) forward-filled data from the Stream and 2) backfilled data from the BackfillSource (if provided). This table is created and managed by Databricks and is deleted when the Stream is deleted
    ingestion_job_id int
    (integer) - The ID of the Databricks Job that performs the forward-fill ingestion
    ingestion_pipeline_id str
    (string) - The ID of the SDP pipeline that continuously copies new events from the streaming source into the ingestion Delta table
    backfill_source GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSource
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    deduplication_columns Sequence[str]
    (list of string) - Column paths used to identify duplicate rows during ingestion; only one row per distinct combination of these values is kept. Use dot notation for nested fields (e.g. value.user_id). Empty list means every column is compared
    backfillJobId Number
    (integer) - The ID of the Databricks Job that performs the historical backfill of the ingestion Delta table
    ingestionDestination Property Map
    (IngestionDestination) - Destination for the Databricks-managed Delta table that holds an offline copy of the streaming data for querying and training. This table contains both 1) forward-filled data from the Stream and 2) backfilled data from the BackfillSource (if provided). This table is created and managed by Databricks and is deleted when the Stream is deleted
    ingestionJobId Number
    (integer) - The ID of the Databricks Job that performs the forward-fill ingestion
    ingestionPipelineId String
    (string) - The ID of the SDP pipeline that continuously copies new events from the streaming source into the ingestion Delta table
    backfillSource Property Map
    (BackfillSource) - A user-provided source for backfilling data. Historical data is used when creating a training set from streaming features linked to this Stream. The backfill data stored in this location will be copied into the ingestion table for offline querying and training. The schema for this source must match exactly that of the key and payload schemas specified for this Stream
    deduplicationColumns List<String>
    (list of string) - Column paths used to identify duplicate rows during ingestion; only one row per distinct combination of these values is kept. Use dot notation for nested fields (e.g. value.user_id). Empty list means every column is compared

    GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSource

    DeltaTableName string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    DeltaTableSource GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSourceDeltaTableSource
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    DeltaTableName string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    DeltaTableSource GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSourceDeltaTableSource
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    delta_table_name string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    delta_table_source object
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    deltaTableName String
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    deltaTableSource GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSourceDeltaTableSource
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    deltaTableName string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    deltaTableSource GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSourceDeltaTableSource
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    delta_table_name str
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    delta_table_source GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSourceDeltaTableSource
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored
    deltaTableName String
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    deltaTableSource Property Map
    (DeltaTableSource, deprecated) - Deprecated: Use deltaTableName instead. Kept for backwards compatibility. The Delta table source containing the historical data to backfill. Only the delta table name is used for backfill, other fields are ignored

    GetFeatureEngineeringKafkaConfigIngestionConfigBackfillSourceDeltaTableSource

    FullName string
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    DataframeSchema string
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    EntityColumns List<string>
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    FilterCondition string
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    TimeseriesColumn string
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    TransformationSql string
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    FullName string
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    DataframeSchema string
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    EntityColumns []string
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    FilterCondition string
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    TimeseriesColumn string
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    TransformationSql string
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    full_name string
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    dataframe_schema string
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    entity_columns list(string)
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    filter_condition string
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    timeseries_column string
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    transformation_sql string
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    fullName String
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    dataframeSchema String
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    entityColumns List<String>
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    filterCondition String
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    timeseriesColumn String
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    transformationSql String
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    fullName string
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    dataframeSchema string
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    entityColumns string[]
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    filterCondition string
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    timeseriesColumn string
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    transformationSql string
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    full_name str
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    dataframe_schema str
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    entity_columns Sequence[str]
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    filter_condition str
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    timeseries_column str
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    transformation_sql str
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe
    fullName String
    (string) - The full three-part (catalog, schema, table) name of the Delta table
    dataframeSchema String
    (string) - Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformationSql is specified. Example: {"type":"struct","fields":[{"name":"colA","type":"integer","nullable":true,"metadata":{}},{"name":"colC","type":"integer","nullable":true,"metadata":{}}]}
    entityColumns List<String>
    (list of string, deprecated) - Deprecated: Use Feature.entity instead. Kept for backwards compatibility. The entity columns of the Delta table
    filterCondition String
    (string) - Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections
    timeseriesColumn String
    (string, deprecated) - Deprecated: Use Feature.timeseries_column instead. Kept for backwards compatibility. The timeseries column of the Delta table
    transformationSql String
    (string) - A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. "SELECT , colA + colB AS colC FROM x.y.z WHERE colA > 0" would have transformationSql ", colA + colB AS colC") If transformationSql is not provided, all columns of the delta table are present in the DataSource dataframe

    GetFeatureEngineeringKafkaConfigIngestionConfigIngestionDestination

    DeltaTableName string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    DeltaTableName string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    delta_table_name string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    deltaTableName String
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    deltaTableName string
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    delta_table_name str
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion
    deltaTableName String
    (string) - The full three-part name (catalog, schema, name) of the Delta table to be created for ingestion

    GetFeatureEngineeringKafkaConfigKeySchema

    JsonSchema string
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    JsonSchema string
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    json_schema string
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    jsonSchema String
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    jsonSchema string
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    json_schema str
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    jsonSchema String
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)

    GetFeatureEngineeringKafkaConfigProviderConfig

    WorkspaceId string
    Workspace ID which the resource belongs to. This workspace must be part of the account which the provider is configured with.
    WorkspaceId string
    Workspace ID which the resource belongs to. This workspace must be part of the account which the provider is configured with.
    workspace_id string
    Workspace ID which the resource belongs to. This workspace must be part of the account which the provider is configured with.
    workspaceId String
    Workspace ID which the resource belongs to. This workspace must be part of the account which the provider is configured with.
    workspaceId string
    Workspace ID which the resource belongs to. This workspace must be part of the account which the provider is configured with.
    workspace_id str
    Workspace ID which the resource belongs to. This workspace must be part of the account which the provider is configured with.
    workspaceId String
    Workspace ID which the resource belongs to. This workspace must be part of the account which the provider is configured with.

    GetFeatureEngineeringKafkaConfigSubscriptionMode

    Assign string
    (string) - A JSON string that contains the specific topic-partitions to consume from. For example, for '{"topicA":[0,1],"topicB":[2,4]}', topicA's 0'th and 1st partitions will be consumed from
    Subscribe string
    (string) - A comma-separated list of Kafka topics to read from. For example, 'topicA,topicB,topicC'
    SubscribePattern string
    (string) - A regular expression matching topics to subscribe to. For example, 'topic.*' will subscribe to all topics starting with 'topic'
    Assign string
    (string) - A JSON string that contains the specific topic-partitions to consume from. For example, for '{"topicA":[0,1],"topicB":[2,4]}', topicA's 0'th and 1st partitions will be consumed from
    Subscribe string
    (string) - A comma-separated list of Kafka topics to read from. For example, 'topicA,topicB,topicC'
    SubscribePattern string
    (string) - A regular expression matching topics to subscribe to. For example, 'topic.*' will subscribe to all topics starting with 'topic'
    assign string
    (string) - A JSON string that contains the specific topic-partitions to consume from. For example, for '{"topicA":[0,1],"topicB":[2,4]}', topicA's 0'th and 1st partitions will be consumed from
    subscribe string
    (string) - A comma-separated list of Kafka topics to read from. For example, 'topicA,topicB,topicC'
    subscribe_pattern string
    (string) - A regular expression matching topics to subscribe to. For example, 'topic.*' will subscribe to all topics starting with 'topic'
    assign String
    (string) - A JSON string that contains the specific topic-partitions to consume from. For example, for '{"topicA":[0,1],"topicB":[2,4]}', topicA's 0'th and 1st partitions will be consumed from
    subscribe String
    (string) - A comma-separated list of Kafka topics to read from. For example, 'topicA,topicB,topicC'
    subscribePattern String
    (string) - A regular expression matching topics to subscribe to. For example, 'topic.*' will subscribe to all topics starting with 'topic'
    assign string
    (string) - A JSON string that contains the specific topic-partitions to consume from. For example, for '{"topicA":[0,1],"topicB":[2,4]}', topicA's 0'th and 1st partitions will be consumed from
    subscribe string
    (string) - A comma-separated list of Kafka topics to read from. For example, 'topicA,topicB,topicC'
    subscribePattern string
    (string) - A regular expression matching topics to subscribe to. For example, 'topic.*' will subscribe to all topics starting with 'topic'
    assign str
    (string) - A JSON string that contains the specific topic-partitions to consume from. For example, for '{"topicA":[0,1],"topicB":[2,4]}', topicA's 0'th and 1st partitions will be consumed from
    subscribe str
    (string) - A comma-separated list of Kafka topics to read from. For example, 'topicA,topicB,topicC'
    subscribe_pattern str
    (string) - A regular expression matching topics to subscribe to. For example, 'topic.*' will subscribe to all topics starting with 'topic'
    assign String
    (string) - A JSON string that contains the specific topic-partitions to consume from. For example, for '{"topicA":[0,1],"topicB":[2,4]}', topicA's 0'th and 1st partitions will be consumed from
    subscribe String
    (string) - A comma-separated list of Kafka topics to read from. For example, 'topicA,topicB,topicC'
    subscribePattern String
    (string) - A regular expression matching topics to subscribe to. For example, 'topic.*' will subscribe to all topics starting with 'topic'

    GetFeatureEngineeringKafkaConfigValueSchema

    JsonSchema string
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    JsonSchema string
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    json_schema string
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    jsonSchema String
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    jsonSchema string
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    json_schema str
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)
    jsonSchema String
    (string) - Schema of the JSON object in standard IETF JSON schema format (https://json-schema.org/)

    Package Details

    Repository
    databricks pulumi/pulumi-databricks
    License
    Apache-2.0
    Notes
    This Pulumi package is based on the databricks Terraform Provider.
    databricks logo
    Viewing docs for Databricks v1.96.0
    published on Thursday, Jun 18, 2026 by Pulumi
    Schema (JSON)
    pulumi/pulumi-databricks

      Try Pulumi Cloud free.
      Your team will thank you.

      Start free trial