Skip to main content
Version: v2.1.0

Credentials configuration

To be able to read data from different data sources, you need to pass credentials either as a parameter to specific methods or via environmental variables.

Specifying using environmental variable​

AWS S3​

  • S3_ACCESS_KEY
  • S3_SECRET_KEY
  • S3_REGION
note

An optional parameter, S3_ROLE_ARN, can be specified. If specified, an AWS IAM role that delegates access to the bucket will be used.

Without an environmental variable or AWS credential, you are still able to access public S3 data.

Minio​

  • S3_ACCESS_KEY
  • S3_SECRET_KEY
  • S3_REGION
  • S3_ENDPOINT
note

An optional parameter, S3_ROLE_ARN , can be specified. If specified, a Minio IAM role that delegates access to the bucket will be used.

S3_ENDPOINT should be provided so that Feature Store can read from the corresponding Minio server.

JDBC Postgres​

  • JDBC_POSTGRES_USER
  • JDBC_POSTGRES_PASSWORD

JDBC Teradata​

  • JDBC_TERADATA_USER
  • JDBC_TERADATA_PASSWORD

Azure credentials​

Feature Store provides three variants for providing Azure Credentials

Azure name and key credentials​

  • AZURE_ACCOUNT_NAME is the name of the Azure storage account where the data source is stored.
  • AZURE_ACCOUNT_KEY is the key for the Azure storage account where the data source is stored.

Azure SAS credentials​

  • AZURE_ACCOUNT_NAME is the name of the Azure storage account where the data source is stored.
  • AZURE_SAS_TOKEN is the Shared Access Signature (SAS) token for the Azure storage account. It grants restricted access to Azure Storage resources. You can use this form of authentication if provided with a SAS.

Azure principal credentials​

  • AZURE_ACCOUNT_NAME is the name of the Azure storage account where the data source is stored.
  • AZURE_SP_CLIENT_ID is the client ID for an Azure Service Principal. It is used to identify and authenticate the Service Principal when it requests access to Azure resources.
  • AZURE_SP_TENANT_ID is the tenant ID for the Azure Active Directory tenant associated with the Service Principal.
  • AZURE_SP_SECRET is the client secret for an Azure Service Principal.

S3 credentials​

  • S3_ACCESS_KEY is the Access Key ID for an Amazon S3 (Simple Storage Service) bucket.
  • S3_SECRET_KEY is the Secret Access Key for the S3 bucket.
  • S3_REGION is the AWS region where the S3 bucket is located.

Snowflake credentials​

Feature Store provides two variants for providing Snowflake Credentials

Snowflake user and password credentials​

  • SNOWFLAKE_USER is the username for accessing the Snowflake database.
  • SNOWFLAKE_PASSWORD is the password for the corresponding user account to authenticate the user when logging in to the Snowflake database.

Snowflake key pair credentials​

  • SNOWFLAKE_USER is the username for accessing the Snowflake database.
  • SNOWFLAKE_PRIVATE_KEY_FILE is the location of private key pem file on users local machine from where private key is read and sent to Snowflake where is checked against public key part that's stored in Snowflake.
  • PRIVATE_KEY_PASSPHRASE in the case the private key pem file is encrypted a passphrase is needed.

Teradata credentials​

  • USER is the username for accessing the Teradata database.
  • PASSWORD is the password for the corresponding user account to authenticate the user when logging in to the Teradata database.

Postgres credentials​

  • USER is the username for accessing the PostgreSQL database.
  • PASSWORD is the password for the corresponding user account to authenticate the user when logging in to the PostgreSQL database.

GCP credentials​

  • GOOGLE_APPLICATION_CREDENTIALS environment variable will be evaluated and the corresponding credentials file will be utilized by default
  • Alternatively, user can create and use a GcpCredentials instance by specifying his/her own file path pointing to stored GCP credentials file.
from featurestore import *
credentials = GcpCredentials.from_file(local_file_path)

Passing credentials as a parameters​

from featurestore import *
credentials = AzureKeyCredentials(AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY)
credentials = AzureSasCredentials(AZURE_ACCOUNT_NAME, AZURE_SAS_TOKEN)
credentials = AzurePrincipalCredentials(AZURE_ACCOUNT_NAME, AZURE_SP_CLIENT_ID, AZURE_SP_TENANT_ID, AZURE_SP_SECRET)
credentials = S3Credentials(S3_ACCESS_KEY, S3_SECRET_KEY, S3_REGION)
credentials = SnowflakeCredentials(SNOWFLAKE_USER, SNOWFLAKE_PASSWORD)
credentails = SnowflakeKeyPairCredentials(SNOWFLAKE_USER, SNOWFLAKE_PRIVATE_KEY_FILE, PRIVATE_KEY_PASSPHRASE)
credentials = TeradataCredentials(USER, PASSWORD)
credentials = PostgresCredentials(USER, PASSWORD)

Passing secrets to environment variables in Databricks Notebook​

You can make use of Databricks dbutils package to inject secrets into environment variables.

The following example shows passing an Azure Storage Account Key from Databricks Secret Vault into the respective environment variable as required by Feature Store.

import os
os.environ["AZURE_ACCOUNT_KEY"] = dbutils.secrets.get("<scope_name>", "<key_name>")
Feedback