Databricks AWS Platform Architect

An administrator is accountable for who can do what. You separate account and workspace roles, bring in users and groups, and set the workspace settings and controls that keep a shared platform predictable.

• Account admins and workspace admins
• Users, groups and service principals
• Single sign-on and automatic provisioning
• Workspace settings and feature controls
• Cluster policies as guardrails

You build An administration plan with roles, groups and workspace settings for three teams.

Console clicks do not scale across many workspaces. You call the account-level API to create the building blocks of a workspace, authenticate as a service principal, and describe the result as code that can be repeated and reviewed.

• What the account API manages
• Authenticating with service principals and OAuth
• Credential, storage and network configurations
• Creating a workspace through the API
• Describing the same setup in Terraform
• Handling errors and cleaning up

You build A script that creates a workspace from configuration objects through the account API.

The default network suits a trial, not a regulated company. You deploy into your own VPC, plan its subnets and security groups, and decide how clusters reach the internet, AWS services and the control plane.

• Why use a customer-managed VPC
• CIDR planning and subnets across zones
• Security group rules that clusters need
• Egress: NAT, firewalls and allowed destinations
• VPC endpoints for AWS services
• Validating a network configuration before use

You build A VPC design with subnets, security groups and endpoints, validated against a workspace.

Keeping traffic off the public internet is a design choice with several parts. You connect users to the workspace and clusters to the control plane privately, and restrict who can reach the workspace at all.

• Front-end and back-end private connectivity
• VPC endpoints for Databricks services
• Private access settings on the workspace
• IP access lists for the workspace and account
• Testing that traffic follows the private path

You build A private connectivity plan for users and clusters, with the checks that confirm it.

Your data already sits in S3 buckets. You grant Databricks access through IAM roles and trust policies, register storage credentials and external locations in Unity Catalog, and govern the files from there.

• Root storage and data buckets
• IAM roles, trust policies and external IDs
• Storage credentials in Unity Catalog
• External locations and volumes
• Bucket policies and least-privilege access
• Diagnosing an access denied error

You build A bucket connected through a storage credential and external location with minimal permissions.

Databricks rarely works alone. You connect clusters to streaming, catalog and warehouse services on AWS, and give workloads credentials through roles rather than stored keys.

• IAM roles and instance profiles for clusters
• Reading streams from Kinesis
• Glue and Redshift as sources for federation
• Storing secrets for external services
• Auditing which service calls a workload made

You build A cluster setup that reads from two AWS services without any stored credentials.

Some organizations must hold the keys to their own data. You apply customer-managed keys to workspace storage and to managed services, write the key policies that allow it, and plan for rotation and revocation.

• What encryption protects and what it does not
• Keys for managed services and workspace storage
• Writing KMS key policies for Databricks
• Rotation and what revoking a key breaks
• Adding keys to an existing workspace

You build A key setup and policy for one workspace, with a plan for rotation and revocation.

You assemble a full deployment from what came before: API-driven creation, a customer-managed VPC with private connectivity, governed storage and customer-managed keys. Then you review it as a security team would.

• Turning requirements into a deployment design
• Creating the network, storage and keys
• Standing up the workspace through the API
• Verifying isolation and access
• Documenting the design for reviewers

You build A repeatable, documented workspace deployment with network isolation and managed keys.