Terraform + EMR Bootstrap PySpark with Anaconda

This code should help to jump start PySpark with Anaconda on AWS using Terraform.

This project is no longer actively maintained and has been archived on Jan 7, 2025.

Getting Started

• Install Terraform on Mac: brew install terraform
• Adjust the scripts (bootstrapactions.sh and pysparkquick_setup.sh) in scripts if necessary
• Set parameters in terraform.tfvars
• Start cluster:

terraform init
terraform apply

• Destroy cluster:

terraform destroy

Notes

• Configure AWS on your local machine: aws configure
• AWS instance cost for eu-central-1

Maintainers

• Dat Tran, github: datitran

Copyright

See LICENSE for details.