The current release of HTC requires python3.7, and the documentation assumes the use of virtualenv. Set this up as follows:
This will create a virtual environment for python execution with dependencies and libraries required. To activate the environment run:
echo "source ~/environment/aws-htc-grid/venv/bin/activate" >> ~/.bashrc
To validate the version of python we are using is the one from the virtual environment we just created you can run, and it should point to
To simplify the deployment we will use a set of environment variables, that later on we will be able to use and replace in multiple commands. We will also store this environment variables in a
load_variables.sh file that we will be able to use to reload them whenever we need them (for example,when opening multiple terminals in Cloud9 IDE).
export HTCGRID_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | jq -r '.region')
for var in TAG HTCGRID_REGION ; do echo "export $var=$(eval "echo \"\$$var\"")" >> load_variables.sh ; done
echo -e "===\nYour variables and configuration have been setup as follows\n===\n$(cat load_variables.sh)"
echo "source ~/environment/aws-htc-grid/load_variables.sh" >> ~/.bashrc
The code above first set the variables and then, saves all of them within the file
load_variables.sh file. Finally it creates a line within the
bashrc file so that whenever we create new terminal the variables get pre-loaded.
As for the variables that we have created:
main as the TAG for our setup. TAG will be used in the naming of the S3 buckets, so it needs to follow S3 naming rules.
The following step creates 3 S3 buckets that will be needed during the installation:
make init-grid-state TAG=$TAG REGION=$HTCGRID_REGION
To validate the creation of the S3 buckets, you can run
aws cloudformation describe-stacks --stack-name $TAG --region $HTCGRID_REGION --query 'Stacks'
That will list the 3 S3 Buckets that we just created.