sagemaker-containers | Please use the SageMaker Training Toolkit | Machine Learning library

When running SageMaker in a local Jupyter notebook, it expects the Docker container to be running on the local machine as well.

The key to ensuring that SageMaker (running in a local notebook) uses the AWS hosted docker container, is to omit the LocalSession object when initializing the Estimator.

A SageMaker training job in "local" is actually executing inside of a Docker container that is isolated from the Python kernel that is executing your notebook. Therefore, the plt.show() in the train_cnn.py script doesn't actually get routed to the notebook UI in the same way that executing that command directly from a notebook would.

Instead of using plt.show(), consider using plt.savefig() to output the plot to an image:

edit 07/10: adding the training key in the local reading path: replacing opt/ml/input/data/orders.csv by opt/ml/input/data/train/orders.csv

You have an error because your data = pd.read_csv(data_location, ...) tries to read from S3. Try replacing by data = pd.read_csv('opt/ml/input/data/orders.csv', ...)

If you use SageMaker, you do not need to read from S3 inside your training script: SageMaker does the copy from S3 to EC2 for you.

Instead, as indicated in the documentation, you only need to read the data from the local path opt/ml/input/data/ where is the key used to name your input in the training call model.fit({'': 's3://my data'}). Note that local here means local to the remote ephemeral SageMaker Training EC2 instance, not to the SageMaker Notebook EC2 instance you may be using for development and orchestration.

Same thing with the artifact copy to s3: you do not need to do it yourself. Just write the artifact in the local path opt/ml/model, and the service will copy it back to S3. Some AWS-provided containers (like the sklearn container) also provide input data path and artefact path in environment variable(SM_CHANNEL_ and SM_MODEL_DIR), which you can optionally use to avoid hard-coding them in your code. You can get inspiration from this random forest demo and adapt it to your case. You do not need s3fs.

For older containers using the deprecated sagemaker_containers, the approach you described is correct.

For newer containers that use sagemaker-training-toolkit, this is how you retrieve information about the environment: https://github.com/aws/sagemaker-training-toolkit#get-information-about-the-container-environment

I found a hint in the AWS docs and came up with this solution:

Following @chepner's comment an example solution would be something like this:

config.ini file: