pachyderm | Data-Centric Pipelines and Data Versioning

Like some of the comments above suggest. It looks like your test_pclean.py file isn't in the docker image. Here's what should fix it.

1. Make sure your test_pclean.py file is in your docker image by having be included as part of the build process. Put this as the last step in your dockerfile:

COPY test_pclean.py .

1. Ensure that your pachyderm pipeline spec has the following for the cmd portion: "cmd": ["python3", "./test_pclean.py"]

2. And this is more of a suggestion than a requirement.... You'll make life easier for yourself if you use image tags as part of your docker build. If you default to latest tag, any future iterations/builds of this step in your pipeline could have negitave affects (new bugs in your code etc.). Therefore the best practice is to use a particular version in your pipeline: mopng-beneficiary-v2-image-7:v1 and mopng-beneficiary-v2-image-7:v2 and so on. That way you can iterate on say version 3 and it won't affect the already running pipeline.

docker build -t avisrivastava254084/mopng-beneficiary-v2-image-7:v1

Then just update your pipeline spec to use avisrivastava254084/mopng-beneficiary-v2-image-7:v1