A project is basically a
git repository. For example, a Project might be organized like:
. ├── .git/ ├── .orchest │ ├── environments/ │ └── pipelines/ ├── california_housing.orchest ├── collect-results.ipynb └── get-data.py
Projects also contain jobs, however, these are not stored in the project’s filesystem.
You can access Project files in your code running inside environments using relative paths. For absolute paths, all files of a project are mounted to the
You can get started with Projects by:
Creating a new project
Importing an existing project using its git repository URL (see how to import a project).
Importing Orchest curated or community contributed examples.
👉 See quickstart tutorial.
.orchest directory should be versioned since it defines the environment in use. This enables the Project to run on every machine.
/data directory can be used to store data locally. The
/data directory can be accessed by all pipelines across all projects, even by jobs.
Secrets should be set with environment variables to avoid them being versioned.
git inside Orchest¶
👉 See video tutorial: versioning using git in Orchest.
git config --global user.name "John Doe" git config --global user.email "firstname.lastname@example.org"
Use the following command to add a private SSH key to your terminal session in JupyterLab:
echo "chmod 400 /data/id_rsa" >> ~/.bashrc echo "ssh-add /data/id_rsa 2>/dev/null" >> ~/.bashrc echo "if [ -z \$SSH_AGENT_PID ]; then exec ssh-agent bash; fi" >> ~/.bashrc mkdir -p ~/.ssh printf "%s\n" "Host github.com" " IdentityFile /data/id_rsa" >> ~/.ssh/config ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
id_rsa private key file is uploaded through the pipeline file manager in the
🚨 Adding a private key file to the
/data folder exposes it to everyone using your Orchest instance.
You can then version using
JupyterLab git extension UI.