How to build and version control pipelines
Before using the pipeline, you need to build it. Building it merges the custom logic of the pipeline itself with the CLI and UI logic so that you can later use it.
Building a pre-trained pipeline
BojAI contains several pre-trained pipelines, which you do not need to code and you can just use to process, train, and deploy pipelines. You can follow the steps below to use them:
-
Run
bojai list --pipelines
to see what pipelines are available. You can also optionally refer to the pipelines’ documentation to get details on what each pipeline does, how it works, and what models it uses. -
Once you know which pipeline to use, run
bojai build --pipeline chosen-pipeline-name
.
Now you have a built pipeline and you can use it either through command-line interface, or visual interface.
Building a custom pipeline
If you want to build your custom pipeline, you need to create it first by following the steps below:
-
Run
bojai create --pipeline your-custom-pipeline --directory directory-where-to-save-pipeline-folder
. This creates a folder with the name you gave in--pipeline
in the directory you specified in--directory
. -
Access the pipeline folder in the directory you specified, and insert your data processing, training, evaluation, and model usage logic. Refer to How to code a custom pipeline for more details.
Once coding of one stage is finished, you can start using your pipeline. To use your pipeline you need to build it then start it. You can build your model by following the steps below:
-
Run
bojai list --build
to see if you had already built your pipeline. If you did, you can rebuild it but the files will be overwritten. -
Run
bojai build --piplein give-a-name --directory custom-pipeline-directory
if this is the first time you build it, or
Runbojai build --directory custom-pipeline-directory --replace
if you had already built it and want to update it.
The directory when building your custom model is the directory you entered in the bojai create command plus the name of your pipeline. For example: if you run bojai create --pipeline mypip --directory ./pipelines
then the build command should be bojai build --directory ./pipelines/mypip
.
Now you have a built pipeline and you can use it either through command-line interface, or visual interface.
Building a modified pipeline or changed UI/CLI
If you modified any of the pre-built pipelines, and/or you changed the CLI or UI using bojai checkout, you can build your model by running the command below:
bojai build --piplein pipeline-name --directory checkout-directiony,modifydirectory
Meaning, you can provide multiple directories, the build will prioritize the files in the provided directories before using the default ones. Meaning, your modified UI/CLI or modified pipelines will have priority.
If you are using your own pipeline and checkout out UI and CLI or both, you can follow the usual build a custom pipeline process along with providing the checkout directories separated by commas in the following style:
bojai build --piplein give-a-name --directory custom-pipeline-directory,checkout-directiony
Version control through git
You can use git to version control your custom pipeline. This is especially useful if you’re actively improving the processor, trainer, or user logic over time, or working with a team.
Version control happens on the custom pipeline’s directory — the one you pass to --directory
when building.
Follow the steps below to version control your pipeline:
-
Go to your custom pipeline directory:
cd path/to/your/custom-pipeline
-
Initialize a git repository in the folder:
git init
-
Add the files you want to track:
git add .
-
Commit your changes:
git commit -m "Initial version of my custom pipeline"
-
(Optional) Connect to a remote repository (like GitHub or GitLab) to back up your work or collaborate:
git remote add origin https://github.com/yourusername/yourrepo.git git push -u origin main
-
Every time you make changes to your code (like improving your trainer or processor), you can repeat the following:
git add . git commit -m "Describe what you changed" git push
-
Finally, after each git update or changes to your pipeline, remember to rebuild your pipeline before using it:
bojai build --directory path/to/your/custom-pipeline --replace
This makes sure the most recent version of your code is what gets run when you use BojAI.
You now have a version-controlled pipeline that is easier to manage, track, and share.