Developing a model with AI4EOSC services
Developing a model with AI4EOSC services. - Borja Esteban Sanchis (KIT)
Why templates?
- Provide Consistency and Uniformity
- Speed-up development time
- Apply best learning practices
- Reduction in Errors
- Let you focus on what is important
- Are a great source of inspiration
Types of templates
- main
This is a minimal template for developing new modules in the AI4OS Platform. It uses cookiecutter to generate the templates. Simple template, with the minimum requirements to integrate your AI model in AI4OS Hub. - child-module
This template specifically tailors users performing a retraining of an existing at AI4OS-Hub module. It only creates a Docker repo whose container is based on an existing modules’ Docker image. - advanced
This is a more advanced template. It makes more assumptions on how to structure projects and adds more files. If you want to integrate an already existing AI code, which you still want to keep in a separate repository, this template is for you.
AI4EOSC Templates main features
- Preset code for coupling with DEEPaaS API
- Basic software tests
- Pre-configured Jenkins CI/CD based on Jenkins-Pipeline Library (JePL2)
- Utils to connect with a remote storage (Nextcloud)
- Containerisation
Main template example
Child template example
Advanced template
- Basic documentation generation files
- Basic testing with Pytest
- Customization of multiple API arguments by Schemas
- Basic files to start implementing CICD with Jenkins
- Basic logging and use of HTTP Responses
Advantages of the advanced template
Multiple integration modes
- From model base template
Want to start your project from the base, following good practices? You are the developer of the model source and the api. You want all to be highly coupled and in a unique same place. - From repository as submodule
Your team have a source model and want to add the API? You are not the developer of the model, but it is not published and you need the source code to run the service. - As requirement for the API
Additional code features
- Templates for responses
- Schemas module for parameters
Inspire users on how to generate the response they need.
Object schemas simplify maintenance and visibility in APIs with large amount of arguments and hyperparameters.
Provide single points where user needs to modify the code.
- Pre-made testing with pytest
- Debug configurations for vscode
We know it is hard and long to generate comprehensible tests. Therefore, we can generate basic testing for you by simply testing the known points of the API in search for failures.
Additionally you can debug with breakpoints if you use vscode.
- Jenkins SQA baseline v2
And more to come
- Data version control (with DVC)
Add data version and control to your repository meanwhile it is stored in your favorite cloud solution.
- Drift detection with Frouros
Monitor and validate your new data before merging into “main” or publish it automatically.
How to start with template?
https://templates.cloud.ai4eosc.eu
Powered by Cookiecutter
Alternatively to using https://templates.cloud.ai4eosc.eu it is possible to use command line with cookiecutter at: https://github.com/ai4os/ai4-template-adv
For example:
- cookiecutter \ https://github.com/ai4os/ai4-template-adv
- cookiecutter -c 2.0.0 \ https://github.com/ai4os/ai4-template-adv
Time for a demo?
