Getting started with cmf-server¶

cmf-server is a key interface for the user to explore and track their ML training runs. It allows users to store the metadata file on the cmf-server. The user can retrieve the saved metadata file and can view the content of the saved metadata file using the UI provided by the cmf-server.

Setup a cmf-server¶

There are two ways to start cmf server -

Using docker compose file
Using docker run

Pre-requisites¶

Clone the Github repository.

git clone https://github.com/HewlettPackard/cmf

Install Docker Engine with non root user privileges.
Install Docker Compose Plugin.

In earlier versions of docker compose, docker compose was independent of docker. Hence, docker-compose was command. However, after introduction of Docker Compose Desktop V2, compose command become part of docker engine. The recommended way to install docker compose is installing a docker compose plugin on docker engine. For more information - Docker Compose Reference.
Docker Proxy Settings are needed for some of the server packages. Refer to the official Docker documentation for comprehensive instructions: Configure the Docker Client for Proxy.

Using `docker compose` file¶

This is the recommended way as docker compose starts cmf-server, postgres db and ui-server in one go. It is neccessary to start postgres db before cmf-server.

Go to root cmf directory.

Replace xxxx with user-name in docker-compose-server.yml available in the root cmf directory.

......
services:
server:
  image: server:latest
  volumes:
     - /home/xxxx/cmf-server/data:/cmf-server/data                 # for example /home/hpe-user/cmf-server/data:/cmf-server/data 
     - /home/xxxx/cmf-server/data/static:/cmf-server/data/static   # for example /home/hpe-user/cmf-server/data/static:/cmf-server/data/static
  container_name: cmf-server
  build:
....

Create a .env file in the same directory as docker-compose-server.yml and add the necessary environment variables.
```
POSTGRES_USER=myuser
POSTGRES_PASSWORD=mypassword
```
Execute following command to start both the containers. IP variable is the IP address and hostname is host name of the machine on which you are executing the following command. You can use either way.
```
IP=200.200.200.200 docker compose -f docker-compose-server.yml up
           OR
hostname=host_name docker compose -f docker-compose-server.yml up
```
Replace docker compose with docker-compose for older versions. Also you can adjust $IP in docker-compose-server.yml to reflect the server IP and run the docker compose command without specifying IP=200.200.200.200.
```
.......
environment:
REACT_APP_MY_IP: ${IP}
......
```

Stop the containers.

  docker compose -f docker-compose-server.yml stop

It is neccessary to rebuild images for cmf-server and ui-server after cmf version update or after pulling latest cmf code from git.

OR

Using `docker run` command¶

Install cmflib on your system.
Go to cmf/server directory.
```
cd server
```
List all docker images.
```
docker images
```
Execute the below-mentioned command to create a cmf-server docker image.
```
Usage:  docker build -t [image_name] -f ./Dockerfile ../
```
Example:
```
docker build -t server_image -f ./Dockerfile ../
```
Note - '../' represents the Build context for the docker image.

Launch a new docker container using the image with directory /home/user/cmf-server/data mounted. Pre-requisite: mkdir /home/<user>/cmf-server/data/static

Usage: docker run --name [container_name] -p 0.0.0.0:8080:80 -v /home/<user>/cmf-server/data:/cmf-server/data -e MYIP=XX.XX.XX.XX [image_name]

Example:

docker run --name cmf-server -p 0.0.0.0:8080:80 -v /home/user/cmf-server/data:/cmf-server/data -e MYIP=0.0.0.0 server_image

After cmf-server container is up, start ui-server, Go to cmf/ui folder.
```
cd cmf/ui
```

Execute the below-mentioned command to create a ui-server docker image.

Usage:  docker build -t [image_name] -f ./Dockerfile ./

Example:

docker build -t ui_image -f ./Dockerfile ./

Launch a new docker container for UI.

Usage: docker run --name [container_name] -p 0.0.0.0:3000:3000 -e REACT_APP_MY_IP=XX.XX.XX.XX [image_name]

Example:

docker run --name ui-server -p 0.0.0.0:3000:3000 -e REACT_APP_MY_IP=0.0.0.0 ui_image

Note: If you face issue regarding Libzbar-dev similar to the snapshot, add proxies to '/.docker/config.json'

Screentshot (115)

{
   proxies: {
        "default": {
                     "httpProxy": "http://web-proxy.labs.xxxx.net:8080",
                     "httpsProxy": "http://web-proxy.labs.xxxx.net:8080",
                     "noProxy": ".labs.xxxx.net,127.0.0.0/8"
             }
         }
 }

To stop the docker container.
```
docker stop [container_name]
```
To delete the docker container.
```
docker rm [container_name] 
```
To remove the docker image.
```
docker image rm [image_name] 
```

API Reference¶

cmf-server APIs are organized around FastAPI. They accept and return JSON-encoded request bodies and responses and return standard HTTP response codes.

List of APIs¶

Method	URL	Description
`Post`	`/mlmd_push`	Used to push Json Encoded data to cmf-server
`Get`	`/mlmd_pull/{pipeline_name}`	Retrieves a mlmd file from cmf-server
`Get`	`/display_executions`	Retrieves all executions from cmf-server
`Get`	`/display_artifacts/{pipeline_name}/{data_type}`	Retrieves all artifacts from cmf-server for resp datat type
`Get`	`/display_lineage/{lineage_type}/{pipeline_name}`	Creates lineage data from cmf-server
`Get`	`/display_pipelines`	Retrieves all pipelines present in mlmd file

HTTP Response Status codes¶

Code	Title	Description
`200`	`OK`	mlmd is successfully pushed (e.g. when using `GET`, `POST`).
`400`	`Bad request`	When the cmf-server is not available.
`500`	`Internal server error`	When an internal error has happened

Getting started with cmf-server¶

Setup a cmf-server¶

Pre-requisites¶

Using docker compose file¶

OR

Using docker run command¶

API Reference¶

List of APIs¶

HTTP Response Status codes¶

Using `docker compose` file¶

Using `docker run` command¶