CRD changes for GitOps support

[sriumcp]

Sample GitOps-enabled experiment.

Before start action is invoked.

apiVersion: iter8.tools/v2alpha1
kind: Experiment
metadata:
  name: experiment-1
spec:
  target: knative-test/sample-application
  strategy:
    testingPattern: Canary
    deploymentPattern: fixed-split
    actions:
      start:
      - library: knative
        task: updateExperimentInGit
        with:
          targetRepo: https://github.com/my/repo
          targetFile: path/to/folder/experiment.yaml
      loop:
      - library: knative
        task: updateKsvcInGit
        with:
          targetRepo: https://github.com/my/repo
          targetFile: path/to/folder/ksvc.yaml
      finish:
      - library: common
        task: replaceGitFolder
        with: 
          sourceFolder: https://github.com/my/repo/source/{{ .name }}
          targetRepo: https://github.com/my/repo
          targetFolder: path/to/app
  criteria:
    objectives:
    - metric: mean-latency
      upperLimit: 2000
    - metric: error-rate
      upperLimit: "0.01"
  duration:
    intervalSeconds: 3600
    maxIterations: 24
    maxLoops: 14

After start action is invoked and GitOps operator has finished syncing.

apiVersion: iter8.tools/v2alpha1
kind: Experiment
metadata:
  name: experiment-1
spec:
  target: knative-test/sample-application
  strategy:
    testingPattern: Canary
    deploymentPattern: fixed-split
    actions:
      start:
      - library: knative
        task: updateExperimentInGit
        with:
          targetRepo: https://github.com/my/repo
          targetFile: path/to/folder/experiment.yaml
      loop:
      - library: knative
        task: updateKsvcInGit
        with:
          targetRepo: https://github.com/my/repo
          targetFile: path/to/folder/ksvc.yaml
      finish:
      - library: common
        task: replaceGitFolder
        with: 
          sourceFolder: https://github.com/my/repo/source/{{ .name }}
          targetRepo: https://github.com/my/repo
          targetFolder: path/to/app
  criteria:
    objectives:
    - metric: mean-latency
      upperLimit: 2000
    - metric: error-rate
      upperLimit: "0.01"
  duration:
    intervalSeconds: 3600
    maxIterations: 24
    maxLoops: 14
  versionInfo:
    baseline:
      name: current # names of versions MUST be unique
      variables:
      - name: revision
        value: sample-application-v1
      weightObjRef:
          apiVersion: serving.knative.dev/v1
          kind: Service
          name: sample-application
          namespace: knative-test
          fieldPath: /spec/traffic/0/percent
    candidates:
      - name: candidate
        variables: 
        - name: revision
          value: sample-application-v2
        weightObjRef:
          apiVersion: serving.knative.dev/v1
          kind: Service
          name: sample-application
          namespace: knative-test
          fieldPath: /spec/traffic/1/percent

[sriumcp]

Sample InCluster experiment.

Before start action is invoked.

apiVersion: iter8.tools/v2alpha1
kind: Experiment
metadata:
  name: experiment-1
spec:
  target: knative-test/sample-application
  strategy:
    testingPattern: Canary
    deploymentPattern: progressive
    actions:
      start:
      - library: knative
        task: updateExperimentInCluster
      finish:
      - library: common
        task: updateInClusterResourcesUsingKubectl
        with: 
          sourceFolder: https://github.com/my/repo/{{ .name }}
  criteria:
    objectives:
    - metric: mean-latency
      upperLimit: 2000
    - metric: error-rate
      upperLimit: "0.01"
  duration:
    intervalSeconds: 3600
    maxIterations: 336

After start action is invoked.

apiVersion: iter8.tools/v2alpha1
kind: Experiment
metadata:
  name: experiment-1
spec:
  target: knative-test/sample-application
  strategy:
    testingPattern: Canary
    deploymentPattern: progressive
    actions:
      start:
      - library: knative
        task: updateExperimentInCluster
      finish:
      - library: common
        task: updateInClusterResourcesUsingKubectl
        with: 
          sourceFolder: https://github.com/my/repo/{{ .name }}
  criteria:
    objectives:
    - metric: mean-latency
      upperLimit: 2000
    - metric: error-rate
      upperLimit: "0.01"
  duration:
    intervalSeconds: 3600
    maxIterations: 336
  versionInfo:
    baseline:
      name: current # names of versions MUST be unique
      variables:
      - name: revision
        value: sample-application-v1
      weightObjRef:
          apiVersion: serving.knative.dev/v1
          kind: Service
          name: sample-application
          namespace: knative-test
          fieldPath: /spec/traffic/0/percent
    candidates:
      - name: candidate
        variables: 
        - name: revision
          value: sample-application-v2
        weightObjRef:
          apiVersion: serving.knative.dev/v1
          kind: Service
          name: sample-application
          namespace: knative-test
          fieldPath: /spec/traffic/1/percent

[sriumcp]

Summarizing CRD changes

• strategy.type => strategy.testingPattern (done in etc3)
• strategy.weights.algorithm => strategy.deploymentPattern (done in etc3)
• versionDetail.tags => versionDetail.variables (done in etc3)
• versionDetail.variables is now a list of objects; each object has name, value fields. (done in etc3)
• remove strategy.weights.weightDistribution (done in etc3)
• introduce strategy.actions (preserve unknown fields in strategy)
• metric CRD -- change params to a list of name-value pairs.
• metric CRD -- change sample_size to sampleSize

---

7. delete rollback handler type (will be consumed by 6)
8. collapse all handler types into one
9. duration.maxLoops (optional; default = 1)
10. etc3 should not update spec (it should still late initialize locally)