Add ability to have extensions to dsc.exe

[SteveL-MSFT]

Summary of the new feature / enhancement

We should allow for community extensions to dsc.exe so they show up as sub-commands. For example we may want to have dsc cache --clear which is supported by PS-Adapter extension.

Proposed technical implementation details (optional)

Extensions should be executables (or maybe an extension manifest?) that have -dsc suffix to the command, like cache-dsc.exe

[michaeltlombardi]

Proposals:

1. Define a set of extension points for where/how the extension can interact with DSC.
2. Require a manifest, with a similar data structure as resources to indicate how the extension integrates.

1. Extension Points

I think we probably want to define a few common extension points to enable hooking into existing DSC commands. Separately, we should consider whether we want to support something similar to the GitHub CLI extension model, where integrating developers can define new commands for DSC.

Proposed built-in extension points

• dsc cache <clear|refresh|init> (Cache) - A cache extension defines three commands - one to initialize a cache, one to refresh it, and one to clear it. Users can invoke the cache commands directly (e.g. dsc cache refresh. Before DSC executes a configuration operation, it follows the code path for initializing all known caches. The init command for a cache extension should return as soon as it knows whether the cache is already initialized. I'm not sure what input, if any, these commands should receive. They should exit with code 0 on success and can send messages, but not output, back to DSC.
• dsc config resolve (Resolver) - A resolving extension defines a single command that operates on and returns a DSC configuration document. DSC calls this hook at the beginning of any dsc config operation, before final validation of the resolved document by DSC. This would enable extensions to pre-process a configuration - for example, you could define a resource instance generator or fill in non-specified default values, or whatever. Theoretically, you could also use this extension to enable authoring custom configuration functions. The only input and output for these extensions is a DSC configuration document.
• dsc <command> [<subcommands>] (Command) - a command extension defines a new command for DSC at the top-level. The names of these commands must be unique and can't conflict with built-in commands.

Theoretically, an extension could do more than one of these, but it's probably better to have separate manifests, even if the app is shared across the three kinds.

Example command extensions

# Extension that shows a node graph for a configuration document or result
cat ./app.config.dsc.yaml | dsc graph # Show node graph for config document
dsc config get | dsc graph            # Show node graph for get result

# Extension that shows terminal-help for resources
dsc explain --resource Microsoft/OSInfo                     # Show full help
dsc explain --resource Microsoft/OSInfo --property codename # Show property help only
dsc explain --manifest ./tstoy.dsc.resource.json            # Inspect manifest for help

# Extension that helps with resource manifest DevX
dsc manifest new --type TailspinToys/tstoy --capabilities Get, Set, Export # Scaffold manifest
dsc manifest new --interactive                                             # Create with prompts
dsc manifest validate ./tstoy.dsc.resource.json                            # Test manifest
dsc manifest show ./tstoy.dsc.resource.json                                # Pretty-print manifest
dsc manifest show ./tstoy.dsc.resource.json --schema                       # Pretty-print schema

# Extension that handles packaging, installing, updating resources
# Validate that the MyResource folder has everything needed to publish a resource package
dsc package validate --folder ./MyResource
# Package the resource for publishing
dsc package build --folder ./MyResource --output-folder ./packages
# Publish the resource to the default repository
dsc package publish  ./packages/MyOrg.MyResource.dsc.package.json
# Search the GitHub container registry for resources
dsc package find resource
# List all locally installed packages
dsc package list
# Update locally installed packages
dsc package update

2. Manifests

I think that we should either require standalone manifests for an extension or extend the resource manifest to enable defining extensions. I'm hesitant to support both for the complexity/ambiguity.

I prefer a standalone manifest from an integration, testing, and packaging perspective.

Cache extension

$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/bundled/extension/manifest.json
type: Microsoft.DSC/PowerShellCache
description: Manage caching adapted PowerShell resources
tags:
- PowerShell
cache:
  init:
    executable: pwsh
    args:
    - -NoLogo
    - -NonInteractive
    - -NoProfile
    - -Command
    - './psDscAdapter/powershell.cache.ps1'
  clear:
    executable: pwsh
    args:
    - -NoLogo
    - -NonInteractive
    - -NoProfile
    - -Command
    - './psDscAdapter/powershell.cache.ps1 -Clear'
  refresh:
    executable: pwsh
    args:
    - -NoLogo
    - -NonInteractive
    - -NoProfile
    - -Command
    - './psDscAdapter/powershell.cache.ps1 -Refresh'

Example resolver extension manifest

$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/bundled/extension/manifest.json
type: Microsoft.DSC/InstanceGenerator
description: Pre-process a configuration to generate instances from a configuration function.
tags:
- DevX
resolve:
  executable: dsc-igen
  input:      stdin

Example custom command extension manifest (args)

$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/bundled/extension/manifest.json
type: Microsoft.DSC.Docs/Explain
tags:
- Docs
- Help
customCommand:
  executable: dsc-explain
  help:
    arg:   --help
    command: help
  args:
    - name: --resource
      type: string
      description: The type of the resource to show help for.
    - name: --property
      type: string
      description: The property of the resource to show help for.
    - name: --manifest
      type: string
      description: The path to the resource manifest to show help for.
  argSets:
    - required: --resource
      args: [--resource, --property]
    - required: --manifest
      args: [--manifest, --property]
  # In this case, we can be relatively lazy and just indicate it returns a Markdown string;
  # If the extension returned structured data, we would need to have the full schema here
  # for users and integrating tools.
  outputSchema:
    $schema: https://json-schema.org/draft/2020-12/schema
    $id:              https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/extensions/explain/output.json
    type:             string
    contentMediaType: text/markdown
    title:            Resource Documentation
    description:      The Markdown documentation for a resource or a property of the resource.

  # Has no subcommands, but use the same definition structure recursively.
  subcommands: []

This is probably the roughest example, as we need to be able to determine the argument definitions from the manifest. This requires much more thorough design and investigation. An alternative would be to require custom commands to specify a JSON schema for input and output both, then generate the CLI arguments from the schema properties (but always send the data to the extension as JSON either over stdin or to the JSON input argument).

Example custom command extension manifest (input schema)

$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/bundled/extension/manifest.json
type: Microsoft.DSC.Docs/Explain
tags:
- Docs
- Help
customCommand:
  executable: dsc-explain
  help:
    arg:   --help
    command: help
  args: []
  input: stdin
  schemas:
    input:
      $schema: https://json-schema.org/draft/2020-12/schema
      $id:              https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/extensions/explain/input.json
      oneOf:
      - required: [resource]
      - required: [manifest]
      properties:
        resource:
          type:        string
          description: The type of the resource to show help for.
        property:
          type:        string
          description: The property of the resource to show help for.
        manifest:
          type:        string
          description: The path to the resource manifest to show help for.
    output:
      $schema:          https://json-schema.org/draft/2020-12/schema
      $id:              https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/extensions/explain/output.json
      type:             string
      contentMediaType: text/markdown
      title:            Resource Documentation
      description:      The Markdown documentation for a resource or a property of the resource.