Integrate Azure carbon emission data into carbon cost reporting

[pauldotyu]

Is your feature request related to a problem? Please describe.

Azure customers should have the ability to view OpenCost's carbon cost reporting based on the same data that is provided by the Carbon optimization dashboards available within the Azure portal. Having data that aligns with what Azure is reporting but sliced down to Kubernetes-centric resources will be a good compliment to the Azure resource-centric emission reports.

Describe the solution you'd like

Azure carbon optimization provides APIs and exports to enable users to track and analyze Scope 1, 2, and 3 emissions associated with Azure down to the resource level.

Can the carbon cost functionality of OpenCost be updated to allow Azure customers to invoke the Carbon optimization REST API to get data on-demand or provide their own CSV file?

The API call will require an Azure Bearer token but it should be easy to obtain using the same service principal that is used to get Azure cost reporting data (as long as proper RBAC is assigned. See this doc for more on access permissions)

Describe alternatives you've considered
Leave current implementation as is which uses carbonlookupdata.csv as basis for report.

Additional context
You can test the Carbon optimization API in your subscription with the following Azure CLI commands:

Ensure you have the proper permissions before you begin: https://learn.microsoft.com/azure/carbon-optimization/permissions

# set your subscription
SUBSCRIPTION=<YOUR_SUBSCRIPTION_ID>

# login and connect to an azure subscription that has access to carbon emission reporting
az login
az account set -s $SUBSCRIPTION

# create a json file that we'll use as the request body
# reports are generated monthly and both the start and
# end dates must be the first of each month
cat <<EOF > carbonEmissionItemDetailsReportRequest.json
{
  "reportType": "ItemDetailsReport",
  "groupCategory": "",
  "subscriptionList": [
    "${SUBSCRIPTION}"
  ],
  "resourceGroupUrlList": [],
  "carbonScopeList": [
    "Scope1",
    "Scope2",
    "Scope3"
  ],
  "dateRange": {
    "start": "2024-04-01",
    "end": "2024-04-01"
  },
  "categoryType": "Resource",
  "sortDirection": "Desc",
  "pageSize": 2000,
  "orderBy": "totalCarbonEmission",
  "skipToken": ""
}
EOF

# invoke the rest api
az rest --method post \
  --uri "https://management.azure.com/providers/Microsoft.Carbon/carbonEmissionReports?api-version=2023-04-01-preview" \
  --body @carbonEmissionItemDetailsReportRequest.json > carbonEmissionItemDetailsReport.json

The emissions data is available for the previous month by the 17th day of the current month and contains JSON payload that looks like this:

{
  "skipToken": "",
  "value": [
    {
      "categoryType": "Resource",
      "changeRatioFor12Months": 0,
      "changeRatioForLastMonth": -0.0533473604144896,
      "changeValueMonthOverMonth": -0.4170808129835182,
      "dataType": "ResourceItemDetailsData",
      "groupName": "",
      "itemName": "aks-agentpool-32387455-vmss",
      "resourceGroup": "mc_test-clusters_cluster1_eastus",
      "resourceId": "/subscriptions/<REDACTTED>/resourcegroups/mc_test-clusters_cluster1_eastus/providers/microsoft.compute/virtualmachinescalesets/aks-agentpool-32387455-vmss",
      "resourceType": "microsoft.compute/virtualmachinescalesets",
      "subscriptionId": "<REDACTTED>",
      "totalCarbonEmission": 7.401128180731483,
      "totalCarbonEmission12MonthsAgo": 0,
      "totalCarbonEmissionLastMonth": 7.818208993715001
    },
    {
      "categoryType": "Resource",
      "changeRatioFor12Months": 0,
      "changeRatioForLastMonth": -0.005475461734918942,
      "changeValueMonthOverMonth": -0.011246653604271,
      "dataType": "ResourceItemDetailsData",
      "groupName": "",
      "itemName": "cluster3",
      "resourceGroup": "staging-clusters",
      "resourceId": "/subscriptions/<REDACTTED>/resourcegroups/staging-clusters/providers/microsoft.containerservice/managedclusters/cluster3",
      "resourceType": "microsoft.containerservice/managedclusters",
      "subscriptionId": "<REDACTTED>",
      "totalCarbonEmission": 2.0427634278738154,
      "totalCarbonEmission12MonthsAgo": 0,
      "totalCarbonEmissionLastMonth": 2.0540100814780864
    },
    {
      "categoryType": "Resource",
      "changeRatioFor12Months": 0,
      "changeRatioForLastMonth": -0.005324604714043389,
      "changeValueMonthOverMonth": -0.00273451709031347,
      "dataType": "ResourceItemDetailsData",
      "groupName": "",
      "itemName": "kubernetes",
      "resourceGroup": "mc_staging-clusters_cluster-private_westus2",
      "resourceId": "/subscriptions/<REDACTTED>/resourcegroups/mc_staging-clusters_cluster-private_westus2/providers/microsoft.network/loadbalancers/kubernetes",
      "resourceType": "microsoft.network/loadbalancers",
      "subscriptionId": "<REDACTTED>",
      "totalCarbonEmission": 0.5108279419409293,
      "totalCarbonEmission12MonthsAgo": 0,
      "totalCarbonEmissionLastMonth": 0.5135624590312428
    },
    {
      "categoryType": "Resource",
      "changeRatioFor12Months": 0,
      "changeRatioForLastMonth": 0.028013219952077676,
      "changeValueMonthOverMonth": 0.01385750932327768,
      "dataType": "ResourceItemDetailsData",
      "groupName": "",
      "itemName": "aks-agentpool-168895aks-agentpool-1688957os__1_b0f74623a22241a7bf7ea3242373110b",
      "resourceGroup": "mc_staging-clusters_cluster-win_westus2",
      "resourceId": "/subscriptions/<REDACTTED>/resourcegroups/mc_staging-clusters_cluster-win_westus2/providers/microsoft.compute/disks/aks-agentpool-168895aks-agentpool-1688957os__1_b0f74623a22241a7bf7ea3242373110b",
      "resourceType": "microsoft.compute/disks",
      "subscriptionId": "<REDACTTED>",
      "totalCarbonEmission": 0.5085349989865072,
      "totalCarbonEmission12MonthsAgo": 0,
      "totalCarbonEmissionLastMonth": 0.49467748966322955
    }
  ]
}

Request payloads for other reports can be found here: https://learn.microsoft.com/azure/carbon-optimization/export-data?tabs=RESTAPI

[mattray]

Adding this functionality similar to the current Cloud Carbon Footprint support should be relatively straightforward. I'll check with Kubecost Engineering