Use VECTOR_SELF_NODE_NAME for internal_metrics

[taind772]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

Should we use VECTOR_SELF_NODE_NAME as host label for Internal metrics in Kubernetes deploys? Or some other label like node?

The current host label in Kubernetes deployment (agent mode) is the same as pod label. As I understand, to get which node vector agent was running on we need to join the metrics with kube_pod_info (metrics from kube-state-metrics) or add a custom VRL transform. While possible, it is tedious and should just work by default.

This piece of information is meaning

Attempted Solutions

No response

Proposal

No response

References

No response

Version

0.36.0
No response

[jszwedko]

Thanks for this report @taind772 .

Currently Vector just uses whatever gethostname returns as the hostname. Can you explain your use-case a bit more? Why do you want to tie it to the node name rather than the pod name?

I think if we do decide to do this enhancement to the "host" detection it should apply equally to all places where Vector annotates the name of the host it is running on in addition to internal_metrics.

[taind772]

Sorry for the bad & incomplete issue, I take a look back and can't figure out what am I about to say at the time xD.

In my use case, I use Vector to scrape and push Kubernetes pod logs to remote sinks, so it is easier to debug if we know which Kubernetes node Vector pod is running in since the Kubernetes node name is static and I only have one Vector agent per node. The Vector pod name changes after a restart - so it's a bit more tedious to debug (we need to map the pod name to the Kubernetes hostname with Kube state metrics or use metrics relabeling).

I think if we do decide to do this enhancement to the "host" detection it should apply equally to all places where Vector annotates the name of the host it is running on in addition to internal_metrics.

It's better to have a way for users to override the default hostname from gethostname since sometimes users have little control over the host Vector running on and want the host label to represent some other semantic.

After a short look back, I'm confused about the issue myself. Maybe I'm just too lazy to add a join query on metrics, so feel free to close the issue.

[jszwedko]

Gotcha, I can see why it'd be useful to have the node in that situation. I think we'd be open to configuring the hostname to use (maybe a global hostname parameter).