Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give user ability to set ingestion Filter Patterns based off of tags #16724

Open
traitortots opened this issue Jun 19, 2024 · 0 comments
Open
Assignees
Labels
enhancement New feature or request profiler

Comments

@traitortots
Copy link

Is your feature request related to a problem? Please describe.
When setting up or fine-tuning an ingestion, sometimes there is not a simple regex pattern that can describe what I want to include or exclude when running an ingestion on a service, so I end up typing in a ton of table names manually.

Describe the solution you'd like
It would be great to be able to use tags to identify and set filter patterns for databases, schemas, or tables during ingestion. For example, if I only want to run Profiler Ingestion on everything tagged as Tier 1, or have a tag called "Profiler-Include" that is then an input parameter for the ingestion. This would also make it easier for users without access to the ingestion setup to still contribute to deciding what is and is not ingested, and will make ingestions more dynamic as certain qualities of the data change (ie, a table is no longer considered Tier 1 and when reclassified, the ingestion manifest does not need to be edited and the table is no longer profiled).

Describe alternatives you've considered
Users continue to manually set up Include or Exclude patterns at a more granular level when regex patterns are not able to capture the needs of the user.

Additional context
N/A

@traitortots traitortots added the enhancement New feature or request label Jun 19, 2024
@TeddyCr TeddyCr self-assigned this Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request profiler
Projects
None yet
Development

No branches or pull requests

2 participants