Ingestions - Profiler - Fail - tagFQN > 128

[AnaMariaChousa]

Affected module
Ingestion Framework

Describe the bug
When ingesting the profile, it gives an error when finding tagFQN greater than 128 characters. I understand that the error may be due to tagFQN of business terms, which we have needed to define nested at several levels and with long names. (But the client asks us that the names be specific and detailed)
We have just migrated from version 1.1.5 to 1.3.4, going through the versions:
1.1.7 (with ingest version 7.17.0), 1.2.0, 1.2.5, 1.3.0, 1.3.4

To Reproduce

Expected behavior
The desired behavior would be for the ingest to accept tagFQN with long values, with a maximum limit of 256 characters. Although it could be 512, to be able to use it occasionally when necessary.
I've done some research and the limit of 256 is commonly used for descriptive names and reasonably deep hierarchies without being excessive.
The 512 characters are used when extremely detailed labels and very deep hierarchies are needed, but it needs to be used carefully as it could impact performance and readability.

Version:

• OS: Linux
• Python version: python3.10
• OpenMetadata version: 1.3.4
• OpenMetadata Ingestion package version: [e.g. `openmetadata-ingestion[docker]==1.3.4]

Additional context

ERROR

Error listing source and entities for database due to [1 validation error for Table columns -> 5 -> tags -> 0 -> tagFQN -> root ensure this value has at most 128 characters (type=value_error.any_str.max_length; limit_value=128)]

LOG

File "/home/airflow/.local/lib/python3.10/site-packages/metadata/profiler/source/metadata.py", line 294, in get_table_entities
yield from self.filter_entities(tables)
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/profiler/source/metadata.py", line 183, in filter_entities
for table in tables:
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/ometa/ometa_api.py", line 448, in list_all_entities
entity_list = self.list_entities(
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/ometa/ometa_api.py", line 411, in list_entities
entities = [entity(**elmt) for elmt in resp["data"]]
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/ometa/ometa_api.py", line 411, in
entities = [entity(**elmt) for elmt in resp["data"]]
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
pydantic.error_wrappers.ValidationError: 1 validation error for Table
columns -> 5 -> tags -> 0 -> tagFQN -> root
ensure this value has at most 128 characters (type=value_error.any_str.max_length; limit_value=128)

[harshach]

@ayush-shah we removed this limit right , can you verify?

[ayush-shah]

We modified the tagName side of the things that individual tags shouldn't have any restrictions, i will remove the tagFQN upper bound restriction as well alongwith some tests

ref: https://github.com/open-metadata/OpenMetadata/pull/14467/files#diff-be944b7efdd9e4992fc3162e13bb8e9d527b58e148d2395cd8f86c363c838e3aL102-L103

[AnaMariaChousa]

ayush-shah thank you very much.
In what version is this improvement incorporated?

[ayush-shah]

Hey @AnaMariaChousa it will be a part of 1.5.0 release