You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running the code above, the stream prompts the following error:
2024-06-19 11:33:46 +0800 28860 execution.flow INFO Current thread is not main thread, skip signal handler registration in AsyncNodesScheduler.
2024-06-19 11:33:46 +0800 28860 execution.flow WARNING Input 'project_scope' of evaluate_with_rai_service is not json serializable, use str to store it.
2024-06-19 11:33:46 +0800 28860 execution.flow WARNING Input 'credential' of evaluate_with_rai_service is not json serializable, use str to store it.
2024-06-19 11:33:46 +0800 28860 execution.flow INFO Start executing nodes in thread pool mode.
2024-06-19 11:33:46 +0800 28860 execution.flow INFO Current thread is not main thread, skip signal handler registration in AsyncNodesScheduler.
2024-06-19 11:33:46 +0800 28860 execution.flow WARNING Failed to serialize inputs or output for flow run because of cannot pickle '_thread.RLock' object.The inputs and output field in api_calls will be None.
2024-06-19 11:33:46 +0800 28860 execution WARNING [evaluate_with_rai_service in line None (index starts from 0)] stderr> AzureCliCredential.get_token failed: Failed to invoke the Azure CLI
2024-06-19 11:33:46 +0800 28860 execution.flow INFO Executing node validate_inputs. node run id: b185f8e5-f22f-4c44-8816-123f5c966134_validate_inputs_afb5ee9d-f2b6-4231-b880-046f1ed40835
2024-06-19 11:33:46 +0800 28860 execution.flow INFO Current thread is not main thread, skip signal handler registration in AsyncNodesScheduler.
2024-06-19 11:33:46 +0800 28860 execution ERROR Node evaluate_with_rai_service in line None failed. Exception: Execution failure in 'evaluate_with_rai_service': (CredentialUnavailableError) Failed to invoke the Azure CLI.
Traceback (most recent call last):
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\site-packages\azure\identity\_credentials\azure_cli.py", line 199, in _run_command
return subprocess.check_output(args, **kwargs)
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\subprocess.py", line 424, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\subprocess.py", line 507, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\subprocess.py", line 1134, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\subprocess.py", line 1526, in _communicate
raise TimeoutExpired(self.args, orig_timeout)
subprocess.TimeoutExpired: Command '['cmd', '/c', 'az account get-access-token --output json --resource https://management.azure.com']' timed out after 10 seconds
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\site-packages\promptflow\_core\flow_execution_context.py", line 182, in _invoke_tool_inner
return f(**kwargs)
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\site-packages\promptflow\tracing\_trace.py", line 556, in wrapped
output = func(*args, **kwargs)
File "C:\Users\xxxx\AppData\Local\anaconda3\envs\project\Lib\site-packages\promptflow\evals\evaluators\_content_safety\flow\evaluate_with_rai_service.py", line 189, in evaluate_with_rai_service
rai_svc_url = get_rai_svc_url(project_scope, credential)
File "C:\Users\xxxx\AppData\Local\anaconda3\envs\project\Lib\site-packages\promptflow\evals\evaluators\_content_safety\flow\evaluate_with_rai_service.py", line 165, in get_rai_svc_url
discovery_url = _get_service_discovery_url(azure_ai_project=project_scope, credential=credential)
File "C:\Users\xxxx\AppData\Local\anaconda3\envs\project\Lib\site-packages\promptflow\evals\evaluators\_content_safety\flow\evaluate_with_rai_service.py", line 148, in _get_service_discovery_url
bearer_token = credential.get_token("https://management.azure.com/.default").token
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\site-packages\azure\identity\_credentials\default.py", line 219, in get_token
token = self._successful_credential.get_token(*scopes, claims=claims, tenant_id=tenant_id, **kwargs)
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\site-packages\azure\identity\_internal\decorators.py", line 33, in wrapper
token = fn(*args, **kwargs)
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\site-packages\azure\identity\_credentials\azure_cli.py", line 112, in get_token
output = _run_command(command, self._process_timeout)
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\site-packages\azure\identity\_credentials\azure_cli.py", line 223, in _run_command
raise error from ex
azure.identity._exceptions.CredentialUnavailableError: Failed to invoke the Azure CLI
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\site-packages\promptflow\_core\flow_execution_context.py", line 90, in invoke_tool
result = self._invoke_tool_inner(node, f, kwargs)
File "c:\Users\xxxx\AppData\Local\anaconda3\envs\project\lib\site-packages\promptflow\_core\flow_execution_context.py", line 206, in _invoke_tool_inner
raise ToolExecutionError(node_name=node_name, module=module) from e
promptflow._core._errors.ToolExecutionError: Execution failure in 'evaluate_with_rai_service': (CredentialUnavailableError) Failed to invoke the Azure CLI
The bug will not be triggered if parallel parameter of content safety evaluator equals to False.
The text was updated successfully, but these errors were encountered:
Wix97
changed the title
[BUG] [promptflow-evals] Failed to invoke the Azure CLI when running
[BUG] [promptflow-evals] Failed to invoke the Azure CLI when running evaluation
Jun 19, 2024
Describe the bug
promptflow-evals version: 0.3.0
When running the code above, the stream prompts the following error:
The bug will not be triggered if
parallel
parameter of content safety evaluator equals toFalse
.The text was updated successfully, but these errors were encountered: