Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] failed #935

Open
matti opened this issue Jun 23, 2024 · 0 comments
Open

[bug] failed #935

matti opened this issue Jun 23, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@matti
Copy link

matti commented Jun 23, 2024

 Starting gateway container on VMs (16/16) in 50.64s
Exception in thread Thread-339:
Traceback (most recent call last):
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/connection.py", line 461, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/http/client.py", line 1395, in getresponse
    response.begin()
  File "/pyenv/versions/3.11.9/lib/python3.11/http/client.py", line 325, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/http/client.py", line 294, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/skyplane/api/tracker.py", line 153, in run
    for chunk in chunk_stream:
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/skyplane/api/transfer_job.py", line 644, in dispatch
    reply = self.http_pool.request(
            ^^^^^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/_request_methods.py", line 118, in request
    return self.request_encode_body(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/_request_methods.py", line 217, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/poolmanager.py", line 444, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/util/retry.py", line 470, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/util/util.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/urllib3/connection.py", line 461, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/http/client.py", line 1395, in getresponse
    response.begin()
  File "/pyenv/versions/3.11.9/lib/python3.11/http/client.py", line 325, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/http/client.py", line 294, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/pyenv/versions/3.11.9/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/skyplane/api/tracker.py", line 166, in run
    UsageClient.log_exception(
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/skyplane/api/usage.py", line 147, in log_exception
    stats = client.make_error(
            ^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/skyplane/api/usage.py", line 304, in make_error
    dest_regions = [tag.split(":")[1] for tag in dest_region_tags]
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/pyenv/versions/3.11.9/lib/python3.11/site-packages/skyplane/api/usage.py", line 304, in <listcomp>
    dest_regions = [tag.split(":")[1] for tag in dest_region_tags]
                    ~~~~~~~~~~~~~~^^^
IndexError: list index out of range
 Dispatching chunks... 500 chunks (~6.38MB dispatched)  0/? bytes18:21:39 [ERROR] Error running copy_log, AWSServer(region_tag=aws:eu-north-1, instance_id=i-022330e924095e459): SSH session not active
18:21:39 [ERROR] Error running copy_log, AWSServer(region_tag=aws:eu-north-1, instance_id=i-04acb2f33cd98ed9a): SSH session not active
18:21:39 [ERROR] Error running copy_log, AWSServer(region_tag=aws:eu-north-1, instance_id=i-08414153357859333): SSH session not active
18:21:39 [ERROR] Error running copy_log, AWSServer(region_tag=aws:eu-north-1, instance_id=i-00de12d08c9130762): SSH session not active
18:21:39 [ERROR] Error running copy_log, AWSServer(region_tag=aws:eu-north-1, instance_id=i-09853ce5b0234da47): SSH session not active
18:21:39 [ERROR] Error running copy_log, AWSServer(region_tag=aws:eu-north-1, instance_id=i-03fa984063c2dac35): SSH session not active
18:21:39 [ERROR] Error running copy_log, AWSServer(region_tag=aws:eu-north-1, instance_id=i-02cb36cebb176785b): SSH session not active
 Dispatching chunks... 500 chunks (~6.38MB dispatched)  0/? bytes18:21:39 [ERROR] Error running copy_log, AWSServer(region_tag=aws:eu-north-1, instance_id=i-04833ba6f324caf25): SSH session not active
⠹ Dispatching chunks... 500 chunks (~6.38MB dispatched)  0/? bytes Traceback (most recent call last) 
 /pyenv/versions/3.11.9/lib/python3.11/site-packages/skyplane/cli/cli_transfer.py:511 in sync   
@matti matti added the bug Something isn't working label Jun 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant