Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[It broke]: High ram usage #112

Open
Charles-Em opened this issue Feb 5, 2024 · 0 comments
Open

[It broke]: High ram usage #112

Charles-Em opened this issue Feb 5, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@Charles-Em
Copy link

Charles-Em commented Feb 5, 2024 •
edited by C9Glax
Loading

What is broken?

Hello,

I had to stop the container because of really high memory usage

image

There may be a memory leak in one the processes.

Investigation trails :

  • got timeout which happend first on mangalife after having downloaded 456 chapters of tower of god 12 hours before I noticed it (ToG have a significant amount of images, those chapters do represent a lot of requests - 50 to 100 images)
  • but the timeout seemed to be affecting more? It lasted for 70+ minutes (see logs, I did cut the middle part, only posted beginnign and ending)
  • Afterwards there was a crash ? (see next part of logs)
  • And in the end there were some other issues (last part of logs)

Log-output

[02/04/2024 20:49:06.463]     MangaLife | /Manga/Tower of God/Tower of God - Vol.3 Ch.45 .cbz 134/133 OK
[02/04/2024 20:49:06.463]     MangaLife | Creating archive /Manga/Tower of God/Tower of God - Vol.3 Ch.45 .cbz
[02/04/2024 20:49:07.685]        Kavita | Updating libraries.
[02/04/2024 20:49:07.686]        Kavita | Getting libraries.
[02/04/2024 20:49:07.717]       JobBoss | Removing Tranga.Jobs.DownloadChapter-dG93ZXJvZmdvZDIwMTA=-45 Chapter: Chapter Tower of God dG93ZXJvZmdvZDIwMTA= 45 
[02/04/2024 20:49:07.717] DownloadChapt | Cancelling Tranga.Jobs.DownloadChapter-dG93ZXJvZmdvZDIwMTA=-45 Chapter: Chapter Tower of God dG93ZXJvZmdvZDIwMTA= 45 
[02/04/2024 20:49:07.718]       JobBoss | Deleting Job-file /usr/share/tranga-api/jobs/Tranga.Jobs.DownloadChapter-dG93ZXJvZmdvZDIwMTA=-45.json
[02/04/2024 20:49:07.718]       JobBoss | Next job in -1.04:21:16.4467621 Tranga.Jobs.DownloadNewChapters-Y2hhaW5zYXdtYW4yMDE4
[02/04/2024 20:49:07.818]     MangaLife | Copy cover Publication Tower of God dG93ZXJvZmdvZDIwMTA=
[02/04/2024 20:49:07.819]     MangaLife | Cover exists Publication Tower of God dG93ZXJvZmdvZDIwMTA=
[02/04/2024 20:49:07.819]     MangaLife | Retrieving chapter-info Chapter Tower of God dG93ZXJvZmdvZDIwMTA= 46  Publication Tower of God dG93ZXJvZmdvZDIwMTA=
[02/04/2024 20:49:07.844]     NetClient | GET https://anonymised-kavita-url/api/Library -> 401: Unauthorized
[02/04/2024 20:49:07.844]        Kavita | No libraries returned
[02/04/2024 20:49:18.169] ChromiumDownl | Could not load Page:
One or more errors occurred. (Timeout of 10000 ms exceeded)
[02/04/2024 20:49:18.237]       JobBoss | Removing Tranga.Jobs.DownloadChapter-dG93ZXJvZmdvZDIwMTA=-46 Chapter: Chapter Tower of God dG93ZXJvZmdvZDIwMTA= 46 
[02/04/2024 20:49:18.237] DownloadChapt | Cancelling Tranga.Jobs.DownloadChapter-dG93ZXJvZmdvZDIwMTA=-46 Chapter: Chapter Tower of God dG93ZXJvZmdvZDIwMTA= 46 
[02/04/2024 20:49:18.238]       JobBoss | Deleting Job-file /usr/share/tranga-api/jobs/Tranga.Jobs.DownloadChapter-dG93ZXJvZmdvZDIwMTA=-46.json
[02/04/2024 20:49:18.238]       JobBoss | Next job in -1.04:21:26.9671174 Tranga.Jobs.DownloadNewChapters-Y2hhaW5zYXdtYW4yMDE4
[02/04/2024 20:49:18.339]     MangaLife | Copy cover Publication Tower of God dG93ZXJvZmdvZDIwMTA=
[02/04/2024 20:49:18.340]     MangaLife | Cover exists Publication Tower of God dG93ZXJvZmdvZDIwMTA=
[02/04/2024 20:49:18.340]     MangaLife | Retrieving chapter-info Chapter Tower of God dG93ZXJvZmdvZDIwMTA= 47  Publication Tower of God dG93ZXJvZmdvZDIwMTA=
[02/04/2024 20:49:18.340] ChromiumDownl | Waiting 0.8292314 seconds
[02/04/2024 20:49:29.346] ChromiumDownl | Could not load Page:
One or more errors occurred. (Timeout of 10000 ms exceeded)

.....

[02/04/2024 21:53:42.179] UpdateMetadat | Could not find Manga Publication Kuroko's Basketball a3Vyb2tvc2Jhc2tldGJhbGwyMDA4
[02/04/2024 21:53:42.280]       JobBoss | Removing Tranga.Jobs.UpdateMetadata-a3Vyb2tvc2Jhc2tldGJhbGwyMDA4 Manga: Publication Kuroko's Basketball a3Vyb2tvc2Jhc2tldGJhbGwyMDA4
[02/04/2024 21:53:42.280] UpdateMetadat | Cancelling Tranga.Jobs.UpdateMetadata-a3Vyb2tvc2Jhc2tldGJhbGwyMDA4 Manga: Publication Kuroko's Basketball a3Vyb2tvc2Jhc2tldGJhbGwyMDA4
[02/04/2024 21:53:42.280]       JobBoss | Deleting Job-file /usr/share/tranga-api/jobs/Tranga.Jobs.UpdateMetadata-a3Vyb2tvc2Jhc2tldGJhbGwyMDA4.json
[02/04/2024 21:53:42.820]       JobBoss | Next job in -00:00:00.0000001 Tranga.Jobs.UpdateMetadata-aGFqaW1lbm9pcHBvMTk4OQ==
[02/04/2024 21:53:42.924] ChromiumDownl | Waiting 0.2550285 seconds
[02/04/2024 21:54:34.423] ChromiumDownl | Could not load Page:
One or more errors occurred. (Timeout of 10000 ms exceeded)
[02/04/2024 21:54:34.426] UpdateMetadat | Could not find Manga Publication Hajime no Ippo aGFqaW1lbm9pcHBvMTk4OQ==
[02/04/2024 21:54:34.526]       JobBoss | Removing Tranga.Jobs.UpdateMetadata-aGFqaW1lbm9pcHBvMTk4OQ== Manga: Publication Hajime no Ippo aGFqaW1lbm9pcHBvMTk4OQ==
[02/04/2024 21:54:34.527] UpdateMetadat | Cancelling Tranga.Jobs.UpdateMetadata-aGFqaW1lbm9pcHBvMTk4OQ== Manga: Publication Hajime no Ippo aGFqaW1lbm9pcHBvMTk4OQ==
[02/04/2024 21:54:34.527]       JobBoss | Deleting Job-file /usr/share/tranga-api/jobs/Tranga.Jobs.UpdateMetadata-aGFqaW1lbm9pcHBvMTk4OQ==.json
[02/04/2024 21:54:34.767]       JobBoss | Next job in -00:00:00.0000001 Tranga.Jobs.UpdateMetadata-dG93ZXJvZmdvZDIwMTA=
[02/04/2024 21:54:34.875] ChromiumDownl | Waiting 0.5506709 seconds
[02/04/2024 21:57:46.871] ChromiumDownl | Could not load Page:
One or more errors occurred. (Timeout of 10000 ms exceeded)
[02/04/2024 21:57:46.873] UpdateMetadat | Could not find Manga Publication Tower of God dG93ZXJvZmdvZDIwMTA=
[02/04/2024 21:57:46.973]       JobBoss | Removing Tranga.Jobs.UpdateMetadata-dG93ZXJvZmdvZDIwMTA= Manga: Publication Tower of God dG93ZXJvZmdvZDIwMTA=
[02/04/2024 21:57:46.973] UpdateMetadat | Cancelling Tranga.Jobs.UpdateMetadata-dG93ZXJvZmdvZDIwMTA= Manga: Publication Tower of God dG93ZXJvZmdvZDIwMTA=
[02/04/2024 21:57:46.973]       JobBoss | Deleting Job-file /usr/share/tranga-api/jobs/Tranga.Jobs.UpdateMetadata-dG93ZXJvZmdvZDIwMTA=.json
[02/04/2024 21:57:54.66 ]       JobBoss | Next job in 00:00:26.3266430 Tranga.Jobs.DownloadNewChapters-Ym9rdW5vaGVyb2FjYWRlbWlhMjAyMw==
[02/04/2024 21:58:20.731]       JobBoss | Adding Job to Queue. Tranga.Jobs.DownloadNewChapters-Ym9rdW5vaGVyb2FjYWRlbWlhMjAyMw== Manga: Publication Boku no Hero Academia Ym9rdW5vaGVyb2FjYWRlbWlhMjAyMw==
[02/04/2024 21:58:20.784]   MangaKatana | Getting new Chapters for Publication Boku no Hero Academia Ym9rdW5vaGVyb2FjYWRlbWlhMjAyMw==
[02/04/2024 21:58:20.798]   MangaKatana | Getting chapters Publication Boku no Hero Academia Ym9rdW5vaGVyb2FjYWRlbWlhMjAyMw==
[02/04/2024 21:58:43.589] HttpDownloadC | Request timed out https://mangakatana.com/manga/boku-no-hero-academia.551.

...


System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 10 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Security.SslStream'.
   at System.Net.Security.SslStream.<ThrowIfExceptional>g__ThrowExceptional|123_0(ExceptionDispatchInfo e)
   at System.Net.Security.SslStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.Write(ReadOnlySpan`1 buffer)
   at System.Net.Http.HttpConnection.WriteToStreamAsync(ReadOnlyMemory`1 source, Boolean async)
   at System.Net.Http.HttpConnection.FlushAsync(Boolean async)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.HttpMessageHandlerStage.Send(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpMessageHandlerStage.Send(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.SocketsHttpHandler.Send(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.Send(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
   at System.Net.Http.HttpClient.Send(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken)
   at Tranga.MangaConnectors.HttpDownloadClient.MakeRequestInternal(String url, String referrer, String clickButton) in /src/Tranga/MangaConnectors/HttpDownloadClient.cs:line 75

...

Object name: 'System.Net.Sockets.Socket'..
   at System.Net.HttpResponseStream.InternalWrite(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.HttpResponseStream.WriteCore(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.Write(ReadOnlySpan`1 buffer)
   at Tranga.Server.SendResponse(HttpStatusCode statusCode, HttpListenerResponse response, Object content) in /src/Tranga/Server.cs:line 638
[02/05/2024 07:43:34.938]        Server | System.Net.HttpListenerException (0x80131620): Unable to write data to the transport connection: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'..
   at System.Net.HttpResponseStream.InternalWrite(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.HttpResponseStream.WriteCore(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.Write(ReadOnlySpan`1 buffer)
   at Tranga.Server.SendResponse(HttpStatusCode statusCode, HttpListenerResponse response, Object content) in /src/Tranga/Server.cs:line 638
[02/05/2024 07:43:34.938]        Server | System.Net.HttpListenerException (0x80131620): Unable to write data to the transport connection: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'..
   at System.Net.HttpResponseStream.InternalWrite(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.HttpResponseStream.WriteCore(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.Write(ReadOnlySpan`1 buffer)
   at Tranga.Server.SendResponse(HttpStatusCode statusCode, HttpListenerResponse response, Object content) in /src/Tranga/Server.cs:line 638
[02/05/2024 07:43:34.938]        Server | System.Net.HttpListenerException (0x80131620): Unable to write data to the transport connection: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'..
   at System.Net.HttpResponseStream.InternalWrite(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.HttpResponseStream.WriteCore(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.Write(ReadOnlySpan`1 buffer)
   at Tranga.Server.SendResponse(HttpStatusCode statusCode, HttpListenerResponse response, Object content) in /src/Tranga/Server.cs:line 638
[02/05/2024 07:43:34.938]        Server | System.Net.HttpListenerException (0x80131620): Unable to write data to the transport connection: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'..
   at System.Net.HttpResponseStream.InternalWrite(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.HttpResponseStream.WriteCore(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.Write(ReadOnlySpan`1 buffer)
   at Tranga.Server.SendResponse(HttpStatusCode statusCode, HttpListenerResponse response, Object content) in /src/Tranga/Server.cs:line 638
[02/05/2024 07:43:35.86 ]        Server | System.Net.HttpListenerException (0x80131620): Unable to write data to the transport connection: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'..
   at System.Net.HttpResponseStream.InternalWrite(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.HttpResponseStream.WriteCore(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.Write(ReadOnlySpan`1 buffer)
   at Tranga.Server.SendResponse(HttpStatusCode statusCode, HttpListenerResponse response, Object content) in /src/Tranga/Server.cs:line 638

Additional stuff

No response
@Charles-Em Charles-Em added the bug Something isn't working label Feb 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Charles-Em