Ultra-slow first request performance

[blecorre-mnp]

Describe the bug
We have been using EdgeDB in production for a while, and a few months ago updated our NodeJS version from v16 to v18 and more on multiple projects. We have been experiencing ultra-slow first request performances, every time our EdgeDB client is trying to execute a request after a while.

Reproduction
As this is a performance issue, I guess it might be kinda hard to replicate. We have a small DB with 56 types and 16380 objects, and this way of running requests.

import { createClient } from '~/edgedb';
import * as EdgeDB from '.../edgedb.ts';

// Single use client
request.run(createClient());

// Or using a shared client (imported from 'edgedb.ts')
request.run(EdgeDB.client);

Expected behavior
Initial request should not take up to 50x more time to run than following requests.

Versions

• OS: MacOS / Linux
• EdgeDB version: 4.5+641a8f3
• EdgeDB CLI version: 4.1.1+a774177
• edgedb-js version: 1.4.1

We tried a lot of Node versions such as v18.16.1, v19.8.1, v20.8.0, v20.11.0, v21.6.0. We also tried Bun v.1.1.0 and got the same results.

Thanks for your help!

[blecorre-mnp]

As an example of what's happening, here are the comparison of request time between Node versions, from Node v16 to the left to the newest versions down the line.

Everytime the server idles a bit, you can see this kind of metrics when a new request comes in.

[jaclarke]

How are you running the edgedb server? If you're using the edgedb cli tool to create the instance, then this is the expected behaviour. Since instances created with the edgedb cli are intended for local development, and we don't want to use all your system resources if you have a lot of instances, the server is configured to shutdown after being idle for some time, and will restart using socket activation on the next query. For production use you should setup the edgedb server using one of these guides: https://docs.edgedb.com/guides/deployment, or use edgedb cloud.

[blecorre-mnp]

We are running EdgeDB servers on bare metal Ubuntu instances. The EdgeDB server did not change between the Node 16 and above transitions we did.

[blecorre-mnp]

I also noticed that this happens on the EdgeDB UI app side of things enabled on the production server. Could this be linked to the complexity of the DB and therefore be more appropriate to talk about this in the EdgeDB repo?