[Feature]: Add flag to disable compression on upstream requests?

[Manouchehri]

The Feature

Similar to #3958 / #3533, LiteLLM might get a performance boost by disabling gzip on upstream LLM requests.

See encode/httpx#2220 (comment) for more details on how to disable compression. tl;dr: set the Accept-Encoding: identity header.

Motivation, pitch

https://medium.com/@juan.deaton/gzip-data-compression-has-a-critical-role-of-increasing-performance-and-smashing-the-monstrous-caae5b4a001c#:~:text=CPUs%20are%20Inefficient%20at%20Performing%20GZIP&text=With%20no%20GZIP%2C%20the%20web,and%20consumes%207x%20more%20energy.

Twitter / LinkedIn details

https://www.linkedin.com/in/davidmanouchehri/

[krrishdholakia]

@Manouchehri can we move these from being issues to discussions?

happy to move this to a feature request once if you can confirm you're seeing a perf boost here

[Manouchehri]

Done! To be honest I'm not super familiar on doing perf benchmarks with LiteLLM; the slowest part for our use-case is definitely the LLM APIs themselves, so it's tricky for us to tell if disabling compression helps or not.

Maybe we could use a mock Cloudflare Pages (or Worker) server to simulate a fake OpenAI server for benchmarking purposes? Do you have anything like this hidden somewhere already? :)