More accurate time logging for ImageEncoder and fix concurrent image processing corruption

[irexyc]

Motivation

#1759

[RunningLeon]

do we still need to convert to cpu ?

[irexyc]

The modification here is mainly to solve the #1759

[lvhan028]

Will it slow down the inference performance?

[irexyc]

test with llava-v1.5-7b and profile_restful_api.py (PR 1662)

Move tensor to cpu doesn't affect the performance. But using asyncio.Task may affects the performance.

# with thread
number of prompt tokens: 248339
number of completion tokens: 240582
token throughput (completion token): 825.260 token/s
token throughput (prompt + completion token): 1677.128 token/s
RPS (request per second): 3.430 req/s
RPM (request per minute): 205.816 req/min

# with asyncio.Task
number of prompt tokens: 248339
number of completion tokens: 240582
token throughput (completion token): 808.719 token/s
token throughput (prompt + completion token): 1643.513 token/s
RPS (request per second): 3.362 req/s
RPM (request per minute): 201.691 req/min

[grimoire]

If gpu utility is high, you can add

        await asyncio.get_event_loop().run_in_executor(None,
                                                       self.stream.synchronize)

after forward so other coroutine can use cpu during forward.

[irexyc]

According to my test, wrapping forward is better to synchronize the stream.

outputs = await asyncio.get_event_loop().run_in_executor(
                None, self.forward, inputs)
# token throughput (completion token): 826.370 token/s

outputs = self.model.forward(inputs)
if isinstance(outputs[0], torch.Tensor):
    stream = torch.cuda.current_stream(outputs[0].device)
    await asyncio.get_event_loop().run_in_executor(
                None, stream.synchronize)
# token throughput (completion token): 815.130 token/s

[RunningLeon]

LGTM

[lvhan028]

@irexyc Does this PR ONLY target #1759?

[irexyc]

Does this PR ONLY target #1759?

and #1730