Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Webhook to handle Replicate's responses for large model #154

Open
1 task
bgmntmail opened this issue May 22, 2024 · 0 comments
Open
1 task

Add a Webhook to handle Replicate's responses for large model #154

bgmntmail opened this issue May 22, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@bgmntmail
Copy link

Describe the feature

Problem Overview

  1. Asynchronous Processing:
  • Replicate API: When you make a request to the Replicate API to process some data, it doesn't give you the result immediately because the processing takes time.
  • Webhook: Instead, the Replicate API allows you to specify a webhook. A webhook is like a callback URL that the API will call once it finishes processing the data.
  1. Timeout Issues:
  • Vercel and Netlify: Our current hosting providers, Vercel and Netlify, will cut off any request that takes too long (25 seconds for Vercel, 10 seconds for Netlify). Since the processing by Replicate can take longer, we can't wait for the response directly in our initial API call.

Solution: Using Webhooks

  1. Initial Request:
  • When the user makes a request to our API to process data with the Replicate API, our API immediately returns a response saying the request has been accepted.
  • Our API also stores some information to track this request.
  1. Handling the Webhook:
  • We set up a new endpoint in our API to act as a webhook. The Replicate API will call this webhook once it has finished processing the data.
  • When our webhook endpoint receives the call, it updates the status of the original request with the result.

Additional context

No response

Participation

  • I am willing to submit a pull request for this feature.
@bgmntmail bgmntmail added the enhancement New feature or request label May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant