The trap
An OpenAI request can work perfectly in local development and still fail in production when it is tied to one serverless HTTP request.
The user clicks a button. Your route calls OpenAI. The model takes longer than expected. The function hits a limit, the browser disconnects, the stream closes, or the user refreshes. Now the product has a worse problem than a failed request.
It has an unknown request.
- Did the job finish?
- Did it fail?
- Did it run twice?
- Should the user retry?
- Will retrying charge again?
- Is the answer sitting somewhere, or was it lost?
Why this hurts AI products more
For AI products, a timeout is not just a backend error. It can mean the user's work has no reliable final state.
That is why simply increasing a timeout is often the wrong mental model. The more durable pattern is to accept the job, return a request ID, run the OpenAI call reliably, and let the app check status until the request completes or fails visibly.
That is the problem ReqRun is built around: durable request IDs, retries, idempotency, async execution, and status visibility around an OpenAI-compatible request.
The problem is not serverless
Serverless is not broken. The mistake is using one live HTTP request as the only container for work that may take longer, fail transiently, or need to survive a refresh.
That design can be fine for a quick classification or a throwaway internal tool. It becomes fragile when the AI task matters: generating a customer-facing report, classifying a support ticket, processing a form submission, summarizing a long document, or running a step inside an agent workflow.
OpenAI calls are not always predictable. Prompt length, model load, tool calls, rate limits, network conditions, and upstream errors can all change how long one request takes.
What actually times out?
When a developer says an OpenAI request timed out, several different things may have happened.
To the user, they all look the same: they clicked, waited, and got nothing useful. To the system, they are very different events, and that difference matters.
- The browser stopped waiting.
- The serverless function reached its maximum duration.
- The stream started, then disconnected.
- The upstream model provider was slow.
- The function returned a 504.
- A retry started a second copy of the same job.
- A user refreshed the page and submitted again.
Current platform limits still matter
You should configure your platform correctly. If your function duration is too low, raise it. If your app streams responses, make sure streaming is actually supported by the runtime and deployment path. If an upstream call can hang, handle the error.
Vercel's current documentation says that if a function does not respond within its duration, it returns a 504 FUNCTION_INVOCATION_TIMEOUT. With Fluid Compute enabled by default, Vercel currently lists 300-second defaults on Hobby, Pro, and Enterprise, with maximums up to 800 seconds on Pro and Enterprise.
AWS Lambda documents the same architectural boundary in a different form: the function runs for a configured amount of time, up to 900 seconds, and timeout risk depends on external service latency and the amount of data being processed.
Why a bigger maxDuration is only a partial fix
A bigger timeout is still just a bigger clock. It does not create a durable request record. It does not dedupe duplicate clicks. It does not preserve state after a refresh. It does not give support a request ID.
It also does not distinguish a provider failure from a user disconnect, or protect you from retrying the same expensive request twice.
For a prototype, increasing the timeout may be enough. For a production AI feature, the better question is not how long a function can wait. The better question is what happens to the user's AI job if the function disappears.
The better pattern: accept, run, check
The production pattern is simple: accept the request, create a durable AI job, return a request ID, run the OpenAI request outside the fragile user-facing HTTP lifecycle, expose status, store the final result, and dedupe repeat submissions.
Developers often arrive at this pattern after fighting timeout issues for a while because it changes the contract with the user from hope to visibility.
- Accepted
- Running
- Retrying
- Completed
- Failed
The direct version
A direct OpenAI call inside a serverless route is still the easiest thing to write, and sometimes that is the right choice.
There is nothing wrong with this shape for a short, cheap request that is safe to lose.
const result = await openai.chat.completions.create({
model: "gpt-5-nano",
messages: [
{ role: "user", content: userMessage }
]
});
return Response.json(result);The durable version
If the request matters, the safer version is to make it durable. Return acceptance now, then reconnect to the outcome later through a request ID and status lookup.
Now the app has something it can hold onto: a request ID, a status, a dedupe key, and a way to check later without guessing.
const run = await reqrun.chat.completions.create({
model: "gpt-5-nano",
messages: [
{ role: "user", content: userMessage }
],
wait: false,
idempotency_key: userActionId
});
return Response.json({
request_id: run.id,
status: "accepted"
});What this looks like with ReqRun
ReqRun's job is narrow: queue the request, retry transient failures, dedupe repeat submissions, return a durable rr_ id, and make the final outcome visible.
That last word matters. Visible. Not maybe completed. Not probably failed. Not lost in logs. Not ask the user to try again and hope it does not duplicate.
Complete or fail visibly is a much better product contract than one long spinner tied to one fragile request.
A production checklist for OpenAI on serverless
Before shipping an AI feature on serverless, ask whether the system can survive a noisy day instead of only a quiet one.
- Can the user refresh the page without losing the result?
- Can a repeated click be deduped?
- Can support look up one AI request by ID?
- Can the frontend show a real status instead of an endless spinner?
- Can you tell the difference between a provider error, a function timeout, and a browser disconnect?
- Do retries have a maximum?
- Do retries use backoff?
- Can failed requests be inspected later?
- Is the final result stored somewhere durable?
- Is the user's original action tied to an idempotency key?
Why do OpenAI requests timeout in serverless functions?
OpenAI requests can timeout in serverless functions because the AI response may take longer than the platform, runtime, browser, proxy, or stream can safely wait.
The issue is usually not OpenAI alone. It is the whole request chain.
Is increasing the serverless timeout enough?
Sometimes. Increasing maxDuration can help for simple long responses.
But it does not solve durable state, duplicate submissions, user refreshes, retry control, or status visibility.
What is the best pattern for long-running OpenAI calls?
The best pattern is to accept the request, return a durable request ID, process the OpenAI call asynchronously, and let the app check status until it completes or fails.
When should I use a direct OpenAI call instead?
Use a direct OpenAI call when the task is short, cheap, synchronous, and safe to lose.
Use a durable request layer when the task is important, expensive, long-running, user-visible, or should not be duplicated.
The simple rule
A short AI call can live inside a normal request. An important AI job needs a record.
That is the serverless timeout trap in one sentence. The failure is not always the timeout itself. The failure is accepting a user's work without creating a durable way to know what happened to it.
Serverless did not create that problem. AI made it easier to notice. Once users depend on an AI result, try again is not a reliability strategy.