Why longer prompts slow down local AI on an RTX 5090

A one-line chat prompt and a request that includes a system prompt, retrieved documents and conversation history exercise the same model differently. NVIDIA lists 32 GB of GDDR7 for the RTX 5090. The model, runtime and other workloads determine how much memory is available for longer requests.

Pick a real task, such as answering a question from a project brief. Keep the model, quantization and generation settings fixed. Run the task with a short prompt, a typical application prompt and a long prompt drawn from the same workflow.

Record prompt token count, peak GPU memory and the time from sending the request to receiving the first token. Measure output tokens per second separately. A model can generate quickly once it starts while still leaving the user waiting for the prompt to be processed.

Warm up the model, then repeat each input size. Compare fresh requests separately from requests that reuse a cached prefix. Test one user first, then repeat at the concurrency your app needs.

Remove repeated instructions, trim old chat turns or retrieve fewer passages. Run the same questions again and check whether the answers still use the right evidence. If a shorter prompt causes mistakes, restore the missing material before comparing speed.

Keep a small set of representative questions and decide how long users should wait for a first token. Re-run that set when the prompt, model or runtime changes so you can spot regressions in both speed and quality.