A artificial intelligence agent it doesn’t spend all its time generating responses. It can start a program, consult an external service, run a test, or stop waiting for user permission. During these pauses, however, there is a risk of losing a shortcut that allows the linguistic models to work faster and with less expense: the prompt cache.
A study published on arXiv proposes to keep it active through short periodic signals, called keepalive. In tests conducted with the services of Anthropic, OpenAI, Google and DeepSeek, this technique allowed a large part of the already processed prefix to be kept in the chatbot cache even during pauses long enough to normally cause its removal. In the most favorable case, the cost of the single request sent after the break can be reduced by a maximum factor of 12.5. However, the overall convenience depends on the time spent, the platform used and the frequency with which keepalives are carried out.
What is prompt caching and why does it make AI more efficient
When a request is sent to a large language model, the system must first parse the received text. The instructions, previous messages and any documents are divided into tokens, i.e. small units, and transformed into an internal state that can be used to produce the response. This initial phase is called prefill.
In long conversations, much of the content remains the same from one call to the next. To avoid repeating the same calculations every time, suppliers can temporarily store the prefix already processedthat is, the initial part of the prompt that remains the same between one request and the following. The model thus only has to deal with the added portion.
This is not the long-term memory with which a chatbot can remember preferences or information about the user. It is a technical mechanism, comparable to leaving documents on the desk that will still be needed and as long as they remain available there is no need to prepare them again.
In the services examined in the study, tokens retrieved from cache can cost much less than those processed from scratch in some cases about a tenth. It also takes less time for the system to start generating the response.
Why AI agent pauses can increase costs
The problem emerges above all in agentic systems, whose work cycle can be summarized in three phases: think, act and wait. After determining what action to perform, the agent uses an external tool. It could, for example, compile some code, launch a series of tests, query a database, or ask for confirmation before sending an email.
These tasks can take several minutes. Meanwhile, the temporary storage associated with the prefix remains unused and the vendor can remove it to free up resources.
If the copy has been removed in the meantime, when the following request arrives the system he is no longer able to recover the calculations already carried outalthough the timeline of the conversation remained almost identical. The context is still present and the agent has not forgotten the task to complete, but the prefill must be run again.
The consequence becomes relevant when the history includes tens of thousands of tokens and the same dynamic is repeated several times. Each removal of the prefix from the cache results in a new prefill phasewith an increase in expense and time required for the answer to begin to appear.
How keepalive works and what the study showed
The analyzed solution consists of resend the same prefix while waitingasking the model to generate the minimum amount of new content possible. Each signal renews the permanence of the data in the cache, preventing the platform from eliminating the internal state produced during the processing of the prompt. The principle is reminiscent of keepalive messages used in network connections to communicate that a session is still active.
To test the effectiveness of the technique, they were used 40,000 and 100,000 token contextsintroducing breaks of 1, 5 and 10 minutes. In one part of the tests no signal was sent during the interval, while in the others the prefix was presented again every 30 seconds.
Measurements were also carried out for up to 40 minutes, with the aim of identifying the moment in which each service tended to delete the retained data.
With keepalives enabled, a median share of at least 98% of the prompt tokens.
In tests, the prefix was completely evicted from Anthropic’s cache after about 6 minutes, while on DeepSeek this happened within 10 minutes. In the case of OpenAI, it became completely “cold” around the half hour mark.
The results for Google were less straightforward. The platform did not show a stable expiration point, probably because requests could be routed to different machines. Pings increased the likelihood of finding data, but a window of economic convenience could not be identified as precisely.
When keeping the cache “warm” really pays off
Keeping the prefix active is not always advantageous. In fact, each signal consumes resources and, if it is sent too often, the additional expense can exceed the savings obtained by avoiding new processing.
The study shows that the 30 second interval used in the tests was about eight times more expensive than necessary for Anthropic. To reduce waste, you need to wait as long as possible without exceeding the period during which the supplier retains the data with sufficient certainty.
Based on measurements, if you choose to use keepalive, the most efficient interval would be about 4 minutes for Anthropic and DeepSeek and 8 minutes for OpenAI. These values include a safety margin with respect to the observed deadline.
The overall duration of the break also plays a decisive role. If the pause is very short, the content still remains available and the keepalive represents only an additional cost. If the wait is prolonged excessively, however, the sum of the signals can become more expensive than a single new prefill.
There is therefore an intermediate range in which keeping the cache “warm” is really convenient. During a 30-minute pause with a 100,000 token context, considering pings and the follow-up request, the technique reduced spending by a factor of 1.56 on Anthropic and up to 2.45 on OpenAIusing the most suitable interval for each platform.
On DeepSeek, the price of new processing was already so low that the series of pings was inconvenient. For Google, however, the absence of a predictable deadline did not allow it to calculate an equally precise strategy.
The maximum value of 12.5 times concerns the cost of the next request and derives from the comparison between the case in which the prefix is still available and the one in which the entire context must be processed from scratch. Also considering the signals sent during the wait, the economic advantage of the complete operation is more limited.
However, the technique offers an additional benefit: in services where the cached copy had been eliminated, the keepalive significantly reduced the time needed to start generating the response.
Because this solution may not last forever
Preserving internal state may be convenient for the individual developer, but could become more problematic if the technique becomes widespread. The available cache spaces are shared and, if each agent continuously sent signals, numerous prefixes would always appear recent and therefore need to be preserved, increasing the pressure on the platforms’ infrastructures.
For this, suppliers could introduce absolute time limits, quotas for each account or specific fees based on the number of tokens kept in memory per hour.
The work is also a pre-publication and has some limitations. The measurements describe the conditions observed with a single test system. The tests on Google were reduced by the available quotas, while DeepSeek was examined through a specific intermediate service.
However, the results show that, in activities composed of actions and dead time, preserving the already elaborated context can avoid repeated calculations. To achieve real savings, however, the frequency of keepalives must be adapted to the behavior and prices of the individual platform.
