Timeouts

Introduction

Not all actions or requests complete immediately. Most of the actions take a bit of time to finish and that time may depend on various variables like user input, current load, etc. At the same time, we don't want to wait forever as any waiting takes resources, and we are limited in terms of resources.

One way to go about this problem is to specify a timeout which is how much time you can wait in the worst case for the action to happen.

Some functionality comes with built-in timeouts. For example, if you are using a HTTP or RPC client, chances are there is timeout parameter in there:

• HTTPX: Timeouts
• AIOHTTP: Timeouts
• gRPC: ClientCallDetails
• IOTHRIFT: Client Timeout

For situations where this is not a case, Hyx provides a decorator and a context manager that can help with setting up timeouts.

Use Case

• Ensure that the caller has to wait no more than a given delay. Local timeouts work best when limiting action that doesn't touch other microservices.
• Limit a microservice request chain by using a distributed timeout

By Locality

Local Timeouts

decoratorcontext manager

import asyncio

from hyx.timeout import timeout


@timeout(1)
async def copy_directory(source_dir: str, destination_dir: str) -> None:
    """
    Copy a huge directory
    """
    ...


asyncio.run(copy_directory(source_dir="~/datasets", destination_dir="~/data"))

import asyncio

from hyx.timeout import timeout


async def copy_directory(source_dir: str, destination_dir: str) -> None:
    """
    Copy a huge directory
    """
    async with timeout(1):
        ...


asyncio.run(copy_directory(source_dir="~/datasets", destination_dir="~/data"))

class hyx.timeout.timeout(timeout_secs, *, name=None, listeners=None, event_manager=None)

Timeout Decontext

Parameters:

• timeout_secs (float) - Max amount of time to wait for the action in seconds
• name (None | str) - A component name or ID (will be passed to listeners and mention in metrics)
• listeners (None | Sequence[TimeoutListener]) - List of listeners of this concreate component state

Warning

For the sake of simplicity, Hyx assumes that you are following AsyncIO best practices and not running CPU-intensive operations in the main thread. Otherwise, the timeout functionality may fire with a delay after the thread is unblocked.

Distributed Timeout

Distributed systems make things more complicated when it comes to timeouts. When you request a downstream service with a timeout, there is a possibility that the request will be queued for a while. Hence, the caller will give up waiting on the timeout and move on handling other requests.

But what will eventually happen with that queued request? If not failed on the load balancing level, it will simply be processed when the turn comes to it. That will be a waste of resources as the result of the request is not going to be used anyhow. That's a problem.

A common solution is to let all microservices in the request chain know how much time they have to respond. If we exceed that limit, microservices can ignore the requests as they are already timed out. This is called a distributed timeout (a.k.a. deadlines, timeout budget).

Info

Hyx doesn't yet provide a way to easily add a distributed budget as it will require to integrate with the API framework of your choice. We have some plans to support that. Let us know if this is useful for you.