url

Server Availability Checker

A robust utility for verifying the reachability of HTTP(S) endpoints while protecting against SSRF attacks. It validates URLs, enforces whitelist/blacklist policies, filters private networks, resolves hostnames securely, and performs configurable HTTP requests with retry, timeout, and SSL handling.

Key Features

• Strict URL scheme validation (only http and https)
• Optional whitelist loading from a file and dynamic blacklist caching from GitHub
• Private‑network and blacklisted‑host protection
• DNS‑over‑HTTPS resolution with fallback to the system resolver
• Customizable SSL context with optional custom CA bundles
• Manual redirect handling with a configurable limit
• Retry strategy with exponential back‑off and timeout control
• Dataclass ServerCheckResult summarizing status, final URL, redirects, and messages

Usage Examples

>>> from plantdb.client.url import is_server_available
>>> result = is_server_available('https://example.com', validate_host=False)
>>> print(result.ok, result.final_url)
True https://example.com/

Returned by:

• Reference API client client rest_api urls  origin_url

ServerCheckResult dataclass

ServerCheckResult(url, status_code, ok, final_url, message, n_redirects=0)

Class containing information about the server's response.

Returned by:

• Reference API client client url  is_server_available

is_server_available

is_server_available(url, timeout=5.0, max_redirects=2, max_retries=3, backoff_factor=0.3, allow_private_ip=False, validate_host=True, cert_path=None, verify_ssl=True)

Check if a server is available by making an HTTP request to the given URL.

Parameters:

Name	Type	Description	Default
url	str	The URL of the server to check.	required
timeout	float	Maximum number of seconds to wait for a response. Default is 5.0.	5.0
max_redirects	int	Maximum number of redirects to follow. Default is 2.	2
max_retries	int	Maximum number of retries in case of failure. Default is 3.	3
backoff_factor	float	Multiplier for the delay between retry attempts. Default is 0.3.	0.3
allow_private_ip	bool	If True, private or non‑routable IPs are accepted.	False
validate_host	bool	Whether to validate the hostname against a whitelist/blacklist. Default is True.	True
cert_path	str or Path	Path to a CA bundle to use. Default is None.	None
verify_ssl	bool	Whether to verify SSL certificates. Default is True.	True

Returns:

Type	Description
ServerCheckResult	An object containing details about the server check result.

Raises:

Type	Description
ValueError	If the supplied URL is malformed or a redirect target is invalid.
RuntimeError	If the maximum redirect limit is exceeded.
ConnectionError	If the HTTP request fails due to network issues.

Examples:

>>> from plantdb.client.url import is_server_available
>>> scr = is_server_available('https://google.com', validate_host=False)
>>> print(scr.final_url)
https://www.google.com/
>>> scr = is_server_available('https://google.com', validate_host=True)
INFO     [url] Validating URL hostname: google.com...
INFO     [url] Validating URL hostname: www.google.com...
print(scr.ok)
True

Source code in plantdb/client/url.py

def is_server_available(
        url: str,
        timeout: float = 5.0,
        max_redirects: int = 2,
        max_retries: int = 3,
        backoff_factor: float = 0.3,
        allow_private_ip: bool = False,
        validate_host: bool = True,
        cert_path: Optional[str] = None,
        verify_ssl: bool = True,
) -> ServerCheckResult:
    """Check if a server is available by making an HTTP request to the given URL.

    Parameters
    ----------
    url : str
        The URL of the server to check.
    timeout : float, optional
        Maximum number of seconds to wait for a response. Default is ``5.0``.
    max_redirects : int, optional
        Maximum number of redirects to follow. Default is ``2``.
    max_retries : int, optional
        Maximum number of retries in case of failure. Default is ``3``.
    backoff_factor : float, optional
        Multiplier for the delay between retry attempts. Default is ``0.3``.
    allow_private_ip : bool, optional
        If ``True``, private or non‑routable IPs are accepted.
    validate_host : bool, optional
        Whether to validate the hostname against a whitelist/blacklist. Default is ``True``.
    cert_path : str or pathlib.Path, optional
        Path to a CA bundle to use. Default is ``None``.
    verify_ssl : bool, optional
        Whether to verify SSL certificates. Default is ``True``.

    Returns
    -------
    plantdb.client.url.ServerCheckResult
        An object containing details about the server check result.

    Raises
    ------
    ValueError
        If the supplied URL is malformed or a redirect target is invalid.
    RuntimeError
        If the maximum redirect limit is exceeded.
    ConnectionError
        If the HTTP request fails due to network issues.

    Examples
    --------
    >>> from plantdb.client.url import is_server_available
    >>> scr = is_server_available('https://google.com', validate_host=False)
    >>> print(scr.final_url)
    https://www.google.com/
    >>> scr = is_server_available('https://google.com', validate_host=True)
    INFO     [url] Validating URL hostname: google.com...
    INFO     [url] Validating URL hostname: www.google.com...
    print(scr.ok)
    True
    """
    # ------------------------------------------------------------------ #
    # 1. Parse the URL
    # ------------------------------------------------------------------ #
    parsed = _parse_url(url)
    if not parsed:
        message = f"URL '{url}' is not a valid URL."
        logger.warning(message)
        return ServerCheckResult(
            url=url,
            status_code=0,
            ok=False,
            final_url=url,
            message=message,
        )

    # ------------------------------------------------------------------ #
    # 2. Build retry strategy
    # ------------------------------------------------------------------ #
    retry_strategy = Retry(
        total=max_retries,
        backoff_factor=backoff_factor,
        status_forcelist=[500, 502, 503, 504],
        allowed_methods={"GET", "HEAD"},
        redirect=max_redirects,
    )

    # ------------------------------------------------------------------ #
    # 3. Configure SSL
    # ------------------------------------------------------------------ #
    cert_path_obj = Path(cert_path) if cert_path is not None else None
    ssl_context, ca_certs, cert_reqs = _build_ssl_context(cert_path_obj, verify_ssl)

    # ------------------------------------------------------------------ #
    # 4. Create HTTP pool manager
    # ------------------------------------------------------------------ #
    http = _configure_http_pool(timeout, retry_strategy, ssl_context, ca_certs, cert_reqs)

    # ------------------------------------------------------------------ #
    # 5. Perform the request and follow redirects manually
    # ------------------------------------------------------------------ #
    try:
        response, final_url, n_redirects = _handle_redirects(http, URL(url).href, max_redirects,
                                                             allow_private_ip, validate_host)
    except (ConnectionError, RuntimeError, ValueError) as exc:
        message = f"Request to {url!r} failed: {exc}"
        logger.error(message)
        return ServerCheckResult(
            url=url,
            status_code=0,
            ok=False,
            final_url=url,
            message=message,
        )

    # ------------------------------------------------------------------ #
    # 6. Build the result object
    # ------------------------------------------------------------------ #
    ok = 200 <= response.status < 400
    return ServerCheckResult(
        url=url,
        status_code=response.status,
        ok=ok,
        final_url=final_url,
        message="Valid URL." if ok else f"HTTP {response.status}",
        n_redirects=n_redirects,
    )