Fastapi & Rate Limiting

Let’s test a simple Rate Limiting Function found on the Net …

main.py

Main App with Rate Limiting Function

# main.py
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from starlette.middleware.base import BaseHTTPMiddleware
from Token_bucket import TokenBucket

app = FastAPI()

class RateLimiterMiddleware(BaseHTTPMiddleware):
    def __init__(self, app, bucket: TokenBucket):
        super().__init__(app)
        self.bucket = bucket

    async def dispatch(self, request: Request, call_next):
        if self.bucket.take_token():
            return await call_next(request)

        # Return a JSON response for rate limit exceeded
        return JSONResponse(status_code=429, content={"detail": "Rate limit exceeded"})

# Token bucket with a capacity of 3 and refill rate of 1 token per second
bucket = TokenBucket(capacity=3, refill_rate=3)

# Apply the rate limiter middleware
app.add_middleware(RateLimiterMiddleware, bucket=bucket)

@app.get("/")
async def read_root():
    return {"message": "Hello World"}

Token_bucket.py

# Token_bucket.py
import time

class TokenBucket:
    def __init__(self, capacity, refill_rate):
        self.capacity = capacity
        self.refill_rate = refill_rate
        self.tokens = capacity
        self.last_refill = time.time()

    def add_tokens(self):
        now = time.time()
        if self.tokens < self.capacity:
            tokens_to_add = (now - self.last_refill) * self.refill_rate
            self.tokens = min (self.capacity, self.tokens + tokens_to_add)
        self.last_refill=now

    def take_token(self):
        self.add_tokens()
        if self.tokens >= 1:
            self.tokens -=1
            return True
        return False

Test Script

Let’s produce some requests …

how to use Python Decorator

Sample Code

main.py

cat << 'EOF' > main.py
#!/usr/bin/env python3

# Sample from https://blog.stoege.net/posts/python_decorator/

# Vars
a = 3
b = 8

# Decorator which logs start and end of a function
def log_function_call(func):
    def wrapper(*args, **kwargs):
        print(f"\nCalling function '{func.__name__}' with {args=} & {kwargs=}")
        result = func(*args, **kwargs)
        print(f"Function '{func.__name__}' finnished\n")
        return result
    return wrapper

@log_function_call
def add(a=int, b=int) -> int:
    result = a + b
    return result

@log_function_call
def sub(a=int, b=int) -> int:
    result = a - b
    return result

@log_function_call
def prod(a=int, b=int) -> int:
    result = a * b
    return result

@log_function_call
def div(a=int, b=int) -> int:
    result = a / b
    return result

if __name__ == "__main__":
    print("Result:", add(a, b))
    print("Result:", sub(a, b))
    print("Result:", prod(a, b))
    print("Result:", div(a, b))

EOF
chmod u+x main.py

##### RUN #####

# ./main.py

###############

Running Code

user@host% ./main.py 

Calling function 'add' with args=(3, 8) & kwargs={}
Function 'add' finnished

Result: 11

Calling function 'sub' with args=(3, 8) & kwargs={}
Function 'sub' finnished

Result: -5

Calling function 'prod' with args=(3, 8) & kwargs={}
Function 'prod' finnished

Result: 24

Calling function 'div' with args=(3, 8) & kwargs={}
Function 'div' finnished

Result: 0.375

sha256: ab6641b101cea8ac7eb6517fcefb029380644a56203723973fa1bc957fafbafb

OpenBSD 7.6 / LibreSSl 4.0.0 / cryptography 43.0.1

seems that cryptography library is broken / not yet ready for libressl 4.0.0

Problems adding Crypto

cd /tmp
poetry new test_crypto
cd test_crypto/
poetry add cryptography

root@host /bin# cd tmp

root@host /tmp# poetry new test_crypto

Created package test_crypto in test_crypto
root@host /tmp# cd test_crypto/

root@host /tmp/test_crypto# poetry add cryptography                                                                                                                                                  
Creating virtualenv test-crypto in /tmp/test_crypto/.venv
Using version ^43.0.1 for cryptography

Updating dependencies
Resolving dependencies... (0.2s)

Package operations: 3 installs, 0 updates, 0 removals

  - Installing pycparser (2.22)
  - Installing cffi (1.17.1)
  - Installing cryptography (43.0.1): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel
  
  Running `maturin pep517 build-wheel -i /tmp/tmpm4f6kesa/.venv/bin/python --compatibility off`
  📦 Including license file "/tmp/tmpzw46sfn4/cryptography-43.0.1/LICENSE"
  📦 Including license file "/tmp/tmpzw46sfn4/cryptography-43.0.1/LICENSE.APACHE"
  📦 Including license file "/tmp/tmpzw46sfn4/cryptography-43.0.1/LICENSE.BSD"
  🍹 Building a mixed python/rust project
  🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
  🐍 Not using a specific python interpreter
  📡 Using build options features, locked from pyproject.toml
     Compiling proc-macro2 v1.0.86
     Compiling target-lexicon v0.12.15
     Compiling unicode-ident v1.0.12
     Compiling pyo3-build-config v0.22.2
     Compiling quote v1.0.36
     Compiling cc v1.1.6
     Compiling syn v2.0.71
     Compiling vcpkg v0.2.15
     Compiling pkg-config v0.3.30
     Compiling once_cell v1.19.0
     Compiling openssl-sys v0.9.103
  error: failed to run custom build command for `openssl-sys v0.9.103`
  
  Caused by:
    process didn't exit successfully: `/tmp/tmpzw46sfn4/cryptography-43.0.1/src/rust/target/release/build/openssl-sys-b9cf452982f5d9f0/build-script-main` (exit status: 101)
    --- stdout
    cargo:rustc-check-cfg=cfg(osslconf, values("OPENSSL_NO_OCB", "OPENSSL_NO_SM4", "OPENSSL_NO_SEED", "OPENSSL_NO_CHACHA", "OPENSSL_NO_CAST", "OPENSSL_NO_IDEA", "OPENSSL_NO_CAMELLIA", "OPENSSL_NO_RC4", "OPENSSL_NO_BF", "OPENSSL_NO_PSK", "OPENSSL_NO_DEPRECATED_3_0", "OPENSSL_NO_SCRYPT", "OPENSSL_NO_SM3", "OPENSSL_NO_RMD160", "OPENSSL_NO_EC2M", "OPENSSL_NO_OCSP", "OPENSSL_NO_CMS", "OPENSSL_NO_COMP", "OPENSSL_NO_SOCK", "OPENSSL_NO_STDIO"))
    cargo:rustc-check-cfg=cfg(openssl)
    cargo:rustc-check-cfg=cfg(libressl)
    cargo:rustc-check-cfg=cfg(boringssl)
    cargo:rustc-check-cfg=cfg(libressl250)
    cargo:rustc-check-cfg=cfg(libressl251)
    cargo:rustc-check-cfg=cfg(libressl252)
    cargo:rustc-check-cfg=cfg(libressl261)
    cargo:rustc-check-cfg=cfg(libressl270)
    cargo:rustc-check-cfg=cfg(libressl271)
    cargo:rustc-check-cfg=cfg(libressl273)
    cargo:rustc-check-cfg=cfg(libressl280)
    cargo:rustc-check-cfg=cfg(libressl281)
    cargo:rustc-check-cfg=cfg(libressl291)
    cargo:rustc-check-cfg=cfg(libressl310)
    cargo:rustc-check-cfg=cfg(libressl321)
    cargo:rustc-check-cfg=cfg(libressl332)
    cargo:rustc-check-cfg=cfg(libressl340)
    cargo:rustc-check-cfg=cfg(libressl350)
    cargo:rustc-check-cfg=cfg(libressl360)
    cargo:rustc-check-cfg=cfg(libressl361)
    cargo:rustc-check-cfg=cfg(libressl370)
    cargo:rustc-check-cfg=cfg(libressl380)
    cargo:rustc-check-cfg=cfg(libressl381)
    cargo:rustc-check-cfg=cfg(libressl382)
    cargo:rustc-check-cfg=cfg(libressl390)
    cargo:rustc-check-cfg=cfg(libressl400)
    cargo:rustc-check-cfg=cfg(ossl101)
    cargo:rustc-check-cfg=cfg(ossl102)
    cargo:rustc-check-cfg=cfg(ossl102f)
    cargo:rustc-check-cfg=cfg(ossl102h)
    cargo:rustc-check-cfg=cfg(ossl110)
    cargo:rustc-check-cfg=cfg(ossl110f)
    cargo:rustc-check-cfg=cfg(ossl110g)
    cargo:rustc-check-cfg=cfg(ossl110h)
    cargo:rustc-check-cfg=cfg(ossl111)
    cargo:rustc-check-cfg=cfg(ossl111b)
    cargo:rustc-check-cfg=cfg(ossl111c)
    cargo:rustc-check-cfg=cfg(ossl111d)
    cargo:rustc-check-cfg=cfg(ossl300)
    cargo:rustc-check-cfg=cfg(ossl310)
    cargo:rustc-check-cfg=cfg(ossl320)
    cargo:rustc-check-cfg=cfg(ossl330)
    cargo:rerun-if-env-changed=X86_64_UNKNOWN_OPENBSD_OPENSSL_LIB_DIR
    X86_64_UNKNOWN_OPENBSD_OPENSSL_LIB_DIR unset
    cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
    OPENSSL_LIB_DIR unset
    cargo:rerun-if-env-changed=X86_64_UNKNOWN_OPENBSD_OPENSSL_INCLUDE_DIR
    X86_64_UNKNOWN_OPENBSD_OPENSSL_INCLUDE_DIR unset
    cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
    OPENSSL_INCLUDE_DIR unset
    cargo:rerun-if-env-changed=X86_64_UNKNOWN_OPENBSD_OPENSSL_DIR
    X86_64_UNKNOWN_OPENBSD_OPENSSL_DIR unset
    cargo:rerun-if-env-changed=OPENSSL_DIR
    OPENSSL_DIR unset
    cargo:rerun-if-env-changed=OPENSSL_NO_PKG_CONFIG
    cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-openbsd
    cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_openbsd
    cargo:rerun-if-env-changed=HOST_PKG_CONFIG
    cargo:rerun-if-env-changed=PKG_CONFIG
    cargo:rerun-if-env-changed=OPENSSL_STATIC
    cargo:rerun-if-env-changed=OPENSSL_DYNAMIC
    cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
    cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
    cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-openbsd
    cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_openbsd
    cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
    cargo:rerun-if-env-changed=PKG_CONFIG_PATH
    cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-openbsd
    cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_openbsd
    cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
    cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
    cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-openbsd
    cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_openbsd
    cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
    cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
    cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
    cargo:rerun-if-env-changed=SYSROOT
    cargo:rerun-if-env-changed=OPENSSL_STATIC
    cargo:rerun-if-env-changed=OPENSSL_DYNAMIC
    cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
    cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
    cargo:rustc-link-lib=ssl
    cargo:rustc-link-lib=crypto
    cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-openbsd
    cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_openbsd
    cargo:rerun-if-env-changed=HOST_PKG_CONFIG
    cargo:rerun-if-env-changed=PKG_CONFIG
    cargo:rerun-if-env-changed=OPENSSL_STATIC
    cargo:rerun-if-env-changed=OPENSSL_DYNAMIC
    cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
    cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
    cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-openbsd
    cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_openbsd
    cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
    cargo:rerun-if-env-changed=PKG_CONFIG_PATH
    cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-openbsd
    cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_openbsd
    cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
    cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
    cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-openbsd
    cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_openbsd
    cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
    cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
    cargo:rerun-if-changed=build/expando.c
    OPT_LEVEL = Some(3)
    TARGET = Some(x86_64-unknown-openbsd)
    OUT_DIR = Some(/tmp/tmpzw46sfn4/cryptography-43.0.1/src/rust/target/release/build/openssl-sys-69c53327ca178616/out)
    HOST = Some(x86_64-unknown-openbsd)
    cargo:rerun-if-env-changed=CC_x86_64-unknown-openbsd
    CC_x86_64-unknown-openbsd = None
    cargo:rerun-if-env-changed=CC_x86_64_unknown_openbsd
    CC_x86_64_unknown_openbsd = None
    cargo:rerun-if-env-changed=HOST_CC
    HOST_CC = None
    cargo:rerun-if-env-changed=CC
    CC = None
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    RUSTC_WRAPPER = None
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    DEBUG = Some(false)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-openbsd
    CFLAGS_x86_64-unknown-openbsd = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_openbsd
    CFLAGS_x86_64_unknown_openbsd = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    cargo:rustc-cfg=osslconf="OPENSSL_NO_BUF_FREELISTS"
    cargo:rustc-cfg=osslconf="OPENSSL_NO_COMP"
    cargo:rustc-cfg=osslconf="OPENSSL_NO_EC2M"
    cargo:rustc-cfg=osslconf="OPENSSL_NO_ENGINE"
    cargo:rustc-cfg=osslconf="OPENSSL_NO_KRB5"
    cargo:rustc-cfg=osslconf="OPENSSL_NO_PSK"
    cargo:rustc-cfg=osslconf="OPENSSL_NO_SRP"
    cargo:rustc-cfg=osslconf="OPENSSL_NO_SSL3_METHOD"
    cargo:rustc-cfg=osslconf="OPENSSL_NO_SEED"
    cargo:rustc-cfg=osslconf="OPENSSL_NO_SCRYPT"
    cargo:conf=OPENSSL_NO_BUF_FREELISTS,OPENSSL_NO_COMP,OPENSSL_NO_EC2M,OPENSSL_NO_ENGINE,OPENSSL_NO_KRB5,OPENSSL_NO_PSK,OPENSSL_NO_SRP,OPENSSL_NO_SSL3_METHOD,OPENSSL_NO_SEED,OPENSSL_NO_SCRYPT
    cargo:rustc-cfg=openssl
    cargo:rustc-cfg=libressl
    cargo:rustc-cfg=libressl251
    cargo:rustc-cfg=libressl252
    cargo:rustc-cfg=libressl261
    cargo:rustc-cfg=libressl270
    cargo:rustc-cfg=libressl271
    cargo:rustc-cfg=libressl273
    cargo:rustc-cfg=libressl280
    cargo:rustc-cfg=libressl281
    cargo:rustc-cfg=libressl291
    cargo:rustc-cfg=libressl310
    cargo:rustc-cfg=libressl321
    cargo:rustc-cfg=libressl332
    cargo:rustc-cfg=libressl340
    cargo:rustc-cfg=libressl350
    cargo:rustc-cfg=libressl360
    cargo:rustc-cfg=libressl370
    cargo:rustc-cfg=libressl380
    cargo:rustc-cfg=libressl381
    cargo:rustc-cfg=libressl382
    cargo:rustc-cfg=libressl390
    cargo:rustc-cfg=libressl400
    cargo:libressl_version_number=4000000f
  
    --- stderr
    thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssl-sys-0.9.103/build/main.rs:420:5:
  
  
    This crate is only compatible with OpenSSL (version 1.0.1 through 1.1.1, or 3), or LibreSSL 2.5
    through 3.9.x, but a different version of OpenSSL was found. The build is now aborting
    due to this version mismatch.
  
  
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  warning: build failed, waiting for other jobs to finish...
  💥 maturin failed
    Caused by: Failed to build a native library through cargo
    Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.11-64bit" PYO3_PYTHON="/tmp/tmpm4f6kesa/.venv/bin/python" PYTHON_SYS_EXECUTABLE="/tmp/tmpm4f6kesa/.venv/bin/python" "cargo" "rustc" "--features" "pyo3/abi3-py37" "--message-format" "json-render-diagnostics" "--locked" "--manifest-path" "/tmp/tmpzw46sfn4/cryptography-43.0.1/src/rust/Cargo.toml" "--release" "--lib"`
  Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/tmp/tmpm4f6kesa/.venv/bin/python', '--compatibility', 'off'] returned non-zero exit status 1
  

  at ~/.local/lib/python3.11/site-packages/poetry/installation/chef.py:164 in _prepare
      160│ 
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│ 
      163│             if error is not None:
    → 164│                 raise error from None
      165│ 
      166│             return path
      167│ 
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with cryptography (43.0.1) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "cryptography (==43.0.1)"'.

root@host 1 /tmp/test_crypto#

Install LibreSSL 3.9.2

cd /root
ftp https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.9.2.tar.gz
tar xzf libressl-3.9.2.tar.gz
cd libressl-3.9.2
./configure
make
make install DESTDIR=/tmp/

root@host /libressl-3.9.2# ./configure
checking build system type... x86_64-unknown-openbsd7.6
checking host system type... x86_64-unknown-openbsd7.6
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/local/bin/gmkdir -p
checking for gawk... no
checking for mawk... no
checking for nawk... no

...

config.status: creating libssl.pc
config.status: creating libtls.pc
config.status: creating openssl.pc
config.status: executing depfiles commands
config.status: executing libtool commands
root@host /libressl-3.9.2#

root@host /libressl-3.9.2# make        
Making all in include
Making all in openssl
echo "generating opensslconf.h ..."
generating opensslconf.h ...
cp ../../include/arch/amd64/opensslconf.h opensslconf.h
make  all-am
Making all in crypto
echo "generating crypto_portable.sym ..."
generating crypto_portable.sym ...
cp ../crypto/crypto.sym crypto_portable.sym
chmod u+w crypto_portable.sym
make  all-am
  CPPAS    aes/libcrypto_la-aes-elf-x86_64.lo
  CPPAS    aes/libcrypto_la-bsaes-elf-x86_64.lo
  CPPAS    aes/libcrypto_la-vpaes-elf-x86_64.lo

...

  CC       netcat.o
  CC       socks.o
  CC       compat/socket.o
  CC       compat/base64.o
  CCLD     nc
ld: warning: netcat.c:470(netcat.o:(main)): warning: mktemp() possibly used unsafely; consider using mkstemp()
Making all in man
Making all in tests
  CC       empty.lo
  CCLD     libtest.la
root@host /libressl-3.9.2#


root@host /libressl-3.9.2# make install DESTDIR=/tmp/
Making install in include
Making install in openssl
make  install-am
 /usr/local/bin/gmkdir -p '/tmp//usr/local/include/openssl'
 /usr/bin/install -c -m 644 opensslconf.h aes.h asn1.h asn1t.h bio.h blowfish.h bn.h buffer.h camellia.h cast.h chacha.h cmac.h cms.h comp.h conf.h conf_api.h crypto.h ct.h curve25519.h des.h dh.h dsa.h dtls1.h ec.h ecdh.h ecdsa.h engine.h err.h evp.h hkdf.h hmac.h idea.h kdf.h lhash.h md4.h md5.h modes.h obj_mac.h objects.h ocsp.h '/tmp//usr/local/include/openssl'
 /usr/bin/install -c -m 644 opensslfeatures.h opensslv.h ossl_typ.h pem.h pem2.h pkcs12.h pkcs7.h poly1305.h posix_time.h rand.h rc2.h rc4.h ripemd.h rsa.h safestack.h sha.h sm3.h sm4.h srtp.h ssl.h ssl2.h ssl23.h ssl3.h stack.h tls1.h ts.h txt_db.h ui.h ui_compat.h whrlpool.h x509.h x509_vfy.h x509v3.h '/tmp//usr/local/include/openssl'
 /usr/local/bin/gmkdir -p '/tmp//usr/local/include'
 /usr/bin/install -c -m 644 tls.h '/tmp//usr/local/include'
Making install in crypto
make  install-am
 /usr/local/bin/gmkdir -p '/tmp//usr/local/lib'
 /bin/sh ../libtool   --mode=install /usr/bin/install -c   libcrypto.la '/tmp//usr/local/lib'
libtool: install: /usr/bin/install -c -m 644 .libs/libcrypto.so.53.0 /tmp//usr/local/lib/libcrypto.so.53.0
libtool: install: /usr/bin/install -c -m 644 .libs/libcrypto.lai /tmp//usr/local/lib/libcrypto.la
libtool: install: /usr/bin/install -c .libs/libcrypto.a /tmp//usr/local/lib/libcrypto.a

...

ln -sf "x509_verify.3" "/tmp//usr/local/share/man/man3/x509_verify_ctx_set_purpose.3"
Making install in tests
make  install-exec-hook
 /usr/local/bin/gmkdir -p '/tmp//usr/local/lib/pkgconfig'
 /usr/bin/install -c -m 644 libtls.pc libcrypto.pc libssl.pc openssl.pc '/tmp//usr/local/lib/pkgconfig'
root@host /libressl-3.9.2# 

Build Cryptography with LibreSSL 3.9.2

export OPENSSL_DIR="/tmp/usr/local/"; time poetry add cryptography@43

root@host /tmp/gugus# export OPENSSL_DIR="/tmp/usr/local/"; time poetry add cryptography@43 
Creating virtualenv gugus in /tmp/gugus/.venv

Updating dependencies
Resolving dependencies... (0.3s)

Package operations: 3 installs, 0 updates, 0 removals

  - Installing pycparser (2.22)
  - Installing cffi (1.17.1)
  - Installing cryptography (43.0.0)

Writing lock file
    1m40.77s real     2m31.68s user     0m15.74s system

root@host /tmp/gugus#

have phun!

Benchmark API’s

i like to work with fastapi. “FastAPI is a modern, fast (high-performance), web framework for building APIs with Python based on standard Python type hints.”. i also know that scripting languages like python are ways slower than compiled languages like c, c++, rust, …

why not build a little “hello world” api, running it on localhost, and then do a benchmark …

Rust

let’s start with rust.

Code

src/main.rs

Sample PyProject from: https://github.com/volfpeter/motorhead/tree/main

PyProject

poetry init
poetry add motor pydantic
poetry add mkdocs-material mkdocstrings[python] mypy ruff poethepoet pytest pytest-asyncio pytest-random-order --group dev

[project]
name = "motorhead"
description = "Async MongoDB with vanilla Pydantic v2+ - made easy."
readme = "README.md"
license = { text = "MIT" }
authors = [
    { name = "Peter Volf", email = "do.volfp@gmail.com" },
]
requires-python = ">=3.10"
dependencies = ["pydantic", "motor"]
classifiers = [
    "Intended Audience :: Information Technology",
    "Operating System :: OS Independent",
    "Programming Language :: Python :: 3",
    "Development Status :: 4 - Beta",
    "Topic :: Internet",
    "Topic :: Software Development :: Libraries",
    "Topic :: Software Development",
    "Typing :: Typed",
    "Environment :: Web Environment",
    "Framework :: FastAPI",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Topic :: Internet :: WWW/HTTP",
]

[project.urls]
homepage = "https://github.com/volfpeter/motorhead"
documentation = "https://volfpeter.github.io/motorhead"
tracker = "https://github.com/volfpeter/motorhead/issues"

[tool.poetry]
name = "motorhead"
version = "0.2403.0"
description = "Async MongoDB with vanilla Pydantic v2+ - made easy."
authors = ["Peter Volf <do.volfp@gmail.com>"]
readme = "README.md"
packages = [{include = "motorhead"}]

[tool.poetry.dependencies]
python = "^3.10"
motor = "^3.1.0"
pydantic = "^2.1.0"

[tool.poetry.group.dev.dependencies]
mkdocs-material = "^9.5.19"
mkdocstrings = {extras = ["python"], version = "^0.25.0"}
mypy = "^1.10.0"
ruff = "^0.4.2"
poethepoet = "^0.26.0"
pytest = "^8.2.0"
pytest-asyncio = "^0.23.6"
pytest-docker = "^3.1.1"
pytest-random-order = "^1.1.1"

[tool.mypy]
strict = true
show_error_codes = true
exclude = ["tree_app"]

[[tool.mypy.overrides]]
module = ["motor.*"]
ignore_missing_imports = true

[tool.ruff]
line-length = 108
lint.exclude = [
    ".git",
    ".mypy_cache",
    ".pytest_cache",
    ".ruff_cache",
    ".venv",
    "dist",
    "docs",
]
lint.select = [
    "E",  # pycodestyle errors
    "W",  # pycodestyle warnings
    "F",  # pyflakes
    "I",  # isort
    "S",  # flake8-bandit - we must ignore these rules in tests
    "C",  # flake8-comprehensions
    "B",  # flake8-bugbear
]

[tool.ruff.lint.per-file-ignores]
"tests/**/*" = ["S101"]  # S101: use of assert detected

[tool.pytest.ini_options]
addopts = "--random-order"

[tool.poe.tasks]
serve-docs = "mkdocs serve"
check-format = "ruff format --check ."
lint = "ruff check ."
mypy = "mypy ."
format = "ruff format ."
lint-fix = "ruff . --fix"
test = "python -m pytest tests --random-order"

static-checks.sequence = ["lint", "check-format", "mypy"]
static-checks.ignore_fail = "return_non_zero"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

sha256: 889b2e95d218175c3503451a55e2c17af3760750a9b2d6e52559b7f336bee043

Setup Project

Poetry

poetry init
poetry add twisted
poetry add twisted-mtr

Build Python Script

cat << 'EOF' > main.py
#!/usr/bin/env python3

'''
    An example usage for twisted-mtr which initiates multiple async traceroutes
    to multiple IPv4 and IPv6 target IP addresses at the same time. You will
    need to set your source IP addresses correctly and have a working dual
    IPv4/IPv6 networking stack to run this example.

'''

import sys
import signal
import logging
import ipaddress
from twisted.internet import reactor
from twisted_mtr import logger, errors, mtr, utils


log = logger.get_logger('trace', level=logging.DEBUG)


if __name__ == '__main__':

    log.info(f'Starting up...')

    # Find mtr-packet in your path
    mtr_binary_name = 'mtr-packet'
    mtr_binary_path = utils.find_binary(mtr_binary_name)

    # Replace with your local IPv4 address
    # Note that if you set this to an IP address not available on your system
    # your traceroutes will simply all time out
    local_ipv4 = ipaddress.IPv4Address('10.1.2.3')

    # Replace with your local IPv6 address
    # Note that if you set this to an IP address not available on your system
    # your traceroutes will simply all time out
    local_ipv6 = ipaddress.IPv6Address('2404:1:2:3:4:5:6:7')

    # Create the TraceRoute Twisted process object
    app_mtr = mtr.TraceRoute(
        mtr_binary_path=mtr_binary_path,
        local_ipv4=local_ipv4,
        local_ipv6=local_ipv6
    )

    # Bind to the Twisted tractor with the mtr-packet binary
    reactor.spawnProcess(app_mtr, mtr_binary_name, [mtr_binary_path], {})

    # Sets to track the traceroutes that have been dispatched and completed
    requested = set()
    completed = set()

    # Success callback
    def _test_traceroute_callback(timestamp, target_ip, protocol, port, hops):
        log.info(f'Completed traceroute started at {timestamp} to: '
                 f'{target_ip} ({protocol}:{port})')
        completed.add(str(target_ip))
        for (hop_num, hop_ip, microseconds) in hops:
            log.info(f' - {hop_num} {hop_ip} {microseconds}')
        if requested == completed:
            log.info('All traces complete, stopping reactor')
            reactor.stop()

    # Error callback
    def _test_trace_error(counter, joined_request, error, extra):
        log.error(f'Error running traceroute: {error}')
        reactor.stop()

    # Queue up our traceroutes
    target_ip = utils.parse_ip('8.1.1.1')  # No route after a few hops to test
    requested.add(str(target_ip))
    app_mtr.trace(_test_traceroute_callback, _test_trace_error, target_ip)

    target_ip = utils.parse_ip('8.8.8.8')
    requested.add(str(target_ip))
    app_mtr.trace(_test_traceroute_callback, _test_trace_error, target_ip, protocol='udp', port=53)

    target_ip = utils.parse_ip('1.1.1.1')
    requested.add(str(target_ip))
    app_mtr.trace(_test_traceroute_callback, _test_trace_error, target_ip, protocol='tcp', port=53, ttl=3)

    target_ip = utils.parse_ip('2404:6800:4015:802::200e')
    requested.add(str(target_ip))
    app_mtr.trace(_test_traceroute_callback, _test_trace_error, target_ip)

    target_ip = utils.parse_ip('2606:4700::6810:7b60')
    requested.add(str(target_ip))
    app_mtr.trace(_test_traceroute_callback, _test_trace_error, target_ip)

    # Polite hook for control+c to abort the traceroutes before they complete
    def signal_handler(sig, frame):
        sys.stdout.write('\n')  # put the ^C on its own line
        log.info(f'Caught keyboard interrupt, shutting down...')
        reactor.stop()

    signal.signal(signal.SIGINT, signal_handler)

    # Start the Twisted reactor event loop
    log.info(f'Starting event loop...')
    reactor.run()

    # If we reach here the reactor has been stopped, all done
    log.info(f'Goodbye')
EOF

don’t forget to set your local_ipv4 and local_ipv6 address correctly

WebServer with Python Twisted

cat << 'EOF' > main.py
from twisted.web import server, resource
from twisted.internet import reactor, endpoints

class Counter(resource.Resource):
    isLeaf = True
    numberRequests = 0

    def render_GET(self, request):

        client_ip = request.getClientAddress().host

        r=request.uri.decode('utf-8')
        if not r =="/favicon.ico":
          self.numberRequests += 1

        request.setHeader(b"content-type", b"text/plain")
        content = u"I am request #{} from {}\n".format(self.numberRequests, client_ip)
        return content.encode("ascii")

endpoints.serverFromString(reactor, "tcp:8080").listen(server.Site(Counter()))
reactor.run()
EOF

Run

poetry init
poetry add twisted
poetry run python main.py

Browse

Open your Browser: http://ip-of-your-host:8080

History

Long time ago, there were huge discussions about Python 2.7 or Python 3.xx. Fortunately, these times are gone and we’ve all gotten over the hurdle to Python 3. But are you on 3.6, 3.7, 3.8 ? or even 3.11 or 3.12 ? That’s the current Version you should use for your daily Projects ?

Status of Python versions

A good indicator is this Website: https://devguide.python.org/versions/

OpenBSD

It’s also recommended to check what our Operating System is installing by default, or what you can get from their Package Repository.

OpenBSD 7.4

… will be released next week (23. Oct 2023). Why not have a look at the upcomming OS and prepare a VM for Software Development ?

Preparation

grab a fresh VM and Install OpenBSD 7.4

os version

puffy74# sysctl kern.version
kern.version=OpenBSD 7.4 (GENERIC.MP) #1396: Sun Oct  8 09:20:40 MDT 2023
    deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

empty vm

puffy74# pkg_info
quirks-6.159        exceptions to pkg_add rules and cache

add go, rust, python

puffy74# pkg_add go rust python3
quirks-6.159 signed on 2023-10-07T10:09:24Z
go-1.21.1: ok
rust-1.72.1:nghttp2-1.56.0: ok
rust-1.72.1:nghttp3-0.15.0: ok
rust-1.72.1:ngtcp2-0.19.1: ok
rust-1.72.1:curl-8.3.0p0: ok
rust-1.72.1:libssh2-1.11.0: ok
rust-1.72.1: ok
python3-3.10p2:xz-5.4.4: ok
python3-3.10p2:sqlite3-3.42.0: ok
python3-3.10p2:libiconv-1.17: ok
python3-3.10p2:gettext-runtime-0.22.2: ok
python3-3.10p2:libffi-3.4.4: ok
python3-3.10p2:bzip2-1.0.8p0: ok
python3-3.10p2:python-3.10.13: ok
python3-3.10p2: ok

show packages

Let’s play with Packages and Libraries

References

• Python Modules
• Poetry Scripts

Switch to Root Folder

cd /some/path/you/want

Create a new Package

poetry new mypackage

add some libraries

poetry add requests

… add some code …

cat << 'EOF' > mypackage/__init__.py
print("importing", __name__)
EOF

cat << 'EOF' > mypackage/main.py
print("importing", __name__)

def test1():
  print("test1")

def test2(name: str):
  print("hello", name)

def test3(name: str, age:int):
  print(f"Hello {name} at age {age}")

if __name__ == "__main__":
  print("This is a Library or Package. You should import it into your Code and not run it directly ...")
EOF

Build Package

poetry build

List Tree

(mypackage-py3.11) stoege@host:~/git/demo/mypackage> tree
.
├── README.md
├── dist
│   ├── mypackage-0.1.0-py3-none-any.whl
│   └── mypackage-0.1.0.tar.gz
├── mypackage
│   ├── __init__.py
│   └── main.py
├── poetry.lock
├── pyproject.toml
└── tests
    └── __init__.py

4 directories, 8 files

you have a package called “mypackage-0.1.0” created. As ’tar.gz and ‘.whl’ File