54 Commits
Author SHA1 Message Date
trex Bot cc2d75c777 fix: resolve the seven bugs pinned by the test audit (#67)
test / unit-tests (push) Successful in 6s
test / komodo-deploy-script (push) Successful in 16s
ci / docker (push) Successful in 1m13s
2026-11-03 16:47:29 -06:00
trex Bot 0c02cd40b0 test: audit suite for meaningfulness, reach 100% coverage (#66)
test / unit-tests (push) Successful in 7s
test / komodo-deploy-script (push) Successful in 16s
ci / docker (push) Successful in 1m17s
2026-11-03 16:47:04 -06:00
trex Bot d8034278d8 fix: resolve the seven bugs pinned by the test audit
test / unit-tests (push) Successful in 14s
ci / docker (push) Successful in 23s
test / unit-tests (pull_request) Successful in 6s
test / komodo-deploy-script (push) Successful in 23s
test / komodo-deploy-script (pull_request) Successful in 16s
- read [ignored_tags] instead of [ignored_domains], so tag-based ignoring
  actually works
- match ignored tags as whole tags, not substrings ('ipt' no longer matches
  'iptorrents')
- send the login-error notification through the Pushover client instead of
  the non-existent module-level send_message()
- handle an empty torrent list instead of raising IndexError before any
  summary is sent
- tag a new torrent once instead of once per code path
- honour toml_path without DOCKER in AppriseClient
- close replaced log handlers so the file descriptor is released

Also add the missing [telemetry] section to config.toml.example, which
qbit-maid.py requires at startup.

The tests that pinned each bug now assert the fixed behaviour, and the
suite still reports 100% coverage.
2026-11-03 16:10:19 -06:00
trex Bot ab46ea1fb2 ci: fix unit-tests job on the Debian-packaged runner pip
ci / docker (push) Successful in 23s
test / unit-tests (pull_request) Successful in 5s
test / komodo-deploy-script (pull_request) Successful in 16s
test / unit-tests (push) Successful in 6s
test / komodo-deploy-script (push) Successful in 17s
The runner image ships pip 24.0 from the Debian package, so
`pip install --upgrade pip` aborts with "Cannot uninstall pip 24.0,
RECORD file not found" and the job failed after 4s. Drop the upgrade
and install with --break-system-packages.
2026-11-03 15:35:39 -06:00
trex Bot 925f9619be test: audit suite for meaningfulness, reach 100% coverage
test / unit-tests (push) Failing after 4s
test / komodo-deploy-script (push) Successful in 16s
test / unit-tests (pull_request) Failing after 3s
test / komodo-deploy-script (pull_request) Successful in 16s
ci / docker (push) Successful in 1m37s
The suite had 8 empty `pass` tests, two tests that made live network
calls to a production healthcheck endpoint, and a write_csv test that
only asserted the file existed. CI ran only test_komodo_deploy, so the
main suite never executed at all.

- Split tests by module: test_qlist, test_qprocess, test_qlogging_extra,
  test_apprise, test_qbt; rewrote test_qbitmaid and test_write_csv.
- qbt_fakes.py: shared fakes (FakeQbt, FakeQbtClient, FakeAPIError,
  StubPushover, FailingPushover, torrent factory).
- test_qbt.py loads the hyphenated qbit-maid.py via importlib and covers
  the __main__ entry point.
- Replaced the live-network tests with a RecordingRequests stub
  (suite time 14s -> 0.035s).
- .coveragerc + CI unit-tests job running the full suite with
  --fail-under=100.

127 unit tests, 100% statement coverage on all 5 first-party modules
(440 stmts), 15/15 mutations caught.

Seven pre-existing bugs are pinned by tests with KNOWN BUG docstrings
and left unfixed pending a decision.
2026-11-03 15:34:57 -06:00
trex Bot 5b54d9e558 Merge pull request 'feat(notify): value-first emoji rows' (#65) from dev-notify-layout into main
test / komodo-deploy-script (push) Successful in 17s
ci / docker (push) Successful in 1m54s
2026-11-03 15:19:40 -06:00
trex Bot dfce115fd8 feat(notify): value-first emoji rows
ci / docker (push) Successful in 24s
test / komodo-deploy-script (push) Successful in 26s
test / komodo-deploy-script (pull_request) Successful in 16s
Switch the summary to the layout jblu picked: each counter renders as
"emoji value label", grouped two per line and joined by a middle dot.

    Scanned 42 torrents in 12s

    🗑️ 7 removed  ·  👻 4 orphaned
    ⏭️ 5 ignored  ·  🐣 3 premature

    🛡️ 10 protected  ·  🌐 20 public

- the value leads, so the numbers are the first thing the eye lands on
- html style now bolds the values instead of the labels
- "Non-protected" reads as "public", matching the non_protected_tag
- use_emoji now defaults to true; false gives the plain variant
- dry runs read "7 would be removed"
2026-11-03 15:10:03 -06:00
trex Bot 5d9da0d22b fix(notify): group counters instead of padding columns
ci / docker (push) Successful in 36s
test / komodo-deploy-script (push) Successful in 16s
test / komodo-deploy-script (pull_request) Successful in 17s
Pushover renders the notification body in a proportional font, so the
space-padded label/value columns never lined up on a phone — the body
collapsed into a ragged flat list with no visual grouping. The html
variant was worse: the padding sat inside the HTML, so it collapsed to a
single space.

Counters are now grouped two per line and joined with a middle dot, which
survives any font. The body drops from 8 lines to 5 and no longer restates
the outcome the title already carries.

- replace _notify_row/LABEL_WIDTH/VALUE_WIDTH with _stat_row
- add docs/notify-preview.py, which renders the real build_summary()
  output as a phone-width PNG so layout changes can be reviewed visually
- tests: assert no space padding, two counters per row, and that the body
  does not repeat the title
2026-11-03 15:02:57 -06:00
jblu 1caa346a7c Merge pull request 'feat(notify): configurable plain/html summary formatting' (#64) from dev-notification-format into main
ci / docker (push) Successful in 1m16s
test / komodo-deploy-script (push) Successful in 1m31s
Reviewed-on: #64
2026-11-02 01:13:22 -06:00
nanobot 881765cdf5 feat(notify): configurable plain/html summary formatting
test / komodo-deploy-script (push) Successful in 17s
test / komodo-deploy-script (pull_request) Successful in 16s
ci / docker (push) Successful in 2m2s
Rework the Pushover/Apprise summary so it reads as a report instead of a
dump of counters.

- Extract build_summary() as the single source of truth for both notifiers,
  removing the duplicated body string and the stray leading indentation
  caused by backslash line continuations.
- Add a 'format' option: 'plain' (aligned columns) or 'html' (bold labels,
  sets html=1 for Pushover). Apprise always receives the plain body.
- Add 'use_emoji' for emoji-prefixed rows.
- Title now states the outcome ('7 torrents removed') instead of
  '--- qbit-maid summary ---'.
- Say 'Would remove' when delete_torrents is false, so a dry run is never
  mistaken for a real deletion.
- Format runtime as 12s / 1m 04s instead of a timedelta with microseconds.
- Add include_hostname, notify_on_empty, priority, sound, retry and expire.
- Log notification failures instead of letting them abort the run.
- Cover the new formatting with unit tests; document the options in the
  README and config.toml.example.
2026-10-27 10:09:35 -05:00
jblu ebec893a85 Merge pull request 'Improve logging: named logger, rotation, richer format' (#62) from logging-improvement into main
test / komodo-deploy-script (push) Successful in 20s
ci / docker (push) Successful in 42s
Reviewed-on: #62
2026-10-27 00:31:29 -05:00
jblu 147758a1e8 Merge pull request 'ci: auto-deploy qbit-maid to Komodo after image push' (#63) from dev-komodo-deploy into main
ci / docker (push) Failing after 21s
test / komodo-deploy-script (push) Successful in 22s
Reviewed-on: #63
2026-10-27 00:31:12 -05:00
Jonathan Branan b2b8f6c10d fix: deterministic exit code for missing Komodo env vars
test / komodo-deploy-script (push) Successful in 16s
ci / docker (push) Successful in 48s
test / komodo-deploy-script (pull_request) Successful in 16s
':?' exits 1 under bash but 2 under dash (the Ubuntu runner's sh),
so the two missing-variable tests failed in CI while passing locally.
Replace with explicit checks so every failure path exits 1.
2026-10-27 00:16:51 -05:00
Jonathan Branan 003217a2b8 test: cover komodo-deploy.sh against a mock Komodo API
ci / docker (push) Successful in 21s
test / komodo-deploy-script (push) Failing after 21s
15 unittest cases exercising the real script end to end: target
selection, auth headers, request shape, success/failure, HTTP 401/400,
unreachable host, polling to completion, and timeout.

Runs in CI via .gitea/workflows/test.yml on push to main/dev* and on
pull requests. No network or live Komodo instance required.
2026-10-27 00:11:53 -05:00
Jonathan Branan 45908b8f6e ci: fix Komodo deploy step permission denied
ci / docker (push) Successful in 5m11s
core.fileMode is false in this repo, so the exec bit on
komodo-deploy.sh was not recorded and the runner hit
'Permission denied'. Mark the file executable in the index and
invoke it via 'sh' so the step no longer depends on the mode bit.
2026-10-26 23:50:28 -05:00
Jonathan Branan ef002b2fce ci: auto-deploy to Komodo stack after image push
ci / docker (push) Failing after 44s
Adds a shared .gitea/scripts/komodo-deploy.sh that calls the Komodo
/execute/DeployStack API after the image is pushed, then polls
/read/GetUpdate until the deploy completes.

A push webhook cannot do this correctly: it fires before the image
exists in the registry, so Komodo would redeploy the previous :latest.

- docker.yml deploys stack 'qbit-maid' (required)
- docker-dev.yml deploys vars.KOMODO_STACK_DEV, skipped when unset
2026-10-26 23:43:42 -05:00
Jonathan Branan f1489ba8a8 docs: plan for automatic Komodo deploys from Gitea Actions 2026-10-26 23:31:04 -05:00
Jonathan Branan ab30d69c7c Improve logging: named logger, rotation, richer format, remove use_log guards
- Add setup_logging() returning a named 'qbit-maid' logger with
  RotatingFileHandler (configurable size/backups) and a console handler
- Add log_torrent() helper for structured torrent context
- Replace basicConfig-based tor_log() with setup_logging()
- Remove redundant 'if self.use_log:' guards across qbit-maid.py,
  qlist.py, qprocess.py, qlogging.py (level filtering handles it)
- Document new logging options in config.toml.example and README
2026-10-26 22:10:35 -05:00
jblu 36d9692086 Merge pull request 'fixed crontab writing jobs multiple times' (#61) from dev-update-crontab into main
ci / docker (push) Successful in 38s
Reviewed-on: #61
2026-04-10 16:40:28 -05:00
jblu 4416ebd924 fixed crontab writing jobs multiple times
ci / docker (push) Successful in 41s
2026-04-10 16:19:36 -05:00
jblu acaf91a301 update readme
ci / docker (push) Successful in 39s
2026-04-09 12:32:59 -05:00
jblu 2213a0655d Merge pull request 'dev-migrate-to-crond' (#60) from dev-migrate-to-crond into main
ci / docker (push) Successful in 40s
Reviewed-on: #60
2026-04-08 20:52:28 -05:00
jblu 6248b7f721 update readme
ci / docker (push) Successful in 38s
2026-04-08 19:12:43 -05:00
jblu e1668dd42d migrated to crond 2026-04-08 19:10:04 -05:00
jblu d15cc8d95c updated healthchecks to support starting and stopping a job
continuous-integration/drone/push Build is passing
2024-06-03 20:32:57 -05:00
jblu 708d56fc98 Updated documentation
continuous-integration/drone/push Build is passing
2023-10-05 08:24:28 -05:00
jblu 6110dade2d Updated help #56
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2023-10-05 07:37:21 -05:00
jblu da7951384f Merge pull request 'dev-fix-calculations' (#59) from dev-fix-calculations into main
continuous-integration/drone/push Build is passing
Reviewed-on: #59
2023-10-05 07:29:00 -05:00
jblu c3865d5fbb Updated test cases
continuous-integration/drone/push Build is passing
2023-10-04 15:03:11 -05:00
jblu 24f6e7b563 updated unit tests and fixed a bug
continuous-integration/drone/push Build is passing
2023-09-20 00:18:26 -05:00
jblu 588382ae14 fixed unit test
continuous-integration/drone/push Build is passing
2023-09-12 14:03:15 -05:00
jblu 01a4c88cc4 Fixed #58
continuous-integration/drone/push Build is failing
2023-09-12 13:04:38 -05:00
jblu 8e70374ae7 test
continuous-integration/drone/push Build is passing
2023-08-09 10:47:27 -05:00
jblu ce5ae0261c changed drone 2023-08-06 23:38:39 -05:00
jblu 4cfb4a88c1 added badge
continuous-integration/drone/push Build is passing
2023-07-23 00:39:39 -05:00
jblu 43c9324a3f updated documentation #56
continuous-integration/drone/push Build is passing
2023-07-23 00:32:15 -05:00
jblu d2ebddf473 Merge pull request 'updated testing' (#54) from dev-update-testing into main
continuous-integration/drone/push Build is passing
Reviewed-on: https://git.jbranan.com/jblu/qbit-maid/pulls/54
2023-06-27 12:46:20 -05:00
jblu a4fc54afe5 updated testing
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is failing
2023-06-27 12:43:10 -05:00
Jonathan Branan 728e429979 fixed build process
continuous-integration/drone/push Build is passing
2023-06-23 04:24:14 -05:00
Jonathan Branan b8a31c43de updated drone
continuous-integration/drone/push Build is passing
2023-06-23 04:20:12 -05:00
jblu 167575fe5d Merge pull request 'migrated to supercronic' (#53) from dev-migrate-supercronic into main
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
Reviewed-on: https://git.jbranan.com/jblu/qbit-maid/pulls/53
2023-06-23 04:05:51 -05:00
Jonathan Branan ca1430b302 migrated to supercronic
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-06-23 04:02:05 -05:00
jblu 660a18a70f disable warnings
continuous-integration/drone/push Build is passing
2023-06-11 14:41:50 -05:00
jblu 7a6787888c tag only latest
continuous-integration/drone/push Build is passing
2023-06-06 15:20:20 -05:00
jblu 65598d97ac enabled drone
continuous-integration/drone/push Build is passing
2023-06-06 15:17:31 -05:00
jblu e2cdca60c3 Merge pull request 'intergrate-drone' (#51) from intergrate-drone into main
continuous-integration/drone/push Build is passing
Reviewed-on: https://git.jbranan.com/jblu/qbit-maid/pulls/51
2023-06-06 15:06:12 -05:00
jblu 978e9326cc updated to execute on main branch
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-06-06 14:47:03 -05:00
jblu beee95caa4 updated to use secret
continuous-integration/drone/push Build is passing
2023-06-06 14:44:56 -05:00
jblu 6ce2add157 added drone support
continuous-integration/drone/push Build is passing
2023-06-06 14:33:07 -05:00
jblu dc533a33b1 updated docker ignore #50 2023-06-06 14:18:11 -05:00
jblu 5e1c963b72 Merge pull request 'added health check support' (#47) from add-healthcheck-support into main
Reviewed-on: https://git.jbranan.com/jblu/qbit-maid/pulls/47
2023-06-05 23:55:01 -05:00
jblu cf64321e49 added health check support 2023-06-05 23:51:49 -05:00
jblu a0ce03335d Merge pull request 'fixed config init' (#46) from fix-config-init into main
Reviewed-on: https://git.jbranan.com/jblu/qbit-maid/pulls/46
2023-06-05 22:43:06 -05:00
jblu b6346ac335 fixed config init 2023-06-05 22:42:00 -05:00
29 changed files with 3253 additions and 282 deletions

No files matched your search

+18
View File
@@ -0,0 +1,18 @@
[run]
source = .
branch = False
omit =
pushover.py
test_*.py
qbt_fakes.py
*/venv/*
/tmp/*
[report]
# Entry-point guards are not reachable from a test process; they are covered
# by the container smoke test instead.
exclude_lines =
pragma: no cover
if __name__ == .__main__.:
show_missing = True
skip_covered = False
+6 -1
View File
@@ -3,7 +3,6 @@ LICENSE
*.log *.log
README.md README.md
requirements.txt requirements.txt
test_*
Dockerfile Dockerfile
*docker-test* *docker-test*
*.log *.log
@@ -11,6 +10,12 @@ Dockerfile
*.csv *.csv
*.toml *.toml
*.git* *.git*
.dockerignore
.DS_Store
.vscode/*
thunder-tests/*
.drone.yml
# Byte-compiled / optimized / DLL files # Byte-compiled / optimized / DLL files
__pycache__/ __pycache__/
*.py[cod] *.py[cod]
+173
View File
@@ -0,0 +1,173 @@
#!/bin/sh
#
# Trigger a Komodo deploy and optionally wait for it to finish.
#
# Required env:
# KOMODO_URL Base URL of Komodo Core, e.g. https://k.jonb.io
# KOMODO_API_KEY Komodo API key
# KOMODO_API_SECRET Komodo API secret
# KOMODO_TARGET Stack name/id (or Procedure name/id)
#
# Optional env:
# KOMODO_TARGET_TYPE "stack" (default) or "procedure"
# KOMODO_REQUIRED "true" to fail when KOMODO_TARGET is empty
# KOMODO_WAIT "true" (default) to poll until the update completes
# KOMODO_WAIT_TIMEOUT Seconds to wait for completion (default 600)
# KOMODO_WAIT_INTERVAL Seconds between polls (default 10)
#
# Exits non-zero if the request is rejected or the deploy fails.
set -eu
KOMODO_TARGET_TYPE="${KOMODO_TARGET_TYPE:-stack}"
KOMODO_REQUIRED="${KOMODO_REQUIRED:-false}"
KOMODO_WAIT="${KOMODO_WAIT:-true}"
KOMODO_WAIT_TIMEOUT="${KOMODO_WAIT_TIMEOUT:-600}"
KOMODO_WAIT_INTERVAL="${KOMODO_WAIT_INTERVAL:-10}"
if [ -z "${KOMODO_TARGET:-}" ]; then
if [ "$KOMODO_REQUIRED" = "true" ]; then
echo "::error::KOMODO_TARGET is empty but KOMODO_REQUIRED=true" >&2
exit 1
fi
echo "KOMODO_TARGET is not set - skipping Komodo deploy."
exit 0
fi
# Explicit checks rather than ':?', whose exit status differs between
# shells (bash exits 1, dash exits 2). Keep every failure path at exit 1.
if [ -z "${KOMODO_URL:-}" ]; then
echo "::error::KOMODO_URL is required" >&2
exit 1
fi
if [ -z "${KOMODO_API_KEY:-}" ]; then
echo "::error::KOMODO_API_KEY is required" >&2
exit 1
fi
if [ -z "${KOMODO_API_SECRET:-}" ]; then
echo "::error::KOMODO_API_SECRET is required" >&2
exit 1
fi
# Normalise: strip any trailing slash so path joins are predictable.
KOMODO_URL="${KOMODO_URL%/}"
case "$KOMODO_TARGET_TYPE" in
stack)
EXECUTE="DeployStack"
PARAMS="{\"stack\": \"${KOMODO_TARGET}\"}"
;;
procedure)
EXECUTE="RunProcedure"
PARAMS="{\"procedure\": \"${KOMODO_TARGET}\"}"
;;
*)
echo "::error::KOMODO_TARGET_TYPE must be 'stack' or 'procedure', got '${KOMODO_TARGET_TYPE}'" >&2
exit 1
;;
esac
tmp=$(mktemp)
trap 'rm -f "$tmp"' EXIT
# api <path> <json-body> -> writes body to $tmp, echoes HTTP status code
api() {
curl -sS -o "$tmp" -w '%{http_code}' \
-X POST "${KOMODO_URL}$1" \
-H 'Content-Type: application/json' \
-H "x-api-key: ${KOMODO_API_KEY}" \
-H "x-api-secret: ${KOMODO_API_SECRET}" \
-d "$2"
}
echo "Requesting ${EXECUTE} for '${KOMODO_TARGET}' on ${KOMODO_URL}"
if ! code=$(api "/execute/${EXECUTE}" "$PARAMS"); then
echo "::error::Could not reach Komodo at ${KOMODO_URL}" >&2
exit 1
fi
body=$(cat "$tmp")
if [ "$code" != "200" ]; then
echo "::error::Komodo rejected ${EXECUTE} for '${KOMODO_TARGET}' (HTTP ${code})" >&2
echo "$body" >&2
exit 1
fi
echo "Komodo accepted the request (HTTP 200)."
# The execute endpoint returns immediately with an Update; the deploy itself
# runs asynchronously. Poll /read/GetUpdate until it reaches a terminal state.
if [ "$KOMODO_WAIT" != "true" ]; then
echo "KOMODO_WAIT is not 'true' - not waiting for completion."
exit 0
fi
if ! command -v python3 >/dev/null 2>&1; then
echo "python3 not available - cannot poll for completion. Deploy was queued."
exit 0
fi
update_id=$(printf '%s' "$body" | python3 -c '
import json, sys
d = json.load(sys.stdin)
print(d.get("_id", {}).get("$oid", ""))
' 2>/dev/null || true)
if [ -z "$update_id" ]; then
echo "Could not read update id from response - not polling. Deploy was queued."
exit 0
fi
echo "Waiting for update ${update_id} (timeout ${KOMODO_WAIT_TIMEOUT}s)..."
elapsed=0
while [ "$elapsed" -lt "$KOMODO_WAIT_TIMEOUT" ]; do
sleep "$KOMODO_WAIT_INTERVAL"
elapsed=$((elapsed + KOMODO_WAIT_INTERVAL))
if ! code=$(api "/read/GetUpdate" "{\"id\": \"${update_id}\"}"); then
echo "::warning::Could not reach Komodo while polling; retrying." >&2
continue
fi
if [ "$code" != "200" ]; then
echo "::warning::GetUpdate returned HTTP ${code}; retrying." >&2
continue
fi
status=$(python3 -c '
import json, sys
print(json.load(sys.stdin).get("status", ""))
' < "$tmp" 2>/dev/null || true)
case "$status" in
Complete)
if python3 -c '
import json, sys
sys.exit(0 if json.load(sys.stdin).get("success") else 1)
' < "$tmp" 2>/dev/null; then
echo "Deploy of '${KOMODO_TARGET}' completed successfully."
exit 0
fi
echo "::error::Deploy of '${KOMODO_TARGET}' failed. Komodo update log:" >&2
python3 -c '
import json, sys
d = json.load(sys.stdin)
for log in d.get("logs", []):
print(f" [{log.get(\"stage\", \"\")}] {log.get(\"msg\", \"\")}")
' < "$tmp" >&2 2>/dev/null || cat "$tmp" >&2
exit 1
;;
Queued|InProgress)
echo " ${elapsed}s: ${status}"
;;
*)
echo " ${elapsed}s: unexpected status '${status}'"
;;
esac
done
echo "::error::Timed out after ${KOMODO_WAIT_TIMEOUT}s waiting for update ${update_id}" >&2
exit 1
+43
View File
@@ -0,0 +1,43 @@
name: ci
on:
push:
branches:
- dev*
jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Login to Docker Hub
uses: docker/login-action@v4
with:
registry: git.jonb.io
username: '${{ gitea.actor }}'
password: '${{ secrets.JONBIO_CI }}'
-
name: Set up QEMU
uses: docker/setup-qemu-action@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
-
name: Build and push
uses: docker/build-push-action@v7
with:
push: true
tags: git.jonb.io/jblu/qbit-maid:dev
-
name: Checkout
uses: actions/checkout@v4
-
name: Deploy to Komodo
env:
KOMODO_URL: '${{ secrets.KOMODO_URL }}'
KOMODO_API_KEY: '${{ secrets.KOMODO_API_KEY }}'
KOMODO_API_SECRET: '${{ secrets.KOMODO_API_SECRET }}'
# Set the KOMODO_STACK_DEV repository variable to your dev stack name.
# When it is empty the deploy step is skipped, so dev builds still pass.
KOMODO_TARGET: '${{ vars.KOMODO_STACK_DEV }}'
run: sh ./.gitea/scripts/komodo-deploy.sh
+42
View File
@@ -0,0 +1,42 @@
name: ci
on:
push:
branches:
- main
jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Login to Docker Hub
uses: docker/login-action@v4
with:
registry: git.jonb.io
username: '${{ gitea.actor }}'
password: '${{ secrets.JONBIO_CI }}'
-
name: Set up QEMU
uses: docker/setup-qemu-action@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
-
name: Build and push
uses: docker/build-push-action@v7
with:
push: true
tags: git.jonb.io/jblu/qbit-maid:latest
-
name: Checkout
uses: actions/checkout@v4
-
name: Deploy to Komodo
env:
KOMODO_URL: '${{ secrets.KOMODO_URL }}'
KOMODO_API_KEY: '${{ secrets.KOMODO_API_KEY }}'
KOMODO_API_SECRET: '${{ secrets.KOMODO_API_SECRET }}'
KOMODO_TARGET: 'qbit-maid'
KOMODO_REQUIRED: 'true'
run: sh ./.gitea/scripts/komodo-deploy.sh
+37
View File
@@ -0,0 +1,37 @@
name: test
on:
push:
branches:
- main
- dev*
pull_request:
jobs:
unit-tests:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install dependencies
# Mirrors the Dockerfile: the container installs qbittorrent-api
# unpinned, so requirements.txt is not used here.
# --break-system-packages: the runner image ships a Debian-packaged
# pip, so a plain install (or `pip install --upgrade pip`) fails
# with "RECORD file not found".
run: |
python3 -m pip install --break-system-packages requests qbittorrent-api coverage
- name: Run the full suite with coverage
run: |
python3 -m coverage run --rcfile=.coveragerc -m unittest \
test_qbitmaid test_write_csv test_qlist test_qprocess \
test_qlogging_extra test_apprise test_qbt
python3 -m coverage report -m --fail-under=100
komodo-deploy-script:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Run komodo-deploy.sh tests
run: python3 -m unittest -v test_komodo_deploy
+2 -5
View File
@@ -11,19 +11,16 @@ def apprise_notify(req_obj, host, port, aurls, title, body):
class AppriseClient: class AppriseClient:
def __init__(self): def __init__(self):
self.config = '' self.config = ''
try: if os.environ.get("DOCKER"):
if os.environ["DOCKER"]:
self.host = os.environ["host"] self.host = os.environ["host"]
self.port = os.environ["port"] self.port = os.environ["port"]
self.aurls = os.environ["aurls"] self.aurls = os.environ["aurls"]
self.title = os.environ["title"] self.title = os.environ["title"]
self.body = os.environ["body"] self.body = os.environ["body"]
if os.environ["toml_path"]: if os.environ.get("toml_path"):
config_file_path=os.environ["toml_path"] config_file_path=os.environ["toml_path"]
with open(config_file_path, 'rb') as c: with open(config_file_path, 'rb') as c:
self.config = load(c) self.config = load(c)
except:
KeyError
if os.path.exists('./config.toml'): if os.path.exists('./config.toml'):
config_file_path = './config.toml' config_file_path = './config.toml'
with open(config_file_path, 'rb') as c: with open(config_file_path, 'rb') as c:
+3 -5
View File
@@ -1,8 +1,6 @@
FROM python:alpine3.18 FROM python:alpine3.23
WORKDIR / WORKDIR /
COPY . opt COPY *.py entrypoint.sh /opt/
RUN pip install requests RUN pip install requests qbittorrent-api
RUN pip install qbittorrent-api
RUN crontab /opt/crontab
RUN chmod +x /opt/entrypoint.sh RUN chmod +x /opt/entrypoint.sh
CMD ["/opt/entrypoint.sh"] CMD ["/opt/entrypoint.sh"]
+168 -76
View File
@@ -1,6 +1,7 @@
# qbit-maid # qbit-maid
> Warning: This application removes torrents that are over the minimum age and that are not part of the ignored categories, domains or tags. Please use the delete_torrents feature set to false when first testing its functionality.
## Warning: This application removes torrents that are over the minimum age and that are not part of the ignored categories, domains or tags. Please use the delete_torrents feature set to false when first testing its functionality. ![Build Status](https://git.jonb.io/jblu/qbit-maid/actions/workflows/docker.yml/badge.svg)
The objective is to remove torrents based on the following criteria: The objective is to remove torrents based on the following criteria:
- tracker domain name - tracker domain name
@@ -8,94 +9,185 @@ The objective is to remove torrents based on the following criteria:
- ratio - ratio
- state - state
```mermaid ## Install
graph TD; ### Docker(Recommended)
qbit-maid.py-->qlogging.py;
qbit-maid.py-->qlist.py; [package](https://git.jonb.io/jblu/-/packages/container/qbit-maid/latest)
qbit-maid.py-->qprocess.py;
qlogging.py-->qbit-maid.py; docker pull git.jonb.io/jblu/qbit-maid:latest
qlist.py-->qbit-maid.py;
qprocess.py-->qbit-maid.py; #### Docker Run Command:
> Please note it is best practice to escape spaces in variables. That is why there is backslashes in the cron schedule.
docker run --name qbit-maid -v /opt/qbit-maid:/config/ -e CRON=0\ 1\ *\ *\ * -e toml_path=/config/config.toml git.jonb.io/jblu/qbit-maid
#### Docker Compose
```
version: '3.3'
services:
qbit-maid:
image: git.jonb.io/jblu/qbit-maid
container_name: qbit-maid
volumes:
- /opt/qbit-maid:/config
environment:
- CRON="0 1 * * *"
- toml_path=/config/config.toml
```
### Via Git
git clone https://git.jonb.io/jblu/qbit-maid.git
Qbit-maid will look for an environment variable *toml_path* for its configuration.If it doesn't find it, it will look for a config.toml file in it's own directory.
##### config.toml
```
[qbittorrent]
host = "192.168.x.x"
port = 8080
username = "user"
password = "pass"
[logging]
use_log = true
log_level = "INFO"
log_path = "./qc.log"
# Optional: log rotation (defaults shown)
log_max_bytes = 5000000
log_backup_count = 3
[app_tags]
protected_tag = "ipt"
non_protected_tag = "public"
[torrent]
age = 2419200
minimum_age = 432000
delete_torrents = false
[pushover]
use_pushover = false
po_key = "<key>"
po_token = "<token>"
# Notification formatting
# format: "plain" (aligned text) or "html" (bold labels, Pushover HTML)
format = "plain"
use_emoji = false
include_hostname = false
notify_on_empty = true
# Optional: -2..2. Priority 2 also needs retry/expire (defaults shown).
# priority = 0
# sound = "pushover"
# retry = 60
# expire = 3600
[apprise]
use_apprise = false
host = "192.168.x.x"
port = 8088
aurls = 'mailto://user:pass@gmail.com'
[dragnet]
enable_dragnet = false
dragnet_outfile = "./orphaned.csv"
[telemetry]
enable_telemetry = false
telemetry_outfile = "./telemetry.csv"
[ignored_categories]
tech = "tech"
books = "books"
general = "general"
[ignored_domains]
iptorrents-empirehost = "ssl.empirehost.me"
iptorrents-stackoverflow = "localhost.stackoverflow.tech"
iptorrents-bgp = "routing.bgp.technology"
[ignored_tags]
save = "save"
[healthcheck]
use_healthcheck = false
healthcheck_url = "https://example.com/ping/<uuid>>"
``` ```
| File | Purpose | ## Logging
| --- | --- |
| qbit-maid.py | Client to the qbit api and calls functions from the other files |
| qlist.py | Builds out torrent lists |
| qlogging.py | Logging and push notification communication |
| qprocess.py | Submits qualifying torrents for deletion |
| test_qbitmaid.py | Unit tests |
| ignored_categories.json | whitelist for categorys to ignore |
| ignored_tags.json | whitelist for torrent tags to ignore |
| ignored_trackers.json | whitelist of fqdn names to ignore |
You will need a config.json in the root directory. qbit-maid uses a named logger (`qbit-maid`) that writes to both the configured
log file and the console (useful for Docker/cron output).
| Key | Value | - `use_log` — enable or disable logging entirely.
| --- | --- | - `log_level` — one of `DEBUG`, `INFO`, `WARN`, `ERROR`, `CRITICAL`.
| host | string, ip or hostname of qbittorrent server | - `log_path` — path to the log file.
| port | number, port of admin gui(used for api aswell) | - `log_max_bytes` — rotate the log file once it reaches this size (default 5 MB).
| username | admin account for qbittorrent | - `log_backup_count` — number of rotated backups to keep (default 3).
| password | password for admin account |
| log_level | is what log messages are written to the log file. INFO or DEBUG are valid entries(case sensitive) | Log lines include the timestamp, level, logger name, function, and line number,
| protected_tag | used to mark torrents to handle with care | for example:
| non_protected_tag | we don't care about these torrents |
| log_path | will write a log in root directory if left as is other wise specify other path using forward slashes |
| age | number, seconds for how long we keep torrents from IPTORRENTS |
| minimum_age | age in seconds torrents should reached before they are removed |
| use_pushover | true or false to enable or disable pushover notification summary |
| use_log | true or false to enable or disable writing to alog file |
| po_key | pushover key |
| po_token | pushover api token |
| delete_torrents | true or false to enable or disable deletion. Useful for dry-runs |
| enable_dragnet | true or false to enable dragnet functionality. Useful for debugging |
It should look something like this:
Config.json
``` ```
{ 10/26/2026 10:10:00 PM [INFO] qbit-maid:tor_processor:24 - Submitted ["My Torrent..."] for deletion.
"host": "192.168.1.1",
"port": 8080,
"username": "admin",
"password": "admin",
"loglevel": "INFO",
"logpath": "./qc.log",
"protected_tag": "ipt",
"non_protected_tag": "public",
"age": 2419200,
"minimum_age": 432000,
"use_pushover": false,
"use_log": true,
"po_key": "",
"po_token": "",
"delete_torrents": false
"enable_dragnet": false,
"dragnet_outfile": "./orphaned.csv"
}
``` ```
You will need a ignored_categories.json in the root directory. This will ignore any of the categories found in the values of the entries. ## Notifications
Pushover and Apprise share one summary builder, so both report the same
numbers. The title states the outcome, the body gives the scanned total, the
actionable counters, then the tracker split.
The default summary looks like this:
``` ```
{ qbit-maid: 7 torrents removed
"example": "general",
"example2": "sonarr" Scanned 42 torrents in 12s
}
🗑️ 7 removed · 👻 4 orphaned
⏭️ 5 ignored · 🐣 3 premature
🛡️ 10 protected · 🌐 20 public
``` ```
You will need a ignored_domains.json in the root directory. This will ignore any torrents from these trackers. Each counter is `emoji value label`, grouped two per line and joined with a
middle dot. The value leads so the numbers are the first thing the eye lands
on, and the separator survives any font — Pushover renders the body in a
proportional font, so space-padded columns collapse into a ragged list on a
phone.
`format = "html"` bolds the values using Pushover's HTML subset (`html=1` is
set automatically). Apprise has no HTML support, so it always receives the
plain body.
`use_emoji` defaults to `true`. Set it to `false` for a plain-text variant:
``` ```
{ Scanned 42 torrents in 12s
"iptorrents-empirehost": "ssl.empirehost.me",
"iptorrents-stackoverflow": "localhost.stackoverflow.tech", 7 removed · 4 orphaned
"iptorrents-bgp": "routing.bgp.technology" 5 ignored · 3 premature
}
10 protected · 20 public
``` ```
You will need a ignored_tags.json in the root directory. This will ignore any torrents with these tags. To preview the result without sending a push, render the real
`build_summary()` output as a phone-width mockup:
``` ```
{ python3 docs/notify-preview.py /tmp/notify.png
"first":"first",
"second":"second",
"third":"third"
}
``` ```
Pass `--mock` to render the design sketches instead of the live output.
Other options:
- `include_hostname` — add a `Host:` line, useful when several instances notify
the same device.
- `notify_on_empty` — set to `false` to skip the push when nothing was removed
or orphaned.
- `priority` / `sound` — passed through to Pushover. Priority `2` automatically
adds `retry` and `expire`.
When `delete_torrents = false` the summary says `Would remove` instead of
`Removed`, so a dry run is never mistaken for a real one.
+38 -14
View File
@@ -1,37 +1,59 @@
[qbittorrent] [qbittorrent]
host = "192.168.4.11" host = "192.168.x.x"
port = 8085 port = 8080
username = "jman" username = "user"
password = "nO^touchy@" password = "pass"
[logging] [logging]
use_log = true use_log = true
log_level = "DEBUG" log_level = "INFO"
log_path = "./qc.log" log_path = "./qc.log"
# Optional: log rotation (defaults shown)
log_max_bytes = 5000000
log_backup_count = 3
[app_tags] [app_tags]
protected_tag = "ipt" protected_tag = "ipt"
non_protected_tag = "public" non_protected_tag = "public"
[torrent] [torrent]
age = 2419200 max_age = 2419200
minimum_age = 432000 min_age = 432000
delete_torrents = true delete_torrents = false
[pushover] [pushover]
use_pushover = false use_pushover = false
po_key = "" po_key = "<key>"
po_token = "" po_token = "<token>"
# Notification formatting
# format: "plain" (grouped text) or "html" (bold values, Pushover HTML)
# Counters render as "emoji value label", grouped two per line and joined
# with a middle dot, because Pushover uses a proportional font and padded
# columns do not align.
format = "plain"
use_emoji = true
include_hostname = false
notify_on_empty = true
# Optional: -2..2. Priority 2 also needs retry/expire (defaults shown).
# priority = 0
# sound = "pushover"
# retry = 60
# expire = 3600
[apprise] [apprise]
use_apprise = false
host = "192.168.x.x" host = "192.168.x.x"
port = 8088 port = 8088
aurls = 'mailto://user:pass@gmail.com' aurls = 'mailto://user:pass@gmail.com'
[dragnet] [dragnet]
enable_dragnet = true enable_dragnet = false
dragnet_outfile = "./orphaned.csv" dragnet_outfile = "./orphaned.csv"
[telemetry]
enable_telemetry = false
telemetry_outfile = "./telemetry.csv"
[ignored_categories] [ignored_categories]
tech = "tech" tech = "tech"
books = "books" books = "books"
@@ -43,6 +65,8 @@ iptorrents-stackoverflow = "localhost.stackoverflow.tech"
iptorrents-bgp = "routing.bgp.technology" iptorrents-bgp = "routing.bgp.technology"
[ignored_tags] [ignored_tags]
first = "first" save = "save"
second = "second"
third = "third" [healthcheck]
use_healthcheck = true
healthcheck_url = "https://example.com/ping/<uuid>>"
-1
View File
@@ -1 +0,0 @@
0 1 * * * . /etc/environment; python /opt/qbit-maid.py >> /logfile
+143
View File
@@ -0,0 +1,143 @@
# Auto-deploy qbit-maid to Komodo
Status: **implemented** on branch `dev-komodo-deploy`.
Komodo instance: `https://k.jonb.io` · Stack: `qbit-maid`
## What this does
Every successful image build now triggers a Komodo stack deploy:
| Workflow | Trigger | Image tag | Komodo target |
|---|---|---|---|
| `.gitea/workflows/docker.yml` | push to `main` | `:latest` | stack `qbit-maid` |
| `.gitea/workflows/docker-dev.yml` | push to `dev*` | `:dev` | `vars.KOMODO_STACK_DEV` (skipped if unset) |
The deploy step runs **after** `Build and push`, so the new image is already in
the registry when Komodo pulls it. If the deploy fails, the workflow goes red.
## Why not just use the existing webhook?
You already have webhooks configured, and this is the important part: **a push
webhook on the stack does not reliably deploy new container builds.**
The webhook fires on `git push`. That happens *before* Gitea Actions has built
and pushed the image — the build takes minutes. So Komodo receives the deploy
request while the registry still holds the *previous* `:latest`. It pulls that,
deploys it, and reports success. The new code never lands until something else
triggers a second deploy.
The ordering is the whole problem, and a webhook cannot fix it because the
webhook is tied to the push event, not the build event.
So the workflow calls the Komodo API directly instead:
```
POST https://k.jonb.io/execute/DeployStack
x-api-key: <key>
x-api-secret: <secret>
{"stack": "qbit-maid"}
```
This is the same call the Komodo UI makes when you click **Deploy**, just
triggered from CI at the right moment.
### What to do with the existing webhook
If you have a stack webhook pointing at
`/listener/github/stack/qbit-maid/deploy`, it is now redundant and will cause a
second, useless deploy on every push (deploying the stale image). **Remove it**
in Gitea → repo Settings → Webhooks.
Keep webhooks only if you later point them at a *Procedure* rather than the
stack — see "Using a Procedure" below.
## One Komodo setting to verify
`DeployStack` runs `docker compose pull` before `up` **only when the stack has
`auto_pull` enabled**. It defaults to `true`, so this is probably already fine —
but if it has been turned off, deploys will silently reuse the cached image and
you will see no change.
Check in the Komodo UI: stack `qbit-maid` → Config → **Auto Pull** should be on.
## Setup (one-time)
### 1. Komodo API key
Komodo → **Settings → API Keys → New**. The key's user needs **Execute**
permission on the `qbit-maid` stack. Copy the key and secret — the secret is
shown once.
### 2. Gitea secrets
Repo → **Settings → Actions → Secrets**:
| Name | Value |
|---|---|
| `KOMODO_URL` | `https://k.jonb.io` |
| `KOMODO_API_KEY` | from step 1 |
| `KOMODO_API_SECRET` | from step 1 |
### 3. Dev stack (optional)
Repo → **Settings → Actions → Variables**:
| Name | Value |
|---|---|
| `KOMODO_STACK_DEV` | your dev stack name, e.g. `qbit-maid-dev` |
If this variable is unset, the dev workflow logs `KOMODO_TARGET is not set -
skipping Komodo deploy` and still passes. Dev builds never fail because of a
missing dev stack.
## The deploy script
`.gitea/scripts/komodo-deploy.sh` is shared by both workflows. It:
1. Skips cleanly if `KOMODO_TARGET` is empty (unless `KOMODO_REQUIRED=true`).
2. POSTs to `/execute/DeployStack` (or `/execute/RunProcedure`).
3. Fails on any non-200, printing Komodo's error body.
4. Polls `/read/GetUpdate` until the update reaches `Complete`, then checks
`success` and prints the Komodo log on failure.
The execute endpoint returns immediately with an `Update` object — a `200` means
"queued", not "deployed". That is why step 4 exists.
Tunable via env: `KOMODO_TARGET_TYPE` (`stack`/`procedure`), `KOMODO_WAIT`
(default `true`), `KOMODO_WAIT_TIMEOUT` (default 600s), `KOMODO_WAIT_INTERVAL`
(default 10s).
### Using a Procedure instead of a direct stack deploy
If you later wrap the deploy in a Komodo Procedure (deploy + healthcheck +
notification, say), point the workflow at it instead of editing the script:
```yaml
env:
KOMODO_TARGET_TYPE: 'procedure'
KOMODO_TARGET: 'qbit-maid deploy'
```
`RunProcedure` takes `{"procedure": "<name>"}` and returns the same `Update`
shape, so the polling logic is unchanged.
## Testing
Verified locally against a mock of the Komodo API — 12 checks covering the
success path, deploy failure, bad credentials (401), unknown stack (400),
empty target skip, required-but-empty target, unreachable host, and the
procedure target type. All pass.
Live verification still needed:
1. Push `dev-komodo-deploy` — it matches `dev*`, so `docker-dev.yml` runs. With
`KOMODO_STACK_DEV` unset it will skip the deploy; set the variable first if
you want to exercise it.
2. Merge to `main` and confirm the Komodo UI shows a new `DeployStack` update
for `qbit-maid`, operator = the API key's user.
3. Confirm the running container picked up the new image digest.
## Rollback
Delete the `Deploy to Komodo` step from the workflow(s). Nothing on the Komodo
side is modified by CI beyond triggering a deploy.
Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

+214
View File
@@ -0,0 +1,214 @@
#!/usr/bin/env python3
"""Render notification layouts as a phone-width PNG mockup.
Pushover renders the body in a proportional font, so space-padded columns do
not line up on a phone. These mockups use a proportional font on purpose so
the preview matches what the user actually sees.
By default the cards are rendered from the real ``build_summary()`` output, so
the preview is ground truth rather than a hand-written approximation. Pass
``--mock`` to render the built-in candidate sketches instead.
Usage: python3 docs/notify-preview.py [--mock] [out.png]
"""
import datetime
import html
import os
import subprocess
import sys
import tempfile
from collections import Counter
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
STATS = {
"total": 42,
"removed": 7,
"orphaned": 4,
"ignored": 5,
"premature": 3,
"protected": 10,
"non_protected": 20,
"runtime": "12s",
}
EMOJI = {
"removed": "\U0001F5D1\uFE0F",
"orphaned": "\U0001F47B",
"ignored": "\u23ED\uFE0F",
"premature": "\U0001F423",
"protected": "\U0001F6E1\uFE0F",
"non_protected": "\U0001F310",
}
def b(text):
return f"<b>{text}</b>"
# --- candidate layouts -----------------------------------------------------
# Each returns (title, body_html). Body uses Pushover's HTML subset.
def current_table(s):
"""Today's layout: space-padded columns."""
rows = [
("Removed", s["removed"]),
("Orphaned", s["orphaned"]),
("Ignored", s["ignored"]),
("Premature", s["premature"]),
]
tail = [("Protected", s["protected"]), ("Non-protected", s["non_protected"])]
out = [f"Scanned {s['total']} torrents in {s['runtime']}", ""]
out += [f"{b(lbl)}{' ' * max(14 - len(lbl), 1)}{val:>4}" for lbl, val in rows]
out.append("")
out += [f"{b(lbl)}{' ' * max(14 - len(lbl), 1)}{val:>4}" for lbl, val in tail]
return "qbit-maid: 7 torrents removed", "\n".join(out)
def compact_inline(s):
"""One line per group, middle-dot separated."""
out = [
f"Scanned {s['total']} torrents in {s['runtime']}",
"",
f"{b('Removed')} {s['removed']} \u00b7 {b('Orphaned')} {s['orphaned']}",
f"{b('Ignored')} {s['ignored']} \u00b7 {b('Premature')} {s['premature']}",
"",
f"{b('Protected')} {s['protected']} \u00b7 {b('Non-protected')} {s['non_protected']}",
]
return "qbit-maid: 7 torrents removed", "\n".join(out)
def emoji_rows(s):
"""Emoji-led rows, no padding."""
out = [f"Scanned {s['total']} torrents in {s['runtime']}", ""]
for key, label in (("removed", "Removed"), ("orphaned", "Orphaned"),
("ignored", "Ignored"), ("premature", "Premature")):
out.append(f"{EMOJI[key]} {b(label)} {s[key]}")
out.append("")
for key, label in (("protected", "Protected"), ("non_protected", "Non-protected")):
out.append(f"{EMOJI[key]} {b(label)} {s[key]}")
return "qbit-maid: 7 torrents removed", "\n".join(out)
def hero_line(s):
"""Actionable numbers up front, everything else compact below."""
out = [
f"{b(str(s['removed']))} removed \u00b7 {b(str(s['orphaned']))} orphaned",
f"{s['total']} torrents scanned in {s['runtime']}",
"",
f"Ignored {s['ignored']} \u00b7 Premature {s['premature']}",
f"Protected {s['protected']} \u00b7 Non-protected {s['non_protected']}",
]
return "qbit-maid: 7 torrents removed", "\n".join(out)
def emoji_inline(s):
"""Compact inline, emoji instead of bold labels."""
out = [
f"Scanned {s['total']} torrents in {s['runtime']}",
"",
f"{EMOJI['removed']} {s['removed']} removed \u00b7 {EMOJI['orphaned']} {s['orphaned']} orphaned",
f"{EMOJI['ignored']} {s['ignored']} ignored \u00b7 {EMOJI['premature']} {s['premature']} premature",
"",
f"{EMOJI['protected']} {s['protected']} protected \u00b7 {EMOJI['non_protected']} {s['non_protected']} public",
]
return "qbit-maid: 7 torrents removed", "\n".join(out)
MOCK_CANDIDATES = [
("A \u2014 current (padded table)", current_table),
("B \u2014 compact inline", compact_inline),
("C \u2014 emoji rows", emoji_rows),
("D \u2014 hero line", hero_line),
("E \u2014 emoji inline", emoji_inline),
]
class Sample:
"""Minimal stand-in for the Qbt object, holding only what notifiers read."""
def __init__(self, **overrides):
self.config = {"pushover": {}}
self.total_torrents = STATS["total"]
self.torrent_hash_delete_list = ["a"] * STATS["removed"]
self.up_tor_counter = STATS["orphaned"]
self.ignored_counter = STATS["ignored"]
self.preme_tor_counter = STATS["premature"]
self.tracker_protected_tag = "ipt"
self.tracker_non_protected_tag = "public"
self.c = Counter({"ipt": STATS["protected"],
"public": STATS["non_protected"]})
self.delete_torrents = True
self.st = datetime.datetime(2026, 10, 27, 9, 54, 0)
self.et = datetime.datetime(2026, 10, 27, 9, 54, 12)
for key, value in overrides.items():
setattr(self, key, value)
def real_candidates():
"""Render the actual build_summary() output for each supported style."""
from qlogging import build_summary
out = []
for style in ("plain", "html"):
for emoji in (False, True):
title, body = build_summary(
Sample(), style=style, use_emoji=emoji, include_hostname=False
)
label = f"{style}" + (" + emoji" if emoji else "")
out.append((label, title, body))
return out
def card(name, title, body):
body_html = "<br>".join(body.split("\n"))
return f"""
<div class="card">
<div class="app">PUSHOVER \u00b7 {html.escape(name)}</div>
<div class="title">{html.escape(title)}</div>
<div class="body">{body_html}</div>
</div>
"""
def build_html(mock=False):
if mock:
cards = "\n".join(card(n, *fn(STATS)) for n, fn in MOCK_CANDIDATES)
else:
cards = "\n".join(card(n, t, b) for n, t, b in real_candidates())
return f"""<!doctype html>
<html><head><meta charset="utf-8"><style>
body {{ margin:0; padding:18px; background:#1b1d21;
font-family:-apple-system,"Segoe UI",Roboto,Helvetica,Arial,sans-serif; }}
.grid {{ display:grid; grid-template-columns:1fr 1fr; gap:16px; }}
.card {{ background:#2b2f36; border-radius:14px; padding:14px 16px;
box-shadow:0 2px 10px rgba(0,0,0,.45); }}
.app {{ font-size:10px; letter-spacing:.14em; color:#8b929c; margin-bottom:6px; }}
.title {{ font-size:15px; font-weight:600; color:#f2f4f7; margin-bottom:8px; }}
.body {{ font-size:14px; line-height:1.45; color:#d7dbe0; white-space:normal; }}
.body b {{ color:#ffffff; }}
</style></head><body><div class="grid">{cards}</div></body></html>
"""
def main():
args = [a for a in sys.argv[1:] if not a.startswith("--")]
mock = "--mock" in sys.argv[1:]
out = args[0] if args else "/tmp/notify-preview.png"
workdir = tempfile.mkdtemp(prefix="notify-preview-")
page = os.path.join(workdir, "preview.html")
with open(page, "w", encoding="utf-8") as fh:
fh.write(build_html(mock=mock))
subprocess.run([
"docker", "run", "--rm", "-v", f"{workdir}:/w", "-w", "/w",
"--entrypoint", "google-chrome", "dgtlmoon/sockpuppetbrowser:latest",
"--headless", "--no-sandbox", "--disable-gpu", "--disable-dev-shm-usage",
"--hide-scrollbars", "--window-size=880,760",
"--screenshot=/w/preview.png", "file:///w/preview.html",
], check=True, capture_output=True)
subprocess.run(["cp", os.path.join(workdir, "preview.png"), out], check=True)
print(out)
if __name__ == "__main__":
main()
+5 -2
View File
@@ -1,5 +1,8 @@
#!/bin/sh #!/bin/sh
printenv | grep -v "no_proxy" >> /etc/environment CRON_CONFIG_FILE="/etc/crontabs/root"
crond -f grep -qF 'python /opt/qbit-maid.py' $CRON_CONFIG_FILE || echo "${CRON} python /opt/qbit-maid.py" >> $CRON_CONFIG_FILE
grep -qF '@reboot' $CRON_CONFIG_FILE || echo "@reboot python /opt/qbit-maid.py" >> $CRON_CONFIG_FILE
exec crond -f
+30 -12
View File
@@ -12,14 +12,16 @@ from collections import Counter
import csv import csv
import requests as r import requests as r
import os import os
import sys r.packages.urllib3.disable_warnings()
class Qbt: class Qbt:
def __init__(self): def __init__(self):
"""Main object, should be calling functions from qlist.py, qlogging.py and qprocess.py""" """Main object, should be calling functions from qlist.py, qlogging.py and qprocess.py"""
# Open the config. Needs a json file with the data in config.json.example # Open the config. Needs a json file with the data in config.json.example
self.st = datetime.datetime.now() self.st = datetime.datetime.now()
config_file_path=os.environ["toml_path"] if os.getenv("toml_path"):
config_file_path=os.getenv("toml_path")
with open(config_file_path, 'rb') as c: with open(config_file_path, 'rb') as c:
self.config = load(c) self.config = load(c)
if os.path.exists('./config.toml'): if os.path.exists('./config.toml'):
@@ -36,7 +38,6 @@ class Qbt:
password=self.config["qbittorrent"]["password"], password=self.config["qbittorrent"]["password"],
) )
# Create the logging and pushover objects # Create the logging and pushover objects
self.tl = logging
self.po = pushover self.po = pushover
self.ct = Counter self.ct = Counter
self.cv = csv self.cv = csv
@@ -46,6 +47,12 @@ class Qbt:
self.use_log = self.config["logging"]["use_log"] self.use_log = self.config["logging"]["use_log"]
self.log_path = self.config["logging"]["log_path"] self.log_path = self.config["logging"]["log_path"]
self.log_level = self.config["logging"]["log_level"] self.log_level = self.config["logging"]["log_level"]
self.log_max_bytes = self.config["logging"].get("log_max_bytes", 5_000_000)
self.log_backup_count = self.config["logging"].get("log_backup_count", 3)
self.tl = setup_logging(
self.use_log, self.log_path, self.log_level,
self.log_max_bytes, self.log_backup_count,
)
#app_tags #app_tags
self.tracker_protected_tag = self.config["app_tags"]["protected_tag"] self.tracker_protected_tag = self.config["app_tags"]["protected_tag"]
@@ -53,8 +60,8 @@ class Qbt:
#torrent #torrent
self.delete_torrents = self.config["torrent"]["delete_torrents"] self.delete_torrents = self.config["torrent"]["delete_torrents"]
self.minimum_age = self.config["torrent"]["minimum_age"] self.min_age = self.config["torrent"]["min_age"]
self.age = self.config["torrent"]["age"] self.max_age = self.config["torrent"]["max_age"]
#pushover #pushover
self.use_pushover = self.config["pushover"]["use_pushover"] self.use_pushover = self.config["pushover"]["use_pushover"]
@@ -71,6 +78,10 @@ class Qbt:
self.enable_dragnet = self.config["dragnet"]["enable_dragnet"] self.enable_dragnet = self.config["dragnet"]["enable_dragnet"]
self.dragnet_outfile = self.config["dragnet"]["dragnet_outfile"] self.dragnet_outfile = self.config["dragnet"]["dragnet_outfile"]
#telemetry
self.enable_telemetry = self.config["telemetry"]["enable_telemetry"]
self.telemetry_outfile = self.config["telemetry"]["telemetry_outfile"]
#ignored_categories #ignored_categories
self.cat_whitelist = self.config["ignored_categories"] self.cat_whitelist = self.config["ignored_categories"]
@@ -78,19 +89,26 @@ class Qbt:
self.tracker_whitelist = self.config["ignored_domains"] self.tracker_whitelist = self.config["ignored_domains"]
#ignored_tags #ignored_tags
self.ignored_tags = self.config["ignored_domains"] self.ignored_tags = self.config["ignored_tags"]
#healthcheck
self.use_healthcheck = self.config["healthcheck"]["use_healthcheck"]
self.healthcheck_url = self.config["healthcheck"]["healthcheck_url"]
# Calling log and notify functions # Calling log and notify functions
tor_log(self)
tor_notify(self) tor_notify(self)
self.t = time self.t = time
#start healthcheck job
if self.use_healthcheck:
send_ping(self, r, self.healthcheck_url.rstrip("/") + "/start" )
# Pulling domain names to treat carefully # Pulling domain names to treat carefully
self.tracker_list = [] self.tracker_list = []
self.up_tor_counter = 0 self.up_tor_counter = 0
self.preme_tor_counter = 0 self.preme_tor_counter = 0
self.ignored_counter = 0 self.ignored_counter = 0
self.torrent_hash_delete_list = [] self.torrent_hash_delete_list = []
if self.use_log:
self.tl.debug(self.tracker_whitelist) self.tl.debug(self.tracker_whitelist)
#logging in #logging in
try: try:
@@ -99,19 +117,17 @@ class Qbt:
self.tl.info('Connected.') self.tl.info('Connected.')
except qbittorrentapi.APIError as e: except qbittorrentapi.APIError as e:
self.tl.exception(e) self.tl.exception(e)
self.po.send_message(e, title="qbit-maid API ERROR") tor_notify_error(self, e)
# Pulling all torrent data # Pulling all torrent data
self.torrent_list = self.qbt_client.torrents_info() self.torrent_list = self.qbt_client.torrents_info()
#Main process block #Main process block
if self.use_log:
list_qbit_api_info(self) list_qbit_api_info(self)
list_first_tor(self) list_first_tor(self)
debug_torrent_list(self)
build_tor_list(self) build_tor_list(self)
process_counts(self) process_counts(self)
if self.use_log:
torrent_count(self) torrent_count(self)
tor_processor(self) tor_processor(self)
if self.use_log:
print_processor(self) print_processor(self)
if self.delete_torrents: if self.delete_torrents:
tor_delete(self) tor_delete(self)
@@ -121,6 +137,8 @@ class Qbt:
tor_notify_summary(self) tor_notify_summary(self)
if self.use_apprise: if self.use_apprise:
tor_notify_apprise(self, r, apprise_notify) tor_notify_apprise(self, r, apprise_notify)
if self.use_healthcheck:
send_ping(self, r, self.healthcheck_url)
# Run # Run
if __name__== "__main__": if __name__== "__main__":
Qbt() Qbt()
+158
View File
@@ -0,0 +1,158 @@
"""Shared fakes for the qbit-maid test suite.
This is test scaffolding, not production code, so it is excluded from the
coverage report (see ``.coveragerc``).
"""
import datetime
import logging
from collections import Counter
# Keep test output clean: the pipeline logs warnings for ignored torrents.
_test_logger = logging.getLogger("qbit-maid-test")
_test_logger.addHandler(logging.NullHandler())
_test_logger.propagate = False
# ---------------------------------------------------------------------------
# qbittorrentapi stand-in
# ---------------------------------------------------------------------------
class FakeApp:
version = "v4.6.2"
web_api_version = "2.9.3"
class FakeAPIError(Exception):
"""Stands in for qbittorrentapi.APIError."""
class FakeQbtClient:
"""Records calls instead of talking to a real qBittorrent instance."""
def __init__(self, torrents=None, login_error=None, **kwargs):
self.app = FakeApp()
self._torrents = list(torrents or [])
self._login_error = login_error
self.init_kwargs = kwargs
self.added_tags = []
self.deleted_tags = []
self.deleted = []
self.logged_in = False
def auth_log_in(self):
if self._login_error is not None:
raise self._login_error
self.logged_in = True
def torrents_info(self):
return list(self._torrents)
def torrents_add_tags(self, tags, torrent_hash):
self.added_tags.append((tags, torrent_hash))
def torrents_delete_tags(self, tags):
self.deleted_tags.append(list(tags))
def torrents_delete(self, delete_files, hashes):
self.deleted.append((delete_files, list(hashes)))
def torrent(name="Example.Torrent.Release", tracker="https://tracker.example/announce",
category="general", tags="", seeding_time=999999, state="uploading",
ratio=2.0, time_active=999999, infohash="abc123", added_on=1):
"""Build a torrent dict shaped like a qbittorrentapi response."""
return {
"name": name,
"tracker": tracker,
"category": category,
"tags": tags,
"seeding_time": seeding_time,
"state": state,
"ratio": ratio,
"time_active": time_active,
"infohash_v1": infohash,
"hash": infohash,
"added_on": added_on,
}
# ---------------------------------------------------------------------------
# Pushover stand-ins
# ---------------------------------------------------------------------------
class StubPushover:
"""Records message() calls instead of hitting the Pushover API."""
def __init__(self):
self.calls = []
def message(self, user, body, **kwargs):
self.calls.append({"user": user, "body": body, "kwargs": kwargs})
class FailingPushover:
def message(self, *args, **kwargs):
raise RuntimeError("pushover is down")
# ---------------------------------------------------------------------------
# The Qbt object
# ---------------------------------------------------------------------------
class FakeQbt:
"""Stand-in for the Qbt object.
Defaults match the notifier tests (42 scanned, 7 removed, 4 orphaned).
Pipeline tests override ``torrent_list`` and ``qbt_client``.
"""
def __init__(self, **overrides):
# --- notifier surface ---
self.config = {"pushover": {}}
self.total_torrents = 42
self.torrent_hash_delete_list = ["a"] * 7
self.up_tor_counter = 4
self.ignored_counter = 5
self.preme_tor_counter = 3
self.tracker_protected_tag = "ipt"
self.tracker_non_protected_tag = "public"
self.c = Counter({"ipt": 10, "public": 20})
self.delete_torrents = True
self.st = datetime.datetime(2026, 10, 27, 9, 54, 0)
self.et = datetime.datetime(2026, 10, 27, 9, 54, 12)
self.po_key = "USERKEY"
self.poc = StubPushover()
self.apprise_host = "localhost"
self.apprise_port = 8088
self.apprise_aurls = "mailto://user:pass@example.com"
self.tl = logging.getLogger("qbit-maid-test")
# --- pipeline surface ---
self.torrent_list = []
self.tracker_list = []
self.qbt_client = FakeQbtClient()
self.cat_whitelist = {}
self.tracker_whitelist = {"tracker.example": "tracker.example"}
self.ignored_tags = {}
self.min_age = 432000
self.max_age = 2419200
self.enable_telemetry = False
self.telemetry_outfile = "telemetry.csv"
self.enable_dragnet = False
self.dragnet_outfile = "orphaned.csv"
self.cv = None
self.use_pushover = False
self.use_apprise = False
self.extm = ""
for key, value in overrides.items():
setattr(self, key, value)
def reset_counters(self):
"""Zero the pipeline counters.
The constructor defaults are the notifier fixture (42 scanned, 7
removed, ...), which pipeline tests must not inherit.
"""
self.up_tor_counter = 0
self.preme_tor_counter = 0
self.ignored_counter = 0
self.torrent_hash_delete_list = []
return self
+31 -25
View File
@@ -7,51 +7,42 @@ def build_tor_list(self):
self.total_torrents = len(self.torrent_list) self.total_torrents = len(self.torrent_list)
while self.torrent_list: while self.torrent_list:
torrent = self.torrent_list.pop() torrent = self.torrent_list.pop()
if self.use_log:
self.tl.debug(f'---Analyzing ["{torrent["name"][0:20]}..."] {torrent["infohash_v1"]}---') self.tl.debug(f'---Analyzing ["{torrent["name"][0:20]}..."] {torrent["infohash_v1"]}---')
# Need a way to tag when the tracker is blank # Need a way to tag when the tracker is blank
if is_tracker_blank(torrent['tracker']): if is_tracker_blank(torrent['tracker']):
if self.use_log: self.tl.warning(f'Torrent doesn\'t have a tracker ["{torrent["name"][0:20]}..."] hash: {torrent["hash"]}')
self.tl.warning(f'Torrent doesn\'t have a tracker ["{torrent["name"][0:20]}..."] [{torrent["tracker"]}]hash: {torrent["hash"]}')
self.ignored_counter += 1 self.ignored_counter += 1
continue continue
elif is_cat_ignored(torrent['category'], self.cat_whitelist.values()): elif is_cat_ignored(torrent['category'], self.cat_whitelist.values()):
if self.use_log:
self.tl.info(f'Ignored category: ["{torrent["name"][0:20]}..."] category:[{torrent["category"]}] hash: {torrent["hash"]}') self.tl.info(f'Ignored category: ["{torrent["name"][0:20]}..."] category:[{torrent["category"]}] hash: {torrent["hash"]}')
self.ignored_counter += 1 self.ignored_counter += 1
continue continue
elif is_ignored_tag(self.ignored_tags.values(),torrent['tags']): elif is_ignored_tag(self.ignored_tags.values(),torrent['tags']):
if self.use_log:
self.tl.info(f'Ignored tag: ["{torrent["name"][0:20]}..."] tags: {torrent["tags"]} hash: {torrent["hash"]}') self.tl.info(f'Ignored tag: ["{torrent["name"][0:20]}..."] tags: {torrent["tags"]} hash: {torrent["hash"]}')
self.ignored_counter += 1 self.ignored_counter += 1
continue continue
if is_tag_blank(torrent['tags']): if is_tag_blank(torrent['tags']):
if self.use_log: tag = torrent_tag_for(
torrent['tracker'],
self.tracker_protected_tag,
self.tracker_non_protected_tag,
self.tracker_whitelist.values(),
)
if tag:
self.tl.debug(f'Tagging new torrent: ["{torrent["name"][0:20]}..."] {torrent["tracker"]}hash: {torrent["hash"]}') self.tl.debug(f'Tagging new torrent: ["{torrent["name"][0:20]}..."] {torrent["tracker"]}hash: {torrent["hash"]}')
if is_protected_tracker(torrent['tracker'], self.tracker_whitelist.values()): self.qbt_client.torrents_add_tags(tag, torrent['hash'])
self.qbt_client.torrents_add_tags(self.tracker_protected_tag,torrent['hash']) if is_preme(torrent['seeding_time'], self.min_age):
elif is_not_protected_tracker(torrent['tracker'], self.tracker_whitelist.values()):
self.qbt_client.torrents_add_tags(self.tracker_non_protected_tag,torrent['hash'])
if is_preme(torrent['added_on'], self.minimum_age, self.t.time()):
self.preme_tor_counter += 1 self.preme_tor_counter += 1
self.tl.debug(f'Premature torrent: ["{torrent["name"][0:20]}..."] hash: {torrent["hash"]}') self.tl.debug(f'Premature torrent: ["{torrent["name"][0:20]}..."] Seconds Seeded: [{torrent["seeding_time"]}] hash: {torrent["hash"]}')
continue continue
elif is_protected_tracker(torrent['tracker'], self.tracker_whitelist.values()): elif is_protected_tracker(torrent['tracker'], self.tracker_whitelist.values()):
if is_tag_blank(torrent['tags']):
self.qbt_client.torrents_add_tags(self.tracker_protected_tag,torrent['hash'])
if self.use_log:
self.tl.debug(f'Tagging Protected torrent: ["{torrent["name"][0:20]}..."] {torrent["tracker"]}hash: {torrent["hash"]}')
self.tracker_list.append(torrent) self.tracker_list.append(torrent)
elif is_not_protected_tracker(torrent['tracker'], self.tracker_whitelist.values()): elif is_not_protected_tracker(torrent['tracker'], self.tracker_whitelist.values()):
if is_tag_blank(torrent['tags']):
self.qbt_client.torrents_add_tags(self.tracker_non_protected_tag,torrent['hash'])
if self.use_log:
self.tl.debug(f'Tagging Non-protected torrent: ["{torrent["name"][0:20]}..."] {torrent["tracker"]}hash: {torrent["hash"]}')
self.tracker_list.append(torrent) self.tracker_list.append(torrent)
def is_preme(added, minage, time): def is_preme(seeding_time, minage):
if added + minage >= time: if seeding_time <= minage:
return True return True
def is_cat_ignored(cat, catlist): def is_cat_ignored(cat, catlist):
@@ -74,11 +65,26 @@ def is_not_protected_tracker(tracker, trackerlist):
if tracker.split('/')[2] not in trackerlist: if tracker.split('/')[2] not in trackerlist:
return True return True
def torrent_tag_for(tracker, protected_tag, non_protected_tag, trackerlist):
"""Return the tag a torrent should carry, or None for an unknown tracker."""
if is_protected_tracker(tracker, trackerlist):
return protected_tag
if is_not_protected_tracker(tracker, trackerlist):
return non_protected_tag
return None
def is_tag_blank(tag): def is_tag_blank(tag):
if tag == '': if tag == '':
return True return True
def is_ignored_tag(igtags, tortags): def is_ignored_tag(igtags, tortags):
for igt in igtags: """True when the torrent carries one of the configured ignored tags.
if igt in tortags:
return True qBittorrent returns tags as a comma-separated string, so the comparison is
per tag: a configured ``ipt`` must not match a torrent tagged
``iptorrents``.
"""
if not tortags:
return False
torrent_tags = {tag.strip() for tag in str(tortags).split(',') if tag.strip()}
return any(igt in torrent_tags for igt in igtags)
+358 -31
View File
@@ -1,14 +1,7 @@
def tor_log(self): import logging
"""Setting up the log file, if self.use_log is set to true and self.loglevel is DEBUG OR INFO""" import socket
if self.use_log: from logging.handlers import RotatingFileHandler
if self.log_level.lower() == 'debug':
self.tl.basicConfig(filename=self.log_path, format='%(asctime)s:%(levelname)s:%(message)s', encoding='utf-8', datefmt='%m/%d/%Y %I:%M:%S %p',level=self.tl.DEBUG)
elif self.log_level.lower() == 'info':
self.tl.basicConfig(filename=self.log_path, format='%(asctime)s:%(levelname)s:%(message)s', encoding='utf-8', datefmt='%m/%d/%Y %I:%M:%S %p',level=self.tl.INFO)
elif self.log_level.lower() == 'warn':
self.tl.basicConfig(filename=self.log_path, format='%(asctime)s:%(levelname)s:%(message)s', encoding='utf-8', datefmt='%m/%d/%Y %I:%M:%S %p',level=self.tl.WARN)
elif self.log_level.lower() == 'error':
self.tl.basicConfig(filename=self.log_path, format='%(asctime)s:%(levelname)s:%(message)s', encoding='utf-8', datefmt='%m/%d/%Y %I:%M:%S %p',level=self.tl.ERROR)
def tor_notify(self): def tor_notify(self):
"""Seting up to use pushover, if self.use_pushover is set to true and """Seting up to use pushover, if self.use_pushover is set to true and
@@ -17,16 +10,14 @@ def tor_notify(self):
self.poc = self.po.Pushover(self.po_token) self.poc = self.po.Pushover(self.po_token)
def tor_notify_apprise(self, req_obj, app_obj): def tor_notify_apprise(self, req_obj, app_obj):
"""Use apprise""" """Use apprise. Apprise has no HTML support, so always use the plain style."""
body = f" Total: {self.total_torrents}\n\ opts = get_notify_options(self)
Premature: {self.preme_tor_counter}\n\ title, body = build_summary(
Ignored: {self.ignored_counter}\n\ self,
Protected: {self.c[self.tracker_protected_tag]}\n\ style="plain",
Non-protected: {self.c[self.tracker_non_protected_tag]}\n\ use_emoji=opts["use_emoji"],
Orphaned: {self.up_tor_counter}\n\ include_hostname=opts["include_hostname"],
Marked for deletion: {len(self.torrent_hash_delete_list)}\n\ )
{self.extm}"
title = "--- qbit-maid summary ---"
app_obj(req_obj, self.apprise_host, self.apprise_port, self.apprise_aurls, title, body) app_obj(req_obj, self.apprise_host, self.apprise_port, self.apprise_aurls, title, body)
def tornotifytest(self): def tornotifytest(self):
@@ -49,19 +40,67 @@ def print_processor(self):
self.tl.info(f'Marked for deletion: {len(self.torrent_hash_delete_list)}') self.tl.info(f'Marked for deletion: {len(self.torrent_hash_delete_list)}')
def tor_notify_summary(self): def tor_notify_summary(self):
"""Main notification method when the app is used in an automated fashion""" """Main notification method when the app is used in an automated fashion.
self.poc.message(self.po_key, f" Total: {self.total_torrents}\n\
Premature: {self.preme_tor_counter}\n\ Formatting is controlled by the ``[pushover]`` config section: ``format``
Ignored: {self.ignored_counter}\n\ (plain|html), ``use_emoji``, ``include_hostname``, ``notify_on_empty``,
Protected: {self.c[self.tracker_protected_tag]}\n\ ``priority``, ``sound``, ``retry`` and ``expire``.
Non-protected: {self.c[self.tracker_non_protected_tag]}\n\ """
Orphaned: {self.up_tor_counter}\n\ logger = _logger(self)
Marked for deletion: {len(self.torrent_hash_delete_list)}\n\ opts = get_notify_options(self)
{self.extm}", title="--- qbit-maid summary ---") title, body = build_summary(
self,
style=opts["style"],
use_emoji=opts["use_emoji"],
include_hostname=opts["include_hostname"],
)
if not opts["notify_on_empty"]:
stats = summary_stats(self)
if not stats["removed"] and not stats["orphaned"]:
logger.info("Nothing removed or orphaned, skipping notification.")
return
kwargs = {}
if opts["style"] == "html":
kwargs["html"] = 1
if opts["priority"] is not None:
kwargs["priority"] = opts["priority"]
if opts["priority"] == 2:
kwargs["retry"] = opts["retry"]
kwargs["expire"] = opts["expire"]
if opts["sound"]:
kwargs["sound"] = opts["sound"]
try:
self.poc.message(self.po_key, body, title=title, **kwargs)
except Exception as e: # never let a notification failure abort a run
logger.error(f"Failed to send pushover notification: {e}")
def tor_notify_error(self, error, title="qbit-maid API ERROR"):
"""Report an error through pushover when it is configured.
The vendored ``pushover`` module only exposes the ``Pushover`` class, so the
message has to go through the client built by :func:`tor_notify`.
"""
logger = _logger(self)
logger.error(f"{title}: {error}")
if not getattr(self, "use_pushover", False):
return
client = getattr(self, "poc", None)
if client is None:
return
try:
client.message(self.po_key, str(error), title=title)
except Exception as exc: # never let a notification failure abort a run
logger.error(f"Failed to send pushover error notification: {exc}")
def list_first_tor(self, index=0): def list_first_tor(self, index=0):
"""Only lists the first torrent""" """Only lists the first torrent"""
self.tl.debug('First torrent in the list:') self.tl.debug('First torrent in the list:')
if not self.torrent_list:
self.tl.debug('No torrents found.')
return
torrent = self.torrent_list[index] torrent = self.torrent_list[index]
for k,v in torrent.items(): for k,v in torrent.items():
self.tl.debug(f'{k}: {v}') self.tl.debug(f'{k}: {v}')
@@ -79,9 +118,297 @@ def torrent_count(self):
def get_script_runtime(self): def get_script_runtime(self):
elapsed_time = self.et - self.st elapsed_time = self.et - self.st
if self.use_log:
self.tl.info(f'Execution time: [{elapsed_time}]') self.tl.info(f'Execution time: [{elapsed_time}]')
if self.use_pushover: if self.use_pushover:
self.extm = f"Execution time: [{elapsed_time}]" self.extm = f"Execution time: [{elapsed_time}]"
if self.use_apprise: if self.use_apprise:
self.extm = f"Execution time: [{elapsed_time}]" self.extm = f"Execution time: [{elapsed_time}]"
# ---------------------------------------------------------------------------
# Notification formatting
# ---------------------------------------------------------------------------
NOTIFY_STYLES = ("plain", "html")
DEFAULT_NOTIFY_STYLE = "plain"
# Pushover renders the body in a proportional font, so space-padded columns
# collapse into a ragged list. Rows are therefore joined with a visible
# separator instead of being aligned with spaces.
ROW_SEPARATOR = " \u00b7 "
NOTIFY_EMOJI = {
"removed": "\U0001F5D1\uFE0F",
"orphaned": "\U0001F47B",
"ignored": "\u23ED\uFE0F",
"premature": "\U0001F423",
"protected": "\U0001F6E1\uFE0F",
"non_protected": "\U0001F310",
}
def _logger(self):
"""Return the configured logger, or a bare one for ad-hoc/test use."""
return getattr(self, "tl", logging.getLogger(LOGGER_NAME))
def format_runtime(value):
"""Format a timedelta as ``12s``, ``1m 04s`` or ``1h 02m 03s``."""
try:
total = int(value.total_seconds())
except AttributeError:
return str(value)
if total < 0:
total = 0
hours, remainder = divmod(total, 3600)
minutes, seconds = divmod(remainder, 60)
if hours:
return f"{hours}h {minutes:02d}m {seconds:02d}s"
if minutes:
return f"{minutes}m {seconds:02d}s"
return f"{seconds}s"
def get_notify_options(self):
"""Read notification options from ``[pushover]``, falling back to defaults."""
config = getattr(self, "config", None) or {}
po = config.get("pushover", None) or {}
style = str(po.get("format", DEFAULT_NOTIFY_STYLE) or "").lower()
if style not in NOTIFY_STYLES:
style = DEFAULT_NOTIFY_STYLE
priority = po.get("priority")
return {
"style": style,
"use_emoji": bool(po.get("use_emoji", True)),
"include_hostname": bool(po.get("include_hostname", False)),
"notify_on_empty": bool(po.get("notify_on_empty", True)),
"priority": None if priority is None else int(priority),
"sound": po.get("sound"),
"retry": int(po.get("retry", 60)),
"expire": int(po.get("expire", 3600)),
}
def summary_runtime(self):
"""Runtime string for the summary, with a fallback to ``self.extm``."""
start = getattr(self, "st", None)
end = getattr(self, "et", None)
if start is not None and end is not None:
return format_runtime(end - start)
extm = getattr(self, "extm", "") or ""
if "[" in extm and "]" in extm:
return extm[extm.index("[") + 1:extm.rindex("]")]
return "unknown"
def summary_stats(self):
"""Collect the counters shown in a notification summary."""
counter = getattr(self, "c", None)
protected_tag = getattr(self, "tracker_protected_tag", "") or ""
non_protected_tag = getattr(self, "tracker_non_protected_tag", "") or ""
def tag_count(tag):
if counter is None or not tag:
return 0
try:
return counter[tag]
except (KeyError, TypeError):
return 0
return {
"total": getattr(self, "total_torrents", 0),
"removed": len(getattr(self, "torrent_hash_delete_list", None) or []),
"orphaned": getattr(self, "up_tor_counter", 0),
"ignored": getattr(self, "ignored_counter", 0),
"premature": getattr(self, "preme_tor_counter", 0),
"protected": tag_count(protected_tag),
"non_protected": tag_count(non_protected_tag),
"runtime": summary_runtime(self),
}
def send_ping(self, req_obj, healthcheck_url):
try:
req_obj.get(healthcheck_url, timeout=5)
except req_obj.RequestException as e:
self.tl.info(f"Ping failed: {e}")
def debug_torrent_list(self):
self.tl.debug(self.torrent_list)
def _stat_row(pairs, style, use_emoji):
"""Render one line of ``emoji value label`` pairs joined by a middle dot.
The value leads and the label trails, so the numbers are the first thing
the eye lands on even though the font is proportional.
Args:
pairs: Sequence of ``(emoji_key, label, value)`` tuples.
style: ``"plain"`` or ``"html"``.
use_emoji: Prefix each pair with its emoji marker.
Returns:
A single line of text, e.g.
``\U0001F5D1\uFE0F 7 removed \u00b7 \U0001F47B 4 orphaned``.
"""
parts = []
for key, label, value in pairs:
emoji = NOTIFY_EMOJI.get(key, "") if use_emoji else ""
prefix = f"{emoji} " if emoji else ""
if style == "html":
parts.append(f"{prefix}<b>{value}</b> {label}")
else:
parts.append(f"{prefix}{value} {label}")
return ROW_SEPARATOR.join(parts)
def build_summary(self, style=DEFAULT_NOTIFY_STYLE, use_emoji=True,
include_hostname=False):
"""Build the ``(title, body)`` pair used by every notifier.
Result-first layout: the title states what happened, the body gives the
scanned total, the actionable counters, then the tracker split. Counters
are grouped two-per-line as ``emoji value label`` and separated by a middle
dot, because Pushover renders a proportional font and space-padded columns
do not line up.
Args:
style: ``"plain"`` for grouped text, ``"html"`` for Pushover HTML.
use_emoji: Prefix each counter with an emoji marker.
include_hostname: Add a ``Host:`` line above the summary.
Returns:
A ``(title, body)`` tuple of strings.
"""
if style not in NOTIFY_STYLES:
style = DEFAULT_NOTIFY_STYLE
stats = summary_stats(self)
delete_enabled = bool(getattr(self, "delete_torrents", False))
# Title: say what happened. Pushover does not render HTML in titles.
removed = stats["removed"]
noun = "torrent" if removed == 1 else "torrents"
if removed:
action = "removed" if delete_enabled else "would be removed"
title = f"qbit-maid: {removed} {noun} {action}"
elif delete_enabled:
title = "qbit-maid: nothing removed"
else:
title = "qbit-maid: nothing to remove"
lines = []
if include_hostname:
host = socket.gethostname()
if style == "html":
lines.append(f"<b>Host:</b> {host}")
else:
lines.append(f"Host: {host}")
lines.append("")
total = stats["total"]
scanned_noun = "torrent" if total == 1 else "torrents"
if style == "html":
lines.append(f"Scanned <b>{total}</b> {scanned_noun} in <b>{stats['runtime']}</b>")
else:
lines.append(f"Scanned {total} {scanned_noun} in {stats['runtime']}")
lines.append("")
removed_label = "removed" if delete_enabled else "would be removed"
lines.append(_stat_row([
("removed", removed_label, removed),
("orphaned", "orphaned", stats["orphaned"]),
], style, use_emoji))
lines.append(_stat_row([
("ignored", "ignored", stats["ignored"]),
("premature", "premature", stats["premature"]),
], style, use_emoji))
lines.append("")
lines.append(_stat_row([
("protected", "protected", stats["protected"]),
("non_protected", "public", stats["non_protected"]),
], style, use_emoji))
return title, "\n".join(lines)
# ---------------------------------------------------------------------------
# New logging infrastructure (Phase 1 & 3)
# ---------------------------------------------------------------------------
LOGGER_NAME = "qbit-maid"
DEFAULT_MAX_BYTES = 5_000_000
DEFAULT_BACKUP_COUNT = 3
def setup_logging(use_log: bool, log_path: str, log_level: str,
max_bytes: int = DEFAULT_MAX_BYTES,
backup_count: int = DEFAULT_BACKUP_COUNT) -> logging.Logger:
"""Configure and return the qbit-maid logger.
Args:
use_log: If False, adds a NullHandler and sets level to CRITICAL.
log_path: Path to the log file.
log_level: One of debug, info, warn, warning, error, critical.
max_bytes: Max size of a log file before rotation (default 5 MB).
backup_count: Number of rotated backups to keep (default 3).
Returns:
A configured :class:`logging.Logger` named ``"qbit-maid"``.
"""
logger = logging.getLogger(LOGGER_NAME)
# Make setup idempotent: clear any handlers from a previous call
for handler in list(logger.handlers):
logger.removeHandler(handler)
handler.close()
if not use_log:
logger.addHandler(logging.NullHandler())
logger.setLevel(logging.CRITICAL)
return logger
level_map = {
"debug": logging.DEBUG,
"info": logging.INFO,
"warn": logging.WARNING,
"warning": logging.WARNING,
"error": logging.ERROR,
"critical": logging.CRITICAL,
}
level = level_map.get(log_level.lower(), logging.INFO)
logger.setLevel(level)
formatter = logging.Formatter(
fmt="%(asctime)s [%(levelname)s] %(name)s:%(funcName)s:%(lineno)d - %(message)s",
datefmt="%m/%d/%Y %I:%M:%S %p",
)
# File handler with rotation
file_handler = RotatingFileHandler(
log_path, maxBytes=max_bytes, backupCount=backup_count, encoding="utf-8"
)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)
# Console handler for Docker / cron visibility
console_handler = logging.StreamHandler()
console_handler.setFormatter(formatter)
logger.addHandler(console_handler)
return logger
def log_torrent(logger: logging.Logger, level: int, torrent: dict, message: str = ""):
"""Log a torrent record with structured extra fields.
Args:
logger: The qbit-maid logger instance.
level: Logging level constant (e.g. ``logging.INFO``).
torrent: A dict with torrent fields (hash, name, tracker, tags, etc.).
message: Optional human-readable message.
"""
extra = {
"torrent_hash": torrent.get("infohash_v1", ""),
"torrent_name": (torrent.get("name", "") or "")[:20],
"torrent_tracker": torrent.get("tracker", "") or "",
"torrent_tags": torrent.get("tags", "") or "",
}
logger.log(level, message, extra=extra)
+12 -17
View File
@@ -3,34 +3,32 @@ def tor_processor(self):
If torrent meets criteria for deletion, its infohash_v1 will be appended to self.torrent_hash_delete_list If torrent meets criteria for deletion, its infohash_v1 will be appended to self.torrent_hash_delete_list
""" """
for canidate in self.tracker_list: for canidate in self.tracker_list:
if self.use_log: if self.enable_telemetry:
header = ['state','ratio','tags','added','hash','name','tracker']
row = [canidate['state'],canidate['ratio'],canidate["tags"],canidate['added_on'],canidate['infohash_v1'],canidate["name"][0:20],canidate['tracker']]
write_csv(self.cv,self.telemetry_outfile,header,row)
self.tl.debug(f'---Reviewing canidate: ["{canidate["name"][0:20]}..."] {canidate["infohash_v1"]}---') self.tl.debug(f'---Reviewing canidate: ["{canidate["name"][0:20]}..."] {canidate["infohash_v1"]}---')
if is_downloading(canidate['state']): if is_downloading(canidate['state']):
if self.use_log:
self.tl.info(f'["{canidate["name"][0:20]}..."] is still downloading and will be skipped.') self.tl.info(f'["{canidate["name"][0:20]}..."] is still downloading and will be skipped.')
continue continue
elif is_protected_under_ratio(canidate['ratio'], 1.05, self.tracker_protected_tag, canidate["tags"]): elif is_protected_under_ratio(canidate['ratio'], 1.05, self.tracker_protected_tag, canidate["tags"]):
if self.use_log:
self.tl.debug(f'["{canidate["name"][0:20]}..."] is below a 1.05 ratio({canidate["ratio"]})') self.tl.debug(f'["{canidate["name"][0:20]}..."] is below a 1.05 ratio({canidate["ratio"]})')
if is_old_tor(canidate['added_on'], self.age, self.t.time()): if is_old_tor(canidate['time_active'], self.max_age):
if self.use_log: self.tl.debug(f'["{canidate["name"][0:20]}..."] Seconds old: {canidate["time_active"]} Delta: {canidate["time_active"] - self.max_age}')
self.tl.debug(f'["{canidate["name"][0:20]}..."] Seconds old: {self.t.time() - self.age - canidate["added_on"]}')
self.torrent_hash_delete_list.append(canidate['infohash_v1']) self.torrent_hash_delete_list.append(canidate['infohash_v1'])
if self.use_log:
self.tl.info(f'Submitted ["{canidate["name"][0:20]}..."] for deletion from the protected list.') self.tl.info(f'Submitted ["{canidate["name"][0:20]}..."] for deletion from the protected list.')
elif is_protected_over_ratio(canidate['ratio'], 1.05, self.tracker_protected_tag, canidate["tags"]): elif is_protected_over_ratio(canidate['ratio'], 1.05, self.tracker_protected_tag, canidate["tags"]):
if self.use_log:
self.tl.debug(f'["{canidate["name"][0:20]}..."] is above a 1.05 ratio({canidate["ratio"]}).') self.tl.debug(f'["{canidate["name"][0:20]}..."] is above a 1.05 ratio({canidate["ratio"]}).')
self.torrent_hash_delete_list.append(canidate['infohash_v1']) self.torrent_hash_delete_list.append(canidate['infohash_v1'])
if self.use_log:
self.tl.info(f'Submitted ["{canidate["name"][0:20]}..."] for deletion from the protected list.') self.tl.info(f'Submitted ["{canidate["name"][0:20]}..."] for deletion from the protected list.')
elif is_not_protected_tor(self.tracker_non_protected_tag, canidate["tags"]): elif is_not_protected_tor(self.tracker_non_protected_tag, canidate["tags"]):
self.torrent_hash_delete_list.append(canidate['infohash_v1']) self.torrent_hash_delete_list.append(canidate['infohash_v1'])
if self.use_log:
self.tl.info(f'Submitted ["{canidate["name"][0:20]}..."] for deletion.') self.tl.info(f'Submitted ["{canidate["name"][0:20]}..."] for deletion.')
else: else:
if self.enable_dragnet: if self.enable_dragnet:
dragnet(self.cv,self.dragnet_outfile,canidate['state'],canidate['ratio'],canidate["tags"],canidate['added_on'],self.age,self.t.time(),canidate['infohash_v1'],canidate["name"][0:20],canidate['tracker']) header = ['state','ratio','tags','added','thash','tname','trname']
row = [canidate['state'],canidate['ratio'],canidate["tags"],canidate['added_on'],canidate['infohash_v1'],canidate["name"][0:20],canidate['tracker']]
write_csv(self.cv,self.dragnet_outfile,header,row)
self.tl.info(f'["{canidate["name"][0:20]}..."] is orphaned.') self.tl.info(f'["{canidate["name"][0:20]}..."] is orphaned.')
self.up_tor_counter += 1 self.up_tor_counter += 1
continue continue
@@ -42,7 +40,6 @@ def tor_delete_tags(self):
def tor_delete(self): def tor_delete(self):
"""Remove torrents, will also delete files, this keeps the filesystem clean. """Remove torrents, will also delete files, this keeps the filesystem clean.
Only pass self.torrent_hash_delete_list if you would like to keep the files.""" Only pass self.torrent_hash_delete_list if you would like to keep the files."""
if self.use_log:
self.tl.debug('Hash list submitted for deletion:') self.tl.debug('Hash list submitted for deletion:')
self.tl.debug(self.torrent_hash_delete_list) self.tl.debug(self.torrent_hash_delete_list)
if self.torrent_hash_delete_list: if self.torrent_hash_delete_list:
@@ -56,8 +53,8 @@ def is_protected_under_ratio(torratio, setratio, settag, tortag):
if torratio < float(setratio) and settag in tortag: if torratio < float(setratio) and settag in tortag:
return True return True
def is_old_tor(toradd, setage, currenttime): def is_old_tor(realage, maxage):
if toradd + setage <= currenttime: if realage >= maxage:
return True return True
def is_protected_over_ratio(torratio, setratio, settag, tortag): def is_protected_over_ratio(torratio, setratio, settag, tortag):
@@ -68,9 +65,7 @@ def is_not_protected_tor(setnonprotectedtag, tortags):
if setnonprotectedtag in tortags: if setnonprotectedtag in tortags:
return True return True
def dragnet(csv_obj,outfile,state,ratio,tags,added,age,time,thash,tname,trname): def write_csv(csv_obj,outfile,header,row):
header = ['state','ratio','tags','added','age','time','thash','tname','trname']
row = [state,ratio,tags,added,age,time,thash,tname,trname]
with open(outfile, 'a+', encoding='UTF8', newline='') as f: with open(outfile, 'a+', encoding='UTF8', newline='') as f:
writer = csv_obj.writer(f) writer = csv_obj.writer(f)
if f.tell() == 0: if f.tell() == 0:
+168
View File
@@ -0,0 +1,168 @@
"""Tests for AppriseClient — the HTTP poster and its config resolution."""
import os
import shutil
import tempfile
import unittest
from unittest import mock
import AppriseClient as apprise_module
from AppriseClient import AppriseClient, apprise_notify
class FakeResponse:
status_code = 200
class FakeRequests:
def __init__(self):
self.calls = []
def post(self, url, json=None, verify=None):
self.calls.append({"url": url, "json": json, "verify": verify})
return FakeResponse()
class TestAppriseNotify(unittest.TestCase):
def test_posts_the_expected_payload(self):
req = FakeRequests()
response = apprise_notify(
req, "apprise.local", 8088, "mailto://u:p@example.com", "Title", "Body"
)
self.assertIsInstance(response, FakeResponse)
self.assertEqual(len(req.calls), 1)
call = req.calls[0]
self.assertEqual(call["url"], "http://apprise.local:8088/notify/")
self.assertEqual(call["json"], {
"urls": "mailto://u:p@example.com",
"title": "Title",
"body": "Body",
})
self.assertFalse(call["verify"])
class TestAppriseClientConfig(unittest.TestCase):
ENV_KEYS = ("DOCKER", "host", "port", "aurls", "title", "body", "toml_path")
def setUp(self):
self.tmp = tempfile.mkdtemp()
self.addCleanup(shutil.rmtree, self.tmp, ignore_errors=True)
self._cwd = os.getcwd()
os.chdir(self.tmp)
self.addCleanup(os.chdir, self._cwd)
saved = {key: os.environ.get(key) for key in self.ENV_KEYS}
for key in self.ENV_KEYS:
os.environ.pop(key, None)
self.addCleanup(self._restore_env, saved)
self.captured = {}
patcher = mock.patch.object(
apprise_module, "apprise_notify", self._fake_notify
)
patcher.start()
self.addCleanup(patcher.stop)
def _restore_env(self, saved):
for key, value in saved.items():
if value is None:
os.environ.pop(key, None)
else:
os.environ[key] = value
def _fake_notify(self, req_obj, host, port, aurls, title, body):
self.captured.update(host=host, port=port, aurls=aurls, title=title, body=body)
return "sent"
def write_config(self, **overrides):
values = {
"host": "config-host",
"port": 9999,
"aurls": "mailto://config@example.com",
"title": "Config Title",
"body": "Config Body",
}
values.update(overrides)
with open("config.toml", "w", encoding="utf-8") as fh:
fh.write(
"[apprise]\n"
f'host = "{values["host"]}"\n'
f"port = {values["port"]}\n"
f'aurls = "{values["aurls"]}"\n'
f'title = "{values["title"]}"\n'
f'body = "{values["body"]}"\n'
)
def test_reads_values_from_environment(self):
os.environ.update({
"DOCKER": "1",
"host": "env-host",
"port": "1234",
"aurls": "mailto://env@example.com",
"title": "Env Title",
"body": "Env Body",
})
AppriseClient()
self.assertEqual(self.captured["host"], "env-host")
self.assertEqual(self.captured["port"], "1234")
self.assertEqual(self.captured["title"], "Env Title")
def test_reads_values_from_config_file(self):
self.write_config()
AppriseClient()
self.assertEqual(self.captured["host"], "config-host")
self.assertEqual(self.captured["port"], 9999)
self.assertEqual(self.captured["aurls"], "mailto://config@example.com")
self.assertEqual(self.captured["title"], "Config Title")
self.assertEqual(self.captured["body"], "Config Body")
def test_config_file_overrides_environment(self):
os.environ.update({
"DOCKER": "1",
"host": "env-host",
"port": "1234",
"aurls": "mailto://env@example.com",
"title": "Env Title",
"body": "Env Body",
})
self.write_config()
AppriseClient()
self.assertEqual(self.captured["host"], "config-host")
self.assertEqual(self.captured["title"], "Config Title")
def _write_elsewhere_config(self):
config_path = os.path.join(self.tmp, "elsewhere.toml")
with open(config_path, "w", encoding="utf-8") as fh:
fh.write(
"[apprise]\n"
'host = "toml-host"\n'
"port = 4321\n"
'aurls = "mailto://toml@example.com"\n'
'title = "Toml Title"\n'
'body = "Toml Body"\n'
)
return config_path
def test_toml_path_is_read_when_docker_is_set(self):
os.environ.update({
"DOCKER": "1",
"host": "env-host",
"port": "1234",
"aurls": "mailto://env@example.com",
"title": "Env Title",
"body": "Env Body",
})
os.environ["toml_path"] = self._write_elsewhere_config()
AppriseClient()
self.assertEqual(self.captured["host"], "toml-host")
self.assertEqual(self.captured["port"], 4321)
def test_toml_path_is_read_without_docker(self):
os.environ["toml_path"] = self._write_elsewhere_config()
AppriseClient()
self.assertEqual(self.captured["host"], "toml-host")
self.assertEqual(self.captured["port"], 4321)
self.assertEqual(self.captured["title"], "Toml Title")
if __name__ == "__main__":
unittest.main()
-21
View File
@@ -1,21 +0,0 @@
from qprocess import dragnet
import csv
import unittest
class TestDragnet(unittest.TestCase):
def test_dragnet_sanity(self):
self.cv = csv
outfile = './test_outfile.csv'
state = 'downloading'
ratio = 1.05
tags = 'ipt'
added = 1
age = 240000
time = 123456
thash = 'asfasdf23412adfqwer'
tname = 'thisismynamehahahah'
trname = 'https://localhost.stackoverflow.tech/317332f1c125bc9c1b9b14fb8e054908/announce'
dragnet(self.cv,outfile,state,ratio,tags,added,age,time,thash,tname,trname)
if __name__ == '__main__':
unittest.main()
+273
View File
@@ -0,0 +1,273 @@
#!/usr/bin/env python3
"""Tests for .gitea/scripts/komodo-deploy.sh.
Runs the real script against a local mock of the Komodo Core API, so no
network access or live Komodo instance is required.
Run with: python3 -m unittest -v test_komodo_deploy
"""
import json
import os
import socket
import subprocess
import threading
import unittest
from http.server import BaseHTTPRequestHandler, HTTPServer
REPO_ROOT = os.path.dirname(os.path.abspath(__file__))
SCRIPT = os.path.join(REPO_ROOT, ".gitea", "scripts", "komodo-deploy.sh")
def _free_port():
"""Return a port that is currently closed, for the unreachable-host test."""
s = socket.socket()
s.bind(("127.0.0.1", 0))
port = s.getsockname()[1]
s.close()
return port
class MockKomodo:
"""Minimal mock of the Komodo Core endpoints used by the deploy script.
``responses`` is a list of ``(status, body)`` pairs consumed in order. A
single-element list repeats forever, which is what the timeout test needs.
Every request is recorded in ``requests`` for assertions.
"""
def __init__(self, responses):
self.responses = list(responses)
self.requests = []
mock = self
class Handler(BaseHTTPRequestHandler):
def do_POST(self):
length = int(self.headers.get("Content-Length", 0))
raw = self.rfile.read(length).decode() if length else ""
mock.requests.append(
{
"path": self.path,
"key": self.headers.get("x-api-key"),
"secret": self.headers.get("x-api-secret"),
"body": raw,
}
)
status, body = (
mock.responses.pop(0)
if len(mock.responses) > 1
else mock.responses[0]
)
payload = json.dumps(body).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
def log_message(self, *args):
pass
self.server = HTTPServer(("127.0.0.1", 0), Handler)
self.port = self.server.server_address[1]
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
def __enter__(self):
self.thread.start()
return self
def __exit__(self, *exc):
self.server.shutdown()
self.server.server_close()
@property
def url(self):
return "http://127.0.0.1:%d" % self.port
def run_script(env_overrides, timeout=60):
"""Run the deploy script with a clean KOMODO_* environment."""
env = {k: v for k, v in os.environ.items() if not k.startswith("KOMODO_")}
env.update(env_overrides)
return subprocess.run(
["sh", SCRIPT],
env=env,
capture_output=True,
text=True,
timeout=timeout,
)
def base_env(url, **extra):
env = {
"KOMODO_URL": url,
"KOMODO_API_KEY": "test-key",
"KOMODO_API_SECRET": "test-secret",
"KOMODO_TARGET": "qbit-maid",
"KOMODO_WAIT_INTERVAL": "1",
}
env.update(extra)
return env
class TestKomodoDeployScript(unittest.TestCase):
# --- target selection -------------------------------------------------
def test_skips_when_target_empty(self):
r = run_script({"KOMODO_TARGET": ""})
self.assertEqual(r.returncode, 0, r.stderr)
self.assertIn("skipping", r.stdout)
def test_fails_when_target_empty_and_required(self):
r = run_script({"KOMODO_TARGET": "", "KOMODO_REQUIRED": "true"})
self.assertEqual(r.returncode, 1)
self.assertIn("KOMODO_REQUIRED=true", r.stderr)
def test_fails_when_url_missing(self):
r = run_script({"KOMODO_TARGET": "qbit-maid"})
self.assertEqual(r.returncode, 1)
self.assertIn("KOMODO_URL is required", r.stderr)
def test_fails_when_api_key_missing(self):
r = run_script(
{
"KOMODO_URL": "http://127.0.0.1:1",
"KOMODO_TARGET": "qbit-maid",
"KOMODO_API_SECRET": "s",
}
)
self.assertEqual(r.returncode, 1)
self.assertIn("KOMODO_API_KEY is required", r.stderr)
def test_invalid_target_type(self):
r = run_script(
{
"KOMODO_URL": "http://127.0.0.1:1",
"KOMODO_API_KEY": "k",
"KOMODO_API_SECRET": "s",
"KOMODO_TARGET": "qbit-maid",
"KOMODO_TARGET_TYPE": "bogus",
}
)
self.assertEqual(r.returncode, 1)
self.assertIn("must be 'stack' or 'procedure'", r.stderr)
# --- request shape ----------------------------------------------------
def test_sends_auth_headers_and_stack_body(self):
with MockKomodo(
[
(200, {"_id": {"$oid": "u1"}}),
(200, {"status": "Complete", "success": True}),
]
) as mock:
r = run_script(base_env(mock.url))
self.assertEqual(r.returncode, 0, r.stderr)
first = mock.requests[0]
self.assertEqual(first["path"], "/execute/DeployStack")
self.assertEqual(first["key"], "test-key")
self.assertEqual(first["secret"], "test-secret")
self.assertEqual(json.loads(first["body"]), {"stack": "qbit-maid"})
def test_procedure_target_type(self):
with MockKomodo(
[
(200, {"_id": {"$oid": "u1"}}),
(200, {"status": "Complete", "success": True}),
]
) as mock:
r = run_script(base_env(mock.url, KOMODO_TARGET_TYPE="procedure"))
self.assertEqual(r.returncode, 0, r.stderr)
self.assertEqual(mock.requests[0]["path"], "/execute/RunProcedure")
self.assertEqual(json.loads(mock.requests[0]["body"]), {"procedure": "qbit-maid"})
# --- success and failure ---------------------------------------------
def test_success_path(self):
with MockKomodo(
[
(200, {"_id": {"$oid": "u1"}}),
(200, {"status": "Complete", "success": True}),
]
) as mock:
r = run_script(base_env(mock.url))
self.assertEqual(r.returncode, 0, r.stderr)
self.assertIn("completed successfully", r.stdout)
def test_deploy_failure_surfaces_logs(self):
with MockKomodo(
[
(200, {"_id": {"$oid": "u1"}}),
(
200,
{
"status": "Complete",
"success": False,
"logs": [{"stage": "Deploy", "msg": "image pull failed"}],
},
),
]
) as mock:
r = run_script(base_env(mock.url))
self.assertEqual(r.returncode, 1)
self.assertIn("failed", r.stderr)
self.assertIn("image pull failed", r.stderr)
def test_unauthorized(self):
with MockKomodo([(401, {"error": "bad credentials"})]) as mock:
r = run_script(base_env(mock.url))
self.assertEqual(r.returncode, 1)
self.assertIn("HTTP 401", r.stderr)
def test_bad_request(self):
with MockKomodo([(400, {"error": "stack not found"})]) as mock:
r = run_script(base_env(mock.url))
self.assertEqual(r.returncode, 1)
self.assertIn("HTTP 400", r.stderr)
def test_unreachable_host(self):
r = run_script(base_env("http://127.0.0.1:%d" % _free_port()))
self.assertEqual(r.returncode, 1)
self.assertIn("Could not reach Komodo", r.stderr)
# --- polling ----------------------------------------------------------
def test_no_wait_returns_after_accept(self):
with MockKomodo([(200, {"_id": {"$oid": "u1"}})]) as mock:
r = run_script(base_env(mock.url, KOMODO_WAIT="false"))
self.assertEqual(r.returncode, 0, r.stderr)
self.assertIn("not waiting", r.stdout)
self.assertEqual(len(mock.requests), 1)
def test_polls_until_complete(self):
with MockKomodo(
[
(200, {"_id": {"$oid": "u1"}}),
(200, {"status": "Queued"}),
(200, {"status": "InProgress"}),
(200, {"status": "Complete", "success": True}),
]
) as mock:
r = run_script(base_env(mock.url))
self.assertEqual(r.returncode, 0, r.stderr)
self.assertIn("Queued", r.stdout)
self.assertIn("InProgress", r.stdout)
self.assertIn("completed successfully", r.stdout)
self.assertEqual(len(mock.requests), 4)
def test_timeout(self):
with MockKomodo(
[
(200, {"_id": {"$oid": "u1"}}),
(200, {"status": "InProgress"}),
]
) as mock:
r = run_script(
base_env(mock.url, KOMODO_WAIT_TIMEOUT="2", KOMODO_WAIT_INTERVAL="1")
)
self.assertEqual(r.returncode, 1)
self.assertIn("Timed out", r.stderr)
if __name__ == "__main__":
unittest.main(verbosity=2)
+376 -72
View File
@@ -1,103 +1,407 @@
import datetime
import unittest import unittest
from qlist import is_preme,is_cat_ignored,is_tracker_blank,is_protected_tracker,is_not_protected_tracker,is_tag_blank,is_ignored_tag
from qprocess import is_downloading,is_protected_under_ratio,is_old_tor,is_protected_over_ratio,is_not_protected_tor
class TestQbitmaid(unittest.TestCase): from qbt_fakes import FailingPushover, FakeQbt, StubPushover
def test_ispreme_sanity(self): from qlist import (
self.assertTrue(is_preme(1,1,1)) is_cat_ignored,
self.assertFalse(is_preme(1,1,3)) is_ignored_tag,
is_not_protected_tracker,
is_preme,
is_protected_tracker,
is_tag_blank,
is_tracker_blank,
)
from qlogging import (
build_summary,
format_runtime,
get_notify_options,
send_ping,
summary_stats,
tor_notify_apprise,
tor_notify_summary,
)
from qprocess import (
is_downloading,
is_not_protected_tor,
is_old_tor,
is_protected_over_ratio,
is_protected_under_ratio,
)
def test_ispreme(self):
class RecordingRequests:
"""Stands in for the requests module without touching the network."""
class RequestException(Exception):
pass pass
def test_iscatignored_sanity(self): def __init__(self):
self.assertTrue(is_cat_ignored('a', ['a','b','c'])) self.calls = []
self.assertTrue(is_cat_ignored('b', ['a','b','c'])) self.timeouts = []
self.assertTrue(is_cat_ignored('c', ['a','b','c']))
self.assertFalse(is_cat_ignored('d', ['a','b','c']))
self.assertFalse(is_cat_ignored(1, ['a','b','c']))
self.assertFalse(is_cat_ignored(1.0000000, ['a','b','c']))
def test_iscatignored(self): def get(self, url, timeout=None):
pass self.calls.append(url)
self.timeouts.append(timeout)
return "ok"
def test_istrackerblank_sanity(self):
self.assertTrue(is_tracker_blank('')) class TestPredicates(unittest.TestCase):
self.assertFalse(is_tracker_blank('a')) def test_is_preme(self):
self.assertTrue(is_preme(1, 1)) # equal to min_age counts as premature
self.assertTrue(is_preme(1, 2))
self.assertFalse(is_preme(2, 1))
def test_is_cat_ignored(self):
cats = ["a", "b", "c"]
for cat in cats:
self.assertTrue(is_cat_ignored(cat, cats))
self.assertFalse(is_cat_ignored("d", cats))
self.assertFalse(is_cat_ignored(1, cats))
self.assertFalse(is_cat_ignored(1.0, cats))
def test_is_tracker_blank(self):
self.assertTrue(is_tracker_blank(""))
self.assertFalse(is_tracker_blank("a"))
self.assertFalse(is_tracker_blank(1)) self.assertFalse(is_tracker_blank(1))
self.assertFalse(is_tracker_blank(1.00000000)) self.assertFalse(is_tracker_blank(1.0))
def test_istrackerblank(self): def test_is_protected_tracker(self):
whitelist = ["a.com", "b.me", "c.io"]
self.assertTrue(is_protected_tracker("https://a.com/", whitelist))
self.assertFalse(is_protected_tracker("https://google.com/", whitelist))
self.assertFalse(is_protected_tracker("https://d.com", whitelist))
self.assertFalse(is_protected_tracker("", whitelist))
def test_is_not_protected_tracker(self):
whitelist = ["a.com", "b.me", "c.io"]
self.assertFalse(is_not_protected_tracker("https://a.com/", whitelist))
self.assertTrue(is_not_protected_tracker("https://google.com/", whitelist))
self.assertTrue(is_not_protected_tracker("https://d.com", whitelist))
self.assertFalse(is_not_protected_tracker("", whitelist))
def test_is_tag_blank(self):
self.assertTrue(is_tag_blank(""))
for value in ("a", 1, 1.0001, False, True):
self.assertFalse(is_tag_blank(value))
def test_is_downloading(self):
self.assertTrue(is_downloading("downloading"))
self.assertFalse(is_downloading("DOWNLOADING"))
self.assertFalse(is_downloading("dOwNlOaDiNg"))
self.assertFalse(is_downloading("uploading"))
def test_is_protected_under_ratio(self):
self.assertTrue(is_protected_under_ratio(0.5, 1, "a", "a,b,c"))
self.assertFalse(is_protected_under_ratio(1.5, 1, "a", "a,b,c"))
self.assertFalse(is_protected_under_ratio(0.5, 1, "z", "a,b,c"))
def test_is_old_tor(self):
self.assertTrue(is_old_tor(1, 1))
self.assertTrue(is_old_tor(2, 1))
self.assertFalse(is_old_tor(1, 2))
def test_is_protected_over_ratio(self):
self.assertTrue(is_protected_over_ratio(2, 1, "a", "a,b,c"))
self.assertFalse(is_protected_over_ratio(0.5, 1, "a", "a,b,c"))
self.assertFalse(is_protected_over_ratio(2, 1, "z", "a,b,c"))
def test_is_not_protected_tor(self):
self.assertTrue(is_not_protected_tor("a", "a,b,c"))
self.assertFalse(is_not_protected_tor("z", "a,b,c"))
def test_is_ignored_tag(self):
self.assertTrue(is_ignored_tag(["save"], "save,public,ipt"))
self.assertFalse(is_ignored_tag(["save"], "public,ipt"))
def test_is_ignored_tag_matches_whole_tags_only(self):
"""A configured tag must match a whole tag, not a substring of one.
'ipt' must not match 'iptorrents' and 'save' must not match 'unsaved'.
"""
self.assertFalse(is_ignored_tag(["ipt"], "iptorrents"))
self.assertFalse(is_ignored_tag(["save"], "unsaved"))
self.assertTrue(is_ignored_tag(["ipt"], "save,ipt"))
self.assertTrue(is_ignored_tag(["save"], " save , public"))
self.assertFalse(is_ignored_tag(["save"], ""))
class TestSendPing(unittest.TestCase):
def test_send_ping_requests_the_url(self):
req = RecordingRequests()
send_ping(FakeQbt(), req, "https://example.com/ping/x")
self.assertEqual(req.calls, ["https://example.com/ping/x"])
def test_send_ping_uses_a_five_second_timeout(self):
req = RecordingRequests()
send_ping(FakeQbt(), req, "https://example.com/ping/x")
self.assertEqual(req.timeouts, [5])
# --- notification formatting -----------------------------------------------
class TestFormatRuntime(unittest.TestCase):
def test_format_runtime(self):
self.assertEqual(format_runtime(datetime.timedelta(seconds=0)), "0s")
self.assertEqual(format_runtime(datetime.timedelta(seconds=12)), "12s")
self.assertEqual(format_runtime(datetime.timedelta(seconds=64)), "1m 04s")
self.assertEqual(format_runtime(datetime.timedelta(seconds=3723)), "1h 02m 03s")
self.assertEqual(format_runtime(datetime.timedelta(seconds=-5)), "0s")
class TestSummaryStats(unittest.TestCase):
def test_summary_stats(self):
stats = summary_stats(FakeQbt())
self.assertEqual(stats["total"], 42)
self.assertEqual(stats["removed"], 7)
self.assertEqual(stats["orphaned"], 4)
self.assertEqual(stats["ignored"], 5)
self.assertEqual(stats["premature"], 3)
self.assertEqual(stats["protected"], 10)
self.assertEqual(stats["non_protected"], 20)
self.assertEqual(stats["runtime"], "12s")
def test_summary_stats_missing_attributes(self):
class Bare:
pass pass
def test_isprotectedtracker_sanity(self): stats = summary_stats(Bare())
self.assertTrue(is_protected_tracker('https://a.com/',['a.com','b.me','c.io'])) self.assertEqual(stats["total"], 0)
self.assertFalse(is_protected_tracker('https://google.com/',['a.com','b.me','c.io'])) self.assertEqual(stats["removed"], 0)
self.assertFalse(is_protected_tracker('https://d.com',['a.com','b.me','c.io'])) self.assertEqual(stats["runtime"], "unknown")
def test_isprotectedtracker(self):
pass
def test_isnotprotectedtracker_sanity(self): class TestBuildSummary(unittest.TestCase):
self.assertFalse(is_not_protected_tracker('https://a.com/',['a.com','b.me','c.io'])) def test_build_summary_plain(self):
self.assertTrue(is_not_protected_tracker('https://google.com/',['a.com','b.me','c.io'])) title, body = build_summary(FakeQbt(), style="plain", use_emoji=False)
self.assertTrue(is_not_protected_tracker('https://d.com',['a.com','b.me','c.io'])) self.assertEqual(title, "qbit-maid: 7 torrents removed")
self.assertEqual(
body,
"Scanned 42 torrents in 12s\n"
"\n"
"7 removed \u00b7 4 orphaned\n"
"5 ignored \u00b7 3 premature\n"
"\n"
"10 protected \u00b7 20 public",
)
def test_isnotprotectedtracker(self): def test_build_summary_plain_emoji(self):
pass _, body = build_summary(FakeQbt(), style="plain", use_emoji=True)
self.assertEqual(
body,
"Scanned 42 torrents in 12s\n"
"\n"
"\U0001F5D1\uFE0F 7 removed \u00b7 \U0001F47B 4 orphaned\n"
"\u23ED\uFE0F 5 ignored \u00b7 \U0001F423 3 premature\n"
"\n"
"\U0001F6E1\uFE0F 10 protected \u00b7 \U0001F310 20 public",
)
def test_istagblank(self): def test_build_summary_plain_has_no_leading_indent(self):
self.assertTrue(is_tag_blank('')) _, body = build_summary(FakeQbt(), style="plain")
self.assertFalse(is_tag_blank('a')) for line in body.splitlines():
self.assertFalse(is_tag_blank(1)) self.assertFalse(line.startswith(" "), f"unexpected indent: {line!r}")
self.assertFalse(is_tag_blank(1.0001))
self.assertFalse(is_tag_blank(False))
self.assertFalse(is_tag_blank(True))
def test_isdownloading_sanity(self): def test_build_summary_has_no_space_padding(self):
self.assertTrue(is_downloading('downloading')) """Pushover uses a proportional font, so padded columns never align."""
for style in ("plain", "html"):
for emoji in (False, True):
_, body = build_summary(FakeQbt(), style=style, use_emoji=emoji)
for line in body.splitlines():
stripped = line.replace(" \u00b7 ", "")
self.assertNotIn(" ", stripped,
f"padded column in {style}/{emoji}: {line!r}")
def test_isdownloading(self): def test_build_summary_groups_two_counters_per_line(self):
self.assertFalse(is_downloading('DOWNLOADING')) _, body = build_summary(FakeQbt(), style="plain")
self.assertFalse(is_downloading('dOwNlOaDiNg')) rows = [ln for ln in body.splitlines() if "\u00b7" in ln]
self.assertEqual(len(rows), 3)
for row in rows:
self.assertEqual(row.count("\u00b7"), 1, row)
def test_isprotectedunderratio_sanity(self): def test_build_summary_value_leads_label(self):
self.assertTrue(is_protected_under_ratio(0.5,1,'a','a,b,c')) """The number comes first so it is the first thing the eye lands on."""
_, body = build_summary(FakeQbt(), style="plain", use_emoji=False)
self.assertIn("7 removed", body)
self.assertIn("20 public", body)
self.assertNotIn("Removed 7", body)
self.assertNotIn("Non-protected", body)
def test_isprotectedunderratio(self): def test_build_summary_html(self):
pass title, body = build_summary(FakeQbt(), style="html")
self.assertEqual(title, "qbit-maid: 7 torrents removed")
self.assertIn("Scanned <b>42</b> torrents in <b>12s</b>", body)
self.assertIn("<b>7</b> removed", body)
self.assertIn("<b>20</b> public", body)
self.assertNotIn("---", title)
def test_isoldtor_sanity(self): def test_build_summary_emoji(self):
self.assertTrue(is_old_tor(1,2,4)) _, plain = build_summary(FakeQbt(), style="plain", use_emoji=True)
self.assertIn("\U0001F5D1\uFE0F 7 removed", plain)
self.assertIn("\U0001F47B 4 orphaned", plain)
_, html = build_summary(FakeQbt(), style="html", use_emoji=True)
self.assertIn("\U0001F5D1\uFE0F <b>7</b> removed", html)
def test_isoldtor(self): def test_build_summary_emoji_off_has_no_emoji(self):
self.assertFalse(is_old_tor(1661150664,2419200,1662049004.2101078)) _, body = build_summary(FakeQbt(), style="plain", use_emoji=False)
self.assertFalse(is_old_tor(1661150664,2419200,1662049004)) for marker in ("\U0001F5D1", "\U0001F47B", "\u23ED", "\U0001F423",
self.assertFalse(is_old_tor(1661150664.000000,2419200.0000000,1662049004.2101078)) "\U0001F6E1", "\U0001F310"):
self.assertNotIn(marker, body)
def test_isprotectedoverratio_sanity(self): def test_build_summary_delete_disabled(self):
self.assertTrue(is_protected_over_ratio(2,1,'a','a,b,c')) title, body = build_summary(FakeQbt(delete_torrents=False))
self.assertEqual(title, "qbit-maid: 7 torrents would be removed")
self.assertIn("7 would be removed", body)
self.assertNotIn("7 removed", body)
def test_isprotectedoverratio(self): def test_build_summary_body_does_not_repeat_title_phrase(self):
pass """The title carries the outcome; the body must not restate it verbatim."""
title, body = build_summary(FakeQbt(), style="plain", use_emoji=False)
self.assertNotIn("torrents removed", body)
self.assertNotIn("torrents would be removed", body)
self.assertIn("7 removed", body)
self.assertTrue(title.endswith("7 torrents removed"))
def test_isnonprotectedtor_sanity(self): def test_build_summary_nothing_to_do(self):
self.assertTrue(is_not_protected_tor('a','a,b,c')) title, _ = build_summary(FakeQbt(torrent_hash_delete_list=[], up_tor_counter=0))
self.assertEqual(title, "qbit-maid: nothing removed")
title, _ = build_summary(
FakeQbt(torrent_hash_delete_list=[], up_tor_counter=0, delete_torrents=False)
)
self.assertEqual(title, "qbit-maid: nothing to remove")
def test_isnonprotectedtor(self): def test_build_summary_singular(self):
pass title, _ = build_summary(FakeQbt(torrent_hash_delete_list=["a"], total_torrents=1))
self.assertEqual(title, "qbit-maid: 1 torrent removed")
def test_isignoredtag_sanity(self): def test_build_summary_hostname(self):
self.assertTrue(is_ignored_tag(['a','b','c'], 'first,second,third,a')) _, body = build_summary(FakeQbt(), include_hostname=True)
self.assertTrue(body.startswith("Host: "))
def test_isignoredtag(self): def test_build_summary_hostname_html(self):
self.assertTrue(is_ignored_tag(['save'], 'save,public,ipt')) _, body = build_summary(FakeQbt(), style="html", include_hostname=True)
self.assertTrue(body.startswith("<b>Host:</b> "))
# def test__sanity(self): def test_build_summary_unknown_style_falls_back(self):
# pass _, body = build_summary(FakeQbt(), style="markdown")
self.assertNotIn("<b>", body)
# def test_(self):
# pass
if __name__ == '__main__': class TestNotifyOptions(unittest.TestCase):
def test_get_notify_options_defaults(self):
opts = get_notify_options(FakeQbt())
self.assertEqual(opts["style"], "plain")
self.assertTrue(opts["use_emoji"])
self.assertFalse(opts["include_hostname"])
self.assertTrue(opts["notify_on_empty"])
self.assertIsNone(opts["priority"])
self.assertIsNone(opts["sound"])
def test_get_notify_options_emoji_can_be_disabled(self):
opts = get_notify_options(FakeQbt(config={"pushover": {"use_emoji": False}}))
self.assertFalse(opts["use_emoji"])
def test_get_notify_options_overrides(self):
fake = FakeQbt(config={"pushover": {
"format": "HTML",
"use_emoji": False,
"include_hostname": True,
"notify_on_empty": False,
"priority": 2,
"sound": "siren",
"retry": 30,
"expire": 600,
}})
opts = get_notify_options(fake)
self.assertEqual(opts["style"], "html")
self.assertFalse(opts["use_emoji"])
self.assertTrue(opts["include_hostname"])
self.assertFalse(opts["notify_on_empty"])
self.assertEqual(opts["priority"], 2)
self.assertEqual(opts["sound"], "siren")
self.assertEqual(opts["retry"], 30)
self.assertEqual(opts["expire"], 600)
def test_get_notify_options_invalid_style(self):
fake = FakeQbt(config={"pushover": {"format": "bogus"}})
self.assertEqual(get_notify_options(fake)["style"], "plain")
class TestTorNotifySummary(unittest.TestCase):
def test_tor_notify_summary_plain_payload(self):
fake = FakeQbt()
tor_notify_summary(fake)
self.assertEqual(len(fake.poc.calls), 1)
call = fake.poc.calls[0]
self.assertEqual(call["user"], "USERKEY")
self.assertEqual(call["kwargs"]["title"], "qbit-maid: 7 torrents removed")
self.assertNotIn("html", call["kwargs"])
def test_tor_notify_summary_html_payload(self):
fake = FakeQbt(config={"pushover": {"format": "html"}})
tor_notify_summary(fake)
call = fake.poc.calls[0]
self.assertEqual(call["kwargs"]["html"], 1)
self.assertIn("<b>", call["body"])
def test_tor_notify_summary_priority_two_adds_retry_expire(self):
fake = FakeQbt(config={"pushover": {"priority": 2, "retry": 30, "expire": 600}})
tor_notify_summary(fake)
kwargs = fake.poc.calls[0]["kwargs"]
self.assertEqual(kwargs["priority"], 2)
self.assertEqual(kwargs["retry"], 30)
self.assertEqual(kwargs["expire"], 600)
def test_tor_notify_summary_priority_one_has_no_retry(self):
fake = FakeQbt(config={"pushover": {"priority": 1}})
tor_notify_summary(fake)
kwargs = fake.poc.calls[0]["kwargs"]
self.assertEqual(kwargs["priority"], 1)
self.assertNotIn("retry", kwargs)
def test_tor_notify_summary_skips_when_empty(self):
fake = FakeQbt(
config={"pushover": {"notify_on_empty": False}},
torrent_hash_delete_list=[],
up_tor_counter=0,
)
tor_notify_summary(fake)
self.assertEqual(fake.poc.calls, [])
def test_tor_notify_summary_sends_when_empty_by_default(self):
fake = FakeQbt(torrent_hash_delete_list=[], up_tor_counter=0)
tor_notify_summary(fake)
self.assertEqual(len(fake.poc.calls), 1)
def test_tor_notify_summary_swallows_send_errors(self):
fake = FakeQbt()
fake.poc = FailingPushover()
tor_notify_summary(fake) # must not raise
class TestTorNotifyApprise(unittest.TestCase):
def test_tor_notify_apprise_uses_plain_body(self):
captured = {}
def app_obj(req_obj, host, port, aurls, title, body):
captured.update(host=host, port=port, aurls=aurls, title=title, body=body)
fake = FakeQbt(config={"pushover": {"format": "html"}})
tor_notify_apprise(fake, None, app_obj)
self.assertEqual(captured["title"], "qbit-maid: 7 torrents removed")
self.assertNotIn("<b>", captured["body"])
self.assertEqual(captured["port"], 8088)
def test_pushover_and_apprise_bodies_match(self):
fake = FakeQbt()
tor_notify_summary(fake)
captured = {}
tor_notify_apprise(
fake, None,
lambda req, host, port, aurls, title, body: captured.update(title=title, body=body),
)
self.assertEqual(fake.poc.calls[0]["body"], captured["body"])
self.assertEqual(fake.poc.calls[0]["kwargs"]["title"], captured["title"])
if __name__ == "__main__":
unittest.main() unittest.main()
+350
View File
@@ -0,0 +1,350 @@
"""End-to-end tests for the Qbt object in qbit-maid.py.
The module name contains a hyphen, so it cannot be imported normally; it is
loaded by path with ``qbittorrentapi`` stubbed out.
"""
import importlib.util
import logging
import os
import shutil
import sys
import tempfile
import types
import unittest
from unittest import mock
from qbt_fakes import (
FailingPushover,
FakeAPIError,
FakeQbtClient,
StubPushover,
torrent,
)
REPO = os.path.dirname(os.path.abspath(__file__))
CONFIG_TEMPLATE = """
[qbittorrent]
host = "127.0.0.1"
port = 8080
username = "user"
password = "pass"
[logging]
use_log = true
log_level = "{log_level}"
log_path = "{log_path}"
[app_tags]
protected_tag = "ipt"
non_protected_tag = "public"
[torrent]
max_age = 2419200
min_age = 432000
delete_torrents = {delete_torrents}
[pushover]
use_pushover = {use_pushover}
po_key = "USERKEY"
po_token = "TOKEN"
[apprise]
use_apprise = {use_apprise}
host = "localhost"
port = 8088
aurls = "mailto://user:pass@example.com"
[dragnet]
enable_dragnet = false
dragnet_outfile = "{dragnet_outfile}"
[telemetry]
enable_telemetry = false
telemetry_outfile = "{telemetry_outfile}"
[ignored_categories]
tech = "tech"
[ignored_domains]
tracker-example = "tracker.example"
[ignored_tags]
save = "save"
[healthcheck]
use_healthcheck = {use_healthcheck}
healthcheck_url = "https://example.com/ping/x"
"""
def load_qbit_maid(as_main=False):
"""Import qbit-maid.py by path, with qbittorrentapi stubbed.
With ``as_main=True`` the module is executed as ``__main__``, which runs
the ``Qbt()`` entry point at import time.
"""
# Install the stub once and reuse it, so tests can patch its Client.
# This keeps the suite off any real qBittorrent even when the real package
# is installed in the environment.
stub = sys.modules.get("qbittorrentapi")
if getattr(stub, "_qbit_maid_stub", False) is not True:
stub = types.ModuleType("qbittorrentapi")
stub.APIError = FakeAPIError
stub.Client = FakeQbtClient
stub._qbit_maid_stub = True
sys.modules["qbittorrentapi"] = stub
if REPO not in sys.path:
sys.path.insert(0, REPO)
spec = importlib.util.spec_from_file_location(
"__main__" if as_main else "qbit_maid", os.path.join(REPO, "qbit-maid.py")
)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
class QbtTestCase(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.mkdtemp()
self.addCleanup(shutil.rmtree, self.tmp, ignore_errors=True)
self._cwd = os.getcwd()
os.chdir(self.tmp)
self.addCleanup(os.chdir, self._cwd)
self.addCleanup(self._close_log_handlers)
self.module = load_qbit_maid()
def _close_log_handlers(self):
logger = logging.getLogger("qbit-maid")
for handler in list(logger.handlers):
logger.removeHandler(handler)
handler.close()
def write_config(self, **overrides):
values = {
"log_level": "CRITICAL", # keep the console handler quiet
"log_path": os.path.join(self.tmp, "qc.log"),
"delete_torrents": "true",
"use_pushover": "false",
"use_apprise": "false",
"use_healthcheck": "false",
"dragnet_outfile": os.path.join(self.tmp, "orphaned.csv"),
"telemetry_outfile": os.path.join(self.tmp, "telemetry.csv"),
}
values.update(overrides)
with open("config.toml", "w", encoding="utf-8") as fh:
fh.write(CONFIG_TEMPLATE.format(**values))
def build(self, torrents, client=None, **config_overrides):
self.write_config(**config_overrides)
client = client or FakeQbtClient(torrents=torrents)
def factory(**kwargs):
client.init_kwargs = kwargs
return client
with mock.patch.object(self.module.qbittorrentapi, "Client",
side_effect=factory):
qbt = self.module.Qbt()
return qbt, client
class TestQbtPipeline(QbtTestCase):
def sample_torrents(self):
return [
torrent(tags="ipt", ratio=0.5, time_active=9999999,
infohash="del-protected"),
torrent(tags="public", tracker="https://other.example/a", ratio=0.1,
infohash="del-public"),
torrent(tags="public", state="downloading", infohash="skip-downloading"),
torrent(tags="other", infohash="orphan-1"),
torrent(tracker="", infohash="ignored-blank-tracker"),
torrent(category="tech", infohash="ignored-category"),
torrent(tags="ipt", seeding_time=1, infohash="premature"),
torrent(tags="", infohash="new-blank-tag"),
]
def test_full_run_counts_every_bucket(self):
qbt, client = self.build(self.sample_torrents())
self.assertEqual(qbt.total_torrents, 8)
self.assertEqual(qbt.ignored_counter, 2)
self.assertEqual(qbt.preme_tor_counter, 1)
self.assertEqual(qbt.up_tor_counter, 2)
self.assertCountEqual(qbt.torrent_hash_delete_list,
["del-protected", "del-public"])
def test_full_run_deletes_the_selected_hashes(self):
_, client = self.build(self.sample_torrents())
self.assertEqual(len(client.deleted), 1)
delete_files, hashes = client.deleted[0]
self.assertTrue(delete_files)
self.assertCountEqual(hashes, ["del-protected", "del-public"])
def test_full_run_tags_new_torrents(self):
_, client = self.build(self.sample_torrents())
self.assertIn(("ipt", "new-blank-tag"), client.added_tags)
def test_dry_run_does_not_delete(self):
qbt, client = self.build(self.sample_torrents(), delete_torrents="false")
self.assertFalse(qbt.delete_torrents)
self.assertEqual(client.deleted, [])
self.assertCountEqual(qbt.torrent_hash_delete_list,
["del-protected", "del-public"])
def test_client_is_constructed_from_config(self):
_, client = self.build([torrent(tags="ipt")])
self.assertEqual(client.init_kwargs["host"], "127.0.0.1")
self.assertEqual(client.init_kwargs["port"], 8080)
self.assertEqual(client.init_kwargs["username"], "user")
def test_client_logs_in(self):
_, client = self.build([torrent(tags="ipt")])
self.assertTrue(client.logged_in)
def test_runtime_is_recorded(self):
qbt, _ = self.build([torrent(tags="ipt")])
self.assertIsNotNone(qbt.et)
self.assertGreaterEqual(qbt.et, qbt.st)
def test_empty_torrent_list_is_handled(self):
"""A qBittorrent instance with zero torrents must not abort the run."""
qbt, client = self.build([])
self.assertEqual(qbt.total_torrents, 0)
self.assertEqual(qbt.torrent_hash_delete_list, [])
self.assertEqual(client.deleted, [])
def test_empty_torrent_list_still_sends_a_summary(self):
stub = StubPushover()
with mock.patch.object(self.module.pushover, "Pushover",
return_value=stub):
self.build([], use_pushover="true")
self.assertEqual(len(stub.calls), 1)
self.assertEqual(stub.calls[0]["kwargs"]["title"],
"qbit-maid: nothing removed")
class TestQbtConfigSources(QbtTestCase):
def test_config_is_read_from_toml_path_env(self):
config_path = os.path.join(self.tmp, "from-env.toml")
self.write_config()
os.replace("config.toml", config_path)
os.environ["toml_path"] = config_path
self.addCleanup(os.environ.pop, "toml_path", None)
with mock.patch.object(self.module.qbittorrentapi, "Client",
return_value=FakeQbtClient(torrents=[torrent(tags="ipt")])):
qbt = self.module.Qbt()
self.assertEqual(qbt.tracker_protected_tag, "ipt")
def test_local_config_toml_wins_over_toml_path(self):
elsewhere = os.path.join(self.tmp, "elsewhere.toml")
with open(elsewhere, "w", encoding="utf-8") as fh:
fh.write(CONFIG_TEMPLATE.format(
log_level="CRITICAL",
log_path=os.path.join(self.tmp, "other.log"),
delete_torrents="true", use_pushover="false",
use_apprise="false", use_healthcheck="false",
dragnet_outfile="a.csv", telemetry_outfile="b.csv",
).replace('protected_tag = "ipt"', 'protected_tag = "from-elsewhere"'))
os.environ["toml_path"] = elsewhere
self.addCleanup(os.environ.pop, "toml_path", None)
self.write_config()
with mock.patch.object(self.module.qbittorrentapi, "Client",
return_value=FakeQbtClient(torrents=[torrent(tags="ipt")])):
qbt = self.module.Qbt()
self.assertEqual(qbt.tracker_protected_tag, "ipt")
class TestQbtNotifiers(QbtTestCase):
def test_pushover_summary_is_sent_when_enabled(self):
stub = StubPushover()
with mock.patch.object(self.module.pushover, "Pushover",
return_value=stub):
qbt, _ = self.build([torrent(tags="public", infohash="h1")],
use_pushover="true")
self.assertEqual(len(stub.calls), 1)
self.assertEqual(stub.calls[0]["kwargs"]["title"],
"qbit-maid: 1 torrent removed")
def test_apprise_is_sent_when_enabled(self):
captured = {}
def fake_apprise(req_obj, host, port, aurls, title, body):
captured.update(host=host, port=port, title=title, body=body)
with mock.patch.object(self.module, "apprise_notify", fake_apprise):
self.build([torrent(tags="public", infohash="h1")],
use_apprise="true")
self.assertEqual(captured["host"], "localhost")
self.assertEqual(captured["port"], 8088)
self.assertIn("1 removed", captured["body"])
def test_healthcheck_is_pinged_twice_when_enabled(self):
calls = []
def fake_ping(obj, req_obj, url):
calls.append(url)
with mock.patch.object(self.module, "send_ping", fake_ping):
self.build([torrent(tags="ipt")], use_healthcheck="true")
self.assertEqual(len(calls), 2)
self.assertTrue(calls[0].endswith("/start"))
self.assertFalse(calls[1].endswith("/start"))
class TestQbtLoginFailure(QbtTestCase):
def test_login_error_notifies_and_continues(self):
client = FakeQbtClient(torrents=[torrent(tags="ipt")],
login_error=FakeAPIError("bad credentials"))
stub = StubPushover()
with mock.patch.object(self.module.pushover, "Pushover",
return_value=stub):
qbt, _ = self.build([torrent(tags="ipt")], client=client,
use_pushover="true")
# The error notification goes out first, then the run summary.
self.assertEqual(len(stub.calls), 2)
self.assertEqual(stub.calls[0]["kwargs"]["title"],
"qbit-maid API ERROR")
self.assertIn("bad credentials", stub.calls[0]["body"])
self.assertEqual(qbt.total_torrents, 1)
def test_login_error_without_pushover_does_not_crash(self):
client = FakeQbtClient(torrents=[torrent(tags="ipt")],
login_error=FakeAPIError("bad credentials"))
qbt, _ = self.build([torrent(tags="ipt")], client=client)
self.assertEqual(qbt.total_torrents, 1)
def test_login_error_survives_a_failing_pushover(self):
client = FakeQbtClient(torrents=[torrent(tags="ipt")],
login_error=FakeAPIError("bad credentials"))
with mock.patch.object(self.module.pushover, "Pushover",
return_value=FailingPushover()):
qbt, _ = self.build([torrent(tags="ipt")], client=client,
use_pushover="true")
self.assertEqual(qbt.total_torrents, 1)
class TestMainGuard(QbtTestCase):
def test_running_as_main_executes_the_pipeline(self):
self.write_config()
client = FakeQbtClient(torrents=[torrent(tags="ipt")])
stub = sys.modules["qbittorrentapi"]
with mock.patch.object(stub, "Client", return_value=client):
load_qbit_maid(as_main=True)
self.assertTrue(client.logged_in)
self.assertEqual(len(client.deleted), 1)
class TestIgnoredTagsConfig(QbtTestCase):
def test_ignored_tags_reads_the_ignored_tags_section(self):
qbt, _ = self.build([torrent(tags="ipt")])
self.assertEqual(qbt.ignored_tags, {"save": "save"})
self.assertNotIn("tracker-example", qbt.ignored_tags)
def test_ignored_tag_from_config_is_skipped(self):
qbt, _ = self.build([torrent(tags="save", infohash="ignored-tag")])
self.assertEqual(qbt.ignored_counter, 1)
self.assertEqual(qbt.tracker_list, [])
if __name__ == "__main__":
unittest.main()
+130
View File
@@ -0,0 +1,130 @@
"""Tests for qlist.build_tor_list — the tagging and sorting pass."""
import unittest
from qbt_fakes import FakeQbt, torrent
from qlist import build_tor_list, torrent_tag_for
class TestTorrentTagFor(unittest.TestCase):
def test_protected_tracker_gets_the_protected_tag(self):
self.assertEqual(
torrent_tag_for("https://tracker.example/a", "ipt", "public",
["tracker.example"]),
"ipt",
)
def test_unknown_tracker_gets_the_public_tag(self):
self.assertEqual(
torrent_tag_for("https://other.example/a", "ipt", "public",
["tracker.example"]),
"public",
)
def test_blank_tracker_gets_no_tag(self):
self.assertIsNone(
torrent_tag_for("", "ipt", "public", ["tracker.example"])
)
class TestBuildTorList(unittest.TestCase):
def run_build(self, torrents, **overrides):
qbt = FakeQbt(torrent_list=list(torrents), **overrides).reset_counters()
build_tor_list(qbt)
return qbt
def test_sets_total_torrents(self):
qbt = self.run_build([torrent(infohash="a"), torrent(infohash="b")])
self.assertEqual(qbt.total_torrents, 2)
def test_blank_tracker_is_ignored(self):
qbt = self.run_build([torrent(tracker="")])
self.assertEqual(qbt.ignored_counter, 1)
self.assertEqual(qbt.tracker_list, [])
def test_ignored_category_is_ignored(self):
qbt = self.run_build(
[torrent(category="tech")],
cat_whitelist={"tech": "tech"},
)
self.assertEqual(qbt.ignored_counter, 1)
self.assertEqual(qbt.tracker_list, [])
def test_ignored_tag_is_ignored(self):
qbt = self.run_build(
[torrent(tags="save,public")],
ignored_tags={"save": "save"},
)
self.assertEqual(qbt.ignored_counter, 1)
self.assertEqual(qbt.tracker_list, [])
def test_ignored_tag_does_not_match_a_longer_tag(self):
qbt = self.run_build(
[torrent(tags="iptorrents", tracker="https://other.example/a")],
ignored_tags={"ipt": "ipt"},
)
self.assertEqual(qbt.ignored_counter, 0)
self.assertEqual(len(qbt.tracker_list), 1)
def test_blank_tag_on_protected_tracker_gets_protected_tag(self):
qbt = self.run_build([torrent(tags="", infohash="h1")])
self.assertIn(("ipt", "h1"), qbt.qbt_client.added_tags)
self.assertEqual(len(qbt.tracker_list), 1)
def test_blank_tag_on_non_protected_tracker_gets_public_tag(self):
qbt = self.run_build(
[torrent(tags="", tracker="https://other.example/announce", infohash="h2")]
)
self.assertIn(("public", "h2"), qbt.qbt_client.added_tags)
self.assertEqual(len(qbt.tracker_list), 1)
def test_blank_tag_is_applied_once_for_non_premature_torrents(self):
"""A blank-tag torrent is tagged once, not once per code path."""
qbt = self.run_build([torrent(tags="", infohash="h1")])
self.assertEqual(qbt.qbt_client.added_tags, [("ipt", "h1")])
def test_premature_blank_tag_torrent_is_tagged_once(self):
"""Premature torrents are tagged by the first block, then skipped."""
qbt = self.run_build([torrent(tags="", seeding_time=1, infohash="h3")])
self.assertEqual(qbt.qbt_client.added_tags, [("ipt", "h3")])
self.assertEqual(qbt.preme_tor_counter, 1)
self.assertEqual(qbt.tracker_list, [])
def test_premature_torrent_is_counted_and_not_queued(self):
qbt = self.run_build([torrent(seeding_time=10, tags="ipt")])
self.assertEqual(qbt.preme_tor_counter, 1)
self.assertEqual(qbt.tracker_list, [])
def test_protected_torrent_with_tag_is_queued_without_retagging(self):
qbt = self.run_build([torrent(tags="ipt")])
self.assertEqual(qbt.qbt_client.added_tags, [])
self.assertEqual(len(qbt.tracker_list), 1)
def test_non_protected_torrent_with_tag_is_queued_without_retagging(self):
qbt = self.run_build(
[torrent(tags="public", tracker="https://other.example/announce")]
)
self.assertEqual(qbt.qbt_client.added_tags, [])
self.assertEqual(len(qbt.tracker_list), 1)
def test_mixed_batch_counts_each_bucket(self):
qbt = self.run_build([
torrent(tracker=""), # ignored
torrent(category="tech"), # ignored
torrent(tags="save"), # ignored
torrent(seeding_time=1, tags="ipt"), # premature
torrent(tags="ipt", infohash="p1"), # queued
torrent(tags="public", tracker="https://o.example/a",
infohash="n1"), # queued
], cat_whitelist={"tech": "tech"}, ignored_tags={"save": "save"})
self.assertEqual(qbt.ignored_counter, 3)
self.assertEqual(qbt.preme_tor_counter, 1)
self.assertEqual(len(qbt.tracker_list), 2)
self.assertEqual(qbt.total_torrents, 6)
def test_consumes_the_input_list(self):
qbt = self.run_build([torrent(tags="ipt")])
self.assertEqual(qbt.torrent_list, [])
if __name__ == "__main__":
unittest.main()
+322
View File
@@ -0,0 +1,322 @@
"""Tests for the qlogging helpers that the notifier tests do not reach."""
import datetime
import logging
import os
import shutil
import tempfile
import unittest
from collections import Counter
from qbt_fakes import FailingPushover, FakeQbt, StubPushover, torrent
from qlogging import (
LOGGER_NAME,
debug_torrent_list,
format_runtime,
get_script_runtime,
list_first_tor,
list_qbit_api_info,
log_torrent,
print_processor,
process_counts,
send_ping,
setup_logging,
summary_runtime,
summary_stats,
tor_notify,
tor_notify_error,
tor_notify_summary,
tornotifytest,
torrent_count,
)
class FakePushoverModule:
"""Stands in for the vendored ``pushover`` module."""
def __init__(self):
self.created = []
def Pushover(self, token): # noqa: N802 - mirrors the library's name
self.created.append(token)
return StubPushover()
class FailingRequests:
class RequestException(Exception):
pass
def __init__(self):
self.calls = []
def get(self, url, timeout=None):
self.calls.append(url)
raise self.RequestException("boom")
class TestNotifySetup(unittest.TestCase):
def test_tor_notify_creates_client_when_enabled(self):
qbt = FakeQbt(use_pushover=True, po_token="TOKEN")
qbt.po = FakePushoverModule()
tor_notify(qbt)
self.assertEqual(qbt.po.created, ["TOKEN"])
self.assertIsInstance(qbt.poc, StubPushover)
def test_tor_notify_is_a_noop_when_disabled(self):
qbt = FakeQbt(use_pushover=False)
qbt.po = FakePushoverModule()
tor_notify(qbt)
self.assertEqual(qbt.po.created, [])
def test_tornotifytest_sends_a_test_message(self):
qbt = FakeQbt()
tornotifytest(qbt)
self.assertEqual(len(qbt.poc.calls), 1)
self.assertEqual(qbt.poc.calls[0]["user"], "USERKEY")
self.assertEqual(qbt.poc.calls[0]["body"], "Test Message")
self.assertEqual(qbt.poc.calls[0]["kwargs"]["title"], "qbit-maid")
class TestTorNotifyError(unittest.TestCase):
def test_sends_through_the_pushover_client(self):
qbt = FakeQbt(use_pushover=True)
tor_notify_error(qbt, RuntimeError("boom"))
self.assertEqual(len(qbt.poc.calls), 1)
self.assertEqual(qbt.poc.calls[0]["user"], "USERKEY")
self.assertIn("boom", qbt.poc.calls[0]["body"])
self.assertEqual(qbt.poc.calls[0]["kwargs"]["title"],
"qbit-maid API ERROR")
def test_skips_pushover_when_disabled(self):
qbt = FakeQbt(use_pushover=False)
tor_notify_error(qbt, RuntimeError("boom"))
self.assertEqual(qbt.poc.calls, [])
def test_skips_pushover_when_no_client_was_built(self):
qbt = FakeQbt(use_pushover=True, poc=None)
tor_notify_error(qbt, RuntimeError("boom")) # must not raise
def test_survives_a_failing_pushover(self):
qbt = FakeQbt(use_pushover=True, poc=FailingPushover())
tor_notify_error(qbt, RuntimeError("boom")) # must not raise
def test_accepts_a_custom_title(self):
qbt = FakeQbt(use_pushover=True)
tor_notify_error(qbt, RuntimeError("boom"), title="Custom")
self.assertEqual(qbt.poc.calls[0]["kwargs"]["title"], "Custom")
class TestCounters(unittest.TestCase):
def test_process_counts_buckets_by_tag(self):
qbt = FakeQbt(
ct=Counter,
tracker_list=[torrent(tags="ipt"), torrent(tags="ipt"), torrent(tags="public")],
)
process_counts(qbt)
self.assertEqual(qbt.c["ipt"], 2)
self.assertEqual(qbt.c["public"], 1)
def test_print_processor_logs_every_counter(self):
qbt = FakeQbt()
with self.assertLogs("qbit-maid-test", level="INFO") as captured:
print_processor(qbt)
joined = "\n".join(captured.output)
for label in ("Total", "Premature", "Ignored", "Protected",
"Non-protected", "Orphaned", "Marked for deletion"):
self.assertIn(label, joined)
def test_torrent_count_logs_both_tags(self):
qbt = FakeQbt()
with self.assertLogs("qbit-maid-test", level="DEBUG") as captured:
torrent_count(qbt)
joined = "\n".join(captured.output)
self.assertIn('tag["ipt"] 10', joined)
self.assertIn('tag["public"] 20', joined)
class TestDebugHelpers(unittest.TestCase):
def test_list_first_tor_logs_each_field(self):
qbt = FakeQbt(torrent_list=[torrent(name="First")])
with self.assertLogs("qbit-maid-test", level="DEBUG") as captured:
list_first_tor(qbt)
self.assertIn("First", "\n".join(captured.output))
def test_list_first_tor_honours_the_index(self):
qbt = FakeQbt(torrent_list=[torrent(name="Zero"), torrent(name="One")])
with self.assertLogs("qbit-maid-test", level="DEBUG") as captured:
list_first_tor(qbt, index=1)
self.assertIn("One", "\n".join(captured.output))
def test_list_first_tor_handles_an_empty_list(self):
qbt = FakeQbt(torrent_list=[])
with self.assertLogs("qbit-maid-test", level="DEBUG") as captured:
list_first_tor(qbt)
self.assertIn("No torrents found", "\n".join(captured.output))
def test_list_qbit_api_info_logs_versions(self):
qbt = FakeQbt()
with self.assertLogs("qbit-maid-test", level="DEBUG") as captured:
list_qbit_api_info(qbt)
joined = "\n".join(captured.output)
self.assertIn("v4.6.2", joined)
self.assertIn("2.9.3", joined)
def test_debug_torrent_list_logs_the_list(self):
qbt = FakeQbt(torrent_list=[torrent(name="Logged")])
with self.assertLogs("qbit-maid-test", level="DEBUG") as captured:
debug_torrent_list(qbt)
self.assertIn("Logged", "\n".join(captured.output))
class TestRuntime(unittest.TestCase):
def test_get_script_runtime_sets_extm_for_pushover(self):
qbt = FakeQbt(use_pushover=True, use_apprise=False)
get_script_runtime(qbt)
self.assertEqual(qbt.extm, "Execution time: [0:00:12]")
def test_get_script_runtime_sets_extm_for_apprise(self):
qbt = FakeQbt(use_pushover=False, use_apprise=True)
get_script_runtime(qbt)
self.assertEqual(qbt.extm, "Execution time: [0:00:12]")
def test_get_script_runtime_without_notifiers_leaves_extm_alone(self):
qbt = FakeQbt(use_pushover=False, use_apprise=False, extm="")
get_script_runtime(qbt)
self.assertEqual(qbt.extm, "")
def test_format_runtime_falls_back_to_str_for_non_timedelta(self):
self.assertEqual(format_runtime(12), "12")
self.assertEqual(format_runtime("n/a"), "n/a")
def test_summary_runtime_falls_back_to_extm(self):
class OnlyExtm:
extm = "Execution time: [5s]"
self.assertEqual(summary_runtime(OnlyExtm()), "5s")
def test_summary_runtime_returns_unknown_without_any_source(self):
class Nothing:
pass
self.assertEqual(summary_runtime(Nothing()), "unknown")
class TestSummaryStatsEdgeCases(unittest.TestCase):
def test_missing_tag_returns_zero(self):
qbt = FakeQbt(c={})
stats = summary_stats(qbt)
self.assertEqual(stats["protected"], 0)
self.assertEqual(stats["non_protected"], 0)
def test_unindexable_counter_returns_zero(self):
qbt = FakeQbt(c=[])
stats = summary_stats(qbt)
self.assertEqual(stats["protected"], 0)
def test_blank_tag_returns_zero(self):
qbt = FakeQbt(tracker_protected_tag="", tracker_non_protected_tag="")
stats = summary_stats(qbt)
self.assertEqual(stats["protected"], 0)
self.assertEqual(stats["non_protected"], 0)
class TestSendPing(unittest.TestCase):
def test_send_ping_swallows_request_errors(self):
qbt = FakeQbt()
failing = FailingRequests()
send_ping(qbt, failing, "https://example.com/ping/x") # must not raise
self.assertEqual(failing.calls, ["https://example.com/ping/x"])
def test_send_ping_logs_the_failure(self):
qbt = FakeQbt()
with self.assertLogs("qbit-maid-test", level="INFO") as captured:
send_ping(qbt, FailingRequests(), "https://example.com/ping/x")
self.assertIn("Ping failed", "\n".join(captured.output))
class TestSetupLogging(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.mkdtemp()
self.addCleanup(shutil.rmtree, self.tmp, ignore_errors=True)
self.log_path = os.path.join(self.tmp, "qc.log")
self.addCleanup(self._reset_logger)
def _reset_logger(self):
logger = logging.getLogger(LOGGER_NAME)
for handler in list(logger.handlers):
logger.removeHandler(handler)
handler.close()
def test_disabled_logging_uses_a_null_handler(self):
logger = setup_logging(False, self.log_path, "INFO")
self.assertEqual(logger.level, logging.CRITICAL)
self.assertEqual(len(logger.handlers), 1)
self.assertIsInstance(logger.handlers[0], logging.NullHandler)
self.assertFalse(os.path.exists(self.log_path))
def test_enabled_logging_writes_to_file_and_console(self):
logger = setup_logging(True, self.log_path, "DEBUG")
self.assertEqual(logger.level, logging.DEBUG)
self.assertEqual(len(logger.handlers), 2)
logger.info("hello from the test")
for handler in logger.handlers:
handler.flush()
with open(self.log_path, encoding="utf-8") as fh:
self.assertIn("hello from the test", fh.read())
def test_unknown_level_falls_back_to_info(self):
logger = setup_logging(True, self.log_path, "shouty")
self.assertEqual(logger.level, logging.INFO)
def test_setup_is_idempotent(self):
first = setup_logging(True, self.log_path, "INFO")
previous_file_handler = first.handlers[0]
self.assertIsNotNone(previous_file_handler.stream)
logger = setup_logging(True, self.log_path, "INFO")
self.assertEqual(len(logger.handlers), 2)
# The replaced handler is closed, so its file descriptor is released.
self.assertIsNone(previous_file_handler.stream)
def test_rotation_parameters_are_applied(self):
logger = setup_logging(True, self.log_path, "INFO",
max_bytes=1234, backup_count=7)
file_handler = logger.handlers[0]
self.assertEqual(file_handler.maxBytes, 1234)
self.assertEqual(file_handler.backupCount, 7)
class TestLogTorrent(unittest.TestCase):
def test_log_torrent_attaches_structured_fields(self):
logger = logging.getLogger("qbit-maid-test")
previous_level = logger.level
logger.setLevel(logging.DEBUG)
self.addCleanup(logger.setLevel, previous_level)
record = {}
class Capture(logging.Handler):
def emit(self, rec):
record.update(rec.__dict__)
handler = Capture()
logger.addHandler(handler)
self.addCleanup(logger.removeHandler, handler)
log_torrent(logger, logging.INFO, torrent(name="A" * 40, infohash="deadbeef"),
message="processing")
self.assertEqual(record["torrent_hash"], "deadbeef")
self.assertEqual(record["torrent_name"], "A" * 20)
self.assertEqual(record["msg"], "processing")
def test_log_torrent_tolerates_missing_fields(self):
logger = logging.getLogger("qbit-maid-test")
log_torrent(logger, logging.INFO, {}) # must not raise
class TestNotifySummarySound(unittest.TestCase):
def test_sound_is_passed_through(self):
qbt = FakeQbt(config={"pushover": {"sound": "siren"}})
tor_notify_summary(qbt)
self.assertEqual(qbt.poc.calls[0]["kwargs"]["sound"], "siren")
if __name__ == "__main__":
unittest.main()
+103
View File
@@ -0,0 +1,103 @@
"""Tests for qprocess — the deletion decision pass and the delete calls."""
import csv
import os
import shutil
import tempfile
import unittest
from qbt_fakes import FakeQbt, torrent
from qprocess import tor_delete, tor_delete_tags, tor_processor
class TestTorProcessor(unittest.TestCase):
def run_processor(self, candidates, **overrides):
qbt = FakeQbt(tracker_list=list(candidates), cv=csv, **overrides).reset_counters()
tor_processor(qbt)
return qbt
def test_downloading_torrent_is_skipped(self):
qbt = self.run_processor([torrent(state="downloading", tags="public")])
self.assertEqual(qbt.torrent_hash_delete_list, [])
self.assertEqual(qbt.up_tor_counter, 0)
def test_protected_under_ratio_and_old_is_deleted(self):
qbt = self.run_processor([
torrent(tags="ipt", ratio=0.5, time_active=9999999, infohash="old1"),
])
self.assertEqual(qbt.torrent_hash_delete_list, ["old1"])
def test_protected_under_ratio_but_young_is_kept(self):
qbt = self.run_processor([
torrent(tags="ipt", ratio=0.5, time_active=1, infohash="young1"),
])
self.assertEqual(qbt.torrent_hash_delete_list, [])
self.assertEqual(qbt.up_tor_counter, 0)
def test_protected_over_ratio_is_deleted(self):
qbt = self.run_processor([
torrent(tags="ipt", ratio=2.0, infohash="over1"),
])
self.assertEqual(qbt.torrent_hash_delete_list, ["over1"])
def test_non_protected_is_deleted(self):
qbt = self.run_processor([
torrent(tags="public", ratio=0.1, infohash="np1"),
])
self.assertEqual(qbt.torrent_hash_delete_list, ["np1"])
def test_unmatched_torrent_is_orphaned(self):
qbt = self.run_processor([torrent(tags="other", ratio=0.1)])
self.assertEqual(qbt.torrent_hash_delete_list, [])
self.assertEqual(qbt.up_tor_counter, 1)
def test_telemetry_writes_a_row_per_candidate(self):
tmp = tempfile.mkdtemp()
self.addCleanup(shutil.rmtree, tmp, ignore_errors=True)
outfile = os.path.join(tmp, "telemetry.csv")
qbt = self.run_processor(
[torrent(tags="public", infohash="t1"), torrent(tags="public", infohash="t2")],
enable_telemetry=True,
telemetry_outfile=outfile,
)
with open(outfile, newline="", encoding="UTF8") as fh:
rows = list(csv.reader(fh))
self.assertEqual(rows[0], ["state", "ratio", "tags", "added", "hash", "name", "tracker"])
self.assertEqual(len(rows), 3) # header + 2 candidates
self.assertEqual(qbt.torrent_hash_delete_list, ["t1", "t2"])
def test_dragnet_writes_orphans_only(self):
tmp = tempfile.mkdtemp()
self.addCleanup(shutil.rmtree, tmp, ignore_errors=True)
outfile = os.path.join(tmp, "orphaned.csv")
qbt = self.run_processor(
[torrent(tags="other", infohash="orph1"), torrent(tags="public", infohash="del1")],
enable_dragnet=True,
dragnet_outfile=outfile,
)
with open(outfile, newline="", encoding="UTF8") as fh:
rows = list(csv.reader(fh))
self.assertEqual(rows[0], ["state", "ratio", "tags", "added", "thash", "tname", "trname"])
self.assertEqual(len(rows), 2) # header + the single orphan
self.assertEqual(rows[1][4], "orph1")
self.assertEqual(qbt.up_tor_counter, 1)
class TestDeleteCalls(unittest.TestCase):
def test_delete_tags_removes_both_tags(self):
qbt = FakeQbt()
tor_delete_tags(qbt)
self.assertEqual(qbt.qbt_client.deleted_tags, [["ipt", "public"]])
def test_delete_removes_files_for_each_hash(self):
qbt = FakeQbt(torrent_hash_delete_list=["h1", "h2"])
tor_delete(qbt)
self.assertEqual(qbt.qbt_client.deleted, [(True, ["h1", "h2"])])
def test_delete_is_a_noop_on_empty_list(self):
qbt = FakeQbt(torrent_hash_delete_list=[])
tor_delete(qbt)
self.assertEqual(qbt.qbt_client.deleted, [])
if __name__ == "__main__":
unittest.main()
+50
View File
@@ -0,0 +1,50 @@
"""Tests for qprocess.write_csv — header-once, append semantics."""
import csv
import os
import shutil
import tempfile
import unittest
from qprocess import write_csv
HEADER = ["state", "ratio", "tags", "added", "hash", "name", "tracker"]
ROW = ["downloading", 1.05, "ipt", 1, "asfasdf23412adfqwer",
"thisismynamehahahah", "https://localhost.stackoverflow.tech/announce"]
class TestWriteCSV(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.mkdtemp()
self.addCleanup(shutil.rmtree, self.tmp, ignore_errors=True)
self.outfile = os.path.join(self.tmp, "out.csv")
def read_rows(self):
with open(self.outfile, newline="", encoding="UTF8") as fh:
return list(csv.reader(fh))
def test_writes_header_then_row(self):
write_csv(csv, self.outfile, HEADER, ROW)
rows = self.read_rows()
self.assertEqual(rows[0], HEADER)
self.assertEqual(len(rows), 2)
self.assertEqual(rows[1][4], "asfasdf23412adfqwer")
def test_header_is_written_only_once(self):
write_csv(csv, self.outfile, HEADER, ROW)
write_csv(csv, self.outfile, HEADER, ROW)
rows = self.read_rows()
self.assertEqual(rows.count(HEADER), 1)
self.assertEqual(len(rows), 3)
def test_creates_the_file_when_missing(self):
self.assertFalse(os.path.exists(self.outfile))
write_csv(csv, self.outfile, HEADER, ROW)
self.assertTrue(os.path.exists(self.outfile))
def test_values_are_round_tripped(self):
write_csv(csv, self.outfile, HEADER, ROW)
self.assertEqual(self.read_rows()[1], [str(value) for value in ROW])
if __name__ == "__main__":
unittest.main()