Commit graph

46 commits

Author SHA1 Message Date
LimeDrive
e8b0976332 refactor(logging): switch info/debug logs to trace across modules
convert info/debug logs to trace across modules for deeper diagnostics
without changing behavior
2026-04-03 12:09:26 +02:00
LimeDrive
09adc19273 feat(tmdb_matcher): implement orphan matcher with history
- add TMDB orphan matcher backend (orphan_matcher.py) and search client (tmdb_search.py)
- introduce Redis-backed run history tracker (history.py) for matcher runs
- extend TorrentItemModel with tmdb_match_attempted_at and index
- add Alembic migration to create tmdb_match_attempted_at and its index
- integrate new tmdb_orphan_matching job into scheduler; expose in UI
- add admin UI: tmdb_matcher.html, history/revert endpoints, and nav entry
- wire settings: scheduler_tmdb_match_interval_hours and batch size
- enhance title normalizer: strip possessive s and remove technical brackets/tokens
- update admin docs to reflect new matcher and deprecation of old cleanup
- update admin base navigation and scheduled tasks page accordingly
2026-04-02 20:43:36 +02:00
LimeDrive
acef8f2516 feat(torrent-group): add initial grouping framework for torrents
introduce initial torrent grouping: add torrent_groups table and
group_id on torrent_items. include migrations, ORM models, and DAOs.
expose admin/web endpoints to group by info_hash and by title/size.
propagate tmdb_id across grouped items. non-destructive and backwards
compatible with existing data
2026-04-02 16:13:09 +02:00
LimeDrive
2a175242a7 feat(admin): add dynamic settings management system with new UI
Introduce a comprehensive settings management system that allows runtime
configuration through a new admin interface. The system includes:

- Database-backed settings with Redis caching for performance
- Settings registry defining types, defaults, and categories
- New admin UI with navigation cards and dedicated sub-pages for each category
  (general, cache, indexers, proxy, system, TMDB)
- Shared view logic for consistent GET/POST handling across sub-pages
- Automatic seeding of defaults and application of overrides at startup
- Validation and type coercion for setting values

The previous static configuration page has been replaced with a modular,
category-based interface that provides better organization and extensibility
for future configuration needs.
2026-04-02 13:09:53 +02:00
LimeDrive
b8399900d5 feat: add mismatches management and unmatched torrents search
- Implemented a new admin page for managing TMDB mismatches, including listing, creating, and deleting mismatches.
- Added functionality to search for unmatched torrents with filters for language and type.
- Enhanced UI with improved filtering options and group toggling for better user experience.
- Introduced toast notifications for user actions such as creating mismatches and assigning TMDB IDs.
- Updated backend routes to handle mismatch creation, deletion, and assignment of TMDB IDs to selected torrents.
2026-04-01 17:51:15 +02:00
LimeDrive
d65366cce2 feat: Ajouter la gestion du cache local Postgres avec des paramètres de résultats minimum et de durée de validité 2026-04-01 15:37:54 +02:00
LimeDrive
15f3d1facb feat: Refactor torrent filtering and TMDB ID assignment logic for improved accuracy 2026-04-01 13:25:37 +02:00
LimeDrive
fa68cb2c6b feat: Implement title matching and language rules management
- Refactored title filtering logic in `filter_results.py` to utilize a new title matching module.
- Introduced normalization functions for search queries in `jackett_service.py` and `yggflix_service.py`.
- Added new endpoints for managing title and language normalization rules in the admin views.
- Implemented loading and reloading of title matching and language rules in the application lifecycle.
- Enhanced release group extraction and language detection in `parser_utils.py` using the new title matching module.
2026-04-01 11:04:04 +02:00
LimeDrive
a282aac42a chore(deps): update aiohttp-socks version and add tmdb_id type conversion
Update aiohttp-socks dependency to support versions >=0.10.0 and add integer type conversion for tmdb_id field in TorrentItemModel to ensure proper data handling
2026-03-30 23:20:33 +02:00
LimeDrive
0d0c238383 feat(trackers): add support for ABN, G3MINI, and TheOldSchool UNIT3D indexers
Remove deprecated Sharewood tracker integration and add three new private trackers:
- ABN (Abnormal): keyword-based search with name filtering
- G3MINI (Gemini Tracker): TMDB/IMDB ID-based filtering
- TheOldSchool: TMDB/IMDB ID-based filtering

Includes new API clients, result parsers, and service layers. Updates frontend UI, settings configuration, and credential handling to support the new indexers with proper passkey management.
2026-03-30 14:23:20 +02:00
LimeDrive
8468bae024 feat(admin): add metadata mappings table and complete admin UI redesign
Introduce a metadata_mappings table with DAO/model layers for admin-managed
IMDB→TMDB/title overrides, and overhaul the admin panel with a modern
Bootstrap 5 sidebar UI.

- Add metadata_mappings table with migrations (search_titles, year_override)
- Add MetadataMappingDAO with standalone session helpers for use outside FastAPI
- Add new admin pages: dashboard, mappings CRUD, maintenance, config, hash/TMDB search
- Integrate DB mapping lookups into Cinemeta and TMDB metadata providers
- Update catalog views to use DB mappings before hitting external APIs
- Rewrite all admin templates with Bootstrap 5, sidebar navigation, French labels
- Add session-based auth for admin routes with Redis session store
2026-03-29 04:46:51 +02:00
LimeDrive
0aba81abb2 feat(peer): add authenticated peer-to-peer cache sharing
Introduces a secure peer-to-peer cache sharing system using HMAC-SHA256 request signatures and Fernet (AES-128-CBC) response encryption.

- New peer_keys table for per-peer credential management (key_id, secret)
- New /api/peer/check endpoint returns encrypted debrid availability data
- New /api/peer/items endpoint returns encrypted torrent metadata
- Rate limiting per peer key with configurable limits and windows
- Admin UI for creating, viewing, revoking, and deleting peer keys
- Peer cache enrichment integrated into BaseDebrid L2.5 lookup pipeline
- StremThruDebrid override with cross-service PG L2 check + peer cache
- Removed old /api/share/cache endpoint (replaced by /api/peer/*)
- Added cryptography package dependency

BREAKING CHANGE: The /api/share/cache endpoint has been removed and replaced with /api/peer/check. Clients must update to use HMAC authentication with X-Peer-Key-Id, X-Peer-Timestamp, and X-Peer-Signature headers.
2026-03-28 05:07:40 +01:00
LimeDrive
0c8bcb48f9 feat(share): add debrid cache availability check endpoint
Add new `/share/cache` API endpoint that returns which info-hashes are confirmed cached on a given debrid service. Uses hash-only projection to avoid leaking file metadata. Includes API key auth and Redis-based rate limiting.
2026-03-26 16:21:55 +01:00
LimeDrive
0c1d305784 feat(debrid): add PostgreSQL L2 cache for persistent debrid availability
Add a two-level caching architecture for debrid service availability lookups:
- L1: Redis (existing) with 10-day TTL for fast in-memory access
- L2: PostgreSQL with 30-day TTL for persistent caching across Redis restarts

Changes:
- Add `debrid_cache` table with unique constraint on (info_hash, service) and indexes for efficient queries
- Create `DebridCacheDAO` with batch get/upsert and invalidation methods
- Add pgCron job to periodically clean up expired cache entries
- Extend `get_availability_bulk_cached` to check PG on Redis miss, warm Redis from PG hits, and write confirmed-cached results to PG
- Add `invalidate_availability_cache` method to remove false positives from both caches when NO_CACHE_VIDEO_URL is returned during playback
- Integrate `db_session` dependency into playback and search views
- Add retry handling to Yggflix API (3 retries, ignore Retry-After headers) and rate-limiting delay (0.3s) between consecutive queries
2026-03-26 15:47:37 +01:00
LimeDrive
9203464680 refactor(logging): downgrade verbose log levels from info to debug/trace
Lower log verbosity across the codebase by converting info-level logs to debug
and debug-level logs to trace. This reduces noise in production while keeping
detailed information available for debugging when needed.

Affected modules: API key DAO, Real Debrid token manager, C411/Torr9/YGG
indexers, debrid services, torrent container, playback/stream handlers,
search views, and utility modules.
2026-03-26 13:49:57 +01:00
LimeDrive
62fb267622 refactor(logging): optimize log levels and enhance secret redaction
Convert verbose debug messages to trace level for cleaner production logs, improve secret redaction patterns to handle JWT tokens and API key parameters while preserving parameter names, and standardize log messages to English for consistency across the codebase.
2026-03-25 16:56:32 +01:00
LimeDrive
5e183c48a0 refactor: async fixes, security hardening, and code deduplication
Async correctness:
- Wrap blocking time.sleep/requests.get calls with asyncio.to_thread()
  in torrent_service.py and torr9_service.py
- Run cache_container_items() as fire-and-forget background task via
  asyncio.create_task(asyncio.to_thread(...)) in search/views.py
- Make RDTokenManager.get_access_token() and new_access_token() async,
  replace requests.post with aiohttp (token_manager.py)
- Make RealDebrid.get_headers() async, await all callers (realdebrid.py)

Security:
- Remove API key, device code, and PIN values from log output
  (auth/views.py, admin/views.py)
- Add WARNING on startup when SESSION_KEY uses the insecure default
  value (settings.py)
- Make admin template dir configurable via settings.admin_template_dir

Performance:
- Parallelize TMDB metadata fetches with asyncio.gather() (catalog/views.py)
- Add tmdb_language setting instead of hardcoded "fr-FR"

Deduplication:
- Extract ensure_uuid() to stream_fusion/web/api/utils.py
- Extract get_client_ip() to stream_fusion/web/utils.py
- Replace 8-way if/elif debrid factory with _SERVICE_MAP dict +
  _build_service() helper (get_debrid_service.py)
- Extract _model_to_schema() in APIKeyDAO to remove 6 repeated blocks
- Simplify ProxyStreamer: remove manual BytesIO buffer, stream directly
  via iter_any() (stream/views.py)

Inspired by commits 823bc77 and e306eb5 from limedrive/develop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 19:05:33 +01:00
Telkaoss
4fc4a5cacc update 2026-03-02 11:32:28 +01:00
Telkaoss
ca8479e924
Create 2025-01-11-00-00_add_proxied_links.py 2025-10-11 14:39:44 +02:00
Telkaoss
3c57fe0011
Delete stream_fusion/services/postgresql/migrations/2025-01-11-00-00_add_proxied_links.py 2025-10-11 14:38:37 +02:00
Telkaoss
3f9039622d
Create 2025-01-11-00-00_add_proxied_links.py 2025-10-11 14:37:53 +02:00
Telkaoss
79c720e8a5
Update apikey_dao.py 2025-06-11 12:38:38 +02:00
Telkaoss
b570822ad6
Update apikey_dao.py 2025-05-23 06:08:48 +02:00
Telkaoss
90cb7e70fd
Update apikey_schemas.py 2025-05-23 05:48:46 +02:00
Telkaoss
7be417f4c1 update 2025-05-23 05:31:13 +02:00
Telkaoss
2c1892c260 update 2025-05-23 04:48:08 +02:00
Telkaoss
202a5531d8
Create __init__.py 2025-05-23 04:03:59 +02:00
Telkaoss
a7bfa9dcd2
Create db_init.py 2025-05-23 04:03:25 +02:00
Telkaoss
ddeec4d5ab
Update apikey_dao.py 2025-05-23 04:01:50 +02:00
Telkaoss
c76df27bca Stremthru 2025-05-05 05:45:09 +02:00
Telkaoss
35d35993a3 bug fix 2025-05-04 12:41:57 +02:00
Telkaoss
1530d0557c Stremthru, async ++
Async Streaming: Converted key methods (fill_buffer, stream_content, close, get_stream_link, head_playback) to async for non-blocking streaming and improved scalability.

🔄 Unified Stremthru: Full refactor with dynamic STORE_CODE_TO_NAME / STORE_NAME_TO_TOKEN_KEY mappings and propagation of store_name in all requests (add_magnet, unrestrict_link, etc.). Updated RealDebrid to delegate via Stremthru.

💾 Cache Persistence: Store valid Stremthru links in Redis (stremthru:working:{store_code}:{info_hash}, TTL 7 days) and leverage this cache to mark links as "working" even on cached results.
2025-05-04 05:34:36 +02:00
LimeDrive
6753fad06a Refactor seeders attribute in TorrentItemModel fix [Bug]some error on selfhost related to postgresql #62 2024-10-24 11:50:09 +00:00
LimeDrive
66dbf4ae60 refactor: 🎨 ++ clean logs and torbox test/rf 2024-10-09 13:19:42 +00:00
LimeDrive
e70dcc5ddc refactor: 🐛 improve rd sdk 2024-10-02 20:15:50 +00:00
LimeDrive
b73f2d9070 log refractor and put redis on configurable database. 2024-09-23 11:23:55 +00:00
LimeDrive
e098aa1d82 Refactor give ability to use multiple debrid service in the mean time, following where the torrent are cached 2024-09-22 17:34:07 +00:00
LimeDrive
d6b5d320aa Refactor API key renewal logic and improve API key table UI 2024-09-22 00:25:56 +00:00
LimeDrive
21b30a46fc Dev Container 2024-09-21 14:07:52 +00:00
LimeDrive
e215e0d666 CacheKey that messup 2024-09-21 01:34:18 +02:00
LimeDrive
47913db1cc Refactor Redis cache close to use async/await 2024-09-21 00:48:53 +02:00
LimeDrive
12eb249508 add alambic and migration system for DB 2024-09-20 15:47:04 +02:00
LimeDrive
e1bf2accab Switch APIKEY the auth on postgres
update 16 files and delete 2 files
2024-09-19 17:51:18 +02:00
LimeCat on the Hub's
b5491853a4 Refactor APIKeyDAO to add revoke_key method 2024-09-19 01:16:47 +00:00
LimeDrive
c52a537320 Refactor PostgreSQL database setup and cleanup logic with inner pg_cron fonction 2024-09-18 19:10:25 +02:00
LimeDrive
86efdd0113 Add PostgreSQL related files and modules 2024-09-18 18:13:42 +02:00