webstreamr
|
523bbc0f41
|
fix(fetcher): use expires FlareSolverr cookie prop (#111)
|
2025-06-05 10:30:26 +00:00 |
|
WebStreamr
|
efb0f96be2
|
chore(fetcher): use well-working node user agent by default again
|
2025-06-04 18:37:51 +00:00 |
|
WebStreamr
|
aa4c672594
|
chore(fetcher): persist and use FlareSolverr cookie and user agent
doesn't seem to work, but this is best practice anyway..
|
2025-06-04 18:29:02 +00:00 |
|
WebStreamr
|
0d3a5b8919
|
chore(fetcher): use queue limit of 10 again :)
queuing too long results in a bad experience. and often there are other
non-blocked sources too, so people can choose those. or self-host.
|
2025-06-04 16:18:03 +00:00 |
|
WebStreamr
|
7387aef764
|
chore(fetcher): use FlareSolverr also for other 403 Cloudflare responses
|
2025-06-04 15:23:15 +00:00 |
|
WebStreamr
|
e6aa95f875
|
chore(fetcher): increase queue limit to 30
|
2025-06-04 15:22:59 +00:00 |
|
WebStreamr
|
2c9faebcdc
|
fix(fetcher): improve Flaresolverr fetching
- use queued fetching to not mix sessions
- log issue responses
|
2025-06-04 14:59:12 +00:00 |
|
WebStreamr
|
af65ed0094
|
feat(fetcher): add FlareSolverr support
|
2025-06-04 14:18:50 +00:00 |
|
WebStreamr
|
422f7d014c
|
chore(fetcher): hard-code user-agent again for debugging purposes
|
2025-06-04 10:54:16 +00:00 |
|
WebStreamr
|
d78dca3345
|
chore: log response headers for blocks
|
2025-06-04 08:45:08 +00:00 |
|
WebStreamr
|
f0817a3dd6
|
chore: implement queued fetching
|
2025-06-03 22:44:57 +00:00 |
|
WebStreamr
|
ef2c45e2cb
|
chore: add more error infos to results
|
2025-06-03 21:56:19 +00:00 |
|
WebStreamr
|
12d91ba0b4
|
test: fix mocked Fetcher real calls by using same headers
|
2025-06-02 09:42:26 +00:00 |
|
WebStreamr
|
c3a2f9fdc6
|
fix(fetcher): use "node" as user agent explicitly
no idea why, but this avoids challenges so far..
|
2025-06-01 19:31:25 +00:00 |
|
WebStreamr
|
2eb09fc65a
|
chore: generalize forbidden/blocked handling
|
2025-06-01 19:31:11 +00:00 |
|
WebStreamr
|
424ed78b93
|
chore(fetcher): hardcode user-agent for now to avoid using "node"
|
2025-06-01 13:40:59 +00:00 |
|
WebStreamr
|
717019e04e
|
chore(fetcher): add response header info to non-expected errors
|
2025-06-01 13:19:38 +00:00 |
|
WebStreamr
|
ca981220c9
|
chore(fetcher): remove Origin header
more is not always better I guess :)
not breaking anything, but that header is not used for normal HTTP GET
requests
|
2025-06-01 13:08:52 +00:00 |
|
WebStreamr
|
fbc1aa994c
|
fix: switch back to fetch and do not generate user agents
seems to be causing less cloudflare challenges
|
2025-05-31 21:13:52 +00:00 |
|
WebStreamr
|
5bfe2f800b
|
chore(fetcher): reduce timeout to 5s
|
2025-05-29 18:32:06 +00:00 |
|
WebStreamr
|
07ddaab7fb
|
chore(fetcher): set more headers to mimick browser requests
|
2025-05-29 17:55:20 +00:00 |
|
WebStreamr
|
02b73b41d7
|
chore(fetcher): use more realistic referer/origin header values
|
2025-05-29 17:36:43 +00:00 |
|
WebStreamr
|
0638ce0a8c
|
fix: detect and report Cloudflare challenge blocks
|
2025-05-29 17:17:17 +00:00 |
|
WebStreamr
|
b313c4eb8d
|
chore(fetcher): set Origin header and add back proxy headers
|
2025-05-23 10:52:49 +00:00 |
|
WebStreamr
|
8e791bbe24
|
chore: better handle not found streams/links
|
2025-05-23 10:40:59 +00:00 |
|
WebStreamr
|
0a99f1e501
|
chore(extractor): use HEAD instead of GET to check external URLs
|
2025-05-22 16:36:28 +00:00 |
|
webstreamr
|
8daabf1f5a
|
chore(fetcher): increase timeout to 15s (#64)
|
2025-05-20 19:22:08 +00:00 |
|
WebStreamr
|
c2c5331861
|
chore(fetcher): remove proxy headers, add default timeout
The proxy headers are not improving anything and only cause issues with
another extractor that using IP-locking.
|
2025-05-19 08:47:52 +00:00 |
|
WebStreamr
|
f635d6c233
|
chore: create ID per request for logging and error response
|
2025-05-17 21:06:51 +02:00 |
|
WebStreamr
|
4a705ce088
|
refactor: cleanup url and extractorRegistry variable/prop names
|
2025-05-17 20:25:51 +02:00 |
|
WebStreamr
|
8b82215c32
|
feat(handler): limited Frembed support
|
2025-05-17 20:20:19 +02:00 |
|
WebStreamr
|
bb395f0375
|
revert: "feat(extractor): implement Mixdrop"
This reverts commit 4c53161e98.
Mixdrop seems to be IP-locked.
|
2025-05-17 01:05:40 +02:00 |
|
WebStreamr
|
4c53161e98
|
feat(extractor): implement Mixdrop
|
2025-05-16 22:34:19 +02:00 |
|
WebStreamr
|
938f6cb2c3
|
refactor(fetcher): use origin for the referrer
|
2025-05-16 12:27:48 +00:00 |
|
WebStreamr
|
e8ed101176
|
fix(fetcher): do not overwrite config
|
2025-05-15 14:25:34 +00:00 |
|
WebStreamr
|
281cba8da1
|
chore(fetcher): always set same URL host as referer
|
2025-05-14 11:51:48 +00:00 |
|
WebStreamr
|
c6d9bfb01b
|
feat(handler): add Eurostream (it series)
|
2025-05-14 10:12:57 +00:00 |
|
WebStreamr
|
41d2f23812
|
refactor(fetcher): switch to axios for hopefully better errors
|
2025-05-13 16:05:27 +00:00 |
|
WebStreamr
|
072ec4ac0a
|
feat(fetcher): work only with URL objects, set 2 more proxy headers
|
2025-05-12 12:16:55 +00:00 |
|
WebStreamr
|
13520aa194
|
chore(logging): simplify format definition, do nothing in test
|
2025-05-12 12:16:32 +00:00 |
|
WebStreamr
|
fc024209a7
|
refactor: drop logging helper in favor of winston using DI
|
2025-05-12 09:44:52 +00:00 |
|
WebStreamr
|
b7a8fc4e76
|
feat: send also X-Real-IP next to X-Forwarded-For
|
2025-05-12 06:49:20 +00:00 |
|
WebStreamr
|
0a57114c7d
|
refactor: better names for the cache properties
|
2025-05-11 21:39:36 +00:00 |
|
WebStreamr
|
8226d0755a
|
feat: cache extractor results for 15 minutes
|
2025-05-11 21:31:43 +00:00 |
|
WebStreamr
|
b114ac11b7
|
feat: use randomized user agent for fetch
|
2025-05-11 20:05:31 +00:00 |
|
WebStreamr
|
efef039cde
|
feat: introduce Context and use it to set X-Forwarded-For in requests
|
2025-05-11 19:44:19 +00:00 |
|
WebStreamr
|
96dde8a634
|
refactor: DI for handlers, extractors and fetcher
|
2025-05-11 19:11:14 +00:00 |
|