Improved
Browsertrix Crawler v1.12.0-beta.1
What's Changed
- Fix default user-agent to not include minor version + set sec-ua-ch-* headers by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/962
- fix issues related to profile directory placed in /profile: by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/969
- add decompress() interceptor, support undici.request() without decompression + keep content-encoding if no decompression by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/970
- Dedup Initial Implementation by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/889
- Fix browser network loading by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/963
- frame behaviors: use frame.evaluate() instead of custom evaluteWithCLI() by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/964
- rollover: check for rollover before writing new records, not after. by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/974
- Support QA with deduped crawls by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/977
- Add Indexer options to commit/cancel single crawl by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/978
- fix regression where behaviors are run prematurely in new pages due to remaining 'framenavigated' listener by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/979
- Fix up comment related to cleaning up event listeners from behaviors by @tw4l in https://github.com/webrecorder/browsertrix-crawler/pull/980
- Add OpenContainers labels to Dockerfile to support Dependabot by @Mr0grog in https://github.com/webrecorder/browsertrix-crawler/pull/972
- Interrupt instead of fail crawl when not fatal by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/973
- Fix streaming response / retry mechanism when loading from browser by @ikreymer in https://github.com/webrecorder/browsertrix-crawler/pull/975
New Contributors
- @Mr0grog made their first contribution in https://github.com/webrecorder/browsertrix-crawler/pull/972
Full Changelog: https://github.com/webrecorder/browsertrix-crawler/compare/v1.12.0-beta.0...v1.12.0-beta.1