⚠️ Breaking Change: TextExtractor Renamed and Refactored

Applies to: Users who directly used, extended, or overrode TextExtractor via textextractor.class in crawler.yaml.

What Changed:

• TextExtractor has been renamed and is now an interface.

• The default implementation is now called JSoupTextExtractor.

• If you previously specified TextExtractor via textextractor.class, you must now use the fully qualified name of the new class:

  textextractor.class: "org.apache.stormcrawler.parse.JSoupTextExtractor"
  

or just remove the line as it is the default anyway.

No Action Needed If:

• You did not override textextractor.class in your crawler.yaml.

• You did not directly extend the old TextExtractor class.

Migration Notes:

• Update custom implementations to implement the new TextExtractor interface.

• Update any references to the old TextExtractor class to JSoupTextExtractor if applicable.

What's Changed

• Rel stormcrawler 3.3.0 rc1 by @tballison in https://github.com/apache/stormcrawler/pull/1507
• Bump junit.version from 5.12.0 to 5.12.1 by @dependabot in https://github.com/apache/stormcrawler/pull/1498
• Bump org.apache:apache from 33 to 34 by @dependabot in https://github.com/apache/stormcrawler/pull/1506
• Bump com.microsoft.playwright:playwright from 1.50.0 to 1.51.0 by @dependabot in https://github.com/apache/stormcrawler/pull/1504
• Bump org.apache.solr:solr-solrj from 9.8.0 to 9.8.1 by @dependabot in https://github.com/apache/stormcrawler/pull/1500
• Bump org.mockito:mockito-core from 5.16.0 to 5.16.1 by @dependabot in https://github.com/apache/stormcrawler/pull/1499
• Bump selenium.version from 4.29.0 to 4.30.0 by @dependabot in https://github.com/apache/stormcrawler/pull/1505
• Regenerated License file after dependency upgrades by @github-actions in https://github.com/apache/stormcrawler/pull/1508
• Bump org.apache.maven.plugins:maven-surefire-plugin from 3.5.2 to 3.5.3 by @dependabot in https://github.com/apache/stormcrawler/pull/1509
• #621 Async queries in Solr by @mvolikas in https://github.com/apache/stormcrawler/pull/1488
• Update README and compiler target to Java 17 in several plugins by @rzo1 in https://github.com/apache/stormcrawler/pull/1518
• #1516 - Add config options to change the response buffer size in OpenSearch by @rzo1 in https://github.com/apache/stormcrawler/pull/1517
• Bump de.thetaphi:forbiddenapis from 3.8 to 3.9 by @dependabot in https://github.com/apache/stormcrawler/pull/1513
• Bump org.jacoco:jacoco-maven-plugin from 0.8.12 to 0.8.13 by @dependabot in https://github.com/apache/stormcrawler/pull/1511
• Bump selenium.version from 4.30.0 to 4.31.0 by @dependabot in https://github.com/apache/stormcrawler/pull/1510
• Bump org.mockito:mockito-core from 5.16.1 to 5.17.0 by @dependabot in https://github.com/apache/stormcrawler/pull/1512
• Regenerated License file after dependency upgrades by @github-actions in https://github.com/apache/stormcrawler/pull/1519
• Bump junit.version from 5.12.1 to 5.12.2 by @dependabot in https://github.com/apache/stormcrawler/pull/1520
• Bump com.ibm.icu:icu4j from 76.1 to 77.1 by @dependabot in https://github.com/apache/stormcrawler/pull/1501
• Regenerated License file after dependency upgrades by @github-actions in https://github.com/apache/stormcrawler/pull/1521
• Fixes Update NOTICE File to Reflect 2025 by @rzo1 in https://github.com/apache/stormcrawler/pull/1522
• #1298 - Re-enable hold on failure (on coverage fail) by @rzo1 in https://github.com/apache/stormcrawler/pull/1523
• Bump testcontainers.version from 1.20.6 to 1.21.0 by @dependabot in https://github.com/apache/stormcrawler/pull/1524
• Bump org.jsoup:jsoup from 1.19.1 to 1.20.1 by @dependabot in https://github.com/apache/stormcrawler/pull/1530

New Contributors

• @quangdutran made their first contribution in https://github.com/apache/stormcrawler/pull/1532

Full Changelog: https://github.com/apache/stormcrawler/compare/stormcrawler-3.3.0...stormcrawler-3.4.0